Platform Engineering

Diagnosing and fixing a slow XenForo community

A forum's performance problem is almost never where the front page suggests. Thread rendering at depth, unread-state queries and search over a long history are the expensive paths, and they are invisible until a community is large enough for them to matter.

The problem

Forum load has a shape that general tuning does not fit

A forum behaves unlike a content site. Its audience arrives in a predictable daily peak, reads deeply rather than browsing, and returns repeatedly — which means the expensive operations are repeated per member rather than served once from cache. Logged-in traffic cannot be cached at the edge, unread-state is computed per user, and search runs across a decade of posts. Generic optimisation advice addresses none of it.

  • Page load times have grown as the community's history accumulated
  • Performance is acceptable at quiet hours and poor during the evening peak
  • Logged-in members experience a slower site than logged-out visitors
  • Thread pages with many replies load disproportionately slowly
  • Search has become noticeably slow across a long posting history
  • The cache hit rate is low and nobody can explain why
  • Adding server resources improved things briefly and then stopped
  • Nobody can say which pages or queries are actually expensive
Who this is for

The people who usually bring us this problem

A community owner whose forum has become slow

It worked fine at a smaller size and the degradation has been gradual enough that nobody noticed when it started.

An operator who has already added hardware

More memory or a faster CPU helped for a while and then stopped, which usually means the constraint is not the machine.

Someone migrating and wanting the destination to be better

You would rather not carry the current performance problems onto new infrastructure.

What it costs

What this costs while it goes unfixed

Engineering faults are rarely confined to the engineering layer. These are the commercial consequences we see most often.

Forum performance degrades with success, not with age

Query cost and storage grow with posting history and attachment volume. An architecture that was comfortable at one size becomes abruptly inadequate at another, which removes the gradual warning a linear decline would give.

Slow pages cost participation, not just patience

A community's value comes from members returning daily. Slowness reduces how often they do, and that decline looks like waning interest rather than a performance problem.

Hardware is the wrong lever when queries are the cause

Provisioning more resource raises the ceiling without reducing the work. It produces an improvement that stops, and a bill that continues — the most common and most expensive wrong turn in this work.

Search and indexation suffer alongside the members

Forum pages are unusually deep and unusually numerous. Server response and latency affect crawl frequency, so a performance problem becomes a discovery problem with a lag of weeks.

What we do about it

Capabilities

Each of these is work we carry out, not an area we advise on.

Profiling against production-scale data

Finding the expensive paths on a copy of the real database. Query cost on a community with a decade of posts is not visible in a development environment with a few thousand rows, which is why performance work on forums has to use real data.

Query and index analysis

Examining the queries the platform issues under load and the indexes supporting them — unread state, thread rendering at depth, member activity, attachment lookups. Missing or misordered indexes are the most common single cause and the cheapest to correct.

Caching strategy for what a forum can actually cache

Establishing the split between cacheable and uncacheable traffic honestly. On a community, a large share of requests are per-member and cannot be cached at all; a configuration that assumes otherwise produces a cache with almost no hit rate and an illusion of optimisation.

Server tuning for the forum's load shape

Runtime, database and connection handling configured for concurrent authenticated readers rather than for sequential cacheable requests. The tuning that suits a content site is frequently wrong here.

Attachment and media handling

Storage that only grows, served through the application when it need not be. Moving attachment delivery out of the render path is often the largest available improvement and is independent of the database.

Search infrastructure

Search over a long posting history is a distinct expensive path and is frequently delegated to a separate system. Whatever the platform uses is treated as part of the platform and tuned as such.

Peak-load verification

Confirming the improvement under the concurrency the community actually produces, not at a quiet hour. A change validated at average load has not been validated against the condition that matters.

Monitoring so it does not regress

Instrumentation on response time and query cost, so the next accumulation is visible as a trend rather than as a complaint from members.

How we work

Engineering methodology

The sequence is deliberate. The order is usually what determines whether the work holds or has to be repeated.

  1. Measure before changing anything

    Which page types, which queries, which hours and which traffic. Forum performance intuition is unreliable: the front page is usually fine, the expensive paths are deeper, and work targeted at the visible page frequently addresses nothing.

  2. Separate cached from uncached, member from guest

    These are different systems sharing a domain. Most forum performance work goes wrong by optimising the path that is already fast and cacheable while the per-member path stays expensive and invisible in aggregate reporting.

  3. Fix the query before the hardware

    Where the constraint is query cost, more resource raises the ceiling without reducing the work, and the improvement expires. Hardware is the right answer when capacity is genuinely the constraint, which is established rather than assumed.

  4. Work against production-scale data

    A replica of the real database, because that is where the expensive plans appear. Changes validated against a small dataset are validated against conditions the community does not operate in.

  5. Verify under the community's own peak

    The improvement is confirmed at the concurrency the forum actually reaches. A quiet-hour measurement is a statement about the quiet hour.

  6. Leave the measurement running

    Response time and query cost instrumented, so the next degradation is caught as a trend. Performance work that is not monitored reverts, and on a forum the revert is attributed to growth rather than to a change.

Deliverables

What an engagement produces

Documentation is a deliverable, not an afterthought. On most of these engagements a large part of the value is a defect report precise enough for another team to act on.

Diagnosis

  • Expensive page types and queries, identified against production-scale data
  • Index review with the missing or misordered indexes named
  • Cacheable versus uncacheable traffic split, stated honestly
  • Peak concurrency measured rather than assumed
  • Which layer is binding: query, application, storage, delivery or capacity

Remediation

  • Index and query corrections
  • Caching configuration matched to what a forum can actually cache
  • Runtime, database and connection tuning for concurrent authenticated readers
  • Attachment and media delivery moved out of the render path where warranted
  • Search path tuning where it is a separate system

Verification

  • Measurement against the pre-change baseline under peak conditions
  • Response-time and query-cost instrumentation left in place
  • A written record of what changed and what it moved
Under the hood

Architecture and technology

Where forum performance actually goes

  • Unread-state queries, computed per member and repeated on every visit
  • Thread rendering at depth, where a long thread is disproportionately expensive
  • Search across a posting history that only grows
  • Attachment and media delivery served through the application
  • Connection handling sized for requests that each hold a connection longer than a cached page would
  • Indexes that suited the data at one size and not at another

What a forum cannot do

  • Edge-cache authenticated member traffic — a large share of requests are per-member
  • Rely on session-level caching for state that is computed per user per visit
  • Treat a small dataset as representative of production
  • Substitute capacity for query cost and expect a permanent improvement
Related work

Where we have done this

Engagements where this capability was the substance of the work rather than a line item.

Online communities

Spain's largest forum

Since 2019, Soludome has managed infrastructure and platform operations for one of Spain's largest online communities. The work covers server and performance optimisation, CDN architecture, load balancing and site security under sustained traffic.

Since 2019continuous engagement
Online communities

Arrse.co.uk

An established community platform had accumulated three separate problems that compounded each other: page loads that had degraded with growth, search visibility that had not kept pace with the community's authority, and a security posture that had not been revisited as the platform's profile grew. The engagement addressed all three concurrently.

4%increase in organic clicks
Adjacent problems

If this is not quite your problem

These overlap at the edges. Sending you to the right page is more useful than having you work it out.

The constraint is the infrastructure rather than the platform

Capacity, serving path and ongoing operational ownership for a community.

Managed XenForo hosting

The cause is a customisation rather than the platform

Extension and template cost, and how to build functionality that does not add it.

Custom XenForo add-on development

Performance is harming search rather than the members

Response time and latency measured through their crawl consequences.

SEO server-error remediation

You need general performance engineering rather than forum-specific

Field-data method and budgets for the whole estate.

Website performance engineering
Questions

Frequently asked

We added more memory and it helped briefly. Why?

That pattern usually means the constraint is the work rather than the resource. More memory raises the ceiling the application hits, so the improvement is real and temporary — the queries still cost the same, and they accumulate until the new limit is reached. Establishing whether the cost is query, storage or delivery comes before recommending any capacity change, and the answer is frequently that a small number of queries need indexes.

Why is our cache hit rate so low?

Because a forum's traffic is largely uncacheable by nature. Members are authenticated, their pages contain their own unread state, and two members viewing the same thread see different things. A caching configuration designed for anonymous content traffic will report a low hit rate on a community and that is accurate rather than broken. The value is in caching what genuinely can be cached and reducing the cost of what cannot.

Can you work on our live forum or do you need a copy?

The diagnosis uses a copy of the production database, because that is where the expensive query plans actually appear and because profiling production adds load to the system already under pressure. The remediation applies to production through the release process, and verification is measured there. What we will not do is tune a live community from measurements taken on a development dataset, because those measurements are about a different site.

Will our forum be offline during this?

For most of the work, no — an index addition runs online, configuration changes apply without taking the forum down, and attachment delivery changes are gradual. Where a change genuinely requires a window it is identified in the assessment and scheduled outside the community's busy hours, which on a forum are known rather than guessed.

How much faster will it be?

We will not put a number on it before the diagnosis, and afterwards we will report what was measured rather than a rounded claim. Our published forum performance work is described without figures because publishing them needs written permission and the exact scope of the changes, which is a constraint we have recorded rather than worked around. What we will do is show you the before and after on your own data.

Bring us the problem you have not been able to fix

Describe what is happening rather than what you think the cause is. If we are not the right people for it, we will say so.