Search Engineering

Technical SEO for sites where the problem is technical

When content is good and rankings still fall, the cause is usually in how the site is served rather than what it says. We diagnose that layer directly — crawl behaviour, rendering, server responses, indexation — and fix it at the source.

The problem

Technical SEO becomes an engineering problem at scale

Most SEO work is content work. On a large site, a dynamic one, or one where the platform itself is unstable, the binding constraint is somewhere else entirely — and it cannot be resolved from a content editor. The symptoms below are the ones that indicate a technical cause rather than a content one.

Pages are crawled but not indexed, and no content change moves them
Crawl rate dropped and never recovered
Recurring 5xx errors at predictable times, or unplanned downtime
A site migration or replatform that lost traffic and never regained it
Content rendered client-side that search engines do not appear to see
Indexation of a templated page type collapsed across the whole estate at once
Impressions fell without any corresponding change in content

Who this is for

The people who usually bring us this problem

Head of SEO / SEO Director

You have diagnosed the problem and cannot get it fixed, because the fix is in the application, the server or the template — not in the CMS. You need someone who can produce a defect report a development team will act on.

CTO / VP Engineering

You have been handed an SEO problem that is really a reliability or architecture problem, and you need the search consequences explained in engineering terms.

CMO / VP Marketing

Organic acquisition has declined and the agency's recommendations have not changed the outcome. You need to know whether the cause is content, or something below it.

What it costs

What this costs while it goes unfixed

Engineering faults are rarely confined to the engineering layer. These are the commercial consequences we see most often.

Growth spending compounds against a ceiling

Content and paid spend both compete for an index the site cannot fully earn. Until the technical fault is removed, every other acquisition investment is capped by it.

Fresh content loses its window

For news, listings and anything time-sensitive, discovery speed is the whole value. Content found late has already lost most of its commercial life.

Impression loss presents as a content problem

When a template-level defect suppresses a whole page type, the symptom looks like declining content quality. Teams often respond by writing more content, which does not address it.

What we do about it

Capabilities

Each of these is work we carry out, not an area we advise on.

Crawl and indexation diagnosis

Establish what is actually being crawled, at what rate, and what is being excluded — from server logs and Search Console data rather than inference. This is the step that distinguishes a crawl problem from an indexing problem, which have different remedies.

Server response and crawl-rate engineering

Recurring 5xx responses and instability reduce how much a search engine is willing to crawl. We instrument the failure window, produce reproducible evidence for the development team, and verify the recovery.

Rendering and JavaScript SEO

Establish what a crawler can see versus what a browser sees, and resolve the difference — server-side rendering, pre-rendering, or restructuring what depends on client-side execution.

Sitemap and discovery architecture

Sitemaps as an architectural signal rather than a formality: separating time-critical content from the permanent estate so crawlers can treat them differently.

Template-level defect remediation

Many of the highest-impact faults are in the template and affect an entire page type at once — metadata, canonical handling, pagination, status codes. We find them at template level and fix them once.

Migration and replatform safety

URL mapping, redirect architecture, parity verification and post-launch monitoring, so a migration does not silently discard the equity the old site had earned.

Structured data at scale

Implement and validate schema that is supported by visible page content, generated from the content model rather than hand-written per page.

SERP and visibility monitoring

Publicly available SERP APIs supply what first-party reporting cannot: where a page actually positions, which features occupy the result — featured snippets, people-also-ask, AI overviews — and how competitor visibility moves around it. We use them to track ranking and feature movement, and to explain impression changes that search-performance data alone leaves unattributed. These are third-party observations of public results, and they are kept distinct from the site's own impression and click reporting, which only the site owner can measure. Using one to describe the other would be a misstatement of source, so the two are never merged in our reporting.

How we work

Engineering methodology

The sequence is deliberate. The order is usually what determines whether the work holds or has to be repeated.

Establish the actual constraint
Start from logs, Search Console data and server behaviour. The reported symptom is frequently two layers above the cause — 'pages are not indexed' is often a crawl-budget symptom, and crawl-budget symptoms are often a server-reliability symptom.
Separate what is broken from what is merely suboptimal
A defect register with severity, so remediation effort goes to the faults that are actually suppressing performance rather than to a long list of advice.
Fix at the level the fault exists at
A template defect is fixed in the template. An infrastructure defect is fixed in the infrastructure. Page-by-page work on a systemic problem produces a permanent maintenance burden and never fully resolves.
Produce evidence the development team can act on
A recurring error at a consistent hour with logs and timestamps is actionable. 'The site seems unstable' is not. A large share of our output on these engagements is defect documentation precise enough to be worked.
Verify the fix, then watch it
Confirm the change reached production, confirm the metric moved, then monitor to establish that it stayed moved. Search infrastructure responds slowly and partly, so a single post-fix check is not evidence.

Deliverables

What an engagement produces

Documentation is a deliverable, not an afterthought. On most of these engagements a large part of the value is a defect report precise enough for another team to act on.

Diagnosis

Crawl and indexation audit from logs and Search Console
Server-response and reliability analysis
Rendering comparison between crawler and browser
Template-level defect register with severity

Remediation

Defect documentation precise enough for a development team
Sitemap and discovery architecture changes
Structured data implementation and validation
Redirect and migration architecture where applicable

Verification

Post-fix indexation and impression monitoring by page type
Crawl-rate and server-response tracking
Ranking and SERP-feature monitoring from publicly available SERP APIs
Before-and-after evidence suitable for internal reporting

Under the hood

Architecture and technology

What we inspect

HTTP status codes across the estate and by template
Crawl rate and crawl distribution across path segments
Rendered HTML versus source HTML
Canonical and pagination handling
Sitemap coverage against the real route inventory
Server error distribution by time of day

What we change

Sitemap structure and freshness signalling
Template-level metadata and canonical logic
Server and caching configuration affecting crawler-visible response
Redirect architecture
Content-model-driven structured data

Related work

Where we have done this

Engagements where this capability was the substance of the work rather than a line item.

Fintech & capital markets

Search engineering at stockbroking scale

A stockbroking platform publishing at news velocity was losing search visibility to problems that had nothing to do with content quality. Two workstreams ran in parallel: sustaining a high-volume editorial output across business and market categories, and diagnosing the technical faults — a domain safety flag, recurring server errors, and a metadata defect on a templated page type — that were suppressing how much of that output search engines could actually reach.

225Msitewide impressions

Online communities

Arrse.co.uk

An established community platform had accumulated three separate problems that compounded each other: page loads that had degraded with growth, search visibility that had not kept pace with the community's authority, and a security posture that had not been revisited as the platform's profile grew. The engagement addressed all three concurrently.

4%increase in organic clicks

Education technology

Educativ.net

A platform that had previously ranked well had declined through a combination of technical and on-page problems: nearly ninety percent of the site failed Google's Core Web Vitals assessment, server response times were high, and metadata and content signals needed work. The engagement addressed those constraints together.

21%increase in search traffic

All case studies →

Adjacent problems

If this is not quite your problem

These overlap at the edges. Sending you to the right page is more useful than having you work it out.

Performance is the cause, not crawlability

Core Web Vitals failures, high TTFB and layout instability affect both users and crawl efficiency. See Website Performance Engineering.

The site cannot absorb its traffic

If the platform is unstable under load, the crawl symptoms are a consequence. See High-Traffic Website Engineering.

You need visibility in AI answers, not just search results

Crawlability is the prerequisite for both. See AI Search Optimisation.

Questions

Frequently asked

How is this different from an SEO agency?

The difference is what we can change. Content-led SEO works within the CMS and produces recommendations for other people to implement. The faults that suppress large sites are usually in the application, the template, the server configuration or the data layer — so the work is defect diagnosis, remediation and verification, delivered by engineers who can make the change rather than describe it.

Do you write content?

Not as a standalone offering. Content strategy belongs with people closer to the subject matter. Where a client needs content at scale we have run high-volume editorial operations alongside the technical work, which is a supply problem rather than a strategy problem.

What access do you need?

Read access to server logs and Search Console at minimum, plus repository and staging access if the remediation is to be implemented rather than handed over. An engagement that stops at diagnosis is available and sometimes the right first step.

How long before we see movement?

Implementation can be fast. Verification is not, because search infrastructure re-evaluates a site gradually and partly. A defect can be fixed in a day and take several weeks to be fully reflected. We separate those two things in reporting rather than presenting an early reading as the outcome.

Related capabilities and work

Bring us the problem you have not been able to fix

Describe what is happening rather than what you think the cause is. If we are not the right people for it, we will say so.

Discuss your engineering challenge View our work