Publishers
A publisher's archive is permanent and its news desk is hourly. Those two estates have opposite discovery requirements, and most publisher search problems begin with someone treating them as one.
What makes engineering for publishers different
These are the conditions that change the shape of the problem. They are why a generic approach underperforms in this sector.
Permanent archives and hourly output share one domain
A publisher holds years of evergreen archive alongside content published minutes ago. They compete for the same crawl attention, and the correct discovery strategy for each is different — which is why a single sitemap and a single internal linking model serve neither well.
Revenue is decoupled from the content that earns it
Ad-funded publishing monetises pageviews while search visibility is driven by article quality and freshness. That misalignment means editorial and technical priorities are frequently set by different people with different measures, and the technical work has to be justified in the currency the business actually uses.
The monetisation layer is the performance problem
Advertising and analytics scripts are the dominant cause of poor Core Web Vitals on publisher sites, and they are also the revenue. The work is therefore sequencing, deferral and consent-aware loading rather than removal — a negotiation with commercial constraints rather than an engineering optimum.
Content appears in more places than the site
Syndication, AMP, app editions and platform distribution all create alternate versions of the same story. Getting canonicals and availability signals right across them is where a large share of publisher indexation problems actually live.
What usually goes wrong
Stories found after their news window has closed
Discovery latency measured in days rather than minutes, which for news content removes most of the traffic the article existed to earn.
The archive underperforming its content quality
Evergreen and historical content that should accumulate durable search traffic, held back by internal linking that never revisits it and canonical signals muddled by syndication.
Core Web Vitals dominated by third-party scripts
Commercial script load producing field metrics that no amount of front-end optimisation can fix while the tags themselves remain synchronous.
Paywalled and metered content handled inconsistently
Restricted content served differently to crawlers and users, producing either indexation loss or a policy problem depending on which way the implementation errs.
How we work in this sector
The work is grouped by objective rather than by service, because in this sector the objectives interact.
Separate the two estates
- News sitemap with freshness signalling, distinct from the archive
- Archive sitemaps segmented by section and age
- Internal linking that surfaces archive content rather than only recent work
Reduce discovery latency
- Publishing pipeline that notifies rather than waits
- Peak-hour publishing stability
- Crawl-rate monitoring during breaking-news windows
Make the commercial layer less expensive
- Script sequencing and deferral against measured field metrics
- Consent-aware loading that does not block first render
- Field-data monitoring per template, not sitewide averages
Resolve alternate versions
- Canonical strategy across syndication and app editions
- Paywall and metering implementation that is consistent for crawlers and readers
- Structured data aligned to what is actually published
Frequently asked
What work is most relevant to publishers?
Our stockbroking engagement covers high-volume news discovery, sitemap architecture and crawl reliability. The sector is fintech, but the publishing and discovery constraints transfer directly to time-sensitive editorial estates.
Can you fix Core Web Vitals without removing our ad scripts?
Usually, but not to an ideal. Most publisher field-metric failures come from scripts loading synchronously in the critical path, which can be restructured — deferred, sequenced, loaded after first render. The ceiling is set by what the commercial tags will tolerate, so the honest answer is that this is a negotiation with a constrained optimum rather than a problem with a clean solution.
How do you approach paywalled content?
Consistently, and with the implementation matched to what the business intends. Google's guidance permits restricted content to be indexed provided what a crawler receives matches what a user receives at the paywall. Most problems come from implementations that drifted — serving crawlers full text and users a hard wall, or blocking everything including the article preview. Both are fixable, and neither requires giving content away.
Our archive is enormous. Where do you start?
With a segmented crawl and index analysis rather than with the archive as a whole. A large archive almost always contains a small proportion of pages carrying most of its search value, and the useful work is internal linking and indexation toward those, plus identifying which parts are consuming crawl attention without any prospect of returning it.
Related capabilities and work
Tell us what your publishers platform is doing
The symptoms are usually more informative than the diagnosis. We will tell you what we think is happening.