A CDN sits between users and an origin and answers whatever it can from its own copy. That is the point of it — and during an outage it means different people are talking to different things while typing the same URL.
Why the same URL gives different answers ¶
A CDN is not one machine. It is dozens or hundreds of points of presence, each with its own cache, each independently deciding whether it has a fresh copy. Two people in the same city can reach different nodes with different contents.
So during an origin outage, the population splits three ways:
- Nodes holding a fresh copy serve it. Those users see nothing wrong.
- Nodes whose copy expired go to the origin, fail, and return an error.
- Nodes configured to serve stale content on failure keep serving the old copy, possibly for hours.
All three states exist simultaneously, and which one you get depends on where you are and what was cached there. No individual is wrong about what they see.
Reading the pattern ¶
| What you observe | Most likely |
|---|---|
| Static pages load, anything dynamic fails | Origin is down; the CDN is serving cached statics |
| Fails, then works on refresh, then fails | Different nodes, or a partly unhealthy origin pool |
| Works for you, fails for a colleague nearby | Different PoP, or different cache state at the same PoP |
| Content is visibly out of date but nothing errors | Stale-while-revalidate or stale-if-error is masking an origin failure |
| Everything fails everywhere including images | The CDN itself, or DNS |
That fourth row is worth dwelling on. A correctly-configured CDN can hide an origin outage completely, which is excellent for users and confusing for everyone trying to diagnose it — the site looks fine and the data is hours old. It is also one more reason two people can honestly disagree about whether a service is working: see why a site works for some people and not others.
Cache-Control is why monitoring disagrees with users ¶
Two directives that are routinely confused, and the confusion changes what your monitoring measures:
max-ageis an instruction to the BROWSER.s-maxageis an instruction to SHARED caches, which is what a CDN is.
A response with only max-age is not cached by the CDN at all. Every request reaches the origin, so the site is exactly as available as the origin and no more — while everyone assumes the CDN is absorbing traffic. We had this on nearly every page of this site: it read as cached, it tested as cached, and every visitor was executing a database query in us-west.
The reverse also bites. A page cached at the edge for five minutes will keep serving after the origin dies, so an external check hitting the CDN reports healthy throughout. If you want to know whether the origin is alive, you have to bypass the cache or check the origin directly — and if you want to know what users experience, you must not.
Both are legitimate questions. Running only one check and believing it answers both is the mistake — the same one described in what each probe actually proves. You can see what any URL returns today, headers included, with our header inspector.
Diagnosing it ¶
- Look at the response headers. Most CDNs report cache status —
cf-cache-status,x-cache,age. A HIT with a largeageduring an incident tells you the origin has been unreachable for at least that long. - Compare a cacheable and an uncacheable URL. The homepage versus an API endpoint or a URL with a random query string. If one works and the other does not, you have found the boundary.
- Try from another network. Mobile data usually reaches a different PoP. Same result means origin; different result means cache state.
- Check whether content is stale. A page showing data from three hours ago is being served from a cache that has given up reaching the origin.
If you operate the site ¶
- Use stale-if-error deliberately. It converts an outage into stale content, which is almost always the better failure — but it also hides the outage from your own external monitoring, so pair it with an origin check.
- Monitor the origin separately from the edge. They fail independently and you need to know which one did.
- Do not let cached error responses stick. A 5xx cached for five minutes turns a ten-second blip into a five-minute outage for everyone who lands on that node.
Cache keys fragment the population further ¶
A CDN does not hold one copy per URL. It holds one per cache KEY, and the
key can include far more than the path — query strings, the Vary
header, device class, country, sometimes a cookie.
Each dimension multiplies the number of distinct cached objects, and each object has its own freshness. So "the page is cached" is rarely true of the page; it is true of one variant. During an incident that produces symptoms that look impossible:
- The site works in English and fails in another language, because only one variant was warm.
- It works on desktop and fails on mobile, for the same reason.
- A URL with a tracking parameter fails while the clean URL works, because the parameterised variant was never cached.
That last one is worth knowing when someone sends you a link that fails and yours works: compare the full URLs before concluding anything.
Purging during an incident is a decision, not a reflex ¶
The instinct once an origin recovers is to purge everything so users see fresh content. Consider what that does: every node loses its copy simultaneously and goes to an origin that has just come back and is already the weakest part of the system. A full purge at the wrong moment is how a recovering service is knocked over a second time.
Prefer purging the narrowest set that matters — the specific paths serving stale or wrong content — and let the rest expire naturally. If a full purge is genuinely needed, do it after the origin has demonstrated it can carry normal load, not during the first minute of recovery.
The one-line summary ¶
A CDN converts an outage from a single fact into a distribution. Anyone diagnosing one has to ask not just whether the service is down, but for whom and from which copy — and inconsistent reports are evidence about the cache, not evidence that someone is mistaken.