Five caches disagreed about time.
One integer settled it.

The expiry was five minutes because of one field. Everything else in that response was editorial copy that changed twice a month, but sitting in the middle of it was a live number that went out of date the moment somebody bought something. So the whole payload inherited the freshness requirement of its most volatile member, and a document that could safely have been cached for hours was thrown away twelve times an hour instead. Then, to hide the cost of throwing it away, somebody proposed a background job to rebuild it every five minutes. That job was the point at which I stopped and looked at the actual shape of the problem.

This is a write-up of a layered caching strategy for a high-traffic consumer platform: a native app and a website reading from a content API, in front of several source systems of record, one of which is a slow third-party transaction engine that cannot be changed. The details are generalised, but the pattern transfers directly. It is the thing I now reach for whenever a team tells me their app is slow and their answer is another cache.

01 · THE PROBLEMEvery layer had an expiry. Nobody owned the policy.

Two things were true at once, and they are true of most platforms that have been alive for more than a year.

The first is that each cache layer existed for a legitimate and different reason. The device cache exists so a returning user sees the previous screen instantly and so the app works when the network does not. The edge cache exists to remove physical distance from the request. The gateway cache exists to absorb identical bursts and protect what is behind it. The distributed cache exists because the upstream call is expensive and several service instances should share the cost of making it once. The read model exists because the record is a join across systems that should never be joined at request time. Five layers, five distinct jobs. None of them is redundant, and the instinct to collapse them into one is wrong.

The second is that nobody had written down a single number. Each expiry had been chosen sensibly and locally, by a different person, at a different time, for a different layer. Stacked, the worst case staleness was the sum of all of them, and that sum had never been calculated. Which meant the only honest answer to a stakeholder asking how out of date a page might be was a shrug. Worse, when somebody did report seeing an old price, there was no way to determine which of the five layers had served it. A cache you cannot attribute is a cache you cannot tune.

Underneath both sat the real defect, which is that freshness was being treated as a property of an endpoint. It is not. It is a property of a field.

02 · THE WRONG TURNAdd a cache. Then a warmer. Then price a bigger box.

The escalation is worth recording exactly, because it is so reasonable at every individual step that no single person in the chain did anything wrong.

A performance complaint arrives from a demo. Someone adds a distributed cache in front of the slow read, which is correct. The expiry is set low, because the payload contains a volatile field and nobody wants to serve a stale one, which is also correct given the payload. The low expiry means the cache is cold often, so the first user after every expiry still waits, which prompts a proposal for a scheduled job to rebuild the cache every five minutes and keep it warm. That job runs on consumption-priced compute, so the next conversation is about whether to move to a bigger, always-on plan to make the executions cheaper and eliminate cold starts.

Four moves. Each locally defensible. Collectively, a scheduled job and a hosting plan upgrade, both introduced to compensate for an expiry that only existed because of one field in one response. Nobody had proposed removing the field.

I should be honest that the first three options on the table when the complaint landed were all infrastructure: a larger hosting plan, a provisioned tier, minimum always-on instances. I was in that room. The actual fix was a payload boundary and a client that renders what it already has on disk, and it cost nothing to run.

03 · THE KEY INSIGHTClassify the data, then give the whole stack one clock

The unlock is two moves that only work together.

The first is to split the payload by volatility. Stop shipping one response that contains both a description and a live count. Ship a cacheable content envelope, and a separate, deliberately uncacheable lookup for the volatile fields, and merge them on the client. The content envelope can then cache for hours at every layer. The volatile lookup caches nowhere and is read live from the system of record every time. If the volatile call is slow or fails, the card renders without that field rather than blocking the page, which is a better failure than either a stale number or a spinner.

The moment that split lands, the argument that produced the scheduled job and the hosting upgrade simply evaporates. Nobody needs to warm a cache that lives for four hours.

The second move is to give every layer one authority on freshness, so the layers stop each keeping their own private opinion. That authority is a single monotonic version stamp, owned by the write side, incremented whenever any source system publishes a change. It then goes three places: into the cache key at the edge and the gateway, into the persisted snapshot on the device, and into the record itself for conflict resolution.

What that buys is the part people find surprising. Because the version is in the key, a content change produces a new key rather than requiring a purge. There is no invalidation call to make, no cache-purge API to integrate, no fan-out to five layers hoping each one honours it. Old entries become unreachable and age out on their own. Which means the expiry can be long everywhere, because expiry is no longer the mechanism that delivers freshness. It is only the backstop for a missed event.

// the entire invalidation protocol: a tiny uncached endpoint
GET /content/version          // ~40 bytes, never cached, single digit ms
200 { "v": 1487 }

// on app foreground
const { v } = await fetch("/content/version").then(r => r.json());

if (v === snapshot.v) {
  render(snapshot);                     // zero content calls. nothing changed.
} else {
  render(snapshot);                     // still render immediately, then
  revalidate({ version: v });           // refetch in the background
}

A returning user on an unchanged day costs one forty-byte request instead of a full content fetch. This is also what makes multi-day device persistence safe to enable.

Expiry is not a freshness mechanism. It is what you fall back on when an event goes missing. Design for the event and the expiry can be generous.

Five classes, and the policy each one gets everywhere

This table is the controlling artefact of the whole strategy. Not the diagram, not the technology choices. Every endpoint and every query key gets exactly one class, and the class determines its behaviour at all five layers. When a new field is added, the first question is which class it belongs to, and the second is whether it needs its own endpoint.

Two boundary rules do most of the work in practice. A field that displays a live count is class D even when it appears on a class C screen, which means the screen and the count are two different requests. And unbounded search is deliberately absent from this table: arbitrary query strings produce arbitrary cache keys, and a cache with unbounded key cardinality is a memory leak with good intentions.

06 · THE DETAIL THAT SEPARATES REAL FROM DEMOWhat has to be true before a layer is allowed to answer

Layered caching fails in production for reasons that never appear in a diagram. Four of them carry most of the risk, and each is structural rather than a matter of being careful.

The first is the stampede. When an entry expires under load, every concurrent request misses at once and they all go to the expensive source together, which means the moment of highest traffic is also the moment of least protection. Two mechanisms fix it and both belong in the distributed layer: single flight, so that concurrent misses on the same key wait on one in-progress fill rather than starting their own; and refresh ahead, so the entry is rebuilt at seventy or eighty per cent of its lifetime, with jitter, before anyone hits an expired key. Refresh ahead is also exactly what makes an external warming job unnecessary. It has no schedule to maintain, it only refreshes keys that are actually being used, and it costs nothing when traffic is quiet.

The second is credential leakage into a shared cache. A response produced for an authenticated user must never be storable in a layer that serves other people. The structural guard is not a policy exclusion list that somebody has to remember to update, it is a separate client for public content that carries no credentials at all, so a credentialed response cannot physically enter the shared path. Private data lives on the device, under a key scoped to the user, and is cleared on logout and account change.

The third is what an edge does to the identity of the caller. Once an edge sits in front of the gateway, the gateway no longer sees users, it sees the edge. Every per-user throttle silently collapses into one shared bucket for the entire internet, which either blocks legitimate traffic or protects nothing. Either the throttle moves to the edge, which knows the real client, or the gateway reads the forwarded header explicitly. Related and just as commonly missed: if the origin still accepts traffic directly, the whole edge including its firewall is optional from an attacker's point of view. Lock the origin to the edge or the security layer is decoration.

The fourth is attribution. Every layer must stamp its response with which layer served it and how old the entry was. Without that, a stale-page report is unfalsifiable and the team ends up tuning by superstition. This is a two-line change that pays for itself the first week.

What the client owes the user before the network answers

The last piece is the one teams skip because it sits in the app rather than the platform, and it is often the single largest perceived improvement available. A returning user should see the previous screen and the previous images before any request completes. That means persisting the allowlisted public classes to disk, hydrating them before deciding whether to show a skeleton, and treating the render and the revalidation as separate concerns.

// persist public classes only. never the private ones.
persist({
  key:     "content-snapshot",
  maxAge:  24 * 60 * 60 * 1000,
  buster:  schemaVersion,                // a shape change must not hydrate garbage
  include: (q) => q.status === "success"
                  && PUBLIC_CLASSES.has(q.class)
                  && q.class !== "search"   // unbounded keys stay out
});

// images: exactly one high priority, the rest bounded
<Image
  src={rendition(item, "card")}      // ask for the size you will draw
  cacheKey={`${item.id}:card`}       // per rendition, never shared with the hero
  priority={isFirstHero ? "high" : "normal"}
  placeholder={item.thumbHash}
/>

Two details worth more than they look: a rendition-specific cache key stops a thumbnail and a full-size hero fighting over one entry, and a schema buster stops an app update from hydrating a snapshot it can no longer parse.

The four moves that make a slow checkout feel fast

Warm the connection before the tap. A large share of the first-byte cost on a third-party host is not the vendor's application at all: it is DNS resolution, the TLS handshake, connection setup and a cold server-side session. All of that can be done speculatively while the user is still reading the page, and both mobile platforms expose the primitive. On Android, a custom tab can be warmed and told which URL is likely, which pre-renders the page in the background. On iOS the equivalent is a pre-warmed web view and pre-established connections. The user then taps into a page that has already painted. Nothing about the vendor changed.

Prepare the session server to server. Instead of handing the browser a bare checkout link and letting it build a cart and an authenticated session from scratch in front of the user, do that work from your own backend while the user is still deciding, then hand over a link that resumes a prepared session. Session creation and identity exchange are often the slowest sequential steps in a checkout, and they do not need to be on the user's critical path at all. This also removes the redirect chain that authentication usually adds.

Own the first frame. On tap, render your own summary screen instantly from data already on the device: what they are buying, how much, which date. It costs nothing because you already have it. The vendor page loads behind that, and you swap when it is ready. A user looking at a correct summary for eight hundred milliseconds is having a completely different experience from a user looking at a white rectangle for the same duration.

Make the return leg yours. After payment, do not send the user back through the vendor for confirmation. Take the completion event, write the order into your own store, and let the app read it from there, cached on the device and available offline. The confirmation screen is then as fast as the rest of your app, which matters because it is the screen people revisit.

// phase 1, speculative. bounded, and only on real intent.
const warm = throttle(async (item) => {
  if (!strongIntent(item)) return;            // dwell or press-in, not render
  if (inflight.size >= MAX_WARM) return;         // never unbounded

  browser.prewarm();                             // DNS + TLS + pre-render shell
  browser.mayLaunch(checkoutHost);

  const s = await api.prepareSession(item.id);      // cart + identity, server side
  prepared.set(item.id, { url: s.resumeUrl, ttl: s.expiresAt });
}, 400);

// phase 1 -> 2, on tap. the wait already happened.
function onTap(item) {
  showOwnedSummary(item);                            // 0 ms, from local data
  const p = prepared.get(item.id);
  browser.open(p?.url ?? coldCheckoutUrl(item));    // prepared, or fall back
}

The fallback line matters more than the optimisation. If preparation failed, expired, or was throttled, the user gets the ordinary cold path and never sees an error. Speculative work must never be load-bearing.

We could not make the vendor faster. We could make sure the user arrived after the slow part had already happened.

08 · WHY NOT JUSTRight-sizing, and the options I turned down

A fair challenge is why any of this beats the simpler moves, so here is the reasoning against each one I rejected.

Just make the expiry longer. That trades correctness for speed, and it does not survive the first stale price. Long expiry is only safe once the version stamp means a change produces a new key, which is why the classification and the version had to land before the expiries were extended, not after.

Just buy a bigger hosting plan. That buys idle capacity, not correctness, and it does nothing for the four layers in front of the compute. Where cold start genuinely is the issue, the fix is almost always configuration rather than a platform migration: always-ready instances, ahead-of-time compilation, a smaller dependency graph. Measure before migrating, because a migration is a quarter and a configuration change is an afternoon.

Just run a job to keep the cache warm. This is the one I want to argue against most clearly, because it is so intuitive. A scheduled warmer needs a schedule, and a schedule needs a calendar of business hours that somebody has to maintain forever. It refreshes keys nobody is asking for and misses keys under real load. It fails silently. Refresh ahead inside the cache does the same job, only for keys in actual use, with no schedule, no calendar and no separate thing to monitor.

Just cache at every layer, more. Two caches on the same path means two expiry clocks, an unattributable stale page and a purge that has to be honoured twice. For any given response, exactly one layer should be the one that answers it, chosen by where the cost actually is: distance goes to the edge, burst goes to the gateway, upstream expense goes to the distributed cache, and instant plus offline goes to the device.

09 · THE PROOFWhat the design guarantees, and what I have not proven

What the strategy commits to, in a form that can be tested rather than asserted.

• Every endpoint carries exactly one data class, and the class determines its behaviour at all five layers rather than each layer choosing independently
• Volatile fields travel on their own endpoint and are stored by no layer, so no cache can serve a number that has already changed
• Private data is keyed to the user, encrypted on device, cleared on account change, and cannot enter a shared cache because the public client carries no credentials
• A content change bumps one monotonic version, which changes the key at every layer, so freshness needs no purge call anywhere in the system
• A returning user renders the previous screen and images before any request completes, and costs one small version check when nothing has changed
• Concurrent misses on the same key collapse to a single upstream fill, and the previous value is served while that fill runs
• Every response reports which layer served it and how old the entry was, so a stale-page report is diagnosable rather than a debate
• The checkout handoff carries a session prepared off the user's critical path, with an unconditional fallback to the ordinary cold path

And the edges, because a design that does not name its own unknowns should not be trusted.

• Per-hop latency across the full path is still to be measured rather than estimated. Every number in a caching argument is guesswork until each hop has its own figure
• Edge cache hit ratio under real traffic shape is unproven. Versioned keys make long expiry safe but they also fragment the key space on every publish, and the trade needs watching
• The version stamp is a single point of coordination. It is small and cheap, but if it lies, five layers lie with it, and it needs the monitoring that implies
• Speculative checkout warming needs observation of the vendor's own load, not just ours. The moment it costs them something, the technique has to be dialled back regardless of what it does for our numbers