Preventing Cache Stampedes: Coalescing, Locks, Early Expiration, and Stale Serving

A cache keeps an expensive query off your database — until the moment the entry expires. If that key serves 2,000 requests per second and the recomputation takes 300 ms, roughly 600 requests arrive during the recomputation window, all miss, and all run the same expensive query simultaneously. The database slows under the burst, so recomputation takes longer, so more requests pile up. That's a cache stampede (or thundering herd, or dog-piling), and it can take down a system that runs comfortably at steady state. (What Is Caching?.)

Stampedes also follow cache restarts (cold cache), deploys that change key formats, and synchronised TTLs (everything cached at startup expires together).

Fix 1: request coalescing (singleflight)

Within one process, ensure only one recomputation per key is in flight; concurrent callers await the same promise.

const inflight = new Map<string, Promise<unknown>>()

async function getOrCompute<T>(key: string, ttlMs: number, compute: () => Promise<T>): Promise<T> {
  const hit = await cache.get<T>(key)
  if (hit !== undefined) return hit

  let p = inflight.get(key) as Promise<T> | undefined
  if (!p) {
    p = (async () => {
      try {
        const value = await compute()
        await cache.set(key, value, ttlMs)
        return value
      } finally {
        inflight.delete(key)
      }
    })()
    inflight.set(key, p)
  }
  return p
}

This reduces a herd of 600 to one per instance. With 20 instances, that's 20 concurrent recomputations — often acceptable. Go's singleflight package and many caching libraries provide this. CDNs and reverse proxies offer the equivalent for HTTP (often called request collapsing). (HTTP Caching Headers.)

Fix 2: a distributed lock for recomputation

To get down to one recomputation across instances, take a short lock in the shared cache:

SET lock:{key} {token} NX PX 5000

• Winner recomputes and writes the value, then releases the lock only if it still holds it (compare the token in a Lua script — never a blind DEL, which could release someone else's lock after yours expired).
• Losers either serve stale (below — best), wait briefly and re-check the cache (with a bounded number of retries), or fall through to compute as a last resort.

Caveats: lock TTLs must exceed normal recomputation time; a process that stalls past the TTL can write a stale value after a newer one — include a version or timestamp in cached values and don't overwrite newer with older. Single-node Redis locks are fine for efficiency (preventing duplicate work); don't use them where duplicate execution would be incorrect — that needs fencing tokens and idempotent writes. (Implementing Rate Limiting uses the same atomic-script technique.)

Fix 3: serve stale while revalidating

The most user-friendly strategy: separate freshness from existence. Store the value with a soft expiry and keep it past that point:

type Entry<T> = { value: T; freshUntil: number }

async function getSWR<T>(key: string, freshMs: number, staleMs: number, compute: () => Promise<T>) {
  const e = await cache.get<Entry<T>>(key)
  const now = Date.now()
  if (e && now < e.freshUntil) return e.value                   // fresh
  if (e) {                                                      // stale: serve it, refresh once in background
    void refreshWithLock(key, freshMs, staleMs, compute)
    return e.value
  }
  return computeAndStore(key, freshMs, staleMs, compute)        // cold miss: coalesce/lock here
}
// cache TTL (hard expiry) = freshMs + staleMs

Users almost never wait on recomputation; one background refresh (guarded by the lock from Fix 2) replaces the value. Combine with stale-if-error: if recomputation fails, keep serving the stale value and retry later, instead of turning a database hiccup into an outage.

Fix 4: probabilistic early expiration (XFetch)

Instead of everyone discovering expiry at the same instant, each request has a small, increasing chance of recomputing before expiry — the closer to expiry and the more expensive the computation, the likelier. The well-known formulation (from Vattani, Chierichetti, and Lowenstein's work on optimal probabilistic cache stampede prevention):

// delta = how long the last recomputation took (ms); beta ≈ 1 (higher = earlier)
function shouldRecomputeEarly(expiry: number, delta: number, beta = 1): boolean {
  return Date.now() - delta * beta * Math.log(Math.random()) >= expiry
}

(Math.log(Math.random()) is negative, so the term pushes "now" forward by a random, exponentially distributed amount scaled by recompute cost.) Store delta alongside the value. It needs no locks and spreads recomputation naturally; on very hot keys, pair it with coalescing.

Fix 5: jitter TTLs

If many keys are written at the same time (a warm-up job, a deploy), identical TTLs make them expire together. Add randomness:

const ttl = baseTtl * (0.9 + Math.random() * 0.2)   // ±10%

Cheap, and eliminates synchronised expiry waves.

Fix 6: refresh ahead for known-hot keys

For a small set of critical keys (homepage data, pricing, feature-flag config), don't wait for traffic to trigger recomputation: a scheduled job refreshes them before expiry, and reads never miss. (Your App Needs Background Jobs.)

Related: negative caching and penetration

Requests for keys that don't exist (random IDs, deleted items, enumeration by bots) always miss and always hit the database — cache penetration. Cache "not found" results too, with a short TTL, and validate IDs before lookup. Bloom filters help at very large scale.

Protect the origin regardless

Stampede defences reduce load spikes; the database should still survive the ones that get through:

• A bounded connection pool, so a burst queues instead of overwhelming Postgres. (Postgres Connection Pooling.)
• Timeouts on recomputation queries; circuit breaking to stale values.
• Indexes that make the recomputation itself cheap — often the real fix. (Database Indexes.)

Testing it

Reproduce the stampede deliberately: warm a hot key, drive steady high concurrency with a load tool, force-expire the key, and watch database query counts and p99 latency around the expiry. Then repeat with each mitigation. (Load Testing Your App.)

Choosing

Situation	Use
Any cache with concurrent traffic	In-process coalescing (always) + TTL jitter
Hot keys, staleness acceptable for seconds–minutes	Stale-while-revalidate + background refresh with a lock
Hot keys, no tolerance for serving stale	Distributed lock with brief wait, or XFetch early recomputation
Handful of critical keys	Refresh-ahead job
Lookups for non-existent items	Negative caching

---

EasySpawn gives each project managed Postgres, with Redis on the Team plan, running alongside your app — so Claude Code can reproduce a stampede under load and verify the fix against the real stack. See pricing or join the waitlist.

Related: Redis: When You Actually Need It · HTTP Caching Headers · Why Is My Website Slow?