How caching improves application performance

Caching is storing the result of expensive work so you can reuse it instead of redoing it. Done well it is the highest-leverage performance tool you have. Done badly it serves stale data and hides bugs.

Why it works

Most workloads are read-heavy and repetitive. The same query, the same page, the same computed value gets requested over and over. Caching turns a slow operation into a fast lookup the second time onward.

Layers you can cache at

The closer the cache is to the user, the bigger the win, because you also skip the network hops in between.

A simple app-level cache

const cache = new Map();

async function getUser(id) {
  if (cache.has(id)) return cache.get(id);
  const user = await db.users.find(id);
  cache.set(id, user);
  return user;
}

The two hard parts

There is an old joke that the two hardest problems in computing are naming things, cache invalidation, and off-by-one errors. Invalidation is the real one.

Expiry (TTL)

Give entries a time to live so they refresh eventually. Short TTLs stay fresher, long TTLs hit the cache more. Pick based on how stale the data is allowed to be.

Invalidation

When the underlying data changes, the cache must be updated or evicted. Common strategies:

• Write-through: update the cache when you update the source
• Evict on write: delete the cached entry and let the next read repopulate it

Watch out for

• Serving stale data after an update, the classic bug
• A thundering herd, where many requests miss at once and all hammer the source, is eased by locking or request coalescing
• Caching per-user data in a shared cache and leaking one user's data to another. Scope keys carefully.