260: Caching I: Cache-Aside, Read-Through, Write-Through/Behind, and Cache Placement

Learning outcomes

By the end of this lesson, you can:

explain and apply cache-aside in a realistic implementation;
explain and apply read-through in a realistic implementation;
explain and apply write-through in a realistic implementation;
explain and apply write-behind in a realistic implementation;
explain and apply cache locations in a realistic implementation.

The aim is not just to name these patterns. You should be able to state the consistency and failure contract, choose a pattern that fits it, and describe what you would measure when the cache or its backing store behaves badly.

Prerequisites and retrieval

This lesson assumes the earlier 01–06 foundation and the preceding lessons in this module. Before reading, retrieve one concrete example from a previous project where the same concern appeared. Perhaps a product page was served through a CDN, a process kept a small in-memory map, or a database query was repeated often enough to become a bottleneck. The exact technology is less important than identifying what was cached, who owned the cache, how it was refreshed, and what happened when the cached value was wrong or unavailable.

The goal is not to memorize terminology. It is to make a defensible decision inside a large-scale distributed service, where requirements, traffic, failure modes, cost, and operational constraints all need to be explicit. A cache that is excellent for a public, mostly static response may be unsafe for user-specific or authorization-sensitive data.

Terminology

Cache-aside: The application checks the cache, loads the value from the source on a miss, and then fills the cache. The application owns the read and fill sequence.
Read-through: The cache layer itself loads missing values through a configured loader. This centralizes fill behavior, but it also makes the application depend more heavily on the cache infrastructure's loader, timeout, error, and concurrency semantics.
Write-through: The write path updates the cache and backing store synchronously. It can keep the cache fresh after a successful write, at the cost of extra latency and tighter coupling between the cache path and the store.
Write-behind: The system acknowledges a cache write before asynchronously persisting it to the backing store. This can improve write throughput, but the cache and its queue become part of the durability and data-loss recovery design.
Cache locations: Browser caches, CDNs, reverse proxies, local process caches, distributed caches, and database buffer caches sit at different points in the request path. They address different latency and load problems, and they have different owners and invalidation rules.
Cacheable data: Data is a good cache candidate when it is deterministic, frequently read, expensive to produce, and allowed to be somewhat stale. User-specific or sensitive data needs narrowly scoped keys and authorization-aware fill behavior.

One useful distinction is between a cache pattern and a cache location. Cache-aside or write-through describes how data moves between a cache and its source. Browser, CDN, process, and distributed caches describe where a copy lives. You can combine them, but every additional copy adds another freshness and invalidation decision.

Mental model

Treat Caching I: Cache-Aside, Read-Through, Write-Through/Behind, and Cache Placement as a design problem with observable inputs, outputs, invariants, and failure modes. Caching reduces repeated expensive work, but it does not make that work disappear. It creates a second representation of the data, with its own freshness, eviction, invalidation, serialization, and availability semantics.

A strong implementation makes assumptions visible and narrows uncertainty at boundaries. It also leaves enough evidence—tests, types, constraints, metrics, logs, or diagrams—to show why the design is safe. For example, “the product can be five minutes stale” is a useful requirement; “we added Redis” is not a consistency contract.

A useful interview and production sequence is:

requirement -> constraints -> model -> implementation -> failure analysis -> verification

Do not jump from a requirement directly to a library call. First state what must remain true. If a successful update must be visible immediately to the updating user, say so. If occasional stale reads are acceptable, define the bound. If a cache outage must not take down reads, define the fallback and its resource limit. Then choose the mechanism that enforces those conditions.

Deep dive

1. Cache-aside

The application checks the cache first. If the key is present, it returns the cached value. If the key is absent, the application loads the value from the source, returns it, and writes a copy into the cache for the next request. The source remains authoritative; the cache is a performance optimization rather than the only copy of the data.

This pattern is simple, explicit, and widely used. It also makes the application responsible for details that are easy to overlook: key construction, serialization, TTL selection, negative caching, source errors, and what happens when several requests miss at once. The first misses can race and all perform the expensive source read. A stale entry can also survive after the source changes unless the write or invalidation path deals with it.

On a cache failure, the application may be able to bypass the cache and read the source, but that fallback needs a timeout and protection against a traffic spike. On a source failure, returning an old value may be acceptable for some data, while returning an error is required for other data. Those are product and consistency decisions, not properties supplied automatically by cache-aside.

Decision rule: Use cache-aside deliberately when it makes the contract or invariant easier to prove. It is a good fit when the application already understands the source and needs explicit control over what is filled, invalidated, or allowed to go stale. If it only reduces typing while hiding an assumption about freshness or failure handling, prefer the more explicit design.

2. Read-through

With read-through, the application asks the cache for a key and the cache layer loads the value from the backing source when the key is missing. The loader centralizes the miss path, so callers do not each need to implement the same “get, load, put” sequence.

That convenience changes where important behavior lives. The cache infrastructure now controls or strongly influences source connection handling, serialization, timeouts, retries, negative results, and concurrent misses. Before adopting it, verify what the loader does when the source is unavailable, whether it prevents a stampede for one hot key, and whether the cache can distinguish “not found” from “loader failed.” Otherwise a short application call can conceal a surprising failure contract.

Read-through is most useful when many consumers share one well-defined loading policy and the cache product exposes the semantics you need. It can be a poor fit when different callers need different authorization checks or freshness rules. A cache loader must never turn a missing or unauthorized value into a reusable entry for the wrong scope.

Decision rule: Use read-through deliberately when it makes the contract or invariant easier to prove. It is appropriate when centralized fill behavior is genuinely an ownership advantage. If it only reduces typing while hiding an assumption about cache infrastructure semantics, source failures, or authorization, prefer the more explicit design.

3. Write-through

In a write-through design, a write goes through the cache path and synchronously updates the backing store as part of that operation. The caller receives success only after the required writes have completed according to the defined contract. A later read can therefore see a fresh cache entry, provided the cache and store update semantics are designed consistently.

The useful trade-off is freshness versus latency and coupling. A write-through operation can be slower because it depends on the backing store on every write. It also raises an atomicity question: what should happen if the store write succeeds but the cache update fails, or if the cache changes and the store write fails? The design needs an ordering, rollback or invalidation behavior where possible, and a repair path for partial failure. “Synchronous” does not by itself mean that two independent systems update atomically.

This pattern can be reasonable for values that are read immediately after being written and where the write rate is manageable. It is less attractive when the cache is merely an optimization and the extra dependency would make a write fail unnecessarily. In some systems, writing the source first and invalidating the cache is safer than trying to make the cache a second synchronous write target.

Decision rule: Use write-through deliberately when it makes the contract or invariant easier to prove. Choose it when the freshness benefit justifies the added write latency and coupling, and when partial failures have a recoverable design. If it only reduces typing while hiding an assumption about atomicity or durability, prefer the more explicit design.

4. Write-behind

Write-behind acknowledges a cache write before asynchronously persisting the change to the backing store. The application can absorb bursts and return quickly, while a queue or worker drains changes to the durable source.

The trade-off is fundamental: the acknowledged value may not yet be durable. A process crash, cache loss, queue loss, worker bug, or ordering problem can lose or reorder writes. The system therefore needs to define what “success” means, how changes are durably queued, whether updates are idempotent, how retries and dead letters work, and how lag is monitored. Concurrent updates to the same key need an ordering or versioning rule; otherwise an older delayed write can overwrite a newer value.

Write-behind can improve throughput, but it makes the cache part of the durability and data-loss recovery design. It is not a safe default for balances, inventory, permissions, or other data where acknowledging a lost write would violate the business invariant. If used, expose queue depth and oldest-event age, and provide reconciliation or replay rather than assuming the worker will always catch up.

Decision rule: Use write-behind deliberately when it makes the contract or invariant easier to prove. It can fit high-volume workloads that explicitly tolerate asynchronous durability and have reliable replay, ordering, and recovery mechanisms. If it only reduces typing while hiding an assumption about durability, ordering, or loss, prefer the more explicit design.

5. Cache locations

Browser, CDN, reverse proxy, local process, distributed cache, and database buffer caches solve different latency and load problems. A browser cache can avoid a network request for one user. A CDN can serve cacheable public content close to many users. A reverse proxy can protect an origin at the edge of a service boundary. A local process cache is fast but exists only in one instance. A distributed cache is shared across instances but adds network latency and an operational dependency. A database buffer cache accelerates storage access inside the database and is not a replacement for an application-level freshness policy.

The location determines who can see the entry, who can invalidate it, and what happens during a deploy or instance restart. A public CDN key must not accidentally vary only by URL when the response contains private data. A local cache can produce different answers on different API instances. A distributed cache can reduce that inconsistency while introducing serialization, network, capacity, and availability concerns.

Multiple layers are sometimes justified, but each layer should have clear ownership. Otherwise a browser or CDN may continue serving an old response after the application cache has been invalidated, or a local cache may hide a correction made in the shared cache. Avoid redundant layers without a measurable latency or load reason and an explicit freshness story.

Decision rule: Use cache locations deliberately when it makes the contract or invariant easier to prove. Place a cache where it addresses the identified bottleneck and where visibility, authorization, TTL, invalidation, and failure behavior are manageable. If it only reduces typing while hiding an assumption about ownership or stale data, prefer the more explicit design.

6. Cacheable data

Cache deterministic, frequently read, expensive-to-produce data with acceptable staleness. A computed product summary or public configuration may be a good candidate. A value that changes constantly, is cheap to calculate, or must always be current may gain little from caching and may cost more to invalidate than it saves.

Sensitive user-specific data needs carefully scoped keys and authorization-aware fill behavior. The key must include every input that affects the result, including tenant, locale, permissions, or version where relevant. Authorization must be enforced independently of whether the value came from the cache. Never assume that a cache hit is safe merely because the original request that populated the entry was authorized.

Also decide whether to cache empty results, how long entries live, and what happens after a schema change. Negative caching can protect a source from repeated requests for a missing item, but an entry can become incorrect when the item is created. Serialization and deserialization failures should be observable and should not silently turn arbitrary data into a valid response.

Decision rule: Use cacheable data deliberately when it makes the contract or invariant easier to prove. Cache only when the key, freshness tolerance, invalidation behavior, and security scope are explicit. If it only reduces typing while hiding an assumption about staleness or data isolation, prefer the more explicit design.

Worked example

Consider a large-scale distributed service whose requirements, traffic, failure modes, cost, and operational constraints must be made explicit. A product-detail API is a useful example because the same response may be cached by several layers and because product data changes, but usually not on every request.

Start by writing the requirement in one sentence, list the input and output contracts, and identify which of the concepts above owns each failure mode. For example: “Serve a product detail response quickly, allow it to be stale for at most five minutes, and never return one tenant's private pricing to another tenant.” That statement immediately affects the key, cache location, TTL, authorization boundary, and fallback behavior.

The important move is separation: parsing or validation belongs at the boundary; domain rules belong in the domain/service layer; persistence rules belong in the database or repository; presentation rules belong in the client. Mixing these concerns makes a happy-path demo look shorter but makes edge cases much harder to reason about. A cache lookup should not replace request validation, and a cache hit should not bypass authorization.

One possible high-level request path is:

Client
  |
DNS -> CDN / Edge
  |
Load Balancer -> API instances -> Cache
                          |          |
                          +------> Primary datastore
                          |
                          +------> Queue / Stream -> Workers

The diagram is intentionally abstract. The CDN might cache a public response, the API instances might use a distributed cache with cache-aside, and the queue might carry write-behind work or invalidation events. Those are separate decisions. Do not infer that every request must traverse every component, or that the queue makes a write durable without knowing its delivery and storage guarantees.

Walk the example with at least four cases: the normal path, an empty or missing value, a duplicate/retry/concurrent path where relevant, and a dependency failure.

On the normal path, state whether the response is a cache hit or miss, which layer owns the fill, and what TTL and key are used.
For an empty or missing product, decide whether the source returns a not-found response, whether that result is negatively cached, and how quickly a newly created product becomes visible.
For duplicate, retry, or concurrent requests, consider whether two misses load the same product simultaneously and whether a repeated write is idempotent. If ordering matters, name the version or ordering mechanism.
For a dependency failure, state whether the API fails, serves a bounded stale value, or bypasses one layer. Include timeouts and what prevents the fallback from overwhelming the datastore.

For each case, state which layer detects the problem and what the caller observes. This is the level of explanation expected in a senior code review or technical interview. It is also what turns a diagram into an operational design rather than a list of boxes.

Production perspective

Production correctness is broader than “the code works on my machine.” Ask how the design behaves during deploys, retries, partial failure, stale clients, concurrent requests, malformed data, schema changes, and high cardinality. A cache can fail open by sending load to the source, fail closed by returning errors, or serve stale data. None is universally correct; the choice must match the requirement.

Prefer explicit contracts, bounded resource usage, structured errors, and measurable behavior. Useful measurements include hit and miss rate, load latency, eviction rate, entry age, invalidation lag, cache errors, source fallback rate, and write-behind queue depth. Optimize only after you can identify the bottleneck or risk with evidence. A high hit rate is not proof of correctness, and a low miss rate does not help if the cached response is incorrectly scoped.

When the topic involves an external dependency, define a timeout and cancellation strategy. When it involves persistence, define transaction and consistency expectations. When it involves user-visible state, define loading, empty, error, stale, and success states. When it involves security, assume the client can be modified and the network input is untrusted. These rules still apply on a cache hit: cached data is input to the response path, not a reason to skip validation or authorization.

Guided lab

Add a distributed cache to a product-detail API. Specify the cache key, TTL, miss-fill behavior, source-failure behavior, and what changes if the same response is also cached at the CDN.

Complete the lab with this discipline:

Write the requirement and two non-requirements. For example, state whether bounded staleness is allowed and whether this lab is required to support write-behind durability.
List input, output, and error contracts before implementation. Include not-found responses, cache timeouts, source errors, and authorization scope.
Implement the smallest correct vertical slice. Keep the source authoritative and make the cache interaction observable.
Add at least one invalid-input test and one edge-case test. A concurrent miss, stale entry, empty result, or unavailable cache is a useful edge case.
Instrument or inspect the behavior instead of guessing. Verify hits, misses, TTL behavior, fallback, and the actual key rather than relying only on a successful response.
Refactor one hidden assumption into an explicit type, constraint, function, or configuration. A named key builder or a configured freshness bound is preferable to scattered string concatenation or an unexplained number.
Explain one alternative design and why you did not choose it. Compare cache-aside with read-through, or a distributed cache with a CDN or local cache, using the stated requirements.
Record a short “what would break at 10× scale?” note. Consider hot keys, stampedes, memory capacity, source protection, invalidation traffic, and operational cost.

There is no single correct cache product or TTL for this lab. The solution is credible when the choice follows from the requirement and when the tests demonstrate the promised behavior. If the API serves private pricing, also demonstrate that a request with different tenant or authorization context cannot reuse the first request's entry.

Edge cases and failure modes

Cache-aside: Test absence, malformed input, duplicates, ordering/concurrency where applicable, and behavior at the smallest and largest credible sizes. Also test a miss when the source fails, a cache write that fails after a successful source read, and concurrent misses for the same hot key.
Read-through: Test absence, malformed input, duplicates, ordering/concurrency where applicable, and behavior at the smallest and largest credible sizes. Verify loader timeouts, loader errors, negative results, concurrent loader calls, and the distinction between “not found” and “could not load.”
Write-through: Test absence, malformed input, duplicates, ordering/concurrency where applicable, and behavior at the smallest and largest credible sizes. Exercise partial failure between cache and store, retries, read-after-write behavior, and invalidation or repair after one side is updated.
Write-behind: Test absence, malformed input, duplicates, ordering/concurrency where applicable, and behavior at the smallest and largest credible sizes. Exercise worker delay, queue overflow, retries, duplicate delivery, out-of-order updates, dead letters, restart recovery, and the boundary between acknowledged and durable data.
Cache locations: Test absence, malformed input, duplicates, ordering/concurrency where applicable, and behavior at the smallest and largest credible sizes. Check visibility across API instances, browser and CDN staleness, private-key scoping, deploy invalidation, eviction, network loss, and capacity exhaustion.

The smallest credible size can reveal assumptions such as an empty dataset or a single cache instance. The largest credible size exposes memory, cardinality, hot-key, payload, and invalidation costs. “Works” at one size does not establish that the key space or failure behavior is safe at another.

Common mistakes and debugging

Solving the example instead of the requirement: a copied pattern can be syntactically correct but architecturally wrong. Start with freshness, durability, visibility, and failure requirements.
Hiding uncertainty with assertions, broad exception handlers, permissive schemas, or “temporary” any values. These can turn malformed cache data or a dependency failure into an apparently valid response.
Testing only the happy path and therefore discovering contracts only after integration. Test hits, misses, stale data, source failures, retries, and concurrent requests.
Optimizing before measuring, or selecting a scalable mechanism without a scale requirement. A distributed cache has network, memory, serialization, and operational costs even when it is fast.
Letting client-side behavior stand in for server-side authorization, validation, or persistence guarantees. A browser or CDN cache is not an authorization boundary.
Treating invalidation as an afterthought. A TTL limits how long an entry can remain, but it does not guarantee immediate visibility after a write.

For debugging, reproduce the smallest failing case, inspect the actual value or execution plan, trace the boundary where the invariant first becomes false, and fix the owning layer rather than adding a downstream patch. Inspect the exact cache key, namespace, TTL, entry age, serialization, and authorization context. Then compare the cache result with the source of truth.

If misses suddenly increase, check eviction, expiration, key changes, serialization errors, and dependency latency before simply adding capacity. If the source is overloaded, inspect stampede behavior and fallback volume. If users see old data, trace every cache layer, including the browser and CDN, instead of checking only the application cache. Logs and metrics should let you distinguish a cache miss, cache error, source error, stale hit, and invalidation delay.

Interview questions

What problem does Cache-aside solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Read-through solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Write-through solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Write-behind solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Cache locations solve, and what trade-off or failure mode would make you choose a different approach?

Answer these with a requirement, not just a definition. A strong answer names the source of truth, acceptable staleness or durability, failure behavior, and the operational signal you would inspect. It should also acknowledge that cache-aside versus read-through is about fill ownership, while browser versus CDN versus server cache is about placement.

Checkpoint

Without notes, explain Caching I: Cache-Aside, Read-Through, Write-Through/Behind, and Cache Placement to another developer in five minutes. Your explanation must include one invariant, one edge case, one production failure mode, and one alternative design. Then implement a small example without copying the lesson code.

Before you finish, make the invariant concrete. For example, “a cached response is never reused across tenant or authorization scope” is testable. “The cache is correct” is not. Explain what the caller sees when the cache is unavailable, and identify which metric or log would tell you that the real system is approaching its limit.

Mastery checklist

I can define the core terms precisely.
I can choose a design from requirements instead of from habit.
I can implement and test the normal path and edge cases.
I can explain the runtime, storage, or complexity cost.
I can identify which layer owns validation, errors, and recovery.
I can compare at least two reasonable alternatives.
I can explain how the design changes at larger scale or stricter reliability.