282: Case Study: News Feed / Timeline

Learning outcomes

By the end of this lesson, you can:

explain and apply social graph in a realistic implementation;
explain and apply fan-out on write in a realistic implementation;
explain and apply fan-out on read in a realistic implementation;
explain and apply hybrid fan-out in a realistic implementation;
explain and apply ranking in a realistic implementation.

Prerequisites and retrieval

This lesson assumes the earlier 01–06 foundation and the preceding lessons in this module. Before you read, retrieve one concrete example from an earlier project where the same concern appeared. It might have been a follower relationship, a notification list, a search result, or any other derived list that had to stay useful while the underlying data changed. The point is not to memorize a set of fashionable terms. The point is to use the terms to make a defensible decision inside a large-scale distributed service, where requirements, traffic, failure modes, cost, and operational constraints are explicit.

Terminology

The same feed can look simple in a product requirement and become expensive once its traffic and consistency requirements are made concrete. These are the terms that describe the main design choices:

Social graph: Store follower relationships according to the access patterns for both directions: who a user follows and who follows an author. The data model and indexes need to support the reads that publishing and feed generation actually perform.
Fan-out on write: When an author publishes, push the post ID into the inbox or feed store for each follower. Reads are fast because much of the work has already happened, but one publish can create a very large number of writes.
Fan-out on read: At read time, fetch recent posts from the authors a user follows, then merge and rank those candidates. Publishing stays cheap, but a read can fan out across many authors and its tail latency can grow with the follow count.
Hybrid fan-out: Precompute feed entries for ordinary accounts and merge posts from celebrity accounts at read time. This balances write amplification against read fan-out instead of choosing one extreme for every author.
Ranking: Chronological order is straightforward, but relevance or learned ranking requires candidate generation, features, freshness handling, experimentation, and a clear explanation of how caches and derived read models affect the result.
Pagination: A cursor based on rank, time, and ID must remain stable enough while new posts arrive. Offset pagination tends to drift because inserts and deletions change the meaning of later offsets.

Mental model

Treat Case Study: News Feed / Timeline as a design problem with observable inputs, outputs, invariants, and failure modes. A feed is not just a query that returns posts. It is a read model assembled from a social graph and post data, often asynchronously, and then presented under latency and freshness constraints. That makes the fan-out-on-write versus fan-out-on-read choice central. It also brings celebrity hotspots, ranking, pagination, and eventual consistency into the same design.

A strong implementation makes its assumptions visible, narrows uncertainty at system boundaries, and leaves enough evidence—tests, types, constraints, metrics, or diagrams—to show why the design is safe. A useful interview and production sequence is:

requirement -> constraints -> model -> implementation -> failure analysis -> verification

Do not jump from a requirement directly to a library call or a named database. First state what must remain true. Then choose the mechanism that enforces it, and explain what happens when that mechanism is slow, duplicated, stale, or unavailable.

Deep dive

The first question is not simply, “Where do we store follows?” It is, “Which traversals must be cheap?” A home timeline needs to find the authors followed by a user. Publishing with fan-out on write needs to enumerate the followers of the author. Those are opposite directions, so a useful graph representation normally supports both access patterns rather than assuming one generic relationship query will scale.

Large follower sets require partitioning and efficient enumeration. The system also needs an explicit answer for duplicate follow requests, unfollow races, and the moment at which a new follow becomes visible in the feed. The graph is often the source of truth, while feed entries derived from it may be temporarily stale.

Decision rule: Use the social graph deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

2. Fan-out on write

With fan-out on write, publishing a post starts work that pushes the post ID into follower inboxes or another feed store. A reader can then retrieve a mostly prepared list with low and predictable read latency. This is attractive when the product has many reads per post and ordinary accounts have manageable follower counts.

The cost is write amplification. A post from a celebrity account may need millions of feed-entry writes, and a retry can duplicate work unless the operation is idempotent. A worker queue helps smooth the burst, but it also means the feed is eventually consistent: a successful publish does not necessarily mean every follower can read the post immediately. Deleted posts, privacy changes, blocked users, and unfollows must be propagated or filtered so that a stale derived entry does not become an authorization bypass.

Decision rule: Use fan-out on write deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

3. Fan-out on read

With fan-out on read, the publish path stores the post once. When a user opens the feed, the service fetches recent posts from the authors that user follows and merges or ranks the candidates. This keeps writes cheap and avoids writing the same post into every follower's inbox.

The work moves to the read path. A user who follows many authors can cause many downstream reads, and the slowest dependency tends to determine tail latency unless the service uses bounded parallelism, timeouts, partial results, and caching. Candidate limits matter too: fetching an unbounded history from every followed author is not a viable implementation. The design must state what happens when one author store is unavailable and whether the feed returns a partial result, a stale result, or an error.

Decision rule: Use fan-out on read deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

4. Hybrid fan-out

Hybrid fan-out recognizes that ordinary and celebrity accounts create different costs. The service can precompute feed entries for ordinary accounts, where the follower set is bounded enough for write amplification to be acceptable, and merge celebrity posts at read time. The threshold is a capacity and product decision, not a universal constant.

This approach reduces the worst write burst without requiring every read to query every followed author. It does add routing and operational complexity: the system needs to classify accounts, merge two candidate sources, deduplicate posts, and keep ranking behavior consistent across precomputed and live candidates. A threshold change, backfill, or celebrity reclassification should be observable and reversible.

Decision rule: Use hybrid fan-out deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

5. Ranking

Chronological order is a useful baseline because it is easy to explain, test, and paginate. A relevance or learned ranking system is a different level of complexity. It needs candidate generation, feature computation, freshness rules, model versions, experimentation, and safeguards for missing or stale features.

Ranking also changes how caches and read models must be understood. A cached feed may contain candidates generated under an older model or before a privacy change. The service needs a policy for invalidation, filtering, and model rollout, and it should be possible to explain why a post appeared. A ranking failure should not silently turn into an empty feed if chronological fallback is acceptable.

Decision rule: Use ranking deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

6. Pagination

Cursor pagination is a natural fit for a feed that changes while the user is reading it. A cursor can encode the last seen rank, timestamp, and a tie-breaker such as post ID. On the next request, the service asks for entries after that position rather than asking for an offset into a list that may have changed.

The cursor does not make a changing feed perfectly immutable. New posts can arrive above the current position, ranking can change, and deletions can create gaps. The contract should define whether duplicates are forbidden within a session, whether missing entries are acceptable, and how a cursor is signed or versioned. Offset pagination drifts in a constantly changing feed because inserts and removals change which item occupies each offset.

Decision rule: Use pagination deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

Worked example

Consider a large-scale distributed service whose requirements, traffic, failure modes, cost, and operational constraints must be made explicit. Start by writing the requirement in one sentence. Then list the input and output contracts and identify which concept above owns each failure mode. For example, the social graph owns follow relationships, the publish path owns durable post creation, workers own asynchronous feed projection, and the read path owns candidate merging and ranking.

The useful design move is separation. Parsing and validation belong at the boundary; domain rules belong in the domain or service layer; persistence rules belong in the database or repository; presentation rules belong in the client. Mixing those concerns can make a happy-path demo look shorter, but it makes privacy changes, retries, stale entries, and other edge cases much harder to reason about.

Client
  |
DNS -> CDN / Edge
  |
Load Balancer -> API instances -> Cache
                          |          |
                          +------> Primary datastore
                          |
                          +------> Queue / Stream -> Workers

The diagram is a conceptual request and projection flow, not a claim that every deployment needs every component. A request reaches an API instance through the edge, reads may use a cache and primary datastore, and publish work can be handed to a queue or stream for workers to build derived feed entries. The queue improves isolation from a burst; it does not remove the need to handle duplicate delivery, lag, or poison messages.

Walk the example through at least four cases: the normal path, an empty or missing value, a duplicate, retry, or concurrent path where relevant, and a dependency failure. For each case, state which layer detects the problem and what the caller observes. For a publish, ask whether a retry creates one post or two. For a deleted or private post, ask whether an old feed entry is filtered before it is returned. For a slow worker or cache outage, state whether the caller receives a stale, partial, or failed response. This is the level of explanation expected in a senior code review or technical interview.

Production perspective

Production correctness is broader than “the code works on my machine.” Ask how the design behaves during deploys, retries, partial failure, stale clients, concurrent requests, malformed data, schema changes, and high cardinality. A feed system also needs visibility into queue lag, fan-out volume, cache hit rate, ranking or fallback usage, duplicate rate, pagination anomalies, and tail latency. Prefer explicit contracts, bounded resource usage, structured errors, and measurable behavior. Optimize only after evidence identifies the bottleneck or risk.

When the topic involves an external dependency, define a timeout and cancellation strategy. When it involves persistence, define transaction and consistency expectations. When it involves user-visible state, define loading, empty, error, stale, and success states. When it involves security, assume the client can be modified and all network input is untrusted. In particular, a feed cache or derived store must never be treated as the authority for whether a user is allowed to see a post.

Guided lab

Design a Twitter-like home timeline for ordinary and celebrity users. Include the follow graph, publish path, hybrid fan-out, feed store and cache, cursor pagination, privacy and deletion propagation, ranking, and backfill.

Complete the lab with this discipline:

Write the requirement and two non-requirements.
List input, output, and error contracts before implementation.
Implement the smallest correct vertical slice.
Add at least one invalid-input test and one edge-case test.
Instrument or inspect the behavior instead of guessing.
Refactor one hidden assumption into an explicit type, constraint, function, or configuration.
Explain one alternative design and why you did not choose it.
Record a short “what would break at 10× scale?” note.

The vertical slice should make one complete path observable, such as follow, publish, project, and read. Then use the scale note to identify what changes first: write amplification, read fan-out, queue lag, storage size, ranking cost, or cache pressure. Backfill should be treated as a separate operational path with rate limits and idempotency, not as an unbounded request-time repair.

Edge cases and failure modes

Social graph: Test absence, malformed input, duplicates, ordering and concurrency where applicable, and behavior at the smallest and largest credible sizes. Include follow/unfollow races and both graph traversal directions.
Fan-out on write: Test absence, malformed input, duplicates, ordering and concurrency where applicable, and behavior at the smallest and largest credible sizes. Include retries, worker lag, celebrity-scale follower sets, deletion, and privacy changes.
Fan-out on read: Test absence, malformed input, duplicates, ordering and concurrency where applicable, and behavior at the smallest and largest credible sizes. Include a large follow count, a slow or unavailable author source, bounded candidate fetches, and partial-result behavior.
Hybrid fan-out: Test absence, malformed input, duplicates, ordering and concurrency where applicable, and behavior at the smallest and largest credible sizes. Include threshold changes, merging and deduplication, reclassification, and backfill.
Ranking: Test absence, malformed input, duplicates, ordering and concurrency where applicable, and behavior at the smallest and largest credible sizes. Include missing features, stale model data, ties, freshness, model rollout, and chronological fallback.

Common mistakes and debugging

Solving the example instead of the requirement: a copied pattern can be syntactically correct but architecturally wrong.
Hiding uncertainty with assertions, broad exception handlers, permissive schemas, or “temporary” any values.
Testing only the happy path and therefore discovering contracts only after integration.
Optimizing before measuring, or selecting a scalable mechanism without a scale requirement.
Letting client-side behavior stand in for server-side authorization, validation, or persistence guarantees.

For debugging, reproduce the smallest failing case, inspect the actual value or execution plan, trace the boundary where the invariant first becomes false, and fix the owning layer rather than adding a downstream patch. In this case study, inspect the source graph and post first, then the queue or stream and worker logs, then the derived feed entry, cache behavior, and final response. A missing post can mean a graph query problem, projection lag, a filtering decision, or a pagination cursor issue; the observed symptom alone does not identify the faulty layer.

Interview questions

What problem does Social graph solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Fan-out on write solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Fan-out on read solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Hybrid fan-out solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Ranking solve, and what trade-off or failure mode would make you choose a different approach?

Checkpoint

Without notes, explain Case Study: News Feed / Timeline to another developer in five minutes. Your explanation must include one invariant, one edge case, one production failure mode, and one alternative design. Then implement a small example without copying the lesson code.

Mastery checklist

I can define the core terms precisely.
I can choose a design from requirements instead of from habit.
I can implement and test the normal path and edge cases.
I can explain the runtime, storage, or complexity cost.
I can identify which layer owns validation, errors, and recovery.
I can compare at least two reasonable alternatives.
I can explain how the design changes at larger scale or stricter reliability.