259: Consistency Models, CAP, PACELC, Quorums, and Session Guarantees

Learning outcomes

By the end of this lesson, you should be able to:

explain and apply strong/linearizable consistency in a realistic implementation;
explain and apply eventual consistency in a realistic implementation;
explain and apply cap theorem in a realistic implementation;
explain and apply pacelc in a realistic implementation;
explain and apply session guarantees in a realistic implementation.

Prerequisites and retrieval

This lesson assumes the 01-06 foundation and the earlier lessons in this module. Before you start, retrieve one concrete example from a previous project where this kind of problem appeared. Perhaps a user updated a record and immediately read an older replica, or a retry caused an operation to run twice. The point is to connect the vocabulary to a system you have actually had to reason about.

Do not treat the terminology as something to memorize in isolation. You are making a decision inside a large-scale distributed service, so the requirements, traffic, failure modes, cost, and operational constraints need to be explicit before a consistency choice is defensible.

Terminology

Strong/linearizable consistency: Operations appear to happen atomically in one global order, and that order respects real time. Once a completed write is visible, a later read cannot legitimately return an older value.
Eventual consistency: Replicas may temporarily disagree, but they converge after updates stop, provided the system has a way to resolve conflicts and repair replicas.
CAP theorem: During a network partition, a distributed system cannot guarantee both linearizable consistency and availability for every request. A design must decide which guarantee to weaken for the affected requests.
PACELC: Even without a partition, a system generally trades consistency against latency. PACELC makes the normal-operation choice visible instead of limiting the discussion to CAP's failure case.
Session guarantees: Read-your-writes, monotonic reads/writes, and writes-follow-reads provide useful guarantees within a user's session. They are often weaker and cheaper than global linearizability.
Conflict resolution: Last-write-wins, version vectors, CRDTs, domain-specific merges, and manual conflict workflows are different tools for different data. The correct choice depends on whether concurrent updates can safely overwrite or must be combined.

Mental model

Treat Consistency Models, CAP, PACELC, Quorums, and Session Guarantees as a design problem with observable inputs, outputs, invariants, and failure modes. Consistency is a spectrum of guarantees that a client can observe; it is not simply a choice between two labels. CAP also does not mean choosing two letters during normal operation. It describes the constraint exposed by a partition, while PACELC asks what you trade during ordinary operation as well.

A strong implementation makes its assumptions visible, limits uncertainty at system boundaries, and leaves evidence behind. That evidence might be a test, a type, a database constraint, a metric, or a diagram that explains why the design is safe. This is especially useful when a stale read or a duplicate retry only appears under load or during a partial outage.

A useful sequence for both interviews and production design is:

requirement -> constraints -> model -> implementation -> failure analysis -> verification

Do not jump from a requirement straight to a library call such as a quorum setting or a consistency flag. First state what must remain true. Then choose the mechanism that enforces that invariant and decide what the caller sees when the mechanism cannot provide its normal guarantee.

Deep dive

1. Strong/linearizable consistency

The problem this model solves is disagreement about what happened first. With strong/linearizable consistency, operations appear to occur atomically in a single global order, and that order respects real time. If one client completes a write before another client starts a read, the read observes a state consistent with that completed write.

This model makes application reasoning simpler, but it is not free. Replicas or nodes must coordinate, and that coordination adds latency. During a failure or a partition, the system may have to reject or delay requests rather than return a result that could violate the guarantee.

Decision rule: Use strong/linearizable consistency deliberately when it makes the contract or invariant easier to prove. If it only saves a little application code while hiding an assumption about ordering, choose the more explicit design instead.

2. Eventual consistency

The problem here is often scale or availability: coordinating every read and write would make the service too slow or too unavailable. Under eventual consistency, replicas can diverge temporarily and converge when updates stop, assuming the system has conflict-resolution and repair mechanisms.

That guarantee says something about convergence, not about what a user sees immediately. A user may read stale data, and concurrent writes may conflict. The application therefore needs to define which stale or conflicting states are acceptable, how long they may remain visible, and how a repair is detected and completed.

Decision rule: Use eventual consistency deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption about stale data or conflict ownership, prefer the more explicit design.

3. CAP theorem

The useful question is what happens when nodes cannot communicate reliably. Under a network partition, a distributed system cannot guarantee both linearizable consistency and availability for every request. Partition tolerance is usually unavoidable across networks: packets can be delayed, links can fail, and two healthy groups of nodes can temporarily lose contact.

The choice is therefore about the behavior of affected requests. A consistency-first design may refuse a write or read until it can establish the required ordering. An availability-first design may continue serving requests with a result that can be stale or require later conflict resolution. Neither choice is universally correct; the domain invariant determines which failure is safer.

Decision rule: Use cap theorem deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption about partitions, availability, or stale results, prefer the more explicit design.

4. PACELC

CAP focuses attention on a partition, but most requests happen when the system is not partitioned. PACELC adds the normal case: even when no partition exists, systems commonly trade consistency against latency. A read that consults more replicas or waits for coordination may provide a stronger guarantee, but it usually takes longer than a local or cached read.

PACELC is a reminder to document both decisions: what the system does during a partition, and what it gives up during normal operation. The second decision affects every request, not just the rare outage. It also helps explain why two systems that make the same CAP choice can still have very different latency and consistency behavior.

Decision rule: Use pacelc deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption about normal-path latency or consistency, prefer the more explicit design.

5. Session guarantees

Global linearizability can be more coordination than a user-facing workflow needs. Session guarantees provide narrower promises that match how one user interacts with a service:

Read-your-writes: after a session successfully writes a value, its later reads do not return a state older than that write;
Monotonic reads: later reads in the same session do not move backward to an older version after the session has observed a newer one;
Monotonic writes: writes from one session are applied in the order issued by that session;
Writes-follow-reads: a write issued after a session has read a value is ordered after the version it depended on.

These guarantees are useful when a user edits a profile, refreshes the page, and expects to see the edit, or when one operation updates data based on a value just read. They remain weaker than global linearizability: another user or session may still see a different state, and the guarantee may require routing, session metadata, or a minimum replica version.

Decision rule: Use session guarantees deliberately when they make the contract or invariant easier to prove. If they only reduce typing while hiding an assumption about session boundaries, routing, or replica state, prefer the more explicit design.

6. Conflict resolution

Replication makes concurrent updates possible, so convergence alone is not enough. Last-write-wins is simple, but a timestamp alone can discard a valid concurrent update, and clocks are not a perfect representation of causality. Version vectors can record causal relationships. CRDTs can merge certain data types without a central coordinator. A domain merge can preserve business meaning, while a manual workflow may be the safest option when no automatic merge is trustworthy.

Choose the resolution rule from the data's semantics. Overwriting a display preference may be acceptable; overwriting two independent edits to a document or a financial record may not be. The system also needs repair behavior, observability, and a way to surface conflicts that cannot be resolved automatically.

Decision rule: Use conflict resolution deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption about lost updates or merge safety, prefer the more explicit design.

Worked example

Consider a large-scale distributed service. Before choosing a consistency model, write the requirement in one sentence, list the input and output contracts, and identify which concept above owns each failure mode. For example, the requirement might say that a successful profile update must be visible to that same user on the next read, while a public activity feed may tolerate a short period of staleness. Those are different guarantees and should not be forced into one system-wide label.

Keep the responsibility boundaries clear. Parsing and validation belong at the boundary; domain rules belong in the domain or service layer; persistence rules belong in the database or repository; presentation rules belong in the client. Mixing these concerns can make a happy-path demo look shorter, but it makes stale data, retries, and edge cases much harder to reason about.

Client
  |
DNS -> CDN / Edge
  |
Load Balancer -> API instances -> Cache
                          |          |
                          +------> Primary datastore
                          |
                          +------> Queue / Stream -> Workers

Walk through the architecture using at least four cases: the normal path; an empty or missing value; a duplicate, retry, or concurrent path where relevant; and a dependency failure. For each case, state which layer detects the problem and what the caller observes. In the consistency cases, also state whether the response may be stale, whether the operation is safe to retry, and what evidence shows that a repair or conflict-resolution path ran. That is the level of explanation expected in a senior code review or technical interview.

Production perspective

Production correctness is broader than “the code works on my machine.” Ask how the design behaves during deploys, retries, partial failure, stale clients, concurrent requests, malformed data, schema changes, and high-cardinality workloads. Prefer explicit contracts, bounded resource usage, structured errors, and measurable behavior. Optimize only after evidence identifies the bottleneck or risk.

For consistency work, metrics should help distinguish a slow request from a stale read, an unavailable dependency from a rejected consistency requirement, and a conflict from an ordinary validation error. When a dependency is involved, define a timeout and cancellation strategy. When persistence is involved, define transaction and consistency expectations. When the state is user-visible, define loading, empty, error, stale, and success states. When security is involved, assume the client can be modified and all network input is untrusted.

Guided lab

For profile edits, likes, bank transfers, and collaborative notes, choose a consistency model or session guarantee for each scenario. Explain which anomaly is acceptable and which anomaly is forbidden. A profile edit may need read-your-writes; a bank transfer needs stronger protection against an incorrect balance; a like may tolerate temporary disagreement; and collaborative notes need an explicit concurrent-edit strategy.

Complete the lab with this discipline:

Write the requirement and two non-requirements.
List input, output, and error contracts before implementation.
Implement the smallest correct vertical slice.
Add at least one invalid-input test and one edge-case test.
Instrument or inspect the behavior instead of guessing.
Refactor one hidden assumption into an explicit type, constraint, function, or configuration.
Explain one alternative design and why you did not choose it.
Record a short “what would break at 10× scale?” note.

Edge cases and failure modes

Strong/linearizable consistency: test absence, malformed input, duplicates, ordering and concurrency where applicable, and behavior at the smallest and largest credible sizes. Include the case where coordination cannot complete and verify that the response does not falsely claim success.
Eventual consistency: test absence, malformed input, duplicates, ordering and concurrency where applicable, and behavior at the smallest and largest credible sizes. Verify stale reads, convergence, conflict resolution, and repair rather than testing only the final converged state.
CAP theorem: test absence, malformed input, duplicates, ordering and concurrency where applicable, and behavior at the smallest and largest credible sizes. Simulate a partition and record whether affected requests are rejected, delayed, or served with weaker guarantees.
PACELC: test absence, malformed input, duplicates, ordering and concurrency where applicable, and behavior at the smallest and largest credible sizes. Measure the normal-path latency and compare it with the stronger-consistency option.
Session guarantees: test absence, malformed input, duplicates, ordering and concurrency where applicable, and behavior at the smallest and largest credible sizes. Check refreshes, retries, session changes, replica routing, and reads after a successful write.

Common mistakes and debugging

Solving the example instead of the requirement: a copied pattern can be syntactically correct but architecturally wrong.
Hiding uncertainty with assertions, broad exception handlers, permissive schemas, or “temporary” any values.
Testing only the happy path and discovering the real contracts only after integration.
Optimizing before measuring, or selecting a scalable mechanism without a scale requirement.
Letting client-side behavior stand in for server-side authorization, validation, or persistence guarantees.
Calling a system “eventually consistent” without defining the convergence, conflict, and repair behavior.
Treating a quorum as an automatic synonym for linearizability without checking the datastore's read, write, failure, and replica-ordering semantics.

For debugging, reproduce the smallest failing case and inspect the actual value, version, replica, timestamp, log sequence, or execution plan involved. Trace the boundary where the invariant first becomes false: source or build, browser or DOM, Network or HTTP, server or route, database or query, or deployment or configuration. Then fix the owning layer rather than adding a downstream patch. During a stale-read incident, compare the write acknowledgment with the read's replica and version; during a duplicate operation, trace the request ID and idempotency behavior.

Interview questions

What problem does Strong/linearizable consistency solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Eventual consistency solve, and what trade-off or failure mode would make you choose a different approach?
What problem does CAP theorem solve, and what trade-off or failure mode would make you choose a different approach?
What problem does PACELC solve, and what trade-off or failure mode would make you choose a different approach?
What problem do Session guarantees solve, and what trade-off or failure mode would make you choose a different approach?

Checkpoint

Without notes, explain Consistency Models, CAP, PACELC, Quorums, and Session Guarantees to another developer in five minutes. Include one invariant, one edge case, one production failure mode, and one alternative design. Then implement a small example without copying the lesson code. Be precise about what the example guarantees, what it may return during a failure, and how you would verify those claims.

Mastery checklist

I can define the core terms precisely.
I can choose a design from requirements instead of from habit.
I can implement and test the normal path and edge cases.
I can explain the runtime, storage, or complexity cost.
I can identify which layer owns validation, errors, and recovery.
I can compare at least two reasonable alternatives.
I can explain how the design changes at larger scale or stricter reliability.