139: MongoDB Consistency — Atomicity, Transactions, Sessions, Read Concern, Write Concern, Read Preference, and Retryable Operations

Learning objectives

You will learn to:

understand single-document atomicity;
use conditional atomic updates;
understand sessions;
use multi-document transactions;
choose transaction boundaries;
understand read concern;
understand write concern;
understand read preference;
understand causal consistency at a practical level;
understand retryable reads/writes;
design idempotent application behavior;
avoid treating transactions as a substitute for good schema design.

The thread running through this lesson is simple: consistency is a property you design deliberately. MongoDB gives you atomic updates, transactions, sessions, and configurable read and write behavior, but each tool has a cost and a scope. The right choice depends on the invariant your application must protect, not on a blanket rule such as "put every request in a transaction."

Single-document atomicity

MongoDB writes to one document atomically. When an update includes a filter and an update document, MongoDB evaluates the filter and applies the update as one operation for that document.

Example:

db.accounts.updateOne(
  {
    _id: accountId,
    balancePaise: {
      $gte: 5000
    }
  },
  {
    $inc: {
      balancePaise: -5000,
      version: 1
    }
  }
)

The predicate and update are evaluated atomically for that document. If the balance is at least 5000 paise when MongoDB evaluates the operation, the decrement and version increment happen together. If the predicate no longer matches, the update does not apply.

This can enforce:

do not reduce balance below zero

without a read-then-write race. The application does not need to read the balance, make a decision in application memory, and then send an unconditional replacement. The condition remains part of the database operation that changes the document.

Read-modify-write race

The problem with a read-modify-write sequence is that two requests can make decisions from the same old value.

Unsafe pattern:

read balance = 100
request A subtracts 30
request B subtracts 50
A writes 70
B writes 50

One update is lost. The final value is 50, even though both requests started from 100 and the intended combined result was 20. The application performed the read and write as separate operations, leaving another request free to change the document between them.

Use atomic update operators/predicate:

{
  $inc: {
    balance: -amount
  }
}

with condition. In practice, the condition should express the invariant, such as a non-negative balance, and the update should express the change. Inspect the matched and modified counts so the application can distinguish a successful change from a condition that did not match.

Or optimistic version:

filter:
{
  _id,
  version: expected
}

update:
{
  $set: patch,
  $inc: {
    version: 1
  }
}

The version in the filter says, "apply this patch only if the document is still the version I read." A different writer increments the version first, so a later stale update no longer matches.

If matched count 0:

conflict

That result is not automatically a server failure. It may mean that the document is missing, the caller lacks the expected state, or another writer won the race. The application should handle those cases according to the workflow instead of silently overwriting the newer document.

When one document is enough

If order + line items + totals are embedded:

mark line served
update order total/status

can often be one atomic document update. This is one of the practical advantages of embedding related data: an invariant that belongs to an order can remain inside the same document and be protected by one atomic operation.

If you normalized into many collections, you may need transaction. The state that must change together is now spread across multiple documents, so a single-document update cannot protect the entire invariant.

This is why modeling and transaction design are connected. A schema is not only about how data is stored or queried; it also determines which business changes can be made atomically and which changes require coordination.

Multi-document transaction

Use when one business invariant spans multiple documents/collections. A transaction is justified when the application must not expose a state in which part of that invariant has committed and another part has not.

Example transfer:

debit account A
credit account B
insert transfer record

should commit together. A debit without a credit, or a transfer record without the corresponding account changes, would leave the business state inconsistent.

Transactions support ACID semantics across involved documents on supported deployments. That does not remove the need to choose a sensible scope, configure concerns appropriately, or handle transient failures. It gives the group of database operations a coordinated commit boundary.

Session

Transactions run in client session. A session gives the driver and server the context needed to associate the operations, transaction state, retry metadata, and, where configured, causal-consistency metadata.

Node driver example is lesson 144.

Shell concept:

const session = db.getMongo().startSession()

session.startTransaction()

try {
  ...
  session.commitTransaction()
} catch (error) {
  session.abortTransaction()
  throw error
} finally {
  session.endSession()
}

The finally block matters because a session is a resource with a lifecycle. The exact shell APIs can vary; driver API is primary application interface. In application code, use the API and retry behavior documented for the driver and MongoDB version you actually run.

Transaction design

Keep transactions:

short;
focused;
bounded;
minimal documents;
no user waiting inside;
no slow external API calls.

Short transactions reduce the time that locks, snapshots, and other transactional resources remain relevant. A focused transaction also makes retry behavior easier to reason about. Keep the database work that protects one invariant inside the transaction and move unrelated work outside it.

Do not:

start Mongo transaction
→ call payment API for 20 seconds
→ wait user confirmation
→ commit

External systems cannot participate in Mongo transaction. MongoDB cannot roll back a payment provider's charge merely because the database commit later fails or times out. Waiting for a user or a slow API also holds the database transaction open while nothing useful is being committed locally.

Use saga/outbox/idempotency architecture for cross-system consistency. These patterns acknowledge that the database and an external system have separate commit mechanisms and make progress, retries, and compensation explicit.

Transaction retry

Transactions can encounter transient errors requiring retry. Failover, transient transaction errors, write conflicts, and network interruptions can make a transaction attempt fail even when retrying the logical operation is appropriate.

Official drivers provide helpers such as withTransaction that handle certain retry semantics. The helper is not permission to retry every error forever; it applies the driver's documented rules and still expects the application callback to be suitable for retry.

Use driver-recommended API. Driver behavior and supported options can differ by language and version, so follow the documentation for the driver used by the application.

Do not write naive infinite retry. Set sensible limits, preserve cancellation and deadlines, and make the failure visible when retrying cannot safely complete.

Keep operation idempotent where possible. If the same logical attempt is run more than once, it should not create an additional debit, event, or resource merely because the first attempt's response was lost.

Transaction performance

Transactions add:

coordination;
snapshot state;
oplog/replication pressure;
memory/storage;
contention.

The cost is workload-dependent, but it is real. Larger transactions touch more documents, remain open longer, and increase the amount of state the database must coordinate and retain.

Do not wrap every CRUD request in transaction “for safety.” A transaction around a single document often adds machinery without adding protection.

Single-document atomicity is cheaper. Start with a schema and update that protect the invariant at the smallest practical scope, then use a multi-document transaction when the business rule genuinely crosses that boundary.

Snapshot isolation concept

Transaction reads see a consistent snapshot according to transaction/read concern semantics. The reads within the transaction are therefore not simply a sequence of unrelated reads against whatever happens to be changing at each instant.

Concurrent changes may cause conflicts/retries. Another operation can still modify data that your transaction needs, and the database may require the transaction to abort or be retried rather than allowing an unsafe result.

Understand that “transaction” does not mean no concurrency; it means controlled consistency/isolation semantics. Other work continues, and the application still has to reason about conflicts, latency, and the possibility that an attempted transaction does not commit.

Read concern

Read concern controls consistency/isolation guarantees of reads. It is about what state a read is allowed to observe and, in supported contexts, how that read relates to replication or a transaction snapshot. It is separate from read preference, which selects where a read is served, and from write concern, which controls how a write is acknowledged.

Common levels include concepts such as:

local
available
majority
linearizable
snapshot

Availability depends on deployment/operation. Not every level applies in exactly the same way to every topology or operation.

Do not memorize names without semantics. When choosing a concern, describe the state the caller is allowed to see, the failure behavior it can tolerate, and the latency or availability cost it accepts.

`local`

Can return latest data on node without guaranteeing majority replication. It is useful when low latency and local availability matter more than knowing that the value has survived replication to a majority.

May see data later rolled back in rare failover scenarios. That possibility is the key trade-off: the node can have applied a value locally even though the deployment does not ultimately retain it as the committed majority state.

`majority`

Returns data acknowledged as committed to majority, according to replication semantics. This is the stronger choice when the application needs to read data that has passed the deployment's majority-commit boundary.

Often chosen when reading durable committed data matters. That does not mean every member is instantly serving the same result under every read preference; placement and lag still matter.

Has latency/availability trade-offs. Waiting for the required replication acknowledgment can take longer or become unavailable during a failure that leaves the necessary majority unreachable.

`linearizable`

Strong single-document read semantics in specific supported contexts, with higher cost/constraints. It is intended for requirements where a read must reflect the ordering guarantees of a current primary operation, not as a default setting for ordinary reads.

Use only when requirement demands. First identify the exact single-document consistency property required and confirm that the deployment and operation support it.

`snapshot`

Used for snapshot-consistent reads/transactions under supported scenarios. It provides a consistent view for the relevant operation or transaction, subject to the deployment and concern rules that apply to it.

Write concern

Write concern controls acknowledgment requirement for writes. It answers the question, "When may the client consider this write acknowledged?" It does not by itself dictate where later reads go or guarantee that every read preference sees the value immediately.

Examples:

{
  w: 1
}

acknowledge by primary.

{
  w: "majority"
}

wait for majority acknowledgment.

Optional:

j: true
wtimeout

depending requirements/version. These options have deployment and version-specific behavior, so confirm their meaning rather than copying an old configuration from a tutorial.

Do not assume acknowledged write = globally visible to every read preference instantly. An acknowledged write and a read from a particular member are different events with different routing and consistency behavior.

Durability trade-off

Higher write concern can increase durability and latency. Waiting for more members or stronger persistence acknowledgment gives the application more information about the write's survival, but it can also make the request wait longer and be more sensitive to member or network failures.

For critical financial-like data, stronger acknowledgment may be appropriate. The exact policy should follow the domain's loss tolerance and recovery requirements, not the label of the collection alone.

For disposable telemetry, different policy could fit. If losing a small amount of data is acceptable, a lower-latency policy may be a reasonable trade-off.

Choose from business durability requirement. Document the decision so a later change to defaults or infrastructure does not silently change the meaning of a critical operation.

Write concern timeout

If required acknowledgment not achieved in time, client can receive timeout even if write may have been applied on some nodes. A timeout is therefore not a reliable statement that the write did not happen. It is an ambiguous result about what the client learned before the deadline.

Application must handle ambiguous outcomes carefully. Retrying an operation that creates a resource or applies a side effect can duplicate the effect unless the operation has a stable identity or another idempotency mechanism.

This is why idempotency keys/version checks matter. They let a retry ask for the same logical operation again without treating a lost response as permission to perform a new operation.

Read preference

Read preference controls which replica-set member can serve reads. It is a routing choice, not a substitute for read concern. A mode that permits secondaries may improve distribution or locality while exposing the application to replication lag.

Common modes:

primary
primaryPreferred
secondary
secondaryPreferred
nearest

Primary gives freshest primary view. Secondary can reduce primary read load or serve regional reads but may be stale.

Do not use secondary reads for authorization/critical read-after-write behavior without understanding lag/consistency. For example, a user whose permission was revoked should not be authorized from a secondary that has not applied the revocation, and a confirmation page should not claim that a just-created record is missing merely because it read from a lagging member.

Stale secondary

Flow:

write primary
immediate read secondary
→ old value

Possible depending lag/read concern. The write may have succeeded on the primary while the selected secondary has not applied it yet.

If user expects immediate confirmation, choose appropriate read path. That may mean reading from the primary, using a session and suitable causal behavior, or otherwise designing the workflow so the freshness expectation is explicit.

Causal consistency

Sessions can support causal relationships so later operations observe prior operations in expected order under supported settings. This is useful when the application wants operations in one logical interaction to respect their known ordering without requiring the strongest global consistency policy for all traffic.

Useful for:

write
then read in same logical session

without strongest global consistency everywhere. The session carries the context that lets the driver and server relate the later operation to the earlier one, subject to the supported configuration.

Drivers manage metadata. Application code should use the driver's session APIs rather than trying to reproduce the internal operation-time metadata itself.

Retryable writes

Driver may automatically retry selected single-document writes after transient network/primary errors. This can recover from a temporary topology change without forcing every application to implement its own retry protocol.

This improves reliability while avoiding duplicate effects because server tracks retryable operation identity in supported sessions. The guarantee is limited to the operations and deployment conditions covered by MongoDB's retryable-write support.

Do not assume arbitrary multi-step application workflow is automatically retry-safe. A sequence such as "insert order, charge card, send email" still needs explicit idempotency and cross-system design even if an individual database write is retryable.

Retryable reads

Selected reads can retry after transient errors. A retry can increase latency during failover but improve success.

Use driver defaults/recommendations. A read retry should be considered alongside the read's consistency and freshness requirements; retrying successfully does not make a secondary read current if the application chose a stale member.

Ambiguous outcome

Imagine:

client sends create
server commits
network drops before response

Client does not know if create happened. From the client's perspective, both "not applied" and "applied but response lost" are possible.

If retry creates another record, duplicate. The problem is not solved by checking only whether the first request returned an error, because the error may describe communication rather than database execution.

Solutions:

client-generated stable operation/resource ID;
unique external ID;
idempotency key;
retryable-write semantics where applicable.

The application should make the logical operation recognizable on a retry and treat a duplicate-key or already-completed result as part of the idempotent workflow where appropriate.

Unique index + idempotency

Webhook event:

{
  "eventId": "evt_123"
}

Create processing record with unique index:

{
  eventId: 1
}
unique

If webhook retries, duplicate key tells you already processed/claimed. The unique index is the database-enforced guard; an application pre-check such as "find, then insert" is not enough because two webhook deliveries can pass the pre-check concurrently.

Still design transaction around side effects. Recording or claiming an event and applying its related database changes must have a clear boundary. External effects such as sending a message or charging a provider need their own idempotency or outbox strategy.

Transaction and outbox

Need:

update order
publish event

Mongo transaction cannot atomically publish to Kafka/external broker. The database commit and broker publish are separate operations, so a failure between them can otherwise leave one completed and the other missing.

Outbox pattern:

Transaction writes:

order update
outbox event document

Then worker publishes outbox reliably and marks sent. The worker must also tolerate retries: a publish attempt can have an ambiguous result, so the event should have a stable identity and the consumer or broker-facing workflow should account for duplicates where necessary.

This connects DB atomicity to external messaging. The transaction makes the order change and the durable intention to publish inseparable inside MongoDB, while the worker handles delivery after the transaction commits.

Change streams alternative

Change streams can observe committed changes, but delivery/resume semantics and business event shape need design. A change stream can be useful for reacting to database changes, but the raw change document may not contain the stable, domain-level event contract that consumers need.

Do not assume raw database change stream equals durable business-event outbox for every requirement. Evaluate resume behavior, retention and recovery expectations, event ordering, filtering, and whether consumers need a purpose-built payload.

Advanced lesson covers.

Isolation and counters

Atomic counter:

findOneAndUpdate(
  { _id: "invoice" },
  { $inc: { next: 1 } },
  { returnDocument: "after" }
)

One hot counter can become contention bottleneck at high scale. Every request that needs the next value competes for the same document, so the operation may be atomic while throughput is still limited by contention on that document.

For strict invoice sequences, domain/legal requirements may demand serialization; design capacity intentionally. Before choosing a counter, establish whether the sequence must be gapless or merely unique and ordered. Strict sequencing can impose a cost that must be reflected in the design and capacity plan.

Distributed transactions and sharding

Transactions work in sharded clusters with additional coordination cost. A transaction that spans shards has more participants and more coordination than one contained within a single shard's data locality.

If every request touches many shards, shard key/data model may be poor. The transaction may be compensating for a data model that does not place commonly updated data together, and the resulting coordination can become a systemic performance problem.

Sharding lesson goes deeper.

Read/write concern defaults

Driver/cluster defaults evolve. Defaults are also not a substitute for understanding the business semantics of a particular operation.

Do not copy ancient options from tutorials. An option that was correct for an older driver, server release, or deployment may be unsupported or may mean something different in the current environment.

Use explicit concerns for operations where business semantics differ from defaults and document why. This makes the durability and visibility decision reviewable instead of leaving it hidden in an inherited configuration.

Error labels

Mongo driver errors can include labels such as transient transaction concepts. Those labels are intended to help application and driver logic classify errors without relying on unstable message text.

Use official driver helpers rather than string matching error messages. Error messages can change across versions and are poor control-flow contracts; use documented error labels and driver APIs.

Failure clinic

The following are common designs to question during review or incident analysis:

transaction around every request;
long external API inside transaction;
read from secondary expecting immediate write;
weaker write concern for critical data without decision;
retry loop on unknown outcome creates duplicates;
application pre-check uniqueness without unique index;
transaction used to compensate for bad embedding;
no idempotency on webhook/payment create;
raw change stream treated as business outbox automatically.

Each failure comes from treating a consistency feature as a universal guarantee. Inspect the invariant, the member that served the read, the acknowledgment received by the client, the retry boundary, and the database constraints before deciding what happened.

Exercises

Build atomic decrement with predicate.
Implement versioned update.
Model transfer requiring transaction.
Identify workflow that should avoid transaction by embedding.
Compare w:1 versus majority business trade-off.
Simulate read-after-write from secondary conceptually.
Design idempotent webhook handling.
Design outbox transaction.
Explain ambiguous create outcome.
Create consistency matrix for payment, profile, analytics event.

For each exercise, be explicit about the invariant being protected and what an unsuccessful, timed-out, or retried operation means. The goal is not only to write the syntax; it is to connect the database behavior to the application decision that follows.

Mastery checklist

Explain:

single-document atomicity;
conditional updates;
sessions;
transactions;
read concern;
write concern;
read preference;
causal consistency;
retryable reads/writes;
ambiguous outcomes;
idempotency;
outbox.

If you can explain these terms but cannot say which member may serve a read, what a timeout means, or how a retry avoids a duplicate, revisit the operational examples. Consistency knowledge is useful when it predicts observable application behavior.