283: Case Study: E-Commerce Orders, Inventory, and Payments

Learning outcomes

By the end of this lesson, you can:

explain and apply cart and pricing in a realistic implementation;
explain and apply inventory reservation in a realistic implementation;
explain and apply payment idempotency in a realistic implementation;
explain and apply order state machine in a realistic implementation;
explain and apply saga/outbox in a realistic implementation.

Prerequisites and retrieval

This lesson assumes the earlier 01–06 foundation and the preceding lessons in this module. Before you read, retrieve one concrete example from an earlier project in which one of these concerns appeared. It might have been a checkout flow, a stock counter, a payment retry, or a background job. The point is not to memorize vocabulary. Use that example to make a defensible decision for a large-scale distributed service, where requirements, traffic, failure modes, cost, and operational constraints all need to be explicit.

Terminology

Cart and pricing: A cart represents mutable user intent. At checkout, turn that intent into an immutable-ish priced snapshot containing the product, discount, tax, and shipping facts needed for an audit. That snapshot must still explain the order if catalog prices change later.
Inventory reservation: Use atomic decrement, reservation, or ledger semantics to prevent overselling. A successful checkout should not cause two buyers to claim the same last unit merely because their requests arrived at nearly the same time.
Payment idempotency: Create or confirm a payment with a provider idempotency key, and persist the provider's reference. If your request succeeds but its response is lost, retrying must not create a second charge.
Order state machine: Represent transitions such as pending, paid, confirmed, cancelled, refunded, and fulfilled explicitly. Reject invalid transitions instead of allowing unrelated booleans to describe contradictory states.
Saga/outbox: Order, payment, inventory, and shipping usually cannot participate in one database transaction. Durable workflow state, an outbox, retries, and compensation provide a way to coordinate work across those boundaries.
Reconciliation: Periodically compare internal payment, order, and inventory ledgers with provider or warehouse truth. This exposes gaps created by outages, delayed messages, or manual operations.

Mental model

Treat Case Study: E-Commerce Orders, Inventory, and Payments as a design problem with observable inputs, outputs, invariants, and failure modes. The hard part is not drawing boxes for a cart or payment service. The hard part is keeping the system correct when prices change, stock is limited, an external payment provider is unavailable, and requests are retried or run concurrently. Order state must also remain auditable after partial failure.

A strong implementation makes its assumptions visible, narrows uncertainty at system boundaries, and leaves enough evidence to support its safety claims. That evidence can be tests, types, database constraints, metrics, logs, or diagrams. Each should help answer not only “what happens on the happy path?” but also “what prevents the dangerous path?”

A useful interview and production sequence is:

requirement -> constraints -> model -> implementation -> failure analysis -> verification

Do not jump directly from a requirement to a library call. First state what must remain true. Then choose the mechanism that enforces it, and finally decide how you will observe or verify that mechanism when it fails.

Deep dive

1. Cart and pricing

A cart is mutable user intent: the customer can add items, remove them, or change quantities. Checkout has a different responsibility. It should create an immutable-ish priced snapshot with the product, discount, tax, and shipping facts that explain the amount the customer was charged. This remains useful for audit and support even after the catalog changes.

The snapshot does not mean that every earlier cart value is automatically trusted. At checkout, validate the current rules and calculate the price according to the contract. Once the order is accepted, persist the facts needed to explain that decision rather than relying on a future catalog lookup.

Decision rule: Use cart and pricing deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

2. Inventory reservation

Limited stock creates a concurrency problem. Two checkout requests can both observe that one unit appears available unless the read and the claim are coordinated. Use atomic decrement, reservation, or ledger semantics to prevent oversell. A reservation also needs an expiry or release path so abandoned checkouts do not hold stock forever. In some systems, reservations are partitioned by SKU and location because the available unit depends on where it is stored.

This is separate from merely displaying an availability number. The displayed number can be stale; the reservation operation is the point at which the system must enforce the inventory invariant. Define what happens when payment fails, a reservation expires, or a later workflow step cannot complete.

Decision rule: Use inventory reservation deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

3. Payment idempotency

Payments cross a trust and reliability boundary. A request may reach the provider, charge the customer, and then lose its response before your service sees it. Retrying without an idempotency key can produce a double charge. Create or confirm the payment through a provider idempotency key, persist the provider reference, and make the local result durable enough for a later retry to find it.

The key must represent the intended operation, not just be a newly generated value on every attempt. Also define how your service handles a retry with the same key but different payment details. Treat that as a contract violation or an explicit conflict, rather than silently changing what the original operation meant.

Decision rule: Use payment idempotency deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

4. Order state machine

An order moves through a finite set of meaningful states: pending, paid, confirmed, cancelled, refunded, and fulfilled. Model those transitions explicitly. Otherwise, separate flags such as isPaid, isCancelled, and isFulfilled can drift into combinations that should never exist, such as a cancelled order also being treated as ready for fulfillment.

Reject invalid transitions at the owning layer and record the transition history needed for audit. A retry of an already-applied transition should have a defined outcome; it should not accidentally apply a different transition. The exact states and permitted paths depend on the business contract, but the discipline is the same: name the states, name the transitions, and make invalid paths observable.

Decision rule: Use order state machine deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

5. Saga/outbox

Order, payment, inventory, and shipping generally live behind separate persistence or service boundaries, so one database transaction cannot atomically commit all of them. A saga records durable workflow state and coordinates a sequence of local transactions. An outbox lets a local database transaction persist a state change and the event describing it together; a worker can then publish or process that event with retries.

At-least-once delivery means handlers must tolerate duplicate messages. When a later step fails, compensation may release inventory, cancel or refund payment, or mark the order for manual handling, depending on what has already happened. Compensation is not a magical rollback: external side effects may not be reversible, and the resulting state must remain visible.

Decision rule: Use saga/outbox deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

6. Reconciliation

Even a carefully designed workflow can encounter provider outages, delayed messages, warehouse mistakes, or manual interventions. Periodically compare the internal payment, order, and inventory ledgers with provider or warehouse truth. Reconciliation should identify discrepancies, preserve enough evidence to investigate them, and route them toward a safe repair or manual decision.

It is not a substitute for idempotency or transactional boundaries. It is the recovery and detection mechanism for the gaps those mechanisms cannot eliminate. Decide how often it runs, what time window it examines, and how a discrepancy is prevented from being “fixed” twice.

Decision rule: Use reconciliation deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

Worked example

Consider a large-scale distributed service whose requirements, traffic, failure modes, cost, and operational constraints must be explicit. Begin by writing the requirement in one sentence. Then list the input and output contracts and identify which concept owns each failure mode. The useful separation is straightforward: parsing and validation belong at the boundary; domain rules belong in the domain or service layer; persistence rules belong in the database or repository; presentation rules belong in the client. Mixing these concerns can make a happy-path demo look shorter, but it makes concurrency and edge cases much harder to reason about.

Client
  |
DNS -> CDN / Edge
  |
Load Balancer -> API instances -> Cache
                          |          |
                          +------> Primary datastore
                          |
                          +------> Queue / Stream -> Workers

Walk through the design with at least four cases: the normal path, an empty or missing value, a duplicate, retry, or concurrent path where relevant, and a dependency failure. For each case, identify the layer that detects the problem and describe what the caller observes. For example, a malformed request should be rejected at the boundary, while a duplicate payment retry requires the payment idempotency contract and persisted provider reference. A queue or worker failure should be handled by durable workflow state and retry or compensation logic, not hidden from operators.

This is the level of explanation expected in a senior code review or technical interview: name the invariant, locate the owner, describe the observable failure, and explain how the system recovers.

Production perspective

Production correctness is broader than “the code works on my machine.” Ask how the design behaves during deploys, retries, partial failure, stale clients, concurrent requests, malformed data, schema changes, and high-cardinality workloads. Prefer explicit contracts, bounded resource usage, structured errors, and measurable behavior. Optimize only after evidence identifies the bottleneck or risk.

When an external dependency is involved, define timeout and cancellation behavior, and decide what a retry is allowed to do. When persistence is involved, define transaction and consistency expectations. When state is user-visible, define loading, empty, error, stale, and success states. When security is involved, assume the client can be modified and all network input is untrusted. These are not separate polish items; they are part of the behavior of the checkout system.

Guided lab

Design checkout for limited-stock products. Include a pricing snapshot, inventory reservation, idempotent payment, order state machine, saga/outbox, refunds, reconciliation, and oversell behavior under concurrency. Your design should make clear which operation owns each invariant and what happens when any dependency fails after a previous step has succeeded.

Complete the lab with this discipline:

Write the requirement and two non-requirements.
List input, output, and error contracts before implementation.
Implement the smallest correct vertical slice.
Add at least one invalid-input test and one edge-case test.
Instrument or inspect the behavior instead of guessing.
Refactor one hidden assumption into an explicit type, constraint, function, or configuration.
Explain one alternative design and why you did not choose it.
Record a short “what would break at 10× scale?” note.

Edge cases and failure modes

Cart and pricing: Test missing values, malformed input, duplicates, ordering and concurrency where applicable, and behavior at the smallest and largest credible sizes. Include catalog or discount changes between cart creation and checkout, and verify that the persisted snapshot still explains the accepted price.
Inventory reservation: Test missing values, malformed input, duplicates, ordering and concurrency where applicable, and behavior at the smallest and largest credible sizes. Include the last available unit, reservation expiry or release, and a failed downstream payment.
Payment idempotency: Test missing values, malformed input, duplicates, ordering and concurrency where applicable, and behavior at the smallest and largest credible sizes. Include a lost provider response, a repeated key with changed input, provider timeout, and a refund path.
Order state machine: Test missing values, malformed input, duplicates, ordering and concurrency where applicable, and behavior at the smallest and largest credible sizes. Include repeated transitions and attempts to move from a terminal or incompatible state.
Saga/outbox: Test missing values, malformed input, duplicates, ordering and concurrency where applicable, and behavior at the smallest and largest credible sizes. Include duplicate delivery, a worker crash, a publish failure, a compensation failure, and the point at which manual intervention is required.

Common mistakes and debugging

Solving the example instead of the requirement: a copied pattern can be syntactically correct but architecturally wrong.
Hiding uncertainty with assertions, broad exception handlers, permissive schemas, or “temporary” any values.
Testing only the happy path and therefore discovering contracts only after integration.
Optimizing before measuring, or selecting a scalable mechanism without a scale requirement.
Letting client-side behavior stand in for server-side authorization, validation, or persistence guarantees.

For debugging, reproduce the smallest failing case first. Inspect the actual value, event, ledger entry, or execution plan rather than the value you expected to exist. Trace the boundary where the invariant first becomes false: source or build, browser or DOM, Network or HTTP, server or route, database or query, or deployment or configuration. Then fix the owning layer instead of adding a downstream patch that only conceals the symptom.

For a suspected duplicate charge, start with the idempotency key and provider reference. For oversell, inspect the reservation operation and concurrent database behavior. For a stuck order, inspect the state transition history, outbox record, worker attempt, and compensation result. The debugging path should follow the same ownership boundaries as the design.

Interview questions

What problem does Cart and pricing solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Inventory reservation solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Payment idempotency solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Order state machine solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Saga/outbox solve, and what trade-off or failure mode would make you choose a different approach?

Checkpoint

Without notes, explain Case Study: E-Commerce Orders, Inventory, and Payments to another developer in five minutes. Include one invariant, one edge case, one production failure mode, and one alternative design. Then implement a small example without copying the lesson code. Your explanation should make the ownership boundary clear: who detects the problem, what state is persisted, what the caller sees, and how recovery or reconciliation proceeds.

Mastery checklist

I can define the core terms precisely.
I can choose a design from requirements instead of from habit.
I can implement and test the normal path and edge cases.
I can explain the runtime, storage, or complexity cost.
I can identify which layer owns validation, errors, and recovery.
I can compare at least two reasonable alternatives.
I can explain how the design changes at larger scale or stricter reliability.