249: Back-of-the-Envelope Estimation: Traffic, Storage, Bandwidth, QPS, and Concurrency

Learning outcomes

By the end of this lesson, you can:

explain and apply active users and actions in a realistic implementation;
explain and apply average and peak qps in a realistic implementation;
explain and apply payload size and bandwidth in a realistic implementation;
explain and apply storage growth in a realistic implementation;
explain and apply concurrency in a realistic implementation.

Prerequisites and retrieval

This lesson assumes the earlier 01–06 foundation and the preceding lessons in this module. Before you read, retrieve one concrete example from a previous project where one of these concerns appeared. It might have been a chat endpoint, a file upload, a URL shortener, or a database-backed page. You are not trying to memorize a set of formulas. You are practicing how to make a defensible decision in a large-scale distributed service, where requirements, traffic, failure modes, cost, and operational constraints need to be explicit.

Terminology

Active users and actions: Begin with daily or monthly active users and the meaningful actions each user performs. An active user count is useful only when it is connected to actual behavior.
Average and peak QPS: Convert operations per day into average requests per second, then apply a plausible peak factor. The average is a planning baseline, not a safe peak capacity target.
Payload size and bandwidth: Estimate request and response bytes and multiply those bytes by throughput. Consider both directions when the service receives significant request data.
Storage growth: Estimate the bytes for each durable object, multiply by daily creation and retention, and then account approximately for indexes, replicas, metadata, compression, and backups.
Concurrency: Concurrent in-flight work is approximately throughput multiplied by latency. This is the useful Little’s Law intuition for thinking about connections, worker pools, and memory pressure.
Sensitivity analysis: Change an uncertain assumption by 10x and ask whether the architecture changes. This tells you which assumptions deserve better measurement.

Mental model

Treat Back-of-the-Envelope Estimation: Traffic, Storage, Bandwidth, QPS, and Concurrency as a design problem with observable inputs, outputs, invariants, and failure modes. Capacity estimates do not need false precision. Their purpose is to turn a vague statement such as “this must support large scale” into an order of magnitude that helps you decide whether one database, a cache, a queue, or partitioning is justified.

A strong implementation makes its assumptions visible, narrows uncertainty at system boundaries, and leaves evidence behind: tests, types, constraints, metrics, or diagrams that show why the design is safe. If you estimate 50,000 daily users, for example, record what “daily user” means and what actions that number represents. Otherwise the number looks precise while carrying no useful information.

A useful interview and production sequence is:

requirement -> constraints -> model -> implementation -> failure analysis -> verification

Do not jump from a requirement directly to a library call or a piece of infrastructure. First state what must remain true. Then choose the mechanism that enforces it. For estimation, that means identifying the workload before choosing instance sizes, database topology, or queue technology.

Deep dive

1. Active users and actions

Start from daily or monthly active users and the meaningful actions per user. “One million users” is not yet a workload: those users may each make one request per month, or they may refresh a feed many times per hour. Write down the activity assumption and separate reads from writes, because they often have very different volume, latency, and infrastructure requirements.

For example, if 100,000 daily active users each create 2 messages and read 20 messages per day, that is 200,000 writes and 2,000,000 reads per day before retries, background work, and administrative traffic. The actions, not just the user count, are what drive the next calculations. Be clear about whether one user action produces one request or several API and worker operations.

Decision rule: Use active users and actions deliberately when they make the contract or invariant easier to prove. If the numbers only reduce typing while hiding an assumption, prefer the more explicit design. State whether the count is unique users, sessions, devices, or requests.

2. Average and peak QPS

Convert operations per day to average requests per second:

average QPS = operations per day / 86,400

Then apply a plausible peak factor. Two million reads per day is about 23 average reads per second, but a 10x peak factor means planning for roughly 230 reads per second. The factor is an assumption about traffic shape, not a universal constant. A product with a morning usage spike, a scheduled job, or a viral event may need a different model.

Design for peak and burst behavior rather than relying on a 24-hour average. Also distinguish user-facing request QPS from internal work QPS: one request may enqueue several tasks, and one retry storm may multiply the load. Rate limits, queue backpressure, autoscaling lag, and capacity headroom all matter when the peak is brief but real.

Decision rule: Use average and peak QPS deliberately when they make the contract or invariant easier to prove. If they only reduce typing while hiding an assumption, prefer the more explicit design. Record the time window and peak factor so another engineer can challenge or replace them.

3. Payload size and bandwidth

Estimate request and response bytes and multiply by throughput:

bandwidth per second = bytes per operation * operations per second

For a 20 KB response at 230 peak reads per second, the outbound rate is about 4.6 MB/s, before protocol overhead, compression differences, and cache misses. A 5 MB media response at the same request rate would be about 1.15 GB/s instead. That distinction is why media systems are frequently bandwidth and storage dominated, while metadata APIs are often request and latency dominated.

Keep transfer size separate from processing time. Compression can reduce bytes on the wire while adding CPU work, and a CDN can reduce origin bandwidth without removing the user's download cost. Estimate request and response directions independently when uploads, synchronization, or large request bodies are involved.

Decision rule: Use payload size and bandwidth deliberately when they make the contract or invariant easier to prove. If they only reduce typing while hiding an assumption, prefer the more explicit design. Include units, compression assumptions, and whether the figure describes origin traffic, edge traffic, or client traffic.

4. Storage growth

Estimate bytes per durable object times daily creation and retention:

storage growth = bytes per object * objects created per day * retention days

If 200,000 messages are created each day and the stored representation averages 1 KB, the raw annual data is roughly 73 GB before indexes, replicas, metadata, compression, and backups. Those additional costs are not details to ignore: a replicated database and its backups may require several times the logical data size. Conversely, compression or expiration may reduce the physical footprint.

Separate hot, retained, and archived data when their access patterns differ. A retention policy is part of the storage design, not an afterthought. Include growth from secondary indexes and derived records, and say whether the estimate is logical storage or allocated storage.

Decision rule: Use storage growth deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design. Make retention, replication, backup, and overhead factors visible instead of presenting one unexplained total.

5. Concurrency

Concurrent in-flight work is approximately throughput times latency:

concurrency = throughput * latency

This is the practical intuition behind Little’s Law. At 230 requests per second and 200 ms of average in-flight time, the service has about 46 requests in flight on average. A latency increase to 2 seconds raises that estimate to about 460, even if QPS has not changed. That directly affects connection counts, worker pools, memory pressure, and the amount of work waiting on a slow dependency.

Use the right latency for the question. Average concurrency helps with baseline sizing, while a high percentile and burst model are more useful for protecting resources. A queue can absorb work, but it does not make the work free; it trades immediate request pressure for queued memory, delay, and operational complexity.

Decision rule: Use concurrency deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design. State whether you are estimating requests, database connections, worker tasks, or another kind of in-flight work.

6. Sensitivity analysis

Vary the uncertain assumption by 10x and ask whether the architecture changes. Try this with active actions, peak factor, payload size, latency, and retention. If a 10x change still fits the same cache, database, and worker shape, stop refining that estimate. If it crosses a threshold, call out that threshold as a design risk and identify what measurement would reduce the uncertainty.

Sensitivity analysis prevents false precision. It is more useful to say “the design changes if peak traffic exceeds 2,000 QPS” than to claim that the service will receive exactly 1,843 QPS. It also shows which assumption is worth instrumenting before launch.

Decision rule: Use sensitivity analysis deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design. Keep the variables and thresholds visible so the estimate can be revisited as real traffic arrives.

Worked example

Consider a large-scale distributed service whose requirements, traffic, failure modes, cost, and operational constraints must be explicit. Start by writing the requirement in one sentence, list the input and output contracts, and identify which of the concepts above owns each failure mode. For a chat-style workload, you might begin with 100,000 daily active users, 2 messages written per user per day, 20 reads per user per day, a 10x peak factor, 1 KB of durable message data, and 200 ms of average request latency. These are assumptions to inspect, not facts to hide.

The resulting rough model is 200,000 writes and 2,000,000 reads per day. That is approximately 2.3 average writes QPS and 23 average reads QPS. With the stated peak factor, plan around 23 write QPS and 230 read QPS. One year of raw message data is about 73 GB before indexes, replicas, metadata, compression, and backups. At 230 reads QPS and 200 ms latency, about 46 read requests are in flight on average. The model now gives concrete questions: do reads need caching, can the primary datastore handle the write rate, and how much headroom is needed for retries and bursts?

The important move is separation: parsing and validation belong at the boundary; domain rules belong in the domain or service layer; persistence rules belong in the database or repository; presentation rules belong in the client. Mixing these concerns can make a happy-path demo look shorter, but it makes edge cases and capacity assumptions much harder to reason about.

Client
  |
DNS -> CDN / Edge
  |
Load Balancer -> API instances -> Cache
                          |          |
                          +------> Primary datastore
                          |
                          +------> Queue / Stream -> Workers

Walk the example through at least four cases: the normal path; an empty or missing value; a duplicate, retry, or concurrent path where relevant; and a dependency failure. For the normal path, follow the request through the edge, API, datastore or cache, and any asynchronous worker. For a duplicate message, ask whether an idempotency key or unique constraint prevents double creation. For a dependency failure, ask whether the API fails fast, retries with a limit, or accepts work into a queue. For each case, state which layer detects the problem and what the caller observes. That is the level of explanation expected in a senior code review or technical interview.

Production perspective

Production correctness is broader than “the code works on my machine.” Ask how the design behaves during deploys, retries, partial failure, stale clients, concurrent requests, malformed data, schema changes, and high cardinality. Estimates should include operational behavior: retries can increase QPS, a slow dependency can increase concurrency, and a queue can shift pressure rather than eliminate it. Prefer explicit contracts, bounded resource usage, structured errors, and measurable behavior. Optimize only after you can identify the bottleneck or risk with evidence.

When the topic involves an external dependency, define a timeout and cancellation strategy. An unbounded wait can consume the concurrency budget even when request QPS is stable. When it involves persistence, define transaction and consistency expectations, including what happens if a write succeeds but the response is lost and the client retries. When it involves user-visible state, define loading, empty, error, stale, and success states. When it involves security, assume the client can be modified and that network input is untrusted.

Guided lab

Estimate a chat or URL-shortener workload from users to read and write QPS, storage per year, bandwidth, and in-flight requests. Produce low, base, and high cases, and identify the first component that would need horizontal scaling. Show the assumptions and units for every case. Do not present a single exact number when the input is uncertain.

Complete the lab with this discipline:

Write the requirement and two non-requirements.
List input, output, and error contracts before implementation.
Implement the smallest correct vertical slice.
Add at least one invalid-input test and one edge-case test.
Instrument or inspect the behavior instead of guessing.
Refactor one hidden assumption into an explicit type, constraint, function, or configuration.
Explain one alternative design and why you did not choose it.
Record a short “what would break at 10× scale?” note.

For the estimate itself, include the action rate, peak factor, payload assumptions, retention period, and latency assumption. Then change one uncertain input by 10x. If the first scaling boundary changes, record that as a risk rather than smoothing it away in an average.

Edge cases and failure modes

Active users and actions: test absence, malformed input, duplicates, ordering or concurrency where applicable, and behavior at the smallest and largest credible sizes.
Average and peak QPS: test absence, malformed input, duplicates, ordering or concurrency where applicable, and behavior at the smallest and largest credible sizes. Include burst traffic and retry amplification rather than testing only a smooth average.
Payload size and bandwidth: test absence, malformed input, duplicates, ordering or concurrency where applicable, and behavior at the smallest and largest credible sizes. Include unusually large payloads, compression differences, and upload as well as download traffic.
Storage growth: test absence, malformed input, duplicates, ordering or concurrency where applicable, and behavior at the smallest and largest credible sizes. Check retention, indexes, replicas, and backup overhead.
Concurrency: test absence, malformed input, duplicates, ordering or concurrency where applicable, and behavior at the smallest and largest credible sizes. Check slow dependencies, timeouts, queue buildup, and bounded worker or connection resources.

Common mistakes and debugging

Solving the example instead of the requirement: a copied pattern can be syntactically correct but architecturally wrong.
Hiding uncertainty with assertions, broad exception handlers, permissive schemas, or “temporary” any values.
Testing only the happy path and therefore discovering contracts only after integration.
Optimizing before measuring, or selecting a scalable mechanism without a scale requirement.
Letting client-side behavior stand in for server-side authorization, validation, or persistence guarantees.
Treating an average as a peak, or treating a logical storage estimate as the physical capacity requirement.

For debugging, reproduce the smallest failing case, inspect the actual value or execution plan, trace the boundary where the invariant first becomes false, and fix the owning layer rather than adding a downstream patch. For capacity problems, compare the estimate with observed metrics: request rate, status codes, payload bytes, latency percentiles, active connections, queue depth, and datastore utilization. A mismatch usually points to a missing action, retry, cache miss, replica, or overhead factor.

Interview questions

What problem does Active users and actions solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Average and peak QPS solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Payload size and bandwidth solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Storage growth solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Concurrency solve, and what trade-off or failure mode would make you choose a different approach?

Checkpoint

Without notes, explain Back-of-the-Envelope Estimation: Traffic, Storage, Bandwidth, QPS, and Concurrency to another developer in five minutes. Your explanation must include one invariant, one edge case, one production failure mode, and one alternative design. Then implement a small example without copying the lesson code.

Mastery checklist

I can define the core terms precisely.
I can choose a design from requirements instead of from habit.
I can implement and test the normal path and edge cases.
I can explain the runtime, storage, or complexity cost.
I can identify which layer owns validation, errors, and recovery.
I can compare at least two reasonable alternatives.
I can explain how the design changes at larger scale or stricter reliability.