141: MongoDB Sharding and Horizontal Scaling — Shard Keys, Chunks/Ranges, `mongos`, Balancing, Zones, and Hotspots

Learning objectives

You will learn to:

understand when sharding is needed;
understand sharded-cluster components;
choose shard keys from workload;
distinguish ranged and hashed sharding;
understand targeted versus scatter-gather queries;
understand chunk/range balancing;
understand zones;
understand hot shards;
understand monotonic keys;
understand unique index/shard-key interactions at a practical level;
understand resharding;
understand transactions across shards;
avoid sharding too early.

This lesson is about the decisions and failure modes that matter once a single replica set is no longer enough. The syntax for enabling sharding is only a small part of the problem. The harder question is how data, reads, writes, and operational work will be distributed across the cluster.

Scale vertically first?

Many applications can scale for years on:

one replica set
strong indexes
good modeling
proper hardware

Before introducing sharding, make sure the current system is using those options well. A larger machine, an appropriate index, a better document model, or a properly configured replica set is usually easier to understand and operate than a distributed cluster.

Sharding introduces distributed-system complexity. Queries may need routing, data may move between servers, and an operation that was local on one replica set can now involve coordination and network traffic.

Do not shard because “Mongo is for sharding.” Sharding is not a default optimization or a badge of maturity. It is a response to a concrete capacity or distribution requirement.

Shard when concrete requirements exceed practical single-replica-set capacity or when the data must be distributed for a specific reason, such as regional placement or independent hardware capacity. The threshold is workload-dependent: storage, write volume, read volume, working-set size, and operational requirements all matter.

Sharded cluster components

Conceptually, a request travels through the routing layer to the appropriate shard or shards. A simplified topology looks like this:

client
  ↓
mongos routers
  ↓
config server replica set
  ↓
shard A replica set
shard B replica set
shard C replica set

Each shard typically is a replica set. The replica set supplies high availability for that shard; it does not mean that the entire sharded cluster is one ordinary replica set.

Config servers store cluster metadata, including the information the routing layer needs to understand the distribution of shard-key ranges. They are part of the cluster's control plane, so their availability and backup requirements deserve the same operational attention as the data-bearing shards.

mongos routes operations. Applications normally connect to mongos, or to the connection endpoint provided by a managed service, rather than choosing a shard themselves. A router examines the operation and cluster metadata, targets one or more shards, and returns or merges the result as needed.

Managed Atlas abstracts much of operation. That removes a great deal of server administration, but it does not remove the need to choose a workable shard key, understand query targeting, or monitor workload distribution.

Shard key

When a collection is sharded, the shard key determines how documents are distributed and how operations are routed. It is the field, or compound set of fields, that gives MongoDB the basis for placing and locating documents across the cluster.

This is one of the most important irreversible-ish architecture decisions, though modern Mongo supports resharding. Resharding provides a recovery path, but it is a substantial operational change rather than a reason to choose casually.

A good key should be evaluated against the workload, not selected from a list of fashionable fields. Consider:

cardinality;
write distribution;
query targeting;
growth;
monotonicity;
tenant distribution;
future scale.

There is no universal perfect key. A key that gives excellent write distribution can be poor for range queries, while a key that keeps a tenant's data together can allow one unusually large tenant to overload a single shard.

Example tenant workload

Suppose documents look like this:

{
  tenantId,
  orderId,
  createdAt,
  ...
}

and the application's queries nearly always include:

tenantId = ?

A plausible first shard key is:

{
  tenantId: 1
}

This lets the router use the tenant value to target the relevant shard or range. It can also keep a tenant's data relatively local, which is useful when most transactions and reads are tenant-scoped.

There is an important problem: one giant tenant can dominate one shard. If megaCorp produces most of the data or traffic, giving every document that tenant key can concentrate the workload even when the overall tenant count looks healthy.

Alternative compound or hashed strategies may distribute data within a tenant, but they affect query targeting and locality. For example, adding an entity or order component can spread one tenant's documents, while queries that provide only the tenant prefix may still need to contact multiple ranges. Choose based on actual tenant sizes and workloads, including their read and write patterns, rather than on the average tenant.

Ranged sharding

With ranged sharding, documents are partitioned by ranges of shard-key values. The cluster divides the shard-key space into ordered intervals and assigns those intervals to shards.

For a simplified key space, the arrangement might look like this:

A-M → shard 1
N-Z → shard 2

Ranged sharding is good for range queries on the shard key because nearby values can remain in nearby ranges. A query for a contiguous interval can often be narrowed to the shards owning that interval.

The same ordering can create a hotspot on a monotonic key. If new values continually arrive at one end of the key space, new writes initially concentrate in the latest range instead of spreading naturally across all shards.

Hashed sharding

With hashed sharding, MongoDB hashes the shard-key value before using it for distribution. Values that are adjacent in their original form are not expected to remain adjacent after hashing.

The main benefit is more even distribution for many equality-oriented workloads. The hash breaks up an ordered sequence that might otherwise land in the same newest range.

The trade-off is poor range locality: adjacent original values spread across shards. A time interval, for example, no longer maps neatly to one contiguous part of the shard-key space.

Equality queries can target when they include the hashed shard-key value, because the router can compute the corresponding hash. That does not make every query targeted; queries on unrelated fields can still become scatter-gather operations.

Monotonic key hotspot

Consider a shard key based only on an ever-increasing timestamp:

{
  createdAt: 1
}

All new writes go to the latest range or shard until the range is split and data is moved. Splitting and balancing can reduce the concentration over time, but they do not make the incoming write point disappear. During sustained ingestion, the newest range can remain the active bottleneck.

That creates a hot shard: one shard receives a disproportionate share of writes, CPU, disk activity, or lock and cache pressure. The cluster may contain several mostly idle shards while the latest shard is overloaded.

A hashed time or compound design can distribute writes, but then time-range targeting changes. You need to evaluate whether the write distribution is worth the extra fan-out for time-based reads, and whether the query can include another shard-key component.

Do not shard on timestamp alone for high-write ingestion without explicitly considering the hotspot. The right answer depends on ingestion rate, query shape, retention, and whether time locality is a hard requirement.

Cardinality

Cardinality is the number of distinct values a field can provide for distribution. A low-cardinality shard key such as this one:

status ∈ {open,done}

cannot distribute data well. There are too few distinct values and ranges to create useful placement options, and a large amount of data can be associated with the same value.

The cluster needs enough distinct values and usable ranges to distribute documents and activity. High cardinality is necessary for many designs, but it is not sufficient by itself: the frequencies of those values and the application's queries still matter.

Frequency distribution

High cardinality is not enough if one value dominates 80% of writes. A field can have millions of possible values while one popular value receives most of the traffic.

For example, a multi-tenant system might have a tenant named megaCorp whose write volume is far above every other tenant. A tenant-based key can have excellent cardinality in aggregate and still create a practical hotspot.

Analyze the distribution, not just the distinct-value count. Inspect tenant sizes, per-key read and write rates, the shape of new data over time, and how those measurements are likely to change as the system grows.

Query targeting

The useful distinction here is between a targeted operation and scatter-gather. When a query includes the full shard key as an equality condition, for example:

{
  tenantId: X
}

the router can target the shard or shards that own that value. The exact number can depend on the shard-key design and distribution, but the router has information it can use instead of asking every shard.

A query missing the shard key, such as:

{
  email: "..."
}

may scatter to all shards. Each shard performs its part of the work and the router gathers the results. An index on email can make each local lookup efficient, but it does not by itself tell the router which shard owns the matching document.

Scatter-gather increases latency and load because the request fans out and results must be coordinated. Critical API queries should include the shard key or should be deliberately designed around another routing strategy. Inspect explain output on the sharded topology rather than assuming that a fast single-shard query remains fast when it is broadcast.

Shard key and multi-tenancy

A tenant key can provide data locality and routing. If most operations are scoped to one tenant, that prefix can keep related work together and make those operations easier to target.

But one large tenant may need finer distribution. A compound design can use the tenant as a logical prefix and another component to spread documents within that tenant:

{
  tenantId: 1,
  entityId: "hashed"
}

Conceptually, this distributes data within a tenant while preserving tenant-prefix behavior according to Mongo's supported shard-key definitions and version. The exact behavior and allowed definitions are version-sensitive, so check current MongoDB documentation before committing to the design.

Check current docs for exact hashed compound constraints. Also test the queries that provide only tenantId, the queries that provide both fields, and any zone or uniqueness requirements. A design that looks good in a schema diagram can still produce broad routing in production.

Chunks/ranges

MongoDB partitions shard-key space into ranges and distributes those ranges across shards. These ranges are the units the cluster tracks and, when appropriate, moves between shards.

Modern terminology and implementation evolve; the cluster balancer migrates ranges as it works to maintain an appropriate distribution. The operational details can vary by MongoDB version, but the application-level model remains useful: documents are placed according to shard-key space, and placement is not a permanent assignment made by the application.

The application should not depend on a specific document living on one shard forever unless zone design is intentionally being used to establish that placement. Routing must remain correct as ranges split and migrate.

Balancer

The balancer redistributes data to maintain balance. It can move ranges between shards as the distribution changes, allowing the cluster to use newly added capacity and correct uneven placement.

Migrations use network, disk, and CPU. They can therefore affect the workload they are intended to support, particularly when the cluster is already close to a resource limit.

Heavy balancing can affect application latency and throughput. Monitor the cluster while balancing is active, and account for replication, cache effects, and the additional traffic generated by movement.

Do not run enormous sharding changes just before peak traffic without a plan. Capacity for the migration itself, a rollback or pause strategy, and a clear observation window are part of the change, not optional operational details.

Zones

Zones assign shard-key ranges to specific shards. They are useful when placement has a business or infrastructure meaning, rather than merely being a balancing concern.

Common use cases include:

data residency;
regional locality;
hardware tiers.

For example:

EU tenants → EU zone
JP tenants → Japan zone

This kind of mapping can help meet residency or latency requirements, but the shard key must expose the information needed to identify the desired range. It is not a substitute for legal review, encryption, or a complete disaster-recovery design.

The architecture must still handle failover and compliance. A zone plan that works during normal operation may not satisfy requirements during maintenance, a regional outage, or a change in tenant location.

Zones complicate capacity. Each zone needs enough headroom for its own traffic and data, and a globally balanced total can still hide an overloaded regional group.

Hashed `_id`

Using a hashed _id can distribute writes. This is attractive when identifiers are generated in an ordered or otherwise concentrated way and the workload benefits more from distribution than from preserving identifier locality.

But queries by another tenant or status field may scatter. A well-distributed identifier does not automatically make the application's important business queries targetable.

The shard key must support dominant query patterns, not only write balance. Evaluate the endpoints and background jobs that matter most, then compare their routing behavior with the expected write distribution.

Compound shard key

A compound shard key is useful when the design needs to combine:

tenant routing
+
distribution

For example, a team might compare these designs:

{
  tenantId: 1,
  orderId: 1
}

and:

{
  tenantId: 1,
  orderId: "hashed"
}

The ordered form preserves more range locality for orderId; the hashed form can distribute values more evenly. Neither is universally better. The correct choice depends on the access patterns and the amount of data generated by each tenant.

Evaluate:

list orders by tenant;
point get by tenant+order;
write distribution;
giant tenants;
zone needs.

Do this evaluation with representative data and explain plans where possible. In particular, verify whether the queries used by the application contain enough of the shard key for useful targeting.

Global indexes concept

Traditional sharded queries often require the shard key for targeted uniqueness and routing. A local index on every shard is not automatically a single cluster-wide uniqueness constraint, because different shards may independently contain the same indexed value.

MongoDB capabilities evolve; check current version features for global indexing and unique constraints. Version-specific behavior matters here, and a feature available in one release or deployment mode should not be assumed to exist in another.

Do not assume a unique index on a field that does not contain the shard key works cluster-wide under all versions and configurations.

Verify MongoDB 8.3 docs. Treat the documentation for the exact server version and deployment configuration as authoritative before designing a business invariant around it.

Unique constraints

Sharded unique index rules differ from non-sharded rules. The shard key and index definition determine what MongoDB can enforce locally or across the distributed collection.

If the application requires global business uniqueness, identify it before sharding. Examples include:

email unique per tenant
external event ID

The collection's shard key and indexing strategy must support those invariants. Sometimes the tenant belongs in the uniqueness definition; sometimes an identifier must be globally unique regardless of tenant.

Do not discover a constraint limitation after sharding production. Validate the intended indexes and duplicate-handling behavior in a representative sharded test environment, including writes routed to different shards.

Scatter-gather

Consider this query:

{
  status: "open"
}

On a cluster sharded by tenant, the query contains no tenant information. It can hit every shard, even if each shard has an efficient local status index. The router must gather the responses and combine them.

If this is an administrator's global dashboard that is used rarely, the trade-off may be acceptable. The decision should be explicit and monitored rather than treated as a free operation.

If it is a hot endpoint, redesign it. Options include:

include tenant;
maintain a materialized global summary;
use a secondary analytics system.

The last option can keep analytical scans away from the online request path. A materialized summary can provide a predictable read path, but it introduces update and freshness decisions that must also be designed.

Aggregation on sharded cluster

Aggregation pipelines on a sharded cluster can execute partially on shards and then merge results. This can reduce the amount of work sent to the router when early stages filter or otherwise reduce the data.

Stages such as $lookup, $group, and $sort can still involve substantial network and merge cost. The placement of data, the pipeline order, and whether the operation can be targeted all affect the result.

Explain on the sharded topology. Do not extrapolate single-node pipeline performance to a distributed cluster. A pipeline that appears fast against one node may become expensive when it fans out, transfers intermediate results, or performs a large merge at the router.

Cross-shard transaction

Cross-shard transactions are supported, but they add coordination. The transaction has to involve multiple participants and maintain its guarantees across network boundaries.

If a transaction frequently touches many shards, latency and availability cost rises. More participants mean more opportunities for delay or failure, and the transaction can place pressure on several shards at once.

The shard key should try to co-locate transactionally related data where possible. This does not mean forcing unrelated data into one shard; it means recognizing transaction boundaries as part of the workload used to select the key.

Shard-key updates

Updating shard-key fields has special behavior and constraints and can cause document movement. It is not equivalent to changing an ordinary indexed field on a single node.

Treat the shard key as stable domain identity where possible. If a field can change frequently, especially as part of ordinary user workflows, include that update behavior in the design review and test it on the MongoDB version being deployed.

Resharding

Modern MongoDB supports resharding collections. This provides a way to correct a shard-key decision as a workload changes or an original assumption proves wrong.

Resharding is still a major operation. Plan for:

capacity;
duration;
oplog/change traffic;
index readiness;
application compatibility.

The operation consumes resources while data is copied and changes are tracked. It can interact with replication, storage, monitoring, maintenance windows, and application traffic. The exact limits and procedure are version-dependent.

Design the key carefully from the start, but know that mistakes can be corrected with operational cost. Resharding is a safety net, not a replacement for workload analysis.

Refine shard key

MongoDB supports refining shard keys in supported versions to add suffix fields without a full reshard in certain cases. This can be useful when the original key needs more granularity as the workload evolves.

Refinement is not a general way to change any shard-key property. Check the current requirements, supported index and key definitions, and the operational effects before relying on it as a future migration path.

Hot shard monitoring

Track these dimensions per shard:

ops/sec
CPU
disk
cache
network
storage
chunk/range distribution
connections
latency

Balanced data size does not guarantee balanced load. A shard can own a similar amount of data to its peers while receiving far more requests, handling more active connections, or serving a much hotter working set.

One tenant may drive all reads. A useful dashboard therefore needs both infrastructure metrics and workload dimensions, such as tenant, operation type, query shape, and whether requests are targeted or scatter-gather. When one shard is hot, correlate the timing with range placement and application traffic before assuming the balancer is broken.

Jumbo ranges/documents

Large ranges that cannot split or migrate under the applicable constraints can complicate balancing. A very large document or a range with insufficiently splittable key values can prevent the cluster from dividing the workload as expected.

Avoid extremely low-cardinality or otherwise unsplittable keys. Check how the selected key behaves with the largest documents and the most frequent values, not only with an average record.

Sharding and indexes

Sharded collections have shard-key index requirements. The shard key is not an optional detail that can be added later without considering the collection's existing indexes and data.

Every shard also needs secondary indexes for its local query patterns. A query can be targeted to the correct shard and still be slow there if the local index does not support its filter and sort.

Index count multiplies storage and write cost across the cluster. Review indexes as a distributed resource: adding an index to a sharded collection can consume resources on every shard and increase the cost of writes and maintenance.

Sharding and backup

Backup must capture consistent distributed cluster metadata and data. Restoring only the documents without the metadata that describes their placement is not a complete sharded-cluster recovery plan.

Use Atlas backup or supported sharded backup procedures. Confirm what the selected product and backup mode cover, how point-in-time recovery behaves, and how a restore will be tested.

mongodump across a huge cluster is not automatically an ideal disaster-recovery solution. It may be useful for particular migrations or smaller datasets, but size, consistency, restore time, and operational impact must be evaluated rather than assumed.

Sharding and connection strings

The application connects to mongos or an Atlas endpoint, not directly to an individual shard. The router is the component that understands cluster placement and can direct operations appropriately.

The driver performs topology discovery and management through the supplied connection configuration. Keep the connection behavior aligned with the supported deployment model.

Do not bypass the router for normal application operations. Direct shard access can defeat routing assumptions and create an operational path that the application was not designed to use.

Capacity planning

Adding shards does not instantly make every query faster. More machines help only when the workload can use the additional capacity and the shard key distributes that workload.

If the bottleneck is:

an unindexed scan on every shard;
a giant scatter-gather;
application CPU;
network;
one hot key;

more shards may worsen coordination. A broadcast query still broadcasts, and a hot key can still concentrate work on one shard. Extra shards can also increase balancing and management overhead.

Fix the query or model first. Then measure whether the remaining constraint is one that horizontal distribution can address.

Failure clinic

These are common ways to create an expensive sharded deployment:

shard too early;
use a timestamp shard key and create a hotspot;
choose a low-cardinality key;
allow a giant tenant to dominate one shard;
let a critical query omit the shard key;
assume even data means even load;
ignore unique index rules;
make every transaction cross-shard;
run global analytics scans constantly against the online cluster;
leave no capacity for balancing or resharding;
connect directly to a shard.

Each failure is a workload or operations problem, not merely a configuration typo. The remedy is usually to inspect routing, distribution, query plans, resource metrics, and domain invariants together.

Exercises

Design a shard key for multi-tenant orders. State the dominant queries, expected tenant distribution, and the trade-off between tenant locality and distribution within a giant tenant.
Compare ranged versus hashed sharding for time-series writes. Include the effect on write hotspots, time-range queries, and query fan-out.
Identify scatter-gather queries. For each query, explain which shard-key information is missing and propose a targeted or intentionally asynchronous alternative.
Model a giant-tenant hotspot. Estimate what happens when one tenant produces most writes or reads, and describe at least one compound-key or workload-isolation response.
Design zone sharding for residency. Map shard-key values to regions, then account for failover, capacity, and compliance requirements.
Evaluate a compound shard key. Compare ordered and hashed suffixes for tenant order listing, point reads, write distribution, giant tenants, and zone needs.
Explain unique constraint implications. Identify which business identifiers need uniqueness per tenant and which need global uniqueness, then verify whether the proposed sharded indexes can enforce them.
Design a materialized global dashboard to avoid scatter. Describe its update path, acceptable freshness, failure behavior, and the online query it replaces.
Simulate a reshard decision. List the evidence required before changing the key, the capacity needed during the operation, and the application or index compatibility checks.
Create monitoring dashboard dimensions per shard. Include resource metrics, range placement, latency, connection counts, targeted versus scatter-gather traffic, and the tenants or query shapes producing load.

Mastery checklist

Explain:

sharded components;
shard key;
cardinality/frequency;
ranged/hashed;
targeting;
scatter-gather;
hot shard;
balancing/zones;
unique-index implications;
cross-shard transactions;
reshard/refine;
why sharding is not first optimization.

You should be able to connect each item to a design or debugging decision. For example, naming scatter-gather is not enough: you should be able to recognize why it occurred, estimate its cost, and decide whether to change the query, model, summary path, or shard key.