147: Node.js + Express + MongoDB Production Architecture — Repositories, Services, Validation, Tenancy, Pagination, Transactions, and Shutdown

Learning objectives

By the end of this lesson, you should be able to:

combine Node.js, Express, the MongoDB native driver or Mongoose without blurring responsibilities;
design application startup and dependency composition;
keep HTTP, domain, persistence, and infrastructure boundaries explicit;
build tenant-safe repositories;
validate request data before Mongo query construction;
design cursor pagination with matching indexes;
map Mongo errors to stable HTTP contracts;
choose atomic writes versus transactions;
integrate idempotency and outbox patterns;
manage Mongo connection lifecycle and graceful shutdown;
test the architecture with real Mongo integration;
avoid duplicated data ownership and hidden ODM/framework coupling.

The goal

You already know the individual pieces:

Node runtime
Express
HTTP contracts
security
authentication
workers
testing
Mongo modeling
indexes
aggregation
transactions
replication/sharding
native driver
Mongoose

Production architecture is the discipline of giving each concern one clear owner. The goal is not to add layers for their own sake; it is to make data flow, security boundaries, and failure behavior understandable.

A robust baseline looks like this:

HTTP request
↓
Express middleware
↓
request schema validation
↓
authentication context
↓
service/application operation
↓
authorization/business rules
↓
repository
↓
MongoDB
↓
repository result
↓
service result
↓
HTTP response mapping

Each step has a reason. Validation keeps malformed input away from query construction, the service applies rules, and the repository owns Mongo-specific mechanics. Do not collapse the whole flow into one 300-line route.

The boundaries also make failures easier to locate. A malformed identifier belongs to the input boundary, a forbidden operation belongs to authorization, and a timeout belongs to persistence or infrastructure. When those concerns are mixed together, the resulting error handling becomes difficult to test and easy to bypass.

Suggested project structure

One reasonable layout is:

src/
├─ app/
│  ├─ create-app.js
│  ├─ create-runtime.js
│  └─ shutdown.js
├─ config/
│  └─ config.js
├─ domain/
│  ├─ errors.js
│  ├─ task-policy.js
│  └─ task-service.js
├─ http/
│  ├─ middleware/
│  │  ├─ authenticate.js
│  │  ├─ request-id.js
│  │  ├─ validate.js
│  │  └─ error-handler.js
│  └─ routes/
│     └─ tasks.js
├─ persistence/
│  ├─ mongo-client.js
│  ├─ task-repository.js
│  └─ indexes.js
├─ jobs/
│  └─ outbox-worker.js
└─ server.js

The names are illustrative, not a mandatory framework. A Mongoose implementation may put models inside repository internals, but the upper-level architecture should still look similar.

The useful test is not whether the folders have these exact names. It is whether a reader can identify where an HTTP concern ends, where a business rule lives, and where Mongo-specific behavior is isolated.

Dependency direction

Keep dependencies moving inward toward application behavior and persistence abstractions:

http
↓
service/domain
↓
repository interface/implementation
↓
Mongo driver/Mongoose

The reverse direction creates coupling that becomes expensive to remove. Avoid patterns such as:

repository imports Express
domain imports req/res
Mongo model sends HTTP response
route reaches into global MongoClient singleton everywhere

Infrastructure details should not leak upward unless the boundary deliberately exposes them. That keeps a service testable and prevents an ODM or web framework from becoming the domain model by accident.

Startup composition

Create long-lived dependencies once, compose them at startup, and pass them explicitly:

async function createRuntime(config) {
  const mongo = await createMongoDatabase(
    config.mongodb,
  );

  const taskRepository =
    createTaskRepository({
      db: mongo.db,
    });

  const taskService =
    createTaskService({
      taskRepository,
    });

  const app = createApp({
    taskService,
  });

  return {
    app,
    mongo,
  };
}

The server then owns the runtime lifecycle:

const config = loadConfig();

const runtime = await createRuntime(config);

const server = runtime.app.listen(
  config.port,
  config.host,
);

installShutdown({
  server,
  mongo: runtime.mongo,
});

No route opens its own database connection. This composition point is also where you can substitute test repositories, configure observability, and make shutdown responsibilities explicit.

Explicit composition is preferable to hidden imports of mutable global state. It gives startup a clear failure point if Mongo cannot be reached and gives shutdown a concrete handle to close later.

Mongo client factory

The Mongo client belongs to application infrastructure. A factory can connect once and return the handles the rest of the runtime needs:

import {
  MongoClient,
} from 'mongodb';

export async function createMongoDatabase({
  uri,
  databaseName,
  clientOptions,
}) {
  const client = new MongoClient(
    uri,
    clientOptions,
  );

  await client.connect();

  const db = client.db(
    databaseName,
  );

  return {
    client,
    db,
  };
}

The URI can contain credentials and other sensitive connection details. Do not log the URI.

Repository responsibility

The repository owns Mongo query details. Callers provide the values needed for the operation; they should not need to know how the collection is queried:

export function createTaskRepository({
  db,
}) {
  const tasks = db.collection(
    'tasks',
  );

  return {
    async findById({
      tenantId,
      taskId,
    }) {
      return tasks.findOne({
        _id: taskId,
        tenantId,
      });
    },
  };
}

The service should not need to know about:

$match
ObjectId conversion details
Mongo duplicate-key error internals

Those details may cross the boundary when the design intentionally treats them as part of the domain/persistence contract, but they should not leak accidentally.

Request ID parsing

An HTTP route receives an identifier as a string. Convert and validate it at the boundary, or in a dedicated input mapper, so malformed input becomes a client error rather than an unexpected database exception:

import {
  ObjectId,
} from 'mongodb';

function parseObjectId(value, field) {
  if (
    typeof value !== 'string' ||
    !ObjectId.isValid(value)
  ) {
    throw new ValidationError({
      [field]: 'Invalid identifier.',
    });
  }

  return ObjectId.createFromHexString(
    value,
  );
}

Do not let a raw BSON cast error become a 500. The boundary should classify invalid identifiers consistently with the rest of request validation.

That distinction matters operationally. A client that sends a malformed ID should receive a stable validation response, while an unavailable database should be visible as an infrastructure failure and handled through a different retry and alerting path.

Tenant ownership

Tenant identity comes from authenticated context, not from arbitrary request data. Authentication middleware can make that context available to the route:

res.locals.auth = {
  userId,
  tenantId,
  permissions,
};

The route passes the actor and the validated identifier to the service:

router.get(
  '/:taskId',
  async (req, res) => {
    const task = await taskService.getTask({
      actor: res.locals.auth,
      taskId: parseObjectId(
        req.params.taskId,
        'taskId',
      ),
    });

    res.json({
      data: {
        task,
      },
    });
  },
);

The service and repository retain the tenant predicate all the way to MongoDB:

const task =
  await taskRepository.findById({
    tenantId: actor.tenantId,
    taskId,
  });

Never do this:

await tasks.findOne({
  _id: taskId,
});

and only report an error after examining a document belonging to another tenant. Scope the lookup first. That both enforces authorization and avoids cross-tenant information leaks.

Tenant field cannot come from body

A body-controlled tenant field is an authorization vulnerability:

await Task.create(req.body);

For example, an attacker could submit:

{
  "tenantId": "victimTenant"
}

Instead, pass trusted actor context separately from validated business input:

await taskService.createTask({
  actor,
  input: validatedBody,
});

The repository constructs the document with the server-derived tenant:

{
  tenantId: actor.tenantId,
  title: input.title,
  ...
}

The server derives tenant ownership. It must not accept ownership from the client and hope validation will make it safe.

Schema validation can confirm that a supplied value has the right type, but it cannot make that value authoritative. Trust and shape are separate questions: the authenticated context supplies ownership, while the request schema describes the editable fields.

Validation before query construction

Consider a public filter request:

GET /tasks?status=open&priority=high

First validate and normalize the allowed query shape:

const ListTaskQuery = z.object({
  status: z
    .enum([
      'open',
      'done',
      'all',
    ])
    .default('all'),

  priority: z
    .enum([
      'low',
      'normal',
      'high',
    ])
    .optional(),

  limit: z.coerce
    .number()
    .int()
    .min(1)
    .max(100)
    .default(25),

  after: z
    .string()
    .optional(),
}).strict();

Then build the Mongo filter yourself. Do not pass the raw query object through:

collection.find(req.query);

The explicit builder is the boundary between a public API contract and Mongo's query language.

Query builder

For example, start with the mandatory tenant scope and add only the filters that the validated contract permits:

function buildTaskFilter({
  actor,
  query,
}) {
  const filter = {
    tenantId: actor.tenantId,
  };

  if (query.status === 'open') {
    filter.completed = false;
  }

  if (query.status === 'done') {
    filter.completed = true;
  }

  if (query.priority) {
    filter.priority = query.priority;
  }

  return filter;
}

There is no arbitrary Mongo operator input. That is useful for correctness and is also a security boundary.

It also keeps the public API stable when the persistence model changes. A client speaks in terms such as status and priority; it does not get to choose Mongo operators, field paths, or collection behavior.

Cursor pagination contract

Offset pagination becomes less attractive as a collection grows and changes while a user is paging through it. Cursor pagination needs a deterministic order. One suitable contract is:

createdAt DESC
_id DESC

The supporting index must match the tenant and common filter fields as well as that order:

{
  tenantId: 1,
  completed: 1,
  createdAt: -1,
  _id: -1
}

Use that shape if completed is commonly filtered. The _id tie-breaker makes records with the same timestamp orderable.

A cursor payload can contain:

{
  "createdAt": "2026-08-27T10:00:00.000Z",
  "id": "..."
}

Encode it as an opaque URL-safe token. After decoding, validate it before using it to construct a query.

Page query

For descending order, the next page contains records older than the cursor timestamp, or records at the same timestamp with a smaller identifier:

function applyAfterCursor(
  filter,
  cursor,
) {
  if (!cursor) {
    return filter;
  }

  return {
    ...filter,

    $or: [
      {
        createdAt: {
          $lt: cursor.createdAt,
        },
      },
      {
        createdAt: cursor.createdAt,
        _id: {
          $lt: cursor.id,
        },
      },
    ],
  };
}

Because the tenant and status predicates remain top-level, inspect the actual explain plan and index behavior. You may structure the query with $and if that makes the construction or planner behavior clearer.

Test pagination with the real query planner, not just an array-based unit test.

An in-memory test can prove that a cursor function returns the expected records for a small fixture. It cannot prove that Mongo selects the intended index, preserves the expected sort, or remains efficient as the collection grows. Those are separate pieces of evidence.

Limit + 1

Request one extra record:

.limit(limit + 1)

If that extra record exists:

hasNextPage = true
nextCursor = last returned item

Return only the requested limit to the client. This tells the API whether another page exists without paying for an exact count on every page.

Total counts

If the UI genuinely needs an exact total, choose an explicit strategy:

separate endpoint;
aggregation facet;
precomputed counter;
approximate count;
no count.

Do not make every list query pay the count cost by default, especially when the normal interaction only needs next-page information.

Index ownership

Required indexes should be defined and deployed explicitly. Treat them like schema migrations rather than relying on application startup side effects.

An example migration or administrative script is:

await tasks.createIndex(
  {
    tenantId: 1,
    completed: 1,
    createdAt: -1,
    _id: -1,
  },
  {
    name:
      'tasks_tenant_completed_created_id',
  },
);

Do not depend on production Mongoose autoIndex for a critical large deployment. Track index changes with the same care as schema changes.

An index is part of the runtime contract, not merely a local development convenience. Review its field order against real filters and sort order, and verify its deployment independently of application startup.

Unique error mapping

Suppose an external task identifier must be unique within each tenant. Define a unique compound index:

{
  tenantId: 1,
  externalId: 1
}

The index is unique. A duplicate commonly surfaces as Mongo error code:

The repository translates that persistence detail into a stable application error:

if (isDuplicateKey(error)) {
  throw new ConflictError(
    'External task already exists.',
    {
      cause: error,
    },
  );
}

Do not expose the index name or raw key values when they could reveal sensitive information.

Database errors versus domain errors

Persistence failures and domain outcomes are different categories. Repository failure classes can include:

DuplicateKey
DatabaseUnavailable
DatabaseTimeout

Service or domain outcomes can include:

NotFound
Conflict
Forbidden
Validation

HTTP mapping might then be:

Do not map every Mongo error to 500 blindly. Classification produces a more useful client contract and more actionable operations data.

Retry semantics

The driver may retry eligible reads and writes. The application may also retry selected transient operations, but those decisions require knowledge of idempotency, side effects, and deadlines.

Do not put an unbounded, generic loop around an arbitrary write:

for (let i = 0; i < 10; i++) {
  try {
    return await operation();
  } catch {}
}

Classify the operation by considering:

driver retryable operation;
idempotent service operation;
external side effects;
overall request deadline.

The same write can be safe to retry at one boundary and unsafe at another if an external side effect has already happened.

Retries also need a deadline. A request that has already used most of its allowed time should not begin another expensive attempt just because the driver reports that an operation is retryable.

Create idempotency

For POST /orders, a client can send:

Idempotency-Key

The server stores a record such as:

{
  tenantId,
  key,
  requestHash,
  status,
  result,
  expiresAt
}

Use a unique index per tenant and key:

{
  tenantId: 1,
  key: 1
}

The workflow must define what happens when two requests with the same key arrive concurrently, including whether one waits, receives the stored result, or receives an in-progress response. Do not store idempotency state only in an in-process Map if the application scales horizontally.

Optimistic concurrency

A task can carry a version number:

{
  version: 7
}

The client sends the version it read with its update:

{
  "version": 7,
  "title": "..."
}

The repository makes the expected version part of the match and increments it atomically:

const result = await tasks.updateOne(
  {
    _id: taskId,
    tenantId,
    version: expectedVersion,
  },
  {
    $set: {
      title,
      updatedAt: new Date(),
    },
    $inc: {
      version: 1,
    },
  },
);

If nothing matched, distinguish a missing resource from a version conflict without revealing a cross-tenant resource.

The version predicate is the concurrency check. A second writer using an old version no longer matches, so it cannot silently overwrite the first writer's update.

Atomic write before transaction

Suppose the invariant is simply:

complete task only if open

One conditional atomic update is enough:

findOneAndUpdate(
  {
    _id: taskId,
    tenantId,
    completed: false,
  },
  {
    $set: {
      completed: true,
    },
  },
);

No transaction is needed. The useful distinction is whether one atomic document operation can enforce the invariant, not whether a transaction feels safer.

Transaction boundary

Use a transaction when the invariant genuinely spans multiple documents. For example:

mark order paid
insert payment
write outbox event

Those writes can be committed together inside a Mongo transaction. A worker can publish the outbox event later.

Do not include external side effects such as:

send WhatsApp
charge external gateway

inside a Mongo retryable transaction callback. The callback may run more than once, so external work there can be repeated.

Outbox

An outbox collection can hold the event until a publisher safely observes it:

{
  _id,
  tenantId,
  type: "order.paid",
  aggregateId: orderId,
  payload: {...},
  createdAt,
  publishedAt: null,
  attempts: 0
}

The transaction performs the state changes together:

update order
insert payment
insert outbox
commit

The publisher claims unpublished events and sends them. A crash or retry can cause delivery to repeat, so consumers must be idempotent.

Outbox claim

Use an atomic findOneAndUpdate and a lease approach. A claim should select an event with:

publishedAt null
lease expired

That lets one worker claim it at a time. Do not let every replica publish the same event simultaneously.

The lease must have an owner and an expiry that the worker can renew or release according to the design. A crashed owner should not make the event permanently invisible, and a slow owner should not be mistaken for a completed publication.

Mongoose integration option

Mongoose changes repository internals, not the application boundaries:

routes
→ services
→ repositories
→ models

Avoid this shape:

routes
→ model everywhere

A repository might call:

Task.findOne(...).lean()
Task.updateOne(...)

The architecture remains the same. The model is an implementation detail behind the repository.

`lean()` in list paths

For a read-only list response, Mongoose can return plain objects:

await Task
  .find(filter)
  .sort(sort)
  .limit(limit + 1)
  .lean();

That is a good fit when the service needs ordinary API data rather than document behavior. If the service needs document methods or save, use hydrated documents intentionally.

Aggregation repository

Reports still need a repository boundary and validated parameters:

return orders.aggregate([
  {
    $match: {
      tenantId,
      createdAt: {
        $gte: start,
        $lt: end,
      },
    },
  },
  ...
]).toArray();

Report parameters are validated before the pipeline is built. Do not expose an arbitrary pipeline endpoint.

Search/vector repository

A search request may expose only an application-level contract:

query
filters
limit

The server builds the corresponding $search or $vectorSearch stage and includes tenant and permission filtering. Do not search globally and discard unauthorized hits after applying the limit. That can leak counts or timing and can also produce poor results.

Connection pool architecture

Repositories share an application-scoped database handle. Do not create a separate client in each repository:

Mongo client in each repository

The pool is shared, but it should not be made huge merely because API concurrency is high. A database can become overloaded even when the application can accept more HTTP requests.

Use:

bounded HTTP concurrency where needed;
pool metrics;
DB capacity.

Readiness

Before declaring the process ready, verify that the database is reachable. One possible check is:

await mongo.client.db('admin').command({
  ping: 1,
});

An initially successful connection can also establish readiness. Readiness should not issue a heavy ping on every request at a high rate; cache or reuse a health strategy.

Liveness has a different purpose. It should not restart an otherwise healthy application merely because of a brief primary election.

Keep readiness and liveness decisions tied to their intended consumers. Readiness controls whether traffic should be sent to this instance; liveness indicates whether the process itself needs replacement.

Graceful shutdown ordering

Shutdown is a dependency-ordering problem. A recommended sequence is:

SIGTERM
↓
set readiness false
↓
stop server accepting
↓
stop outbox/job pollers
↓
finish/cancel in-flight with deadline
↓
close change streams/cursors
↓
close MongoClient/Mongoose
↓
flush telemetry
↓
exit

Do not close Mongo first while HTTP requests are still running. Otherwise requests that are already admitted can fail simply because their dependency disappeared underneath them.

Background jobs

An outbox worker must stop claiming new jobs during shutdown:

must stop claiming new jobs on shutdown

It can then finish the current bounded jobs. Use AbortController for cancellation. If durable jobs matter, use leases so a crashed process leaves work that can become claimable after the lease expires.

Observability

For each query category, track enough information to understand latency and failure without turning sensitive data into logs or metric labels:

operation name
duration
result count
error code
tenant? avoid high-cardinality metric label

Tracing should show the request across its boundaries:

HTTP
→ service
→ Mongo
→ external publish

Do not log raw filters containing sensitive fields. Redaction and deliberate field selection matter more than collecting every input.

Query names and error categories are usually more useful for dashboards than complete request payloads. If a diagnostic value is necessary, record a safe classification or bounded summary instead of copying user-controlled data into logs.

Repository tests

Use a real Mongo test deployment for behavior that depends on Mongo semantics, including:

compound unique indexes;
ObjectId behavior;
cursor ordering;
transaction;
duplicate error;
aggregation;
tenant scope;
change streams if topology supports.

Pure mocks cannot prove Mongo semantics. They can still be useful for narrow service tests, but they should not be the only evidence for persistence behavior.

Integration test example

This test verifies that tenant scope is enforced by the repository:

test(
  'cannot load another tenant task',
  async () => {
    const task =
      await repository.insert({
        tenantId: tenantB,
        title: 'Secret',
      });

    const result =
      await repository.findById({
        tenantId: tenantA,
        taskId: task._id,
      });

    assert.equal(
      result,
      null,
    );
  },
);

This is a critical security test, not merely a repository style preference.

Failure injection

Exercise the architecture under failures such as:

Mongo unavailable
primary election
duplicate key
transaction transient failure
slow query
pool saturation
SIGTERM mid-request
outbox publish failure
consumer duplicate

For each case, define the expected user-visible and system behavior. A failure test is useful only when the team knows what the correct degraded behavior is.

For example, a transient database failure might produce a retryable service response, while a duplicate consumer delivery should produce one durable effect and a successful acknowledgement. The expected result should be explicit before the failure is injected.

Failure clinic

Mongo model imported directly in every route

This creates coupling and makes security rules inconsistent.

tenant filter added only in some repository methods

That is a data leak waiting to happen.

raw query params become Mongo filter

That turns an API input boundary into an injection boundary.

count + skip on every large list

This can become a performance problem.

auto retry transaction sends email twice

That is a side-effect bug caused by putting non-transactional work in retryable transaction logic.

pool closes before HTTP drains

Requests fail during deployment because their database dependency was closed too early.

in-memory idempotency/outbox state

This breaks across replicas and restarts.

Mongo errors sent raw to client

That can leak implementation details and sensitive information.

Exercises

Build runtime dependency composition.
Implement tenant-safe repository.
Add cursor pagination with matching index.
Add duplicate-key error mapping.
Implement optimistic version update.
Decide atomic update versus transaction for five workflows.
Build order/payment/outbox transaction.
Implement outbox lease publisher.
Add graceful shutdown order.
Write real Mongo security/integration tests.
Run explain for list query and attach evidence to code review.

Mastery checklist

Explain:

layer boundaries;
runtime composition;
tenant-scoped repository;
validation before query;
cursor/index alignment;
Mongo error mapping;
retry/idempotency;
atomic vs transaction;
outbox;
pool lifecycle;
graceful shutdown;
integration testing.