279: Case Study: URL Shortener

Learning outcomes

By the end of this lesson, you should be able to:

explain and apply short-code generation in a realistic implementation;
explain and apply the redirect read path in a realistic implementation;
explain and apply the creation path in a realistic implementation;
explain and apply caching in a realistic implementation;
explain and apply analytics in a realistic implementation.

Prerequisites and retrieval

This lesson builds on the 01-06 foundation and the preceding lessons in this module. Before you read, retrieve one concrete example from an earlier project where one of these concerns appeared. Maybe you generated identifiers, cached a read-heavy endpoint, or pushed work to a queue. That example gives you something to compare against as the design becomes more demanding.

The goal here is not to memorize a set of terms. It is to make a defensible decision inside a large-scale distributed service. That means making the requirements, traffic, failure modes, cost, and operational constraints explicit before choosing an implementation.

Terminology

Short-code generation: Choose random/base-N, sequence-derived, or distributed-ID codes, then reason about collisions, predictability, length, and the coordination required to generate them.
Redirect read path: Reads dominate this service. An edge or cache, followed by a key-value lookup, maps a code to its destination and returns the selected redirect status.
Creation path: Validate the destination and abuse/security policy, handle an optional custom alias or expiry, and reserve code uniqueness atomically.
Caching: Cache code -> URL mappings aggressively with a bounded TTL. Negative caching can protect against random-code scans, but it must not hide a newly created alias for too long.
Analytics: Emit redirect events asynchronously to a stream or warehouse instead of putting heavyweight synchronous counters on the latency-critical redirect path.
Abuse and safety: Phishing or malware links, enumeration, spam creation, rate limits, takedowns, and privacy retention are product requirements in their own right. They do not disappear just because the service scales.

Mental model

Treat Case Study: URL Shortener as a design problem with observable inputs, outputs, invariants, and failure modes. A URL shortener is a compact way to practice API design, identifier generation, key-value access, caching, analytics, abuse controls, and availability trade-offs in one system.

A strong implementation makes its assumptions visible, narrows uncertainty at system boundaries, and leaves enough evidence to support its safety: tests, types, constraints, metrics, or diagrams. If a code must be unique, the design should show where that uniqueness is enforced. If analytics may be delayed, the design should show that they are not part of the redirect's synchronous success condition.

A useful sequence for both interviews and production design is:

requirement -> constraints -> model -> implementation -> failure analysis -> verification

Do not jump from a requirement directly to a library call. First state what must remain true. Then choose the mechanism that enforces that invariant, and finally decide how you will observe whether the mechanism is working.

Deep dive

1. Short-code generation

The short code is the compact identifier a user places in the URL. You can generate it randomly with a base-N alphabet, derive it from a sequence, or obtain it from a distributed-ID scheme. Each choice changes the collision story, predictability, code length, and amount of coordination needed between writers.

Random codes may require collision detection and retry. Sequence-derived codes make uniqueness easier to reason about, but they can reveal ordering or growth. Distributed IDs reduce central coordination, but their raw form may be longer or may need encoding and additional thought about information leakage. The right choice depends on the contract, not on which generator is most familiar.

Decision rule: Use short-code generation deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

2. Redirect read path

Redirects are the dominant operation in this service. A request carrying a short code may be served at the edge or from a cache; on a miss, a key-value lookup maps the code to its destination. The service then returns the redirect status selected by the product contract.

Hot links and cacheability are central here. A fast read path should not make analytics, reporting, or other secondary work a synchronous prerequisite for returning the redirect. It also needs a clear response for an absent, expired, disabled, or otherwise invalid mapping.

Decision rule: Use a redirect read path deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

3. Creation path

Creation is more than inserting a destination. Validate the destination, apply abuse and security policy, handle an optional custom alias and expiry, and reserve code uniqueness atomically. Retries and concurrent requests matter: two writers must not both be allowed to claim the same alias merely because each checked for availability before either wrote it.

Decision rule: Use a creation path deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

4. Caching

Cache code -> URL mappings aggressively, but give the cache a bounded TTL. A cache hit keeps popular redirects away from the datastore. Negative caching, which temporarily caches a missing code, can also protect the system from random-code scans. Its TTL must be chosen carefully: a negative entry must not conceal a newly created alias for an unacceptable amount of time.

Decision rule: Use caching deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

5. Analytics

A redirect event can contain useful information for reporting, but collecting it should not turn every redirect into a slow, fragile transaction. Emit events asynchronously to a stream or warehouse and let workers process them. This gives analytics its own delivery and processing behavior while keeping the latency-critical redirect focused on resolving and returning the destination.

Decision rule: Use analytics deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

6. Abuse and safety

A URL shortener can be used to distribute phishing or malware links, enumerate valid codes, or create spam at high volume. A production design therefore needs rate limits, policy checks, takedown behavior, and privacy retention rules alongside its scaling plan. These controls affect the creation and read paths, and they can introduce their own availability, latency, and operational trade-offs.

Decision rule: Use abuse and safety deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design.

Worked example

Consider a large-scale distributed service whose requirements, traffic, failure modes, cost, and operational constraints are explicit rather than implied. Start by writing the requirement in one sentence. Then list the input and output contracts, and identify which concept above owns each failure mode.

The useful separation is by responsibility. Parsing and basic validation belong at the boundary. Domain rules belong in the domain or service layer. Persistence rules belong in the database or repository. Presentation rules belong in the client. Mixing these concerns can make a happy-path demo look shorter, but it makes edge cases, retries, and failures much harder to reason about.

Client
  |
DNS -> CDN / Edge
  |
Load Balancer -> API instances -> Cache
                          |          |
                          +------> Primary datastore
                          |
                          +------> Queue / Stream -> Workers

Read this diagram as a flow, not as a claim that every deployment must have exactly these components. The client reaches the service through DNS and possibly a CDN or edge layer. API instances use the cache for common mappings and the primary datastore for the authoritative lookup. Redirect events move through a queue or stream to workers, keeping secondary analytics work away from the synchronous path.

Walk through at least four cases: the normal path, an empty or missing value, a duplicate/retry/concurrent path where relevant, and a dependency failure. For each case, state which layer detects the problem and what the caller observes. For example, a uniqueness conflict belongs to the creation and persistence contract, while an unavailable analytics worker should not necessarily prevent a redirect. This is the level of explanation expected in a senior code review or technical interview.

Production perspective

Production correctness is broader than “the code works on my machine.” Ask how the design behaves during deploys, retries, partial failures, stale clients, concurrent requests, malformed data, schema changes, and high-cardinality workloads. Prefer explicit contracts, bounded resource usage, structured errors, and measurable behavior. Optimize only after evidence identifies a bottleneck or risk.

When the topic involves an external dependency, define a timeout and cancellation strategy. When it involves persistence, define transaction and consistency expectations. When it involves user-visible state, define loading, empty, error, stale, and success states. When it involves security, assume that the client can be modified and that network input is untrusted.

Guided lab

Design a global shortener for read-heavy traffic. Include estimates, code generation, storage and sharding, CDN/cache behavior, the analytics pipeline, custom aliases, expiration, abuse controls, SLOs, and failure behavior.

Complete the lab with this discipline:

Write the requirement and two non-requirements.
List input, output, and error contracts before implementation.
Implement the smallest correct vertical slice.
Add at least one invalid-input test and one edge-case test.
Instrument or inspect the behavior instead of guessing.
Refactor one hidden assumption into an explicit type, constraint, function, or configuration value.
Explain one alternative design and why you did not choose it.
Record a short “what would break at 10× scale?” note.

Edge cases and failure modes

For each concern, test more than the successful request. Check absence, malformed input, duplicates, ordering or concurrency where applicable, and behavior at the smallest and largest credible sizes.

Short-code generation: Check collisions, malformed or unsupported code input, duplicate allocation, concurrent generation, and the limits of the chosen length or alphabet.
Redirect read path: Check a missing, malformed, expired, or disabled mapping; duplicate requests; concurrent reads; and the smallest and largest credible destination or code sizes.
Creation path: Check absent or malformed destinations, duplicates, retries, concurrent alias claims, expiration boundaries, and the smallest and largest credible request sizes.
Caching: Check misses and negative entries, stale values, concurrent fills, expiration ordering, and behavior when the cache or datastore is unavailable.
Analytics: Check delayed, duplicated, reordered, malformed, or dropped events, and verify that a stream or worker failure does not silently change the redirect contract.

Common mistakes and debugging

Solving the example instead of the requirement: a copied pattern can be syntactically correct but architecturally wrong.
Hiding uncertainty with assertions, broad exception handlers, permissive schemas, or “temporary” any values.
Testing only the happy path and therefore discovering contracts only after integration.
Optimizing before measuring, or selecting a scalable mechanism without a scale requirement.
Letting client-side behavior stand in for server-side authorization, validation, or persistence guarantees.

For debugging, reproduce the smallest failing case first. Inspect the actual value or execution plan, trace the boundary where the invariant first becomes false, and fix the layer that owns the rule rather than adding a downstream patch. In this system, that may mean checking the generated code and uniqueness constraint, the cache key and TTL, the redirect response, or the event stream separately instead of treating the entire request as one opaque operation.

Interview questions

What problem does Short-code generation solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Redirect read path solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Creation path solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Caching solve, and what trade-off or failure mode would make you choose a different approach?
What problem does Analytics solve, and what trade-off or failure mode would make you choose a different approach?

Checkpoint

Without notes, explain Case Study: URL Shortener to another developer in five minutes. Include one invariant, one edge case, one production failure mode, and one alternative design. Then implement a small example without copying the lesson code.

Mastery checklist

I can define the core terms precisely.
I can choose a design from requirements instead of from habit.
I can implement and test the normal path and edge cases.
I can explain the runtime, storage, or complexity cost.
I can identify which layer owns validation, errors, and recovery.
I can compare at least two reasonable alternatives.
I can explain how the design changes at larger scale or stricter reliability.