250: Networking Foundations for System Design: IP, TCP/UDP, TLS, HTTP, and Connection Costs

Learning outcomes

By the end of this lesson, you can:

explain and apply IP and routing in a realistic implementation;
explain and apply TCP in a realistic implementation;
explain and apply UDP in a realistic implementation;
explain and apply TLS in a realistic implementation;
explain and apply HTTP versions in a realistic implementation.

These outcomes are deliberately practical. The goal is not simply to recite what each protocol does. You should be able to look at a request path, identify which layer is responsible for a behavior, and make a choice that fits the service's latency, reliability, security, and operational requirements.

Prerequisites and retrieval

This lesson assumes the earlier 01–06 foundation and the preceding lessons in this module. Before reading, retrieve one concrete example from a previous project where one of these concerns appeared. Perhaps a request went through a load balancer, a client reused an HTTP connection, a certificate expired, or a retry made a duplicate operation. You do not need to have used the protocol terminology at the time. The retrieval exercise is meant to connect the terminology to behavior you have already seen.

The goal is not to memorize vocabulary. It is to make a defensible decision inside a large-scale distributed service, where requirements, traffic, failure modes, cost, and operational constraints must be explicit. Networking choices are not isolated facts: they affect one another. For example, the HTTP version influences how requests are multiplexed, while the transport and connection strategy influence the latency the application actually observes.

Terminology

IP and routing: IP provides best-effort packet delivery across networks. Routing determines the next network hop for reaching an address; IP itself does not promise that a packet arrives, arrives once, or arrives in order.
TCP: TCP provides an ordered, reliable byte stream with connection setup, congestion control, retransmission, and flow control. It gives the application a stream of bytes, not message boundaries.
UDP: UDP is connectionless datagram transport without TCP-style ordering or reliability. Each datagram retains message boundaries, but delivery, ordering, and duplicate suppression are not provided by UDP.
TLS: TLS authenticates peers, typically the server, negotiates cryptography, and protects data in transit. It provides confidentiality and integrity for the protected connection; it does not make an application request authorized or trustworthy by itself.
HTTP versions: HTTP/1.1, HTTP/2, and HTTP/3 define application-level request and response semantics while using different connection and transport behavior. Treat the version as a precise engineering choice, not merely vocabulary.
Connection management: Keep-alive, connection pools, DNS changes, timeouts, and proxies determine real request latency and resource usage. The connection setup path can cost more than the application work for a small request.

Mental model

Treat Networking Foundations for System Design: IP, TCP/UDP, TLS, HTTP, and Connection Costs as a design problem with observable inputs, outputs, invariants, and failure modes. A high-level architecture diagram is only useful if you can explain what happens when a packet is lost, a connection is reused, a certificate is rejected, or one dependency becomes slow.

The network sits underneath the application, but its behavior is still visible at the service boundary. Handshakes add latency. Connection reuse can remove that cost. Packet loss can trigger retransmission. Encryption protects the bytes in transit but introduces certificate and key lifecycle work. Multiplexing changes how concurrent requests share a connection. Head-of-line effects and protocol directionality influence latency and reliability. A strong implementation makes assumptions visible, narrows uncertainty at boundaries, and leaves enough evidence - tests, types, constraints, metrics, or diagrams - to explain why the design is safe.

A useful interview and production sequence is:

requirement -> constraints -> model -> implementation -> failure analysis -> verification

For example, "the API should be fast" is not enough to select a protocol. Define what fast means, which requests are latency-sensitive, how much loss is acceptable, how clients behave during retries, and where TLS terminates. Then the protocol choice has something concrete to satisfy.

Do not jump from a requirement directly to a library call. First state what must remain true. Then choose the mechanism that enforces it. If an operation must not be silently reordered, identify where ordering is guaranteed. If a client must be authenticated, identify how identity is established and how the application authorizes the resulting request.

Deep dive

1. IP and routing

When an application sends data to another service, it usually addresses an IP endpoint rather than knowing every physical network segment between the two machines. Routing tables and network devices choose a path one hop at a time. That abstraction is useful, but it should not be mistaken for a delivery guarantee.

IP provides best-effort packet delivery across networks. Application architecture should therefore assume that packets can be delayed, duplicated, reordered, or lost beneath higher-level reliable protocols. Firewalls, NAT, route changes, and overloaded links can also affect whether a path is available. A successful DNS lookup does not prove that the destination is reachable, and reaching an IP address does not prove that the application will accept the request.

The practical consequence is that reliability and request semantics must be owned by a higher layer. TCP can provide a reliable ordered byte stream, and an application can add request identifiers or idempotency keys so that a retry does not create an unintended duplicate. Neither mechanism makes a failed operation automatically safe; the service still needs a clear contract.

Decision rule: Use IP and routing deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design. In a design review, be able to say which addresses are stable, where routing changes are expected, and how the caller distinguishes a network failure from an application response.

2. TCP

An application that reads from a TCP socket receives a byte stream. TCP does not preserve the boundaries of the writes made by the sender, so a protocol running over TCP must define framing, such as a length prefix, delimiter, or self-contained HTTP message format. This distinction is a common source of bugs in custom protocols.

TCP provides an ordered reliable byte stream with connection setup, congestion control, retransmission, and flow control. A slow or lost segment can affect ordered delivery: later data may have arrived at the receiver but cannot be delivered to the application until the missing portion is handled. This is one form of head-of-line behavior. TCP's reliability also does not mean the remote application completed the operation. A connection can fail after the server processed a request but before the client received the response.

Connection setup, TLS setup when used above TCP, retransmission, and slow-start behavior all contribute to the cost of a request. Keep-alive and connection pooling can amortize that cost, but pools must be bounded and must react to dead or outdated connections. Timeouts are part of the TCP client's application contract; without them, a request can consume resources while waiting for a peer that will never respond.

Decision rule: Use TCP deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design. TCP is a natural choice for protocols that need ordered reliable delivery, but document message framing, timeout behavior, retry safety, and the resource limits around the connections.

3. UDP

UDP exposes datagrams rather than one continuous stream. That preserves message boundaries and avoids TCP's connection and ordering machinery, but it also leaves more responsibility to the protocol above it. A datagram may be lost, duplicated, reordered, or rejected because it is too large for some path. If an application needs reliability, sequencing, authentication, congestion handling, or retransmission, those behaviors must come from a higher-level protocol.

UDP is connectionless datagram transport without TCP-style ordering or reliability. It can be useful when a stale update is less valuable than a fresh update delayed by retransmission, or when the application needs a transport on which it can define its own stream behavior. DNS commonly uses UDP for small queries, and real-time systems may prefer dropping an old update over waiting for it. Those choices are workload-specific, not automatic advantages.

Higher-level protocols such as QUIC can build different reliability and stream semantics on top of UDP. QUIC is not simply "UDP with no trade-offs": it still has protocol state, congestion control, encryption, and implementation complexity. The useful comparison is the behavior the complete protocol provides, not the name of the lowest transport layer.

Decision rule: Use UDP deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design. Before choosing it, state what happens to a lost or late datagram, how the receiver limits abusive traffic, and which layer provides integrity and identity.

4. TLS

An encrypted connection is not automatically a trusted business interaction. TLS authenticates peers, typically the server, negotiates cryptography, and protects data in transit. Certificate validation lets a client verify that it is connecting to the intended server identity under the configured trust model. Encryption helps prevent observers from reading or modifying the protected traffic, but application authorization still has to decide whether the caller may perform the requested action.

TLS authenticates peers (typically server), negotiates cryptography, and protects data in transit. Certificate lifecycle, termination point, and mTLS for service identity are architecture concerns. Decide where TLS terminates at an edge, load balancer, proxy, or service, and document which later hops are also protected. If a proxy terminates TLS and sends plaintext onward, that may be acceptable only when the network boundary and threat model explicitly allow it.

Mutual TLS adds client authentication using certificates and can provide service identity between internal components. It also adds certificate issuance, rotation, revocation or expiry handling, trust-store management, and debugging work. These operational responsibilities are part of the design rather than an implementation detail to defer until after deployment.

Decision rule: Use TLS deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design. Never treat a successful TLS handshake as proof that the request is authorized, valid, or safe to retry. Treat network input as untrusted even after transport authentication.

5. HTTP versions

HTTP/1.1 connection reuse, HTTP/2 multiplexing, and HTTP/3 over QUIC have different transport behavior, but application semantics still depend on methods, caching, headers, and idempotency. HTTP/1.1 commonly uses one request or response sequence at a time per connection, although clients can open several connections. HTTP/2 can multiplex many streams over one TCP connection, reducing the need for parallel connections, while loss on that TCP connection can still affect delivery at the transport level. HTTP/3 uses QUIC over UDP and changes the transport behavior, but it does not remove the need to define correct application semantics.

The version does not decide whether a POST is safe to repeat, whether a cache may serve a response, or whether a user is authorized. Those properties come from the method, headers, server contract, and application behavior. A retrying client therefore needs more than a protocol label: it needs to know which failures are safe to retry and how the server handles duplicate requests.

Decision rule: Use HTTP versions deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design. Compare the complete request path, including proxy and client support, connection reuse, multiplexing behavior, observability, and operational cost rather than choosing a version from a benchmark headline.

6. Connection management

Keep-alive, connection pools, DNS changes, timeouts, and proxies determine real request latency. A new request may require DNS resolution, a TCP connection, a TLS handshake, and proxy or load-balancer processing before application code runs. Reusing a healthy connection avoids some of that work, but reuse is not free: idle connections can become stale, a DNS answer can change, and a pool can hold more sockets than the service or operating system can support.

A service can exhaust sockets or connections long before CPU reaches 100%. Set explicit limits for pool size, idle time, queueing, response time, and connection establishment. Separate a connection timeout from a request or response timeout when the client library allows it, and propagate cancellation so abandoned work does not continue indefinitely. During deploys and failovers, the client should be able to discard broken connections and resolve current endpoints rather than assuming that an old address remains valid forever.

Proxies add another boundary where headers, timeouts, TLS termination, connection reuse, and failure reporting may differ from the application's assumptions. Inspect the behavior at each hop. A timeout that is longer in the client than in an intermediate proxy can appear to the caller as a generic connection reset rather than the application error the service intended to return.

Decision rule: Use connection management deliberately when it makes the contract or invariant easier to prove. If it only reduces typing while hiding an assumption, prefer the more explicit design. The design should state who owns connections, how many may exist, how they are retired, and what the caller observes when a connection or dependency fails.

Worked example

Consider a large-scale distributed service whose requirements, traffic, failure modes, cost, and operational constraints must be made explicit. Start by writing the requirement in one sentence. Then list the input and output contracts and identify which of the concepts above owns each failure mode. For example, a route or proxy may report that a destination is unreachable, the transport may report a timeout, TLS may reject the peer identity, and the application may return a structured authorization or validation error. Those are different observations and should not be collapsed into one generic "network error" during design or debugging.

The important move is separation: parsing or validation belongs at the boundary; domain rules belong in the domain or service layer; persistence rules belong in the database or repository; presentation rules belong in the client. Networking concerns cross those layers, but ownership still matters. The client can display a timeout, the transport client can enforce a deadline, and the service can decide whether a request identifier makes a retry safe. Mixing these concerns makes a happy-path demo look shorter while making edge cases much harder to reason about.

Client
  |
DNS -> CDN / Edge
  |
Load Balancer -> API instances -> Cache
                          |          |
                          +------> Primary datastore
                          |
                          +------> Queue / Stream -> Workers

Walk the example with at least four cases: the normal path, an empty or missing value, a duplicate, retry, or concurrent path where relevant, and a dependency failure. For each case, state which layer detects the problem and what the caller observes. Also identify which connections can be reused and where a timeout applies. This is the level of explanation expected in a senior code review or technical interview: not just naming components, but explaining their contracts and failure boundaries.

Production perspective

Production correctness is broader than "the code works on my machine." Ask how the design behaves during deploys, retries, partial failure, stale clients, concurrent requests, malformed data, schema changes, and high cardinality. A connection that works in a local process may behave differently behind a proxy, across a load balancer, or after DNS changes. Prefer explicit contracts, bounded resource usage, structured errors, and measurable behavior. Optimize only after you can identify the bottleneck or risk with evidence.

When the topic involves an external dependency, define a timeout and cancellation strategy. Decide whether a timeout means the operation is unknown rather than definitely failed, because the remote service may have completed work before the response was lost. When it involves persistence, define transaction and consistency expectations. When it involves user-visible state, define loading, empty, error, stale, and success states. When it involves security, assume the client can be modified and the network input is untrusted.

Record enough telemetry to distinguish DNS, connection establishment, TLS negotiation, time waiting for an available pooled connection, server processing, and response transfer when those timings matter. Without that breakdown, teams often optimize application code while the dominant cost is a handshake, a saturated pool, or a slow dependency. Metrics and traces should also avoid leaking credentials, tokens, or sensitive request data.

Guided lab

Draw a client-to-service request including DNS, TCP/QUIC, TLS, load balancer, and application. Mark where connections can be reused and list timeout values you would need at each hop. For each boundary, note whether failure is reported as a DNS failure, a connection failure, a TLS validation failure, a timeout, or an HTTP response. The point is to observe the request path as a sequence of contracts instead of treating the network as one opaque operation.

Complete the lab with this discipline:

Write the requirement and two non-requirements.
List input, output, and error contracts before implementation.
Implement the smallest correct vertical slice.
Add at least one invalid-input test and one edge-case test.
Instrument or inspect the behavior instead of guessing.
Refactor one hidden assumption into an explicit type, constraint, function, or configuration.
Explain one alternative design and why you did not choose it.
Record a short "what would break at 10x scale?" note.

When you write the scale note, include at least one resource limit and one failure mode. For example, consider what happens when the connection pool fills, when a DNS answer changes, when a retry arrives after the original request completed, or when one dependency becomes slower than the caller's timeout.

Edge cases and failure modes

IP and routing: test absence, malformed input, duplicates, ordering or concurrency where applicable, and behavior at the smallest and largest credible sizes. Include unreachable destinations, changing routes, and the distinction between a name-resolution failure and a reachable host that rejects the application request.
TCP: test absence, malformed input, duplicates, ordering or concurrency where applicable, and behavior at the smallest and largest credible sizes. Include message-framing mistakes, connection resets, partial reads, timeouts, and a response that is lost after the server may have processed the request.
UDP: test absence, malformed input, duplicates, ordering or concurrency where applicable, and behavior at the smallest and largest credible sizes. Include loss, reordering, duplicate datagrams, oversized messages, and the behavior the application chooses for late data.
TLS: test absence, malformed input, duplicates, ordering or concurrency where applicable, and behavior at the smallest and largest credible sizes. Include expired or mismatched certificates, trust-store differences, certificate rotation, failed mTLS identity, and an incorrectly assumed termination boundary.
HTTP versions: test absence, malformed input, duplicates, ordering or concurrency where applicable, and behavior at the smallest and largest credible sizes. Include connection reuse, multiplexed requests, retries of non-idempotent operations, cache behavior, proxy compatibility, and protocol negotiation failure.

The generic test categories are a starting point, not a substitute for protocol-specific tests. A test that passes over a healthy local connection says little about loss, stale pooled connections, certificate rotation, or a dependency that completes work after the caller has timed out.

Common mistakes and debugging

Solving the example instead of the requirement: a copied pattern can be syntactically correct but architecturally wrong.
Hiding uncertainty with assertions, broad exception handlers, permissive schemas, or "temporary" any values.
Testing only the happy path and therefore discovering contracts only after integration.
Optimizing before measuring, or selecting a scalable mechanism without a scale requirement.
Letting client-side behavior stand in for server-side authorization, validation, or persistence guarantees.

For debugging, reproduce the smallest failing case, inspect the actual value or execution plan, trace the boundary where the invariant first becomes false, and fix the owning layer rather than adding a downstream patch. Start by determining whether the failure occurs during DNS, route or connection establishment, TLS negotiation, request transmission, server processing, response transfer, or client parsing. Logs, traces, connection-pool metrics, and packet-level tools can answer different parts of that question; do not infer a protocol failure solely from the final error message.

If a retry is involved, determine whether the first request could have reached or completed at the server. If the issue appears only after deployment or failover, inspect DNS caching, pooled connections, proxy behavior, and certificate configuration. If latency is high without an obvious error, separate queueing for a connection from network transfer and application processing before changing the protocol.

Interview questions

What problem does IP and routing solve, and what trade-off or failure mode would make you choose a different approach?
What problem does TCP solve, and what trade-off or failure mode would make you choose a different approach?
What problem does UDP solve, and what trade-off or failure mode would make you choose a different approach?
What problem does TLS solve, and what trade-off or failure mode would make you choose a different approach?
What problem do HTTP versions solve, and what trade-off or failure mode would make you choose a different approach?

Answer each question with a requirement, not only a definition. Include the guarantee the mechanism provides, the guarantee it does not provide, and one operational consequence. For example, TCP's reliable ordering does not prove that a business operation was executed exactly once, and TLS encryption does not prove that the caller is authorized.

Checkpoint

Without notes, explain Networking Foundations for System Design: IP, TCP/UDP, TLS, HTTP, and Connection Costs to another developer in five minutes. Your explanation must include one invariant, one edge case, one production failure mode, and one alternative design. Then implement a small example without copying the lesson code.

As a self-check, make sure your explanation distinguishes the network path from application semantics. You should be able to say where delivery is best-effort, where ordering or reliability is added, where peer identity is checked, where HTTP meaning is defined, and how connection reuse or timeouts change the observed cost.

Mastery checklist

I can define the core terms precisely.
I can choose a design from requirements instead of from habit.
I can implement and test the normal path and edge cases.
I can explain the runtime, storage, or complexity cost.
I can identify which layer owns validation, errors, and recovery.
I can compare at least two reasonable alternatives.
I can explain how the design changes at larger scale or stricter reliability.