Skip to content
AyoKoding

Intermediate Examples

Examples 29-60 leave the terminal and go straight to the socket API: hand-rolled TCP servers and clients, message framing over a byte stream, a tiny command protocol, nc, hand-crafted HTTP requests, curl-driven HTTP methods/status codes/headers, and UDP's connectionless contrast to everything TCP guarantees. Every .py file below is complete, self-contained, and was actually run to produce its **Output** block -- no fabricated output anywhere.


Example 29: TCP Echo Server

ex-29 · exercises co-10, co-07

The classic TCP server sequence -- bind, listen, accept -- claims a port, marks the socket passive, then blocks until one client connects. This example pairs with Example 30's client: run this file first, in the background, then run the client against it.

%% Color Palette: Blue #0173B2, Orange #DE8F05, Teal #029E73
sequenceDiagram
    participant C as client (Ex 30)
    participant S as server (Ex 29)
    S->>S: bind() + listen()
    Note over S: passive, waiting
    C->>S: connect() -- co-07 handshake
    S->>C: accept() returns conn
    C->>S: sendall(message)
    S->>C: sendall(echo)
# learning/code/ex-29-tcp-echo-server/server.py
"""Example 29: TCP Echo Server."""
 
import socket  # => stdlib Berkeley sockets API (co-10)
 
HOST = "127.0.0.1"  # => loopback only -- this server never leaves the local machine
PORT = 50029  # => an ephemeral, unregistered port well above the well-known range (co-05)  # fmt: skip
 
 
def run_server() -> None:  # => binds, listens, accepts ONE client, echoes, then exits
    # socket.AF_INET selects IPv4; SOCK_STREAM selects TCP (co-07 reliable byte stream).
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as server_sock:
        # => "with" scopes the socket's lifetime to this block -- it closes automatically
        # => even if an exception fires below, so a crashed server never leaks the fd
        # SO_REUSEADDR lets an immediate restart reuse a port stuck in TIME_WAIT --
        # Example 38 examines this option in depth; every server script in this topic
        # sets it so repeated runs never collide on a leftover TIME_WAIT socket.
        server_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        # => must be set BEFORE bind() -- setting it after bind() has no effect at all
        server_sock.bind((HOST, PORT))  # => claims (HOST, PORT) -- fails if already in use  # fmt: skip
        server_sock.listen(1)  # => marks the socket passive: ready to queue incoming connections  # fmt: skip
        print(f"listening on {HOST}:{PORT}")  # => a signal the client script waits for
        conn, addr = server_sock.accept()  # => BLOCKS until a client connects (co-07 handshake)  # fmt: skip
        # => conn is a NEW socket dedicated to this one client; addr is the client's (ip, port)
        with conn:
            # => conn's own "with" is a SEPARATE lifetime from server_sock's -- closing
            # => this one client's socket never touches the still-listening server_sock
            print(f"accepted connection from {addr}")
            # => proves accept() genuinely returned, not merely that a SYN packet arrived
            data = conn.recv(1024)  # => reads up to 1024 bytes sent by the client, blocking  # fmt: skip
            print(f"received: {data!r}")
            # => shows the raw bytes BEFORE echoing, so the transcript reads as a clear pair
            conn.sendall(data)  # => echoes the EXACT bytes back -- sendall loops until all sent  # fmt: skip
 
 
if __name__ == "__main__":  # => only runs when invoked directly, not when imported
    run_server()
    # => the guard above is WHY this call never fires if another script imports this module

Run: python3 server.py & (background), then Example 30's client.py

Output (server):

listening on 127.0.0.1:50029
accepted connection from ('127.0.0.1', 54821)
received: b'hello from ex-30 client\n'

Key takeaway: bind claims the address, listen flips the socket passive, accept blocks until a real TCP handshake completes -- three distinct steps, each with a distinct, checkable failure mode.

Why it matters: Every framework's app.listen(port) or app.run() call is this exact three-step sequence, wrapped up in one line -- knowing what happens underneath is what turns "the server won't start" from a mystery into a specific, diagnosable failure (wrong port? already bound? never called listen?). Node's http.createServer().listen() and Flask's app.run() hide this identical sequence behind one method call, but the failure modes remain the same underneath. Recognizing them individually turns a vague production incident into a checklist a new engineer can work through alone.


Example 30: TCP Echo Client

ex-30 · exercises co-10, co-01

socket.create_connection is a convenience wrapper around resolve-and-connect: give it a (host, port) tuple and it performs the full TCP handshake in one call, mirroring the client half of co-01's client-server model against Example 29's server.

# learning/code/ex-30-tcp-echo-client/client.py
"""Example 30: TCP Echo Client."""
 
import socket  # => same stdlib sockets API the server uses (co-10)
 
HOST = "127.0.0.1"  # => must match the server's bound address exactly
PORT = 50029  # => must match the server's bound port exactly (co-05)
 
 
def run_client(message: bytes) -> bytes:  # => connects, sends one message, returns the echo  # fmt: skip
    # socket.create_connection is a convenience wrapper: resolve + connect in one call (co-01).
    with socket.create_connection((HOST, PORT)) as sock:  # => performs the TCP handshake  # fmt: skip
        sock.sendall(message)  # => writes every byte of message onto the connection
        return sock.recv(1024)  # => reads the server's echoed reply, up to 1024 bytes
 
 
if __name__ == "__main__":  # => only runs when invoked directly, not when imported
    # run_client is a plain function returning bytes -- kept separate from the printing/
    # asserting below so it could be imported and reused elsewhere without any side effects
    reply = run_client(b"hello from ex-30 client\n")  # => Example 29's server must already be up  # fmt: skip
    print(f"client received: {reply!r}")  # => confirms the echoed bytes match what was sent  # fmt: skip
    assert reply == b"hello from ex-30 client\n"  # => proves the round trip preserved every byte  # fmt: skip
    print("ex-30 OK")  # => a final marker confirming every assertion above passed, not just ran  # fmt: skip

Run: python3 client.py (with server.py already running)

Output:

client received: b'hello from ex-30 client\n'
ex-30 OK

Key takeaway: The client's exact bytes reappeared, unmodified, in reply -- confirming the full send -> server-recv -> server-send -> client-recv loop preserved every byte, over a real (if local) TCP connection.

Why it matters: This client/server pair is the smallest complete instance of co-01's request/response model this topic offers -- every later, fancier example (a command protocol, a JSON API, a redirect-following HTTP client) is this same shape with more logic layered on top. Production client libraries in every language -- Python's requests, Node's fetch, Go's net/http -- ultimately reduce to this same connect-send-receive sequence, just with far more convenience layered on top. Seeing the raw shape once means that convenience layer never feels like unexplainable magic again.


Example 31: socket.bind / .listen / .accept, Annotated

ex-31 · exercises co-10, co-07

This example runs server and client in ONE process, using a background threading.Thread and a threading.Event for synchronization -- a pattern used throughout the rest of this tier so every example stays self-contained and leaves no stray process running.

# learning/code/ex-31-socket-bind-listen-accept/example.py
"""Example 31: socket.bind / .listen / .accept, Annotated."""
 
import socket  # => same stdlib module used by every server/client in this tier
import threading  # => used only for the ready-signal + background thread below, not real load
 
HOST = "127.0.0.1"  # => loopback -- no traffic ever leaves this machine
PORT = 50031  # => an unregistered ephemeral port (co-05)
 
 
def server(ready: threading.Event) -> None:  # => runs on a background thread (co-10)
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:  # => AF_INET+SOCK_STREAM = TCP  # fmt: skip
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)  # => allow instant re-bind  # fmt: skip
        sock.bind((HOST, PORT))  # => bind: CLAIMS this (address, port) pair for this process  # fmt: skip
        # => after bind, the socket has an address but is not yet accepting connections
        sock.listen(1)  # => listen: flips the socket into PASSIVE mode, backlog of 1 pending conn  # fmt: skip
        # => after listen, the OS will queue up to 1 incoming SYN before accept() is even called
        print("state: bound and listening")  # => Output line 1
        ready.set()  # => signals the main thread it is safe to connect now
        conn, addr = sock.accept()  # => accept: BLOCKS until a client completes the handshake  # fmt: skip
        # => accept() returns a NEW socket (conn) distinct from the listening socket (sock) --
        # => sock keeps listening for MORE clients; conn is this one client's private channel
        print(f"state: accepted a connection from {addr}")  # => Output line 2
        with conn:
            conn.recv(16)  # => drains whatever the client sent, so it can close cleanly
 
 
ready_event = threading.Event()  # => co-11: coordinates "server is ready" without a sleep guess  # fmt: skip
server_thread = threading.Thread(target=server, args=(ready_event,), daemon=True)
# => daemon=True means this thread never blocks process exit if something above hangs
server_thread.start()  # => starts the server concurrently with the code below
ready_event.wait(timeout=5)  # => blocks the main thread until bind()+listen() have both run  # fmt: skip
 
with socket.create_connection((HOST, PORT), timeout=5) as client_sock:  # => triggers accept() above  # fmt: skip
    client_sock.sendall(b"hi")  # => a few bytes so the server's recv() has something to drain  # fmt: skip
    print("state: client connected successfully")  # => Output line 3
 
server_thread.join(timeout=5)  # => waits for the server thread to finish handling that one client  # fmt: skip
print("ex-31 OK")  # => confirms the full bind/listen/accept sequence completed cleanly

Run: python3 example.py

Output:

state: bound and listening
state: client connected successfully
state: accepted a connection from ('127.0.0.1', 54900)
ex-31 OK

Key takeaway: The "client connected successfully" line prints BEFORE the server's "accepted a connection" line -- a real race between the main thread's next statement and the server thread's print, proving these two sides genuinely run concurrently rather than in lockstep.

Why it matters: threading.Event (co-11) is exactly how this entire tier avoids the classic "sleep(1) and hope the server is ready" anti-pattern -- a real coordination primitive instead of a guessed delay is the difference between a test that's reliable and one that's merely usually-passing. Production test suites hit this exact problem constantly when spinning up a local server for integration tests -- a flaky sleep()-based wait is a common, avoidable source of intermittent CI failures. Reaching for threading.Event (or its equivalent in any language) instead turns a coin-flip test into a deterministic one.


Example 32: socket.connect / .sendall / .recv, Annotated

ex-32 · exercises co-10

connect() performs the actual three-way handshake (SYN, SYN-ACK, ACK); sendall() and recv() are the byte-stream read/write primitives that follow. This example calls out each one's specific contract, including the subtlety that .recv()'s argument is a maximum, not a guarantee.

# learning/code/ex-32-socket-connect-send-recv/example.py
"""Example 32: socket.connect / .sendall / .recv, Annotated."""
 
import socket  # => same stdlib module as every other socket example in this tier
import threading  # => only for the ready-signal + background thread, not real concurrency
 
HOST = "127.0.0.1"  # => loopback -- this whole exchange stays on the local machine
PORT = 50032  # => co-05: a fresh ephemeral port, distinct from every other example's port  # fmt: skip
 
 
def server(ready: threading.Event) -> None:  # => a minimal echo server, backgrounded (co-10)  # fmt: skip
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
        # => IPv4 + TCP -- scoped to this "with" block so the fd always closes, even on error
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        # => must be set BEFORE bind(), or an immediate re-run could fail with "address in use"
        sock.bind((HOST, PORT))
        # => claims this exact (HOST, PORT) pair for this process -- fails if already taken
        sock.listen(1)
        # => flips the socket passive: it can now queue ONE pending connection before accept()
        ready.set()
        # => signals the main thread it is safe to connect -- avoids a guessed sleep() delay
        conn, _ = sock.accept()
        # => BLOCKS here until the client's connect() below completes the three-way handshake
        with conn:
            # => conn is this ONE client's private socket -- distinct from the listening sock
            data = conn.recv(1024)  # => reads whatever the client sends
            conn.sendall(data)  # => echoes it straight back
 
 
ready_event = threading.Event()
# => a real synchronization primitive -- co-11: avoids the "sleep(1) and hope" anti-pattern
server_thread = threading.Thread(target=server, args=(ready_event,), daemon=True)
# => daemon=True: this background thread never blocks process exit if something hangs
server_thread.start()
# => runs the server concurrently with the connect/send/recv code that follows below
ready_event.wait(timeout=5)
# => blocks the main thread here until bind()+listen() have both genuinely completed
 
# create_connection wraps getaddrinfo + socket() + connect() in one call (co-01, co-10).
with socket.create_connection((HOST, PORT), timeout=5) as sock:  # => connect: the TCP handshake  # fmt: skip
    # => connect() blocks until the three-way handshake (SYN, SYN-ACK, ACK) completes (co-07)
    print("state: connected")  # => Output line 1
    sock.sendall(b"round trip\n")  # => sendall: loops internally until EVERY byte is written  # fmt: skip
    # => a plain .send() can write FEWER bytes than requested; .sendall() never does
    reply = sock.recv(1024)  # => recv: reads up to 1024 bytes, blocking until at least 1 arrives  # fmt: skip
    # => recv's argument is a MAXIMUM, not a guarantee -- it can return fewer bytes than asked
    print(f"state: received {reply!r}")  # => Output line 2
 
server_thread.join(timeout=5)
# => waits here for the server thread to finish handling that one client before exiting
 
assert reply == b"round trip\n"  # => confirms the full round trip preserved every byte
print("ex-32 OK")  # => a final marker confirming the assertion above passed, not just ran  # fmt: skip

Run: python3 example.py

Output:

state: connected
state: received b'round trip\n'
ex-32 OK

Key takeaway: .sendall() guarantees every byte is written before returning; .recv(n) guarantees AT MOST n bytes -- the two calls are NOT symmetric, and confusing them is a common source of "my message got cut off" bugs.

Why it matters: This asymmetry is exactly why Example 33 needs a read_line() helper at all -- a single .recv() call can never be trusted to return "one whole message," only "however many bytes happened to arrive on this one call." Production bugs from assuming .recv() returns a complete message are common enough that most real protocols define an explicit framing scheme rather than trusting one call to deliver everything. Internalizing this asymmetry early avoids chasing intermittent, hard-to-reproduce truncation bugs later in a project's life.


Example 33: Line Framing with \n Delimiters

ex-33 · exercises co-11, co-07

TCP delivers a raw byte stream with NO built-in message boundaries -- the protocol running on top must invent its own framing. This example deliberately reads only 4 bytes at a time to force multiple recv() calls, then reassembles them into one line using \n as the delimiter.

%% Color Palette: Blue #0173B2, Orange #DE8F05, Teal #029E73
graph LR
    A["send#40;full line#41;"]:::blue --> B["recv#40;4#41;<br/>chunk 1"]:::orange
    B --> C["recv#40;4#41;<br/>chunk 2..N"]:::orange
    C --> D["buffer accumulates<br/>until \n found"]:::teal
    D --> E["one complete line"]:::teal
 
    classDef blue fill:#0173B2,stroke:#000000,color:#FFFFFF,stroke-width:2px
    classDef orange fill:#DE8F05,stroke:#000000,color:#FFFFFF,stroke-width:2px
    classDef teal fill:#029E73,stroke:#000000,color:#FFFFFF,stroke-width:2px
# learning/code/ex-33-line-framing/example.py
"""Example 33: Line Framing with \\n Delimiters."""
 
import socket  # => stdlib sockets -- this example builds framing ON TOP of it, not inside it
import threading  # => only the ready-signal + background thread, not real concurrency
 
HOST = "127.0.0.1"  # => loopback -- keeps this framing demo local and deterministic
PORT = 50033  # => co-05: a fresh ephemeral port, unique to this example
 
 
def read_line(sock: socket.socket, buffer: bytearray) -> bytes:
    # => a "buffer" parameter, not a local variable, is WHY this survives across MULTIPLE
    # => calls -- leftover bytes past one \n stay available for the NEXT call to read_line
    # A TCP byte stream has NO message boundaries -- co-11 says the protocol must invent
    # its own framing. Here, "one message" means "bytes up to the next newline."
    while b"\n" not in buffer:  # => keep reading until a full line has actually arrived
        chunk = sock.recv(4)  # => a DELIBERATELY tiny read size to force multiple recv() calls  # fmt: skip
        if not chunk:  # => the peer closed before a full line arrived
            raise ConnectionError("peer closed mid-line")
        buffer.extend(chunk)  # => accumulate bytes across possibly many small reads
    line, _, rest = buffer.partition(b"\n")  # => split off exactly one line, keep the remainder  # fmt: skip
    buffer[:] = rest  # => whatever came AFTER the newline stays buffered for the next call  # fmt: skip
    return bytes(line)  # => bytearray.partition returns bytearray -- normalize to plain bytes  # fmt: skip
 
 
def server(ready: threading.Event) -> None:  # => backgrounded so the client below can run inline  # fmt: skip
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
        # => IPv4 + TCP, scoped to this "with" block so the fd always closes on exit
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        # => set BEFORE bind() -- lets an immediate re-run reuse a TIME_WAIT'd port
        sock.bind((HOST, PORT))
        # => claims (HOST, PORT) for this process -- must happen before listen()
        sock.listen(1)
        # => flips the socket passive, ready to queue one pending connection
        ready.set()
        # => unblocks the main thread's wait() below -- no guessed sleep() needed
        conn, _ = sock.accept()
        # => BLOCKS until the client's connect() completes the TCP handshake
        with conn:
            buf = bytearray()  # => per-connection leftover-bytes buffer (co-11)
            line = read_line(conn, buf)  # => reassembles ONE full line from many small recv()s  # fmt: skip
            print(f"server framed: {line!r}")  # => proves the tiny 4-byte reads still yield one line  # fmt: skip
 
 
ready_event = threading.Event()
thread = threading.Thread(target=server, args=(ready_event,), daemon=True)
# => daemon=True: this thread never blocks process exit if something above hangs
thread.start()
# => runs the server concurrently with the client connection code that follows below
ready_event.wait(timeout=5)
# => blocks here until bind()+listen() genuinely completed, avoiding a race with connect()
 
with socket.create_connection((HOST, PORT), timeout=5) as sock:
    # Sent as ONE call, but the server above reads it back 4 bytes at a time --
    # framing is what makes "a line" meaningful regardless of how recv() chunks it.
    sock.sendall(b"this line spans many tiny reads\n")
    # => sendall guarantees the WHOLE 33-byte line is written, even though the server
    # => on the other end will only ever see it arrive in small, independent chunks
 
thread.join(timeout=5)
# => waits for the server thread to finish handling that one client before exiting
print("ex-33 OK")  # => confirms read_line() reassembled the tiny reads without loss

Run: python3 example.py

Output:

server framed: b'this line spans many tiny reads'
ex-33 OK

Key takeaway: read_line() reassembled a 32-byte line from 4-byte reads with no loss and no wrong split point -- the \n delimiter, not the size of any one recv() call, is what actually defines "one message" here.

Why it matters: Every line-based protocol in this topic (Example 36's PING/TIME, the capstone's command server) relies on exactly this read_line() pattern -- once you trust it works regardless of how the OS happens to chunk the underlying bytes, building a protocol on top of it becomes straightforward. Line-based protocols remain common in production systems -- Redis's inline command protocol, SMTP, and many internal RPC formats all use this exact newline-delimited framing. A read_line() helper like this one is typically the first piece of infrastructure any hand-rolled protocol implementation needs to get right.


Example 34: Handle Partial recv() -- Reassemble a Large Message

ex-34 · exercises co-11

For a FIXED-size payload (as opposed to a delimited line), the correct pattern is recv_exact(): loop, accumulating bytes, until exactly the expected count has arrived -- since a single recv() call can legally return far fewer bytes than requested, especially for large payloads.

%% Color Palette: Blue #0173B2, Orange #DE8F05, Teal #029E73
graph LR
    A["recv_exact#40;5000#41;<br/>requested"]:::blue --> B["recv#40;#41; #35;1<br/>1500 bytes"]:::orange
    B --> C["recv#40;#41; #35;2<br/>1500 bytes"]:::orange
    C --> D["recv#40;#41; #35;N<br/>remaining bytes"]:::orange
    D --> E["exactly 5000 bytes<br/>assembled"]:::teal
 
    classDef blue fill:#0173B2,stroke:#000000,color:#FFFFFF,stroke-width:2px
    classDef orange fill:#DE8F05,stroke:#000000,color:#FFFFFF,stroke-width:2px
    classDef teal fill:#029E73,stroke:#000000,color:#FFFFFF,stroke-width:2px
# learning/code/ex-34-handle-partial-recv/example.py
"""Example 34: Handle Partial recv() -- Reassemble a Large Message."""
 
import socket  # => stdlib sockets -- recv_exact below is built entirely on top of it
import threading  # => only the ready-signal + background thread, not real concurrency
 
HOST = "127.0.0.1"  # => loopback -- keeps this large-payload demo local and deterministic  # fmt: skip
PORT = 50034  # => co-05: a fresh ephemeral port, unique to this example
PAYLOAD_SIZE = 200_000  # => far bigger than any single recv() call typically returns
 
 
def recv_exact(sock: socket.socket, count: int) -> bytes:
    # co-11: recv() may return FEWER bytes than requested for a large payload -- looping
    # until exactly `count` bytes have arrived is the only correct way to read a fixed size.
    chunks: list[bytes] = []  # => collects each partial read
    remaining = count  # => how many more bytes are still needed
    while remaining > 0:  # => keeps looping until the full payload has arrived
        chunk = sock.recv(min(65536, remaining))  # => never over-read past the target size  # fmt: skip
        if not chunk:  # => peer closed before sending everything -- a real, checkable failure  # fmt: skip
            raise ConnectionError("peer closed before sending the full payload")
        chunks.append(chunk)
        # => appended, not concatenated, here -- string/bytes concatenation in a loop is O(n^2)
        remaining -= len(chunk)  # => shrinks toward 0 as more bytes arrive
    return b"".join(chunks)  # => reassembles every partial read into the original bytes, once  # fmt: skip
 
 
def server(ready: threading.Event) -> None:  # => backgrounded so the client below can run inline  # fmt: skip
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:  # => same triple as Ex 29+  # fmt: skip
        # => IPv4 + TCP, scoped to this "with" block so the fd always closes on exit
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        # => set BEFORE bind() -- lets an immediate re-run reuse a TIME_WAIT'd port
        sock.bind((HOST, PORT))
        # => claims (HOST, PORT) for this process -- must happen before listen()
        sock.listen(1)
        # => flips the socket passive, ready to queue one pending connection
        ready.set()
        # => unblocks the main thread's wait() below -- no guessed sleep() needed
        conn, _ = sock.accept()
        # => BLOCKS until the client's connect() completes the TCP handshake
        with conn:  # => this one connection's socket -- closes automatically on block exit  # fmt: skip
            data = recv_exact(conn, PAYLOAD_SIZE)  # => reassembles the full 200,000-byte payload  # fmt: skip
            intact = data == b"x" * PAYLOAD_SIZE  # => confirms every one of the 200,000 bytes matches  # fmt: skip
            print(f"server received {len(data)} bytes, intact: {intact}")  # => size AND content check  # fmt: skip
 
 
ready_event = threading.Event()  # => the same ready-signal pattern used since Example 34's peers  # fmt: skip
thread = threading.Thread(target=server, args=(ready_event,), daemon=True)
# => daemon=True: this thread never blocks process exit if something above hangs
thread.start()
# => runs the server concurrently with the client connection code that follows below
ready_event.wait(timeout=5)
# => blocks here until bind()+listen() genuinely completed, avoiding a race with connect()
 
with socket.create_connection((HOST, PORT), timeout=5) as sock:  # => a plain, ordinary connect()  # fmt: skip
    sock.sendall(b"x" * PAYLOAD_SIZE)  # => one logical send; the OS may still split it on the wire  # fmt: skip
 
thread.join(timeout=5)
# => waits for the server thread to finish handling that one client before exiting
print("ex-34 OK")  # => confirms recv_exact() reassembled all 200,000 bytes without loss

Run: python3 example.py

Output:

server received 200000 bytes, intact: True
ex-34 OK

Key takeaway: All 200,000 bytes arrived correctly, byte-for-byte, despite the OS almost certainly splitting them into multiple TCP segments underneath -- recv_exact()'s loop is what turns "some bytes" into "exactly the bytes I asked for."

Why it matters: This is the delimiter-free sibling of Example 33's read_line() -- any protocol that sends a fixed-size header (a length prefix, a hash, a fixed record) needs recv_exact()'s pattern, not a single hopeful recv() call. Binary protocols with fixed-length headers -- length-prefixed framing, protocol buffers over a raw socket, custom RPC formats -- all depend on exactly this recv_exact() pattern to read a known byte count reliably. Skipping it is a common source of subtle, load-dependent corruption bugs that only surface under real network conditions.


Example 35: Multiple Messages, One Connection, In Order

ex-35 · exercises co-11, co-07

TCP's byte-stream ordering guarantee (co-07) means multiple request/response pairs can be exchanged over ONE connection, strictly in the order they were sent -- no need to open a fresh connection per message.

%% Color Palette: Blue #0173B2, Orange #DE8F05, Teal #029E73
sequenceDiagram
    participant C as client
    participant S as server (Ex 35)
    Note over C,S: ONE connection carries all three messages
    C->>S: sendall(first)
    S->>C: reply: FIRST
    C->>S: sendall(second)
    S->>C: reply: SECOND
    C->>S: sendall(third)
    S->>C: reply: THIRD
# learning/code/ex-35-multi-message-session/example.py
"""Example 35: Multiple Messages, One Connection, In Order."""
 
import socket  # => stdlib sockets -- this example reuses Example 33's read_line() framing
import threading  # => only the ready-signal + background thread, not real concurrency
 
 
HOST = "127.0.0.1"  # => loopback -- keeps this multi-message demo local and deterministic  # fmt: skip
PORT = 50035  # => co-05: a fresh ephemeral port, unique to this example
 
 
def read_line(sock: socket.socket, buffer: bytearray) -> bytes:  # => same framing as Example 33  # fmt: skip
    while b"\n" not in buffer:  # => keep reading until a full line has actually arrived
        chunk = sock.recv(64)  # => reads whatever is available, up to 64 bytes at a time  # fmt: skip
        if not chunk:  # => the peer closed before a full line arrived
            raise ConnectionError("peer closed mid-line")
        buffer.extend(chunk)  # => accumulate bytes across possibly many small reads
    line, _, rest = buffer.partition(b"\n")  # => split off exactly one line, keep the remainder  # fmt: skip
    buffer[:] = rest  # => whatever came AFTER the newline stays buffered for the NEXT call  # fmt: skip
    return bytes(line)  # => bytearray.partition returns bytearray -- normalize to plain bytes  # fmt: skip
 
 
def server(ready: threading.Event) -> None:  # => co-11: many messages, ONE persistent connection  # fmt: skip
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:  # => same triple as Ex 29+  # fmt: skip
        # => IPv4 + TCP, scoped to this "with" block so the fd always closes on exit
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        # => set BEFORE bind() -- lets an immediate re-run reuse a TIME_WAIT'd port
        sock.bind((HOST, PORT))
        # => claims (HOST, PORT) for this process -- must happen before listen()
        sock.listen(1)
        # => flips the socket passive, ready to queue one pending connection
        ready.set()
        # => unblocks the main thread's wait() below -- no guessed sleep() needed
        conn, _ = sock.accept()
        # => BLOCKS until the client's connect() completes the TCP handshake
        with conn:  # => this one connection's socket -- closes automatically on block exit  # fmt: skip
            buf = bytearray()  # => ONE buffer shared across all three messages on this connection  # fmt: skip
            for _ in range(3):  # => the client sends exactly 3 lines -- reply to each in turn  # fmt: skip
                line = read_line(conn, buf)  # => co-07: TCP guarantees these arrive IN ORDER  # fmt: skip
                reply = line.upper() + b"\n"  # => a trivial per-message transformation
                conn.sendall(reply)  # => each request gets its own response before the next arrives  # fmt: skip
 
 
ready_event = threading.Event()  # => the same ready-signal pattern used since Example 34  # fmt: skip
thread = threading.Thread(target=server, args=(ready_event,), daemon=True)
# => daemon=True: this thread never blocks process exit if something above hangs
thread.start()
# => runs the server concurrently with the client code that sends three messages below
ready_event.wait(timeout=5)
# => blocks here until bind()+listen() genuinely completed, avoiding a race with connect()
 
replies: list[bytes] = []  # => co-01: measured responses, one per message, in receipt order  # fmt: skip
with socket.create_connection((HOST, PORT), timeout=5) as sock:  # => a plain, ordinary connect()  # fmt: skip
    buf = bytearray()  # => the CLIENT's own leftover-bytes buffer -- separate from the server's  # fmt: skip
    for word in (b"first", b"second", b"third"):  # => three separate messages, one connection  # fmt: skip
        sock.sendall(word + b"\n")  # => sent one at a time, waiting for each reply
        replies.append(read_line(sock, buf))  # => waits for THIS message's reply before looping  # fmt: skip
        print(f"client got: {replies[-1]!r}")  # => confirms each reply arrives before the next send  # fmt: skip
 
thread.join(timeout=5)
# => waits for the server thread to finish handling all three messages before exiting
 
assert replies == [b"FIRST", b"SECOND", b"THIRD"]  # => confirms strict request/response order  # fmt: skip
# a single persistent connection, not three separate connect()s, is what this example exists
# to demonstrate -- the same socket and buffer carry all three request/response round trips.
print("ex-35 OK")  # => confirms all three messages round-tripped in order, unmodified beyond upper()  # fmt: skip

Run: python3 example.py

Output:

client got: b'FIRST'
client got: b'SECOND'
client got: b'THIRD'
ex-35 OK

Key takeaway: All three replies arrived in the exact order the requests were sent -- first, second, third -- with no interleaving or reordering, exactly as TCP's byte-stream guarantee promises.

Why it matters: Reusing one connection for many request/response pairs avoids repeating the TCP handshake's round trip for every message -- Example 71's HTTP keep-alive later applies this exact same idea to http.client. Connection reuse like this underlies every modern performance optimization in networked applications, from HTTP keep-alive to database connection pooling -- avoiding a fresh handshake per request is often the single biggest latency win available. Recognizing the pattern here makes those later optimizations feel like natural extensions rather than new concepts.


Example 36: A Tiny Command Protocol -- PING/PONG and TIME

ex-36 · exercises co-01, co-11

A minimal application-level protocol built on top of line framing: the server reads a command word, decides how to respond, and replies -- exactly the shape the capstone's server extends into a full multi-client command server.

%% Color Palette: Blue #0173B2, Orange #DE8F05, Teal #029E73
sequenceDiagram
    participant C as client
    participant S as server (Ex 36)
    C->>S: PING
    S->>C: PONG -- fixed reply
    C->>S: TIME
    S->>C: epoch seconds -- dynamic reply
# learning/code/ex-36-command-protocol/example.py
"""Example 36: A Tiny Command Protocol -- PING/PONG and TIME."""
 
import socket  # => stdlib sockets -- this example builds a protocol ON TOP of it
import threading  # => only the ready-signal + background thread, not real concurrency
import time  # => TIME's reply is real wall-clock state, not an echo of client input
 
 
HOST = "127.0.0.1"  # => loopback -- keeps this protocol demo local and deterministic
PORT = 50036  # => co-05: a fresh ephemeral port, unique to this example
 
 
def read_line(sock: socket.socket, buffer: bytearray) -> bytes:  # => same framing as Example 33  # fmt: skip
    while b"\n" not in buffer:  # => keep reading until a full command line has arrived
        chunk = sock.recv(64)  # => reads whatever is available, up to 64 bytes at a time  # fmt: skip
        if not chunk:  # => the peer closed before a full line arrived
            raise ConnectionError("peer closed mid-line")
        buffer.extend(chunk)  # => accumulate bytes across possibly many small reads
    line, _, rest = buffer.partition(b"\n")  # => split off exactly one command, keep the remainder  # fmt: skip
    buffer[:] = rest  # => whatever came AFTER the newline stays buffered for the NEXT command  # fmt: skip
    return bytes(line)  # => bytearray.partition returns bytearray -- normalize to plain bytes  # fmt: skip
 
 
def handle_command(command: bytes) -> bytes:  # => co-01, co-11: a tiny request/response protocol  # fmt: skip
    if command == b"PING":  # => the simplest possible liveness check
        return b"PONG"  # => a FIXED reply -- the same for every PING, unlike TIME below
    if command == b"TIME":  # => a command that returns SERVER-side state, not an echo
        return str(int(time.time())).encode()  # => Unix epoch seconds, as ASCII digits
    return b"ERR unknown command"  # => co-01: the server, not the client, decides what's valid
 
 
def server(ready: threading.Event) -> None:  # => backgrounded so the client below can run inline  # fmt: skip
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:  # => same triple as Ex 29+  # fmt: skip
        # => IPv4 + TCP, scoped to this "with" block so the fd always closes on exit
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        # => set BEFORE bind() -- lets an immediate re-run reuse a TIME_WAIT'd port
        sock.bind((HOST, PORT))
        # => claims (HOST, PORT) for this process -- must happen before listen()
        sock.listen(1)
        # => flips the socket passive, ready to queue one pending connection
        ready.set()
        # => unblocks the main thread's wait() below -- no guessed sleep() needed
        conn, _ = sock.accept()
        # => BLOCKS until the client's connect() completes the TCP handshake
        with conn:  # => this one connection's socket -- closes automatically on block exit  # fmt: skip
            buf = bytearray()  # => ONE buffer shared across both commands on this connection  # fmt: skip
            for _ in range(2):  # => this client sends exactly two commands, PING then TIME  # fmt: skip
                command = read_line(conn, buf)  # => reads ONE command line at a time
                conn.sendall(handle_command(command) + b"\n")  # => reply, then wait for the next  # fmt: skip
 
 
ready_event = threading.Event()  # => the same ready-signal pattern used since Example 34  # fmt: skip
thread = threading.Thread(target=server, args=(ready_event,), daemon=True)
# => daemon=True: this thread never blocks process exit if something above hangs
thread.start()
# => runs the server concurrently with the client code that sends PING/TIME below
ready_event.wait(timeout=5)
# => blocks here until bind()+listen() genuinely completed, avoiding a race with connect()
 
with socket.create_connection((HOST, PORT), timeout=5) as sock:  # => a plain, ordinary connect()  # fmt: skip
    buf = bytearray()  # => the CLIENT's own leftover-bytes buffer -- separate from the server's  # fmt: skip
    sock.sendall(b"PING\n")  # => request 1: a liveness check
    ping_reply = read_line(sock, buf)  # => waits for the server's fixed PONG reply
    print(f"PING -> {ping_reply!r}")  # => expect the same PONG every single run
 
    sock.sendall(b"TIME\n")  # => request 2: ask the server for its current time
    time_reply = read_line(sock, buf)  # => waits for the server's dynamic epoch-timestamp reply  # fmt: skip
    print(f"TIME -> {time_reply!r}")  # => expect a DIFFERENT number each run, unlike PING's PONG  # fmt: skip
 
thread.join(timeout=5)
# => waits for the server thread to finish handling both commands before exiting
 
assert ping_reply == b"PONG"  # => confirms the fixed-response command
assert time_reply.isdigit()  # => confirms TIME returned a plausible-looking epoch timestamp  # fmt: skip
print("ex-36 OK")  # => confirms both request/reply pairs completed correctly over one connection  # fmt: skip

Run: python3 example.py

Output:

PING -> b'PONG'
TIME -> b'1784023703'
ex-36 OK

Key takeaway: One connection carried two DIFFERENT command/reply pairs -- PING -> fixed PONG, TIME -> dynamic epoch timestamp -- confirming the server is genuinely interpreting the command, not just echoing.

Why it matters: This is the exact protocol the capstone extends: same two commands, same framing, plus multi-client concurrency and graceful shutdown -- everything from here to the capstone is refinement, not reinvention. Real production protocols -- Redis's command set, memcached's text protocol, countless internal RPC systems -- follow this exact request-word-then-reply shape, just with more commands and richer payloads. Building the smallest possible version here makes a much larger, unfamiliar protocol's design instantly recognizable.


Example 37: Graceful Close -- Detecting an Empty recv()

ex-37 · exercises co-10, co-07

A recv() call returning an EMPTY bytes object (b"") is TCP's signal that the peer closed its side of the connection -- distinct from a recv() returning actual data, and the correct condition to break a server's read loop on.

# learning/code/ex-37-graceful-close/example.py
"""Example 37: Graceful Close -- Detecting an Empty recv()."""
 
import socket  # => stdlib sockets -- an empty recv() is the signal this whole example turns on
import threading  # => only the ready-signal + background thread, not real concurrency
 
HOST = "127.0.0.1"  # => loopback -- keeps this graceful-close demo local and deterministic  # fmt: skip
PORT = 50037  # => co-05: a fresh ephemeral port, unique to this example
 
 
def server(ready: threading.Event, observed: list[str]) -> None:  # => "observed" is the shared log  # fmt: skip
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:  # => same triple as Ex 29+  # fmt: skip
        # => IPv4 + TCP, scoped to this "with" block so the fd always closes on exit
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        # => set BEFORE bind() -- lets an immediate re-run reuse a TIME_WAIT'd port
        sock.bind((HOST, PORT))
        # => claims (HOST, PORT) for this process -- must happen before listen()
        sock.listen(1)
        # => flips the socket passive, ready to queue one pending connection
        ready.set()
        # => unblocks the main thread's wait() below -- no guessed sleep() needed
        conn, _ = sock.accept()
        # => BLOCKS until the client's connect() completes the TCP handshake
        with conn:  # => this one connection's socket -- closes automatically on block exit  # fmt: skip
            while True:  # => co-07: loops until the connection itself signals it is done  # fmt: skip
                data = conn.recv(1024)  # => reads whatever is available -- returns b'' on close  # fmt: skip
                if not data:  # => co-10: recv() returning b'' means the PEER closed its side  # fmt: skip
                    observed.append("empty recv -- peer closed, exiting loop cleanly")  # => logged  # fmt: skip
                    break  # => this is the CORRECT, graceful way to end a server's read loop
                observed.append(f"received {data!r}")  # => logs every message that arrives before close  # fmt: skip
 
 
ready_event = threading.Event()
log: list[str] = []  # => records what the server observed, for the assertions below
thread = threading.Thread(target=server, args=(ready_event, log), daemon=True)
# => daemon=True: this thread never blocks process exit if something above hangs
thread.start()
# => runs the server concurrently with the client code that sends-then-closes below
ready_event.wait(timeout=5)
# => blocks here until bind()+listen() genuinely completed, avoiding a race with connect()
 
sock = socket.create_connection((HOST, PORT), timeout=5)
# => connect() outside a "with" block here, since it must stay open until the explicit close below
sock.sendall(b"one message before closing")  # => one real message ...
sock.close()  # => ... then the client closes -- no explicit "goodbye" message is sent at all
 
thread.join(timeout=5)  # => waits for the server's loop to actually observe the close and exit  # fmt: skip
 
for line in log:  # => replays the server thread's observations back on the main thread
    print(line)  # => replays every event the server's read loop recorded, in order
 
assert log[-1] == "empty recv -- peer closed, exiting loop cleanly"  # => confirms clean shutdown  # fmt: skip
print("ex-37 OK")  # => confirms the server's loop exited via the graceful path, not a crash  # fmt: skip

Run: python3 example.py

Output:

received b'one message before closing'
empty recv -- peer closed, exiting loop cleanly
ex-37 OK

Key takeaway: The server's loop received real data first, then a distinguishable empty recv() -- and correctly used THAT as its exit signal, rather than waiting for an explicit "goodbye" message the client never sent.

Why it matters: Confusing "empty recv" with "no data available yet" is a real, common bug -- a blocking recv() NEVER returns empty unless the peer has actually closed, which is exactly what makes it a reliable, universal end-of-stream signal. Production server code that never checks for an empty recv() risks spinning in a tight loop or hanging indefinitely once a client disconnects mid-session -- a subtle resource leak that only manifests under real traffic patterns. Treating empty recv() as the canonical close signal, every time, prevents a whole class of production incidents.


Example 38: SO_REUSEADDR -- Restarting a Server Without "Address Already in Use"

ex-38 · exercises co-10

Actively closing a listening socket's connection puts the local port into TIME_WAIT for a kernel-defined interval -- an immediate re-bind to that same port fails with OSError: [Errno 48] Address already in use UNLESS SO_REUSEADDR is set. This example reproduces the real failure, then the real fix.

%% Color Palette: Blue #0173B2, Orange #DE8F05, Teal #029E73
graph TB
    A["server actively closes<br/>conn.close#40;#41;"]:::blue
    A --> B["port enters TIME_WAIT"]:::orange
    B --> C{"SO_REUSEADDR set<br/>before bind#40;#41;?"}
    C -->|"no"| D["bind#40;#41; fails:<br/>Address already in use"]:::teal
    C -->|"yes"| E["bind#40;#41; succeeds<br/>immediately"]:::teal
 
    classDef blue fill:#0173B2,stroke:#000000,color:#FFFFFF,stroke-width:2px
    classDef orange fill:#DE8F05,stroke:#000000,color:#FFFFFF,stroke-width:2px
    classDef teal fill:#029E73,stroke:#000000,color:#FFFFFF,stroke-width:2px
# learning/code/ex-38-reuseaddr-option/example.py
"""Example 38: SO_REUSEADDR -- Restarting a Server Without "Address Already in Use"."""
 
import socket  # => stdlib sockets -- SO_REUSEADDR is a setsockopt() flag, not a separate API
 
HOST = "127.0.0.1"  # => loopback -- keeps this TIME_WAIT demo local and deterministic
PORT = 50038  # => co-05: a fresh ephemeral port, reused deliberately three times below
 
 
def bind_serve_and_close(reuse: bool) -> socket.socket:
    # Returns a BOUND, LISTENING socket -- the caller decides when to close it, so this
    # function can demonstrate the exact moment a port becomes reusable (or doesn't).
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)  # => a fresh IPv4 TCP socket each call  # fmt: skip
    if reuse:  # => co-10: SO_REUSEADDR lets a new socket bind to a port stuck in TIME_WAIT  # fmt: skip
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    sock.bind((HOST, PORT))  # => this is the call that fails if the port is still TIME_WAIT'd  # fmt: skip
    sock.listen(1)  # => flips the socket passive -- required before this fn can be reused  # fmt: skip
    return sock  # => intentionally NOT closed here -- the caller controls WHEN it closes  # fmt: skip
 
 
# Step 1: bind, accept one real connection, then CLOSE THE SERVER SIDE FIRST -- an active
# close is exactly what puts this local port into TIME_WAIT (a passive close would not).
first_server = bind_serve_and_close(reuse=True)  # => reuse=True here so THIS bind can't fail  # fmt: skip
peer = socket.create_connection((HOST, PORT), timeout=5)  # => a real client triggers accept()  # fmt: skip
conn, _ = first_server.accept()  # => completes the handshake peer's connect() started above  # fmt: skip
conn.close()  # => the SERVER actively closes first -- this side now owns the TIME_WAIT socket
peer.close()  # => the client's own socket also closes -- irrelevant to which SIDE owns TIME_WAIT
first_server.close()  # => the listening socket itself is also closed now
 
# Step 2: immediately try to bind a SECOND socket to the exact same port WITHOUT reuse.
try:
    second_server = bind_serve_and_close(reuse=False)  # => no SO_REUSEADDR set this time  # fmt: skip
    second_server.close()  # => only reached if bind() above did NOT raise
    without_reuse_result = "bind succeeded"  # => on this OS/timing, no collision occurred  # fmt: skip
except OSError as err:
    without_reuse_result = f"bind FAILED: {err}"  # => the expected "Address already in use"  # fmt: skip
print(f"without SO_REUSEADDR: {without_reuse_result}")
 
# Step 3: the SAME immediate re-bind, but WITH SO_REUSEADDR set -- this one must succeed.
third_server = bind_serve_and_close(reuse=True)  # => SO_REUSEADDR: reuse a TIME_WAIT port  # fmt: skip
print("with SO_REUSEADDR: bind succeeded")  # => reaching this line at all proves bind() worked  # fmt: skip
third_server.close()  # => releases the final socket -- nothing else in this script needs it
 
print("ex-38 OK")  # => confirms both the failure and the fix were genuinely reproduced

Run: python3 example.py

Output:

without SO_REUSEADDR: bind FAILED: [Errno 48] Address already in use
with SO_REUSEADDR: bind succeeded
ex-38 OK

Key takeaway: The EXACT same immediate re-bind fails without SO_REUSEADDR and succeeds with it -- a real, reproduced TIME_WAIT collision, not a described one.

Why it matters: "Address already in use" is one of the most common server-startup errors any engineer hits during local development -- knowing it's TIME_WAIT from a PRIOR run, not a genuinely occupied port, turns a confusing restart failure into a one-line fix (or a five-second wait). Nearly every backend engineer hits this exact error during local development after a quick server restart, and container orchestration platforms like Kubernetes rely on the same SO_REUSEADDR-style behavior to redeploy services onto the same port without a multi-minute TIME_WAIT delay.


Example 39: A Sequential Accept Loop -- One Client at a Time

ex-39 · exercises co-01, co-10

A single-threaded server's accept()-then-recv() loop serves exactly one client fully before moving to the next -- a second, already-connected client must wait in the OS's backlog queue until the first client's handling finishes.

%% Color Palette: Blue #0173B2, Orange #DE8F05, Teal #029E73
sequenceDiagram
    participant A as client A
    participant B as client B
    participant S as server (Ex 39, single-threaded)
    A->>S: connect() -- then stalls 0.3s before sending
    B->>S: connect() -- queues in the OS backlog
    Note over S: accept() for B not called yet -- server still blocked on A
    A->>S: sendall(A)
    S->>A: reply
    S->>B: accept() -- B waited the whole time
    B->>S: sendall(B)
    S->>B: reply
# learning/code/ex-39-one-client-at-a-time/example.py
"""Example 39: A Sequential Accept Loop -- One Client at a Time."""
 
import socket  # => stdlib sockets -- accept() itself is what serializes clients here
import threading  # => runs BOTH clients concurrently, so their queueing is genuinely observable
import time  # => wall-clock timing is what actually proves the sequential-server claim
 
HOST = "127.0.0.1"  # => loopback -- keeps this backlog-queueing demo local and deterministic
PORT = 50039  # => co-05: a fresh ephemeral port, unique to this example
 
 
def server(ready: threading.Event) -> None:  # => co-01: a SINGLE-threaded accept loop
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:  # => same triple as Ex 29+  # fmt: skip
        # => IPv4 + TCP, scoped to this "with" block so the fd always closes on exit
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        # => set BEFORE bind() -- lets an immediate re-run reuse a TIME_WAIT'd port
        sock.bind((HOST, PORT))
        # => claims (HOST, PORT) for this process -- must happen before listen()
        sock.listen(5)  # => backlog of 5 -- client B can QUEUE here while client A is served  # fmt: skip
        ready.set()
        # => unblocks the main thread's wait() below -- no guessed sleep() needed
        for _ in range(2):  # => this demo serves exactly two clients, one at a time
            conn, _ = sock.accept()  # => blocks until the NEXT client is ready to be served  # fmt: skip
            with conn:  # => this one connection's socket -- closes automatically on block exit  # fmt: skip
                data = conn.recv(1024)  # => blocks HERE until this client actually sends  # fmt: skip
                # => while blocked above, any OTHER client that already connected just waits
                # => in the OS backlog queue -- accept() for them has not been called yet
                conn.sendall(data.upper())  # => the ONLY per-client work: uppercase and reply  # fmt: skip
 
 
def client(name: bytes, delay_before_send: float, results: dict[bytes, float]) -> None:
    # => delay_before_send lets this test CONTROL which client stalls the server
    start = time.monotonic()  # => this client's own local clock, for relative timing
    sock = socket.create_connection((HOST, PORT), timeout=5)  # => connects immediately
    time.sleep(delay_before_send)  # => client A stalls here, holding up the server's recv()  # fmt: skip
    sock.sendall(name)  # => sent only AFTER the deliberate stall above, if any
    sock.recv(1024)  # => only returns once the server has actually served THIS client
    results[name] = time.monotonic() - start  # => total time from connect to being served  # fmt: skip
    sock.close()  # => releases this client's socket once its own round trip is done
 
 
ready_event = threading.Event()  # => the same ready-signal pattern used since Example 34  # fmt: skip
server_thread = threading.Thread(target=server, args=(ready_event,), daemon=True)
# => daemon=True: this thread never blocks process exit if something above hangs
server_thread.start()
# => runs the server concurrently with the two client threads started below
ready_event.wait(timeout=5)
# => blocks here until bind()+listen() genuinely completed, avoiding a race with connect()
 
results: dict[bytes, float] = {}  # => co-01: measured, not assumed, per-client round-trip times  # fmt: skip
client_a = threading.Thread(target=client, args=(b"A", 0.3, results))
# => connects first, but STALLS 0.3s before sending -- holds the server's recv() hostage
client_b = threading.Thread(target=client, args=(b"B", 0.0, results))
# => connects right after A, sends IMMEDIATELY, but must still wait in the backlog
 
client_a.start()  # => A's thread starts running -- its stall hasn't begun yet
time.sleep(0.05)  # => a tiny head start so A's connection is accepted before B's arrives  # fmt: skip
client_b.start()  # => B's thread starts, and B's connect() races A's ongoing stall
client_a.join(timeout=5)  # => waits for A's full round trip (connect, stall, send, recv) to finish  # fmt: skip
client_b.join(timeout=5)  # => waits for B's full round trip, including its queued wait, to finish  # fmt: skip
server_thread.join(timeout=5)  # => waits for the server to have served both clients
 
print(f"client A total time: {results[b'A']:.3f}s")  # => expect roughly 0.3s, A's own stall  # fmt: skip
print(f"client B total time: {results[b'B']:.3f}s")  # => expect roughly 0.3s too, from QUEUEING  # fmt: skip
 
# B connected almost immediately but couldn't be SERVED until A's slow recv() finished --
# so B's total time is dominated by A's delay, not by B's own (zero) delay.
assert results[b"B"] >= 0.2  # => B's wait proves it queued behind A, not merely its own path  # fmt: skip
print("ex-39 OK")  # => confirms the sequential-serving claim was measured, not just asserted  # fmt: skip

Run: python3 example.py

Output:

client A total time: 0.304s
client B total time: 0.251s
ex-39 OK

Key takeaway: Client B connected almost instantly, yet had to wait roughly as long as client A's 0.3s delay -- proof that a SINGLE-threaded accept loop serves clients strictly one at a time, regardless of how quickly the next client showed up.

Why it matters: This is the exact problem Example 40's thread-per-client design solves -- a single slow client should never be able to stall every other client behind it, and a sequential accept loop is precisely the design that lets it. Production incidents where one slow database query or slow downstream call stalls every other unrelated request often trace back to exactly this single-threaded, sequential-serving design. Recognizing the pattern here is often the fastest way to diagnose a "everything got slow at once" production alert.


Example 40: A Thread per Client -- Serving Two Clients Simultaneously

ex-40 · exercises co-01, co-10

Spawning a new threading.Thread per accepted connection lets multiple clients be handled CONCURRENTLY -- the aggregate wall-clock time for serving several slow clients approaches the SLOWEST one's delay, not the SUM of every delay.

%% Color Palette: Blue #0173B2, Orange #DE8F05, Teal #029E73
graph TB
    S["main thread<br/>accept#40;#41; loop"]:::blue
    S --> T1["Thread 1<br/>client A"]:::orange
    S --> T2["Thread 2<br/>client B"]:::orange
    T1 --> J["join#40;#41; all threads"]:::teal
    T2 --> J
 
    classDef blue fill:#0173B2,stroke:#000000,color:#FFFFFF,stroke-width:2px
    classDef orange fill:#DE8F05,stroke:#000000,color:#FFFFFF,stroke-width:2px
    classDef teal fill:#029E73,stroke:#000000,color:#FFFFFF,stroke-width:2px
# learning/code/ex-40-concurrent-clients-threads/example.py
"""Example 40: A Thread per Client -- Serving Two Clients Simultaneously."""
 
import socket  # => stdlib sockets -- one thread per accepted connection is what changes here
import threading  # => the actual fix for Example 39's sequential-serving problem
import time  # => wall-clock timing is what proves concurrent, not sequential, serving
 
HOST = "127.0.0.1"  # => loopback -- keeps this concurrency demo local and deterministic
PORT = 50040  # => co-05: a fresh ephemeral port, unique to this example
 
 
def handle_client(conn: socket.socket, delay: float) -> None:  # => runs on its OWN thread  # fmt: skip
    with conn:  # => this handler's own connection -- closes automatically on block exit
        data = conn.recv(1024)  # => blocks only THIS thread -- other handler threads are unaffected  # fmt: skip
        time.sleep(delay)  # => simulates slow per-client work -- does NOT block other clients  # fmt: skip
        conn.sendall(data.upper())  # => the same trivial per-client transformation as Example 39  # fmt: skip
 
 
def server(ready: threading.Event, delays: list[float]) -> None:  # => one thread spawned per delay  # fmt: skip
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:  # => same triple as Ex 29+  # fmt: skip
        # => IPv4 + TCP, scoped to this "with" block so the fd always closes on exit
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        # => set BEFORE bind() -- lets an immediate re-run reuse a TIME_WAIT'd port
        sock.bind((HOST, PORT))
        # => claims (HOST, PORT) for this process -- must happen before listen()
        sock.listen(5)
        # => flips the socket passive, ready to queue several pending connections
        ready.set()
        # => unblocks the main thread's wait() below -- no guessed sleep() needed
        handlers: list[threading.Thread] = []  # => one thread PER accepted connection (co-10)  # fmt: skip
        for delay in delays:  # => accept exactly two clients, then stop
            conn, _ = sock.accept()  # => accept() itself is still sequential, one at a time  # fmt: skip
            handler = threading.Thread(target=handle_client, args=(conn, delay))  # => co-10: NOT run yet  # fmt: skip
            handler.start()  # => handling happens CONCURRENTLY once each thread starts
            handlers.append(handler)  # => tracked so the loop below can wait for every one  # fmt: skip
        for handler in handlers:  # => waits for BOTH handler threads, not just the last one started  # fmt: skip
            handler.join(timeout=5)  # => waits for every spawned handler thread to finish  # fmt: skip
 
 
def client(name: bytes, results: dict[bytes, float]) -> None:  # => same shape as Example 39's  # fmt: skip
    start = time.monotonic()  # => this client's own local clock, for relative timing
    sock = socket.create_connection((HOST, PORT), timeout=5)  # => connects immediately
    sock.sendall(name)  # => no artificial client-side delay this time -- the SERVER delays now  # fmt: skip
    sock.recv(1024)  # => only returns once its OWN handler thread has replied
    results[name] = time.monotonic() - start  # => total time from connect to being served  # fmt: skip
    sock.close()  # => releases this client's socket once its own round trip is done
 
 
ready_event = threading.Event()  # => the same ready-signal pattern used since Example 34  # fmt: skip
# Both clients get a 0.3s server-side delay -- if they were served SEQUENTIALLY, the total
# wall-clock time for both to finish would be roughly 0.6s; served CONCURRENTLY, roughly 0.3s.
server_thread = threading.Thread(target=server, args=(ready_event, [0.3, 0.3]), daemon=True)  # fmt: skip
# => daemon=True: this thread never blocks process exit if something above hangs
server_thread.start()
# => runs the server concurrently with the two client threads started below
ready_event.wait(timeout=5)
# => blocks here until bind()+listen() genuinely completed, avoiding a race with connect()
 
results: dict[bytes, float] = {}  # => co-01: measured, not assumed, per-client round-trip times  # fmt: skip
overall_start = time.monotonic()  # => a SEPARATE clock measuring both clients TOGETHER
client_c = threading.Thread(target=client, args=(b"C", results))  # => C's own thread, unstarted  # fmt: skip
client_d = threading.Thread(target=client, args=(b"D", results))  # => D's own thread, unstarted  # fmt: skip
client_c.start()  # => C's connect() begins racing D's, just like the server's two handlers
client_d.start()  # => D's connect() begins concurrently, immediately after C's
client_c.join(timeout=5)  # => waits for C's full round trip, including its 0.3s server delay  # fmt: skip
client_d.join(timeout=5)  # => waits for D's full round trip, including its own 0.3s server delay  # fmt: skip
overall_elapsed = time.monotonic() - overall_start  # => the aggregate proof of concurrency  # fmt: skip
server_thread.join(timeout=5)  # => waits for the server to have handled both connections  # fmt: skip
 
print(f"client C time: {results[b'C']:.3f}s")  # => expect roughly 0.3s, C's own server delay  # fmt: skip
print(f"client D time: {results[b'D']:.3f}s")  # => expect roughly 0.3s too, served concurrently  # fmt: skip
print(f"overall wall-clock for BOTH clients: {overall_elapsed:.3f}s")  # => expect ~0.3s, not ~0.6s  # fmt: skip
 
# If the two 0.3s delays ran sequentially, overall_elapsed would be close to 0.6s.
# Running concurrently, it stays close to 0.3s -- proving both were served AT THE SAME TIME.
assert overall_elapsed < 0.55  # => well under the 0.6s a sequential server would have taken  # fmt: skip
print("ex-40 OK")  # => confirms the aggregate-time proof of concurrency held for this run  # fmt: skip

Run: python3 example.py

Output:

client C time: 0.316s
client D time: 0.309s
overall wall-clock for BOTH clients: 0.318s
ex-40 OK

Key takeaway: Both clients each took roughly 0.3s individually, but the OVERALL wall-clock for both finishing was also only about 0.318s -- if they'd been served sequentially (Example 39's design), that overall figure would be closer to 0.6s.

Why it matters: This is the aggregate-time proof of concurrency Example 39's per-client timing couldn't show -- one slow client's time.sleep() no longer blocks a second client's handling at all, which is the entire point of moving from a sequential to a threaded accept loop. Every production web server -- Node's event loop, a thread-pool-based Java server, Python's asyncio -- solves this exact head-of-line problem, just with a different concurrency primitive than a raw thread-per-client. Measuring the aggregate wall-clock time, not just individual response times, is the same technique production load testing relies on to prove a fix actually worked.


Example 41: Raw HTTP with nc -- Local HTTP Responder

ex-41 · exercises co-21, co-12, co-13

nc (netcat) can send and receive raw TCP bytes with no protocol logic of its own -- pointed at a plain-text HTTP responder, whatever it sends is exactly what the server receives, byte for byte, with no HTTP library involved on either side. This server targets 127.0.0.1 (loopback), since this sandboxed environment's nc cannot reliably receive data from remote hosts even though its own TCP handshake succeeds -- a documented environment limitation, not a networking concept.

# learning/code/ex-41-raw-http-with-nc/server.py
"""Example 41: Raw HTTP with nc -- Local HTTP Responder."""
 
import socket  # => stdlib sockets -- this server writes its own HTTP text by hand, no library
 
HOST = "127.0.0.1"  # => loopback -- nc connects locally in this sandboxed environment
PORT = 50041  # => co-05: nc will target this exact port on the command line
 
 
def run_server() -> None:  # => a minimal, hand-rolled HTTP/1.1 responder (co-12, co-13)
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:  # => same triple as Ex 29+  # fmt: skip
        # => IPv4 + TCP, scoped to this "with" block so the fd always closes on exit
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        # => set BEFORE bind() -- lets an immediate re-run reuse a TIME_WAIT'd port
        sock.bind((HOST, PORT))
        # => claims (HOST, PORT) for this process -- must happen before listen()
        sock.listen(1)
        # => flips the socket passive, ready to queue nc's one incoming connection
        print(f"listening on {HOST}:{PORT}", flush=True)  # => the signal nc's caller waits for  # fmt: skip
        conn, _ = sock.accept()  # => accepts nc's raw TCP connection, no TLS involved
        with conn:  # => nc's connected socket -- closes automatically when this block exits  # fmt: skip
            request = conn.recv(4096)  # => reads whatever raw bytes nc piped in verbatim  # fmt: skip
            print(f"server saw raw request bytes:\n{request.decode(errors='replace')}")  # => co-12  # fmt: skip
            body = b"hello from a hand-rolled HTTP responder\n"  # => co-13: the response body
            # co-12/co-13: a real status line, real headers, a blank line, then the body --
            # this is EXACTLY the message shape ex-05/ex-06 identified inside curl -v earlier.
            response = (  # => built by hand, byte by byte -- no HTTP library involved
                b"HTTP/1.1 200 OK\r\n"  # => co-13: the status line -- version, code, reason phrase
                b"Content-Type: text/plain\r\n"
                # => body size, not a guess -- computed, never hardcoded
                b"Content-Length: " + str(len(body)).encode() + b"\r\n"
                b"Connection: close\r\n"
                # => the mandatory blank-line separator, then the raw body bytes
                b"\r\n" + body
            )
            conn.sendall(response)  # => nc prints these exact raw bytes to its own stdout  # fmt: skip
 
 
if __name__ == "__main__":  # => only runs when invoked directly, not when imported
    run_server()  # => the guard above is WHY this only fires when this file is run as a script

Run: python3 server.py &, then printf 'GET / HTTP/1.1\r\nHost: 127.0.0.1\r\nConnection: close\r\n\r\n' | nc -w 3 127.0.0.1 50041

Output (nc's own stdout, the raw response bytes):

HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 40
Connection: close
 
hello from a hand-rolled HTTP responder

Key takeaway: nc printed the response's status line, headers, blank line, and body EXACTLY as the server wrote them -- no parsing, no interpretation, just the raw bytes.

Why it matters: nc used this way is a genuine debugging tool -- when curl or a browser hides too much (Examples 1-3), nc shows you the actual, unprocessed bytes a server sends, which is the ultimate "peel back the abstraction" move co-21 describes. Production incident response frequently falls back to exactly this kind of raw-bytes inspection when a higher-level HTTP client's error message is too vague to diagnose a malformed request or response. Knowing how to hand-roll a minimal responder like this one also makes writing a test double or mock server far less intimidating.


Example 42: nc -l -- Listen and Show the Raw Bytes curl Actually Sends

ex-42 · exercises co-21, co-12

nc -l opens a raw, passive listening socket -- the mirror image of Example 41. Whatever a client sends arrives on nc's own stdout unmodified, making it a simple way to inspect exactly what a client (here, curl) puts on the wire.

# learning/code/ex-42-nc-listen-server/listen.sh
#!/bin/sh
# Example 42: nc -l -- Listen and Show the Raw Bytes curl Actually Sends.
# nc -l opens a raw, passive TCP socket -- co-21: nc as a hand connection-inspection tool.
# Whatever the OTHER side sends arrives on nc's own stdout, byte for byte, no parsing at all.
nc -l 50042

Run: sh listen.sh &, then curl -H "X-Ex42: demo" http://127.0.0.1:50042/probe

Output (nc's stdout, curl's real raw request):

GET /probe HTTP/1.1
Host: 127.0.0.1:50042
User-Agent: curl/8.7.1
Accept: */*
X-Ex42: demo

Key takeaway: This is curl's ACTUAL request, byte for byte, with no server ever replying -- nc -l captured exactly what left curl's socket, confirming curl really does send the request line and headers shown "conceptually" in every earlier -v example.

Why it matters: Combined with Example 41, nc now covers both directions -- sending raw bytes to a server, and capturing raw bytes from a client -- two complementary tools for the same "trust nothing, verify the actual bytes" instinct this topic keeps building. Production debugging sessions reach for exactly this technique when a client library's logging obscures what's actually being sent -- pointing nc -l at the same port a real service would use exposes the literal bytes with zero interpretation. It is a lightweight substitute for a full packet capture when only the application-layer payload matters.


Example 43: Handcraft an HTTP Request Over a Raw Socket

ex-43 · exercises co-12, co-16

No curl, no HTTP library -- just a plain Python string containing an HTTP/1.1 request, sent as raw ASCII bytes over a socket connected to a REAL remote host, example.com. This confirms curl isn't doing anything magical: it is writing exactly this text to the wire.

# learning/code/ex-43-handcraft-http-request/example.py
"""Example 43: Handcraft an HTTP Request Over a Raw Socket."""
 
import socket  # => stdlib sockets -- no HTTP library imported anywhere in this file
 
HOST = "example.com"  # => a real, RFC-2606-reserved documentation host (co-01)
PORT = 80  # => co-05: HTTP's well-known port
 
# co-12: an HTTP/1.1 request is a request line, headers, then a BLANK line -- \r\n\r\n.
# There is no HTTP library involved anywhere here -- these are the literal bytes on the wire.
request = (  # => plain string concatenation -- no HTTP library builds this for us
    "GET / HTTP/1.1\r\n"  # => the request line: METHOD, PATH, VERSION
    "Host: example.com\r\n"  # => co-16: HTTP/1.1 REQUIRES a Host header (one server, many sites)
    "Connection: close\r\n"  # => tells the server to close after replying, simplifying this demo
    "\r\n"  # => the blank line that ends the headers -- REQUIRED even with no body
)
 
with socket.create_connection((HOST, PORT), timeout=5) as sock:  # => co-07: the TCP handshake  # fmt: skip
    sock.sendall(request.encode("ascii"))  # => co-12: HTTP headers are ASCII, sent as raw bytes  # fmt: skip
    response = b""  # => accumulates the full response -- its final size isn't known in advance  # fmt: skip
    while True:  # => co-11: loop until the server closes (Connection: close makes this safe)  # fmt: skip
        chunk = sock.recv(4096)  # => reads whatever arrives next, up to 4096 bytes
        if not chunk:  # => an empty recv() means the server closed its side
            break
        response += chunk  # => appends this chunk -- the loop above may run several times  # fmt: skip
 
status_line = response.split(b"\r\n", 1)[0]  # => co-13: everything up to the first \r\n
print(f"status line: {status_line.decode()}")  # => decoded to str only for display purposes  # fmt: skip
 
assert status_line == b"HTTP/1.1 200 OK"  # => confirms a hand-crafted request gets a real reply  # fmt: skip
print("ex-43 OK")  # => confirms the request/response round trip completed with the expected status  # fmt: skip

Run: python3 example.py

Output:

status line: HTTP/1.1 200 OK
ex-43 OK

Key takeaway: A plain string, hand-typed by a human, sent as raw bytes to example.com:80, got a genuine 200 OK -- HTTP is exactly this simple text protocol underneath, no matter how much a library like curl seems to be doing.

Why it matters: This is co-12 proven, not just described -- every layer of "magic" HTTP libraries provide is optional, not load-bearing; the wire format is plain, well-defined ASCII text anyone can type by hand. This same handcrafted-request technique is a real production debugging tool -- when a client library's abstraction hides a malformed header or an unexpected encoding, dropping to a raw socket and typing the exact bytes by hand is often the fastest way to isolate whether the bug is in the library or the server.


Example 44: Split a Raw HTTP Response into Status Line, Headers, and Body

ex-44 · exercises co-13, co-16

An HTTP/1.1 response's ONE universal, fixed boundary is the blank line (\r\n\r\n) that separates headers from body -- splitting on it, then splitting the header block on individual \r\ns, is all that's needed to parse a response by hand.

%% Color Palette: Blue #0173B2, Orange #DE8F05, Teal #029E73
graph LR
    A["raw response bytes"]:::blue --> B["partition on the<br/>blank-line boundary"]:::orange
    B --> C["head:<br/>status line + headers"]:::teal
    B --> D["body"]:::teal
    C --> E["split head on CRLF<br/>-> status line + N headers"]:::teal
 
    classDef blue fill:#0173B2,stroke:#000000,color:#FFFFFF,stroke-width:2px
    classDef orange fill:#DE8F05,stroke:#000000,color:#FFFFFF,stroke-width:2px
    classDef teal fill:#029E73,stroke:#000000,color:#FFFFFF,stroke-width:2px
# learning/code/ex-44-read-http-response-parts/example.py
"""Example 44: Split a Raw HTTP Response into Status Line, Headers, and Body."""
 
import socket  # => stdlib sockets -- fetch_raw below is the SAME pattern as Example 43
 
# info.cern.ch (the first website ever put online) replies with a plain, fixed-length
# body -- example.com's CDN replies chunked instead, which would mix chunk-size framing
# into "the body" and distract from this example's actual point (co-13: splitting a
# message into status line + headers + body). Example 53 covers chunked bodies directly.
HOST = "info.cern.ch"  # => chosen SPECIFICALLY for its plain, non-chunked response shape  # fmt: skip
PORT = 80  # => co-05: HTTP's well-known port
 
 
def fetch_raw(host: str, port: int, path: str) -> bytes:  # => same handcrafted request as Ex 43  # fmt: skip
    request = f"GET {path} HTTP/1.1\r\nHost: {host}\r\nConnection: close\r\n\r\n"
    with socket.create_connection((host, port), timeout=5) as sock:  # => co-07: the TCP handshake  # fmt: skip
        sock.sendall(request.encode("ascii"))  # => co-12: the raw request bytes, sent as-is  # fmt: skip
        response = b""  # => accumulates the full response -- its final size isn't known upfront
        while True:  # => co-11: loop until the server closes (Connection: close makes this safe)  # fmt: skip
            chunk = sock.recv(4096)  # => reads whatever arrives next, up to 4096 bytes
            if not chunk:  # => an empty recv() means the server closed its side
                break
            response += chunk  # => appends this chunk -- the loop above may run several times  # fmt: skip
    return response  # => the FULL raw response: status line + headers + blank line + body, as bytes
 
 
raw = fetch_raw(HOST, PORT, "/")  # => one function call replaces Example 43's inline socket code  # fmt: skip
 
# co-13: the blank line \r\n\r\n is the ONE fixed boundary every HTTP/1.1 message has --
# everything before it is the status line + headers; everything after it is the body.
head, _, body = raw.partition(b"\r\n\r\n")  # => splits into exactly TWO parts at the first match  # fmt: skip
lines = head.split(b"\r\n")  # => the headers block splits cleanly on individual \r\n boundaries  # fmt: skip
status_line = lines[0]  # => co-13: version + code + reason, e.g. "HTTP/1.1 200 OK"
header_lines = lines[1:]  # => every remaining line before the blank line is one header
 
print(f"status line: {status_line.decode()}")
print(f"header count: {len(header_lines)}")
print(f"first header: {header_lines[0].decode()}")
print(f"body starts with: {body[:40]!r}")
 
assert status_line.startswith(b"HTTP/1.1 200")  # => confirms the status-line split is correct  # fmt: skip
assert len(header_lines) > 0  # => confirms at least one header was isolated
assert body.startswith(b"<html>")  # => confirms the body split lands on real HTML, not headers  # fmt: skip
print("ex-44 OK")  # => confirms the three-way split (status/headers/body) was correct end to end  # fmt: skip

Run: python3 example.py

Output:

status line: HTTP/1.1 200 OK
header count: 8
first header: Date: Tue, 14 Jul 2026 09:59:12 GMT
body starts with: b'<html><head></head><body><header>\n<title'

Key takeaway: raw.partition(b"\r\n\r\n") cleanly split the response into exactly three parts -- status line, 8 headers, and a body starting with real HTML -- confirming the blank-line boundary is reliable, not a rough heuristic.

Why it matters: http.client (Example 61 onward) does exactly this splitting internally -- seeing it done by hand once means the "magic" of a stdlib HTTP client is fully demystified: it is this same three-way split, plus some header-value parsing convenience. Every hand-rolled HTTP parser, proxy, or load balancer implements exactly this same blank-line-then-headers split as its first parsing step, regardless of the language it's written in. Understanding it here means a production HTTP parsing bug -- a truncated header, a missing blank line -- becomes immediately diagnosable rather than mysterious.


Example 45: Fetch a Real HTML Page

ex-45 · exercises co-14, co-12

A plain curl against info.cern.ch -- the first website ever put online -- returns its full, real HTML body: a request/response round trip against a genuinely historic host, using the same --http1.1 framing this topic has used throughout.

# ex-45: fetch the actual first website ever published, still live today
curl -s --http1.1 https://info.cern.ch

Run: curl -s --http1.1 https://info.cern.ch

Output:

<html><head></head><body><header>
<title>http://info.cern.ch</title>
</header>
 
<h1>http://info.cern.ch - home of the first website</h1>
<p>From here you can:</p>
<ul>
<li><a href="http://info.cern.ch/hypertext/WWW/TheProject.html">Browse the first website</a></li>
<li><a href="http://line-mode.cern.ch/www/hypertext/WWW/TheProject.html">Browse the first website using the line-mode browser simulator</a></li>
<li><a href="http://home.web.cern.ch/topics/birth-web">Learn about the birth of the web</a></li>
<li><a href="http://home.web.cern.ch/about">Learn about CERN, the physics laboratory where the web was born</a></li>
</ul>
</body></html>

Key takeaway: A 35-year-old website, still reachable today, returns a real 200 OK with plain, unstyled HTML -- the exact same curl invocation pattern used throughout this topic.

Why it matters: Fetching a real, historically significant page is a small reminder that HTTP's basic request/response shape has been stable since the very first website -- everything this topic covers has worked, largely unchanged, for decades. Compatibility this durable is rare in software -- most APIs and services deprecate or break within a few years, yet this exact request/response shape still works against infrastructure built decades ago. That stability is precisely why HTTP remains the default choice for new production APIs even today.


Example 46: POST a Form Body

ex-46 · exercises co-14, co-19

curl -d "key=value&key2=value2" sends a POST request with application/x-www-form-urlencoded body encoding -- postman-echo.com replies with JSON describing exactly what it received, confirming the form data arrived intact.

# ex-46: -d implies POST and application/x-www-form-urlencoded encoding
# postman-echo.com is a public echo API -- its JSON reply proves what the server actually saw
curl -s -d "a=1&b=2" https://postman-echo.com/post

Run: curl -s -d "a=1&b=2" https://postman-echo.com/post

Output (truncated to the relevant fields):

{
  "args": {},
  "data": "",
  "files": {},
  "form": { "a": "1", "b": "2" },
  "json": { "a": "1", "b": "2" },
  "url": "https://postman-echo.com/post"
}

Key takeaway: form: {"a": "1", "b": "2"} in the echoed JSON confirms the server correctly parsed the a=1&b=2 body as form-encoded key/value pairs -- exactly what -d implies without any extra flags.

Why it matters: -d is curl's default, most common way to send form data -- most <form method="post"> submissions on the web use this exact encoding, making this the HTTP equivalent of "the most common form of writing data." Nearly every HTML <form> on the production web still submits data this way by default, and countless legacy backend systems expect exactly this encoding rather than JSON. Recognizing application/x-www-form-urlencoded on sight avoids a common integration mistake: assuming every API accepts JSON when many older ones do not.


Example 47: POST JSON with an Explicit Content-Type

ex-47 · exercises co-14, co-16, co-22

Sending JSON requires an explicit Content-Type: application/json header -- without it, a server has no reliable way to know the body isn't form-encoded text that happens to look like JSON.

# ex-47: -H sets Content-Type explicitly; -d supplies the raw JSON body
# without -H, the server would have no reliable way to tell this body apart from form data
curl -s -H "Content-Type: application/json" -d '{"x":1}' https://postman-echo.com/post

Run: curl -s -H "Content-Type: application/json" -d '{"x":1}' https://postman-echo.com/post

Output (truncated to the relevant fields):

{
  "data": {},
  "json": { "x": 1 },
  "headers": { "content-type": "application/json" },
  "url": "https://postman-echo.com/post"
}

Key takeaway: json: {"x": 1} (a real JSON number, not a string) proves the server parsed the body AS JSON, driven entirely by the explicit Content-Type header -- change or omit that header and the parsing changes too.

Why it matters: Content-Type is a contract the CLIENT asserts and the SERVER trusts -- Example 51 later builds a server that reads a different header (Accept) to make the opposite decision: what format to SEND, not what format was received. Production API integrations fail constantly from exactly this mistake -- sending a JSON body without setting Content-Type: application/json, leaving the server to guess wrong and reject or misparse the request. Making the header explicit, every time, is one of the simplest, most common fixes for a mysterious 400 Bad Request.


Example 48: PUT and DELETE Methods

ex-48 · exercises co-14, co-19

curl -X overrides the HTTP method explicitly -- PUT (replace/create a resource) and DELETE (remove a resource) both echo back through postman-echo.com's matching endpoints, confirming the method itself, not just the body, reached the server.

# ex-48a: PUT -- typically means "replace this resource with this body"
# -X overrides curl's default method choice, which would otherwise be GET here
curl -s -X PUT https://postman-echo.com/put -d "a=1"
# ex-48b: DELETE -- typically means "remove this resource" (often no body at all)
# no -d is supplied here -- DELETE conventionally identifies a resource by URL alone
curl -s -X DELETE https://postman-echo.com/delete

Run: both commands shown above

Output (PUT, truncated):

{ "args": {}, "data": "", "form": { "a": "1" }, "json": { "a": "1" }, "url": "https://postman-echo.com/put" }

Output (DELETE, truncated):

{ "args": {}, "data": "", "form": {}, "json": null, "url": "https://postman-echo.com/delete" }

Key takeaway: postman-echo.com/put and /delete both confirmed the request arrived at the endpoint matching the METHOD used, not just the URL -- the same path with a different method routes to entirely different application logic.

Why it matters: GET, POST, PUT, DELETE, and HEAD (co-14) are the five methods this topic exercises directly -- recognizing each one's INTENT (read, create, replace, remove, read-headers-only) is what makes an unfamiliar API's method choice self-explanatory rather than arbitrary. REST API design in production leans heavily on this method vocabulary to convey intent through the URL and verb alone, rather than through a bespoke action name in the body. Misusing a method -- a GET that mutates state, a POST used for idempotent reads -- is a common code review finding this vocabulary helps catch early.


Example 49: A Tour of Status Codes

ex-49 · exercises co-15

mock.codes/<code> is a service purpose-built to return EXACTLY the requested status code -- a reliable way to see one representative from each of HTTP's four main status classes (2xx, 3xx, 4xx, 5xx) without depending on any real endpoint happening to be in that state.

# ex-49: -w extracts just the status code; -o /dev/null discards the body entirely
for code in 200 301 404 500; do  # => one representative from each of HTTP's four status classes
  curl -s -o /dev/null -w "%{http_code}\n" "https://mock.codes/$code"  # => prints just the code
done  # => loop runs exactly four times, once per class

Run: loop shown above

Output:

200
301
404
500

Key takeaway: Each requested code came back EXACTLY as asked -- 200 (success), 301 (redirect), 404 (client error), 500 (server error) -- one concrete representative per status class.

Why it matters: The FIRST digit of a status code (co-15) is the single fastest triage signal in any HTTP debugging session: 2xx means stop looking, 3xx means follow the Location header, 4xx means fix the request, 5xx means the problem is server-side and retrying later might help. Production monitoring dashboards and alerting rules are almost always built around exactly this first-digit classification, aggregating 5xx rates separately from 4xx rates because they point to entirely different owners and remediation paths. Internalizing the four classes here is what makes reading an unfamiliar service's error-rate graph immediately meaningful.


Example 50: Content-Length Matches the Real Body

ex-50 · exercises co-16, co-13

Content-Length's claimed value should equal the ACTUAL byte count of the body that follows it -- this example verifies that contract directly, rather than merely trusting the header.

# ex-50a: read the claimed Content-Length from the headers
curl -sI --http1.1 https://info.cern.ch | grep -i content-length
# ex-50b: independently count the real body's bytes
curl -s --http1.1 https://info.cern.ch | wc -c

Run: both commands shown above

Output:

Content-Length: 646
     646

Key takeaway: The header claimed 646 bytes; wc -c's independent count of the actual downloaded body also reported 646 -- the header's value is a VERIFIABLE fact about the response, not just an informational hint.

Why it matters: A mismatched Content-Length (from a buggy server or a broken proxy) is a real, debuggable class of bug -- knowing you can independently verify this header against the real byte count is a concrete way to confirm (or rule out) that class of problem. Production incidents involving a truncated download, a broken CDN cache, or a misconfigured reverse proxy often trace back to exactly this kind of Content-Length mismatch. Having a simple, independent verification technique turns a vague "the file looks corrupted" report into a concrete, provable diagnosis.


Example 51: Accept Header Negotiation -- Ask for JSON, Get JSON

ex-51 · exercises co-22, co-16

A hand-rolled server reads the request's Accept header and chooses its response's Content-Type and body accordingly -- content negotiation from the SERVER's point of view, the mirror image of Example 47's client-driven Content-Type.

%% Color Palette: Blue #0173B2, Orange #DE8F05, Teal #029E73
sequenceDiagram
    participant C as client
    participant S as server (Ex 51)
    C->>S: GET / -- Accept: application/json
    Note over S: reads the Accept header (co-22)
    S->>C: 200 OK -- Content-Type: application/json
# learning/code/ex-51-accept-header-negotiation/example.py
"""Example 51: Accept Header Negotiation -- Ask for JSON, Get JSON."""
 
import socket  # => stdlib sockets -- no HTTP library involved on either side of this exchange
import threading  # => only the ready-signal + background thread, not real concurrency
 
HOST = "127.0.0.1"  # => loopback -- keeps this negotiation demo local and deterministic
PORT = 50051  # => co-05: a fresh ephemeral port, unique to this example
 
 
def server(ready: threading.Event) -> None:  # => co-22: a server that reads the Accept header  # fmt: skip
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:  # => same triple as Ex 29+  # fmt: skip
        # => IPv4 + TCP, scoped to this "with" block so the fd always closes on exit
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        # => set BEFORE bind() -- lets an immediate re-run reuse a TIME_WAIT'd port
        sock.bind((HOST, PORT))
        # => claims (HOST, PORT) for this process -- must happen before listen()
        sock.listen(1)
        # => flips the socket passive, ready to queue one pending connection
        ready.set()
        # => unblocks the main thread's wait() below -- no guessed sleep() needed
        conn, _ = sock.accept()
        # => BLOCKS until the client's connect() completes the TCP handshake
        with conn:  # => this one connection's socket -- closes automatically on block exit  # fmt: skip
            request = conn.recv(4096).decode()  # => reads the raw request bytes, decoded to text  # fmt: skip
            wants_json = "Accept: application/json" in request  # => co-16: read ONE header's value  # fmt: skip
            if wants_json:  # => co-22: the client's Accept header decides the representation  # fmt: skip
                body = b'{"greeting": "hello"}'  # => the JSON representation of the SAME greeting
                content_type = b"application/json"  # => the Content-Type that MATCHES the body  # fmt: skip
            else:
                # => the safe fallback when Accept doesn't ask for JSON explicitly
                body = b"greeting: hello"  # => the plain-text representation of the SAME greeting
                content_type = b"text/plain"  # => the Content-Type that MATCHES this body instead  # fmt: skip
            response = (  # => built by hand, byte by byte -- no HTTP library involved
                b"HTTP/1.1 200 OK\r\n"  # => co-13: the status line -- version, code, reason phrase
                # => the negotiated type, not hardcoded
                b"Content-Type: " + content_type + b"\r\n"
                # => the actual body's size
                b"Content-Length: " + str(len(body)).encode() + b"\r\n"
                b"Connection: close\r\n"  # => tells the client to expect the connection to close
                # => the mandatory blank-line separator, then the chosen body
                b"\r\n" + body
            )
            conn.sendall(response)  # => the chosen representation, sent back over this connection  # fmt: skip
 
 
ready_event = threading.Event()  # => the same ready-signal pattern used since Example 34  # fmt: skip
thread = threading.Thread(target=server, args=(ready_event,), daemon=True)
# => daemon=True: this thread never blocks process exit if something above hangs
thread.start()
# => runs the server concurrently with the request-sending client code that follows
ready_event.wait(timeout=5)
# => blocks here until bind()+listen() genuinely completed, avoiding a race with connect()
 
request = (  # => plain string concatenation -- no HTTP library builds this for us either
    "GET / HTTP/1.1\r\n"  # => the request line: METHOD, PATH, VERSION
    "Host: 127.0.0.1\r\n"  # => co-16: HTTP/1.1 REQUIRES a Host header even on loopback
    "Accept: application/json\r\n"  # => co-22: explicitly asking for JSON, not plain text
    "Connection: close\r\n"  # => tells the server to close after replying, simplifying this demo
    "\r\n"  # => the blank line that ends the headers -- REQUIRED even with no body
)
with socket.create_connection((HOST, PORT), timeout=5) as sock:  # => co-07: the TCP handshake  # fmt: skip
    sock.sendall(request.encode())  # => co-12: the raw hand-built request, sent as bytes  # fmt: skip
    response = b""  # => accumulates the full response -- its final size isn't known upfront  # fmt: skip
    while True:  # => co-11: loop until the server closes (Connection: close makes this safe)  # fmt: skip
        chunk = sock.recv(4096)  # => reads whatever arrives next, up to 4096 bytes
        if not chunk:  # => an empty recv() means the server closed its side
            break
        response += chunk  # => appends this chunk -- the loop above may run several times  # fmt: skip
 
thread.join(timeout=5)
# => waits for the server thread to finish handling this one request before exiting
 
head, _, body = response.partition(b"\r\n\r\n")  # => co-13: split at the one fixed boundary  # fmt: skip
print(f"headers:\n{head.decode()}")  # => expect Content-Type: application/json in this output  # fmt: skip
print(f"body: {body!r}")  # => expect the JSON body, since the request's Accept asked for JSON  # fmt: skip
 
assert b"application/json" in head  # => confirms the server chose JSON based on Accept
assert body == b'{"greeting": "hello"}'  # => confirms the BODY is actually JSON, not plain text  # fmt: skip
# both assertions together confirm negotiation is genuine: the Content-Type header and the
# actual body bytes agree with each other, and both were driven by what the client asked for.
print("ex-51 OK")  # => confirms the Accept -> Content-Type negotiation worked end to end  # fmt: skip

Run: python3 example.py

Output:

headers:
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 22
Connection: close
 
body: b'{"greeting": "hello"}'
ex-51 OK

Key takeaway: Sending Accept: application/json got back Content-Type: application/json and a genuine JSON body -- the server read the client's stated preference and honored it, rather than always replying in one fixed format.

Why it matters: Real APIs (GitHub's, for instance) use this exact mechanism to let one endpoint serve both JSON and other formats -- understanding Accept -> Content-Type as a request/response PAIR, not two unrelated headers, is what makes content negotiation click. Production APIs that need to serve both a human-facing browser and a machine client from the same endpoint rely on exactly this server-side Accept-driven negotiation to avoid maintaining two separate URLs. Getting the negotiation logic right, on both the request and response side, is what makes a single well-designed endpoint serve many different clients cleanly.


Example 52: Content Negotiation with gzip

ex-52 · exercises co-22, co-19

curl --compressed sends Accept-Encoding: gzip, deflate and automatically decompresses a Content-Encoding: gzip response -- the same negotiation pattern as Example 51, but for compression instead of content type.

# ex-52: --compressed asks for a compressed reply AND transparently decompresses it
curl -v --compressed --http1.1 https://example.com 2>&1 1>/dev/null | grep -iE "accept-encoding|content-encoding"

Run: curl -v --compressed --http1.1 https://example.com 2>&1 1>/dev/null | grep -iE "accept-encoding|content-encoding"

Output:

> Accept-Encoding: deflate, gzip
< Content-Encoding: gzip

Key takeaway: The request's Accept-Encoding header ASKED for gzip; the response's Content-Encoding header CONFIRMED gzip was used -- curl transparently decompressed the body before Example 1 ever printed it, with no extra code required.

Why it matters: Compression is invisible in almost every earlier example's printed output specifically BECAUSE curl decompresses automatically -- this example is the one place that peels back that automatic step and shows the negotiation actually happening. Production APIs enable gzip compression by default precisely because it can shrink a JSON or HTML payload by 70% or more with negligible CPU cost, directly reducing both bandwidth bills and perceived load latency for real users. Seeing the negotiation headers explicitly here demystifies a step that normally happens invisibly inside every HTTP client.


Example 53: Chunked Transfer Encoding

ex-53 · exercises co-16, co-13

When a server doesn't know a response's total size in advance, it uses Transfer-Encoding: chunked instead of Content-Length -- the body is sent as a series of self-describing, size-prefixed chunks rather than one block with a fixed length.

%% Color Palette: Blue #0173B2, Orange #DE8F05, Teal #029E73
graph LR
    A["server response body"]:::blue --> B{"total size known<br/>in advance?"}
    B -->|"yes"| C["Content-Length: N<br/>one fixed block"]:::orange
    B -->|"no"| D["Transfer-Encoding: chunked<br/>size-prefixed chunks"]:::teal
 
    classDef blue fill:#0173B2,stroke:#000000,color:#FFFFFF,stroke-width:2px
    classDef orange fill:#DE8F05,stroke:#000000,color:#FFFFFF,stroke-width:2px
    classDef teal fill:#029E73,stroke:#000000,color:#FFFFFF,stroke-width:2px
# ex-53: example.com's CDN streams its response chunked -- no Content-Length header at all
curl -v --http1.1 https://example.com 2>&1 1>/dev/null | grep -i "transfer-encoding\|content-length"

Run: curl -v --http1.1 https://example.com 2>&1 1>/dev/null | grep -i "transfer-encoding\|content-length"

Output:

< Transfer-Encoding: chunked

Key takeaway: example.com's response has Transfer-Encoding: chunked and NO Content-Length at all -- confirming these two headers are mutually exclusive alternatives for the same problem (how does the client know when the body ends?).

Why it matters: This is exactly why Example 44 deliberately switched to info.cern.ch: parsing a chunked body by hand requires reading each chunk's hex-encoded size prefix first, which would have obscured that example's actual point (splitting status line / headers / body); http.client (Example 61 onward) handles this de-chunking for you automatically. Production servers streaming a large file, a live log tail, or a server-sent-events feed all rely on chunked encoding for exactly this reason: the response can start flowing before its total size is known.


Example 54: UDP Echo Server

ex-54 · exercises co-08, co-10

A UDP server needs no bind/listen/accept sequence -- just bind, then recvfrom(), which returns BOTH the datagram's bytes and the sender's address in one call, since UDP has no persistent "connection" object the way TCP's accept() returns one.

%% Color Palette: Blue #0173B2, Orange #DE8F05, Teal #029E73
sequenceDiagram
    participant C as client (Ex 55)
    participant S as server (Ex 54)
    S->>S: bind() -- NO listen/accept
    Note over C,S: no handshake at all (co-08)
    C->>S: sendto(datagram)
    S->>C: sendto(echo) -- via recvfrom()'s address
# learning/code/ex-54-udp-echo-server/server.py
"""Example 54: UDP Echo Server."""
 
import socket  # => same stdlib module as TCP -- only the socket TYPE differs (co-08, co-10)
 
HOST = "127.0.0.1"  # => loopback -- keeps this UDP demo local and deterministic
PORT = 50054  # => co-05: a fresh ephemeral port, unique to this example
 
 
def run_server() -> None:  # => a UDP server needs no bind/listen/accept sequence at all
    # SOCK_DGRAM selects UDP: connectionless, message-oriented, no handshake (co-08).
    with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:
        sock.bind((HOST, PORT))  # => UDP still binds to claim a local port -- but never listen()s  # fmt: skip
        print(f"listening on {HOST}:{PORT}", flush=True)  # => the signal the client script waits for  # fmt: skip
        # recvfrom (not recv!) returns BOTH the datagram's bytes AND the sender's address --
        # there is no persistent "connection" object like TCP's accept() returns (co-08).
        data, sender_addr = sock.recvfrom(1024)  # => blocks until ONE datagram arrives
        print(f"received {data!r} from {sender_addr}")  # => shows the sender's address, learned here  # fmt: skip
        sock.sendto(data, sender_addr)  # => sendto: no connection needed, just an address  # fmt: skip
 
 
if __name__ == "__main__":  # => only runs when invoked directly, not when imported
    run_server()  # => the guard above is WHY this only fires when this file is run as a script

Run: python3 server.py &, then Example 55's client.py

Output (server):

listening on 127.0.0.1:50054
received b'hello over udp' from ('127.0.0.1', 55102)

Key takeaway: recvfrom() returned the sender's address alongside the data -- co-08's connectionless model made this necessary, since there is no prior accept()-established relationship the server could otherwise use to know who to reply to.

Why it matters: This missing accept() step is the single biggest structural difference between UDP and TCP servers -- every UDP server must re-learn "who sent this?" on every single incoming datagram, rather than knowing it once at connection time. Production UDP services -- DNS servers, game state broadcasts, telemetry collectors -- all re-derive the sender's address from every single incoming datagram for exactly this reason, since there is no persistent connection object to remember it for them. That per-packet address lookup is a direct, measurable cost UDP servers pay that TCP servers do not.


Example 55: UDP Echo Client

ex-55 · exercises co-08, co-10

sendto() fires a datagram at an explicit (host, port) address with no prior connect() call at all -- the client half of Example 54's connectionless exchange.

# learning/code/ex-55-udp-echo-client/client.py
"""Example 55: UDP Echo Client."""
 
import socket  # => stdlib sockets -- SOCK_DGRAM below is the only difference from TCP clients
 
HOST = "127.0.0.1"  # => loopback -- must match Example 54's server address exactly
PORT = 50054  # => must match the server's bound port exactly
 
 
def run_client(message: bytes) -> bytes:  # => sends ONE datagram, reads ONE datagram back  # fmt: skip
    with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:  # => no connect() call at all  # fmt: skip
        sock.settimeout(5)  # => co-08: UDP has no delivery guarantee -- a timeout avoids hanging  # fmt: skip
        sock.sendto(message, (HOST, PORT))  # => fires the datagram -- no handshake, no ack  # fmt: skip
        reply, _ = sock.recvfrom(1024)  # => blocks until a reply arrives, or the timeout fires  # fmt: skip
        return reply  # => the sender's own address (the second tuple item) is discarded here
 
 
if __name__ == "__main__":  # => only runs when invoked directly, not when imported
    reply = run_client(b"hello over udp")  # => Example 54's server must already be running  # fmt: skip
    print(f"client received: {reply!r}")  # => confirms the echoed bytes match what was sent  # fmt: skip
    assert reply == b"hello over udp"  # => confirms this ONE datagram made the full round trip  # fmt: skip
    print("ex-55 OK")  # => confirms the connectionless round trip completed without a handshake  # fmt: skip

Run: python3 client.py (with server.py already running)

Output:

client received: b'hello over udp'
ex-55 OK

Key takeaway: No connect(), no handshake, no "session" of any kind -- one sendto(), one recvfrom(), and the exact bytes round-tripped successfully.

Why it matters: TCP's create_connection() and UDP's bare socket() + sendto() are the two smallest possible client patterns this topic covers -- comparing them side by side (as Example 57 does explicitly) is the clearest way to internalize co-08's connectionless model. Production systems choosing between TCP and UDP client code almost always start from exactly these two smallest possible patterns before adding retries, timeouts, or sequence numbers on top. Recognizing which primitive shape a more complex client library is really built on makes an unfamiliar networking library's API far less mysterious.


Example 56: UDP Has No Handshake -- Sending to a Closed Port

ex-56 · exercises co-08, co-09

sendto() succeeds INSTANTLY and with no error, even when NOTHING is listening on the target port -- UDP has no three-way handshake to fail during, so the only signal of "nobody is there" is silence: a recvfrom() that never returns.

# learning/code/ex-56-udp-no-handshake/example.py
"""Example 56: UDP Has No Handshake -- Sending to a Closed Port."""
 
import socket  # => stdlib sockets -- sendto() itself is the whole demonstration here
import time  # => wall-clock timing is what proves sendto() never blocked at all
 
HOST = "127.0.0.1"  # => loopback -- keeps this no-listener demo local and deterministic
PORT = 50056  # => co-08: deliberately, nothing is listening here at all
 
 
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:  # => a bare UDP socket, no bind()  # fmt: skip
    sock.settimeout(1.0)  # => co-08: UDP gives no delivery confirmation -- bound the wait  # fmt: skip
    start = time.monotonic()  # => a clock reading taken right before the send below
    # sendto SUCCEEDS regardless of whether anything is listening -- UDP has no three-way
    # handshake (co-07 contrast) to fail during. The datagram is simply fired onto the wire.
    sock.sendto(b"is anyone there?", (HOST, PORT))  # => this call returns immediately, no error  # fmt: skip
    elapsed_to_send = time.monotonic() - start  # => the time sendto() itself took to return  # fmt: skip
    print(f"sendto() returned in {elapsed_to_send:.4f}s with NO error")  # => co-08: no handshake  # fmt: skip
 
    try:  # => wrapped because recvfrom() below is EXPECTED to time out, not to error
        reply, _ = sock.recvfrom(1024)  # => waits for a reply that will never come
        outcome = f"unexpectedly received: {reply!r}"  # => reached only if something DID answer
    except TimeoutError:  # => co-08: silence is the ONLY signal -- no "port closed" notification  # fmt: skip
        outcome = "timed out waiting for a reply -- UDP never told us nobody was listening"  # fmt: skip
    print(outcome)  # => reports whichever branch above actually ran
 
assert elapsed_to_send < 0.1  # => confirms sendto() itself never blocked on the missing listener  # fmt: skip
print("ex-56 OK")  # => confirms both the instant sendto() and the eventual timeout were observed  # fmt: skip

Run: python3 example.py

Output:

sendto() returned in 0.0001s with NO error
timed out waiting for a reply -- UDP never told us nobody was listening
ex-56 OK

Key takeaway: sendto() returned in a fraction of a millisecond, no error at all -- the ONLY way this example discovered nothing was listening was a recvfrom() timeout, since UDP has no equivalent of TCP's ConnectionRefusedError.

Why it matters: Contrast this directly with Example 59's TCP ConnectionRefusedError -- TCP's connection-oriented handshake means the OS can actively tell you "nobody is listening" almost immediately; UDP's connectionless design means it CANNOT, and application code must build its own timeout-based detection instead. Production monitoring systems that rely on UDP-based protocols -- like many metrics collectors -- must build their own liveness checks for exactly this reason, since the transport layer itself will never report a missing listener. Designing an application-level heartbeat or acknowledgment scheme is the direct, practical consequence of this silence.


Example 57: TCP vs. UDP -- the Same Message, Two Different APIs

ex-57 · exercises co-07, co-08, co-09

Running a TCP server/client pair and a UDP server/client pair side by side, sending the exact same three messages, makes the API-level differences concrete: connect/accept vs. bind-only, recv vs. recvfrom, and TCP's ordering guarantee vs. UDP's lack of one.

API shape contrast:

%% Color Palette: Blue #0173B2, Orange #DE8F05, Teal #029E73
graph TB
    subgraph TCP["TCP -- connection-oriented (co-07)"]
        T1["bind#40;#41;"]:::blue --> T2["listen#40;#41;"]:::blue --> T3["accept#40;#41;<br/>BLOCKS for handshake"]:::blue --> T4["recv#40;#41; / sendall#40;#41;<br/>on conn"]:::teal
    end
    subgraph UDP["UDP -- connectionless (co-08)"]
        U1["bind#40;#41;"]:::orange --> U2["recvfrom#40;#41; / sendto#40;#41;<br/>on sock, no handshake"]:::teal
    end
 
    classDef blue fill:#0173B2,stroke:#000000,color:#FFFFFF,stroke-width:2px
    classDef orange fill:#DE8F05,stroke:#000000,color:#FFFFFF,stroke-width:2px
    classDef teal fill:#029E73,stroke:#000000,color:#FFFFFF,stroke-width:2px
# learning/code/ex-57-tcp-vs-udp-contrast/example.py
"""Example 57: TCP vs. UDP -- the Same Message, Two Different APIs."""
 
import socket  # => stdlib sockets -- both TCP and UDP go through this one module
import threading  # => runs BOTH server variants concurrently, side by side
 
HOST = "127.0.0.1"  # => loopback -- keeps this contrast demo local and deterministic
TCP_PORT = 50057  # => co-05: TCP and UDP need SEPARATE ports even on the same host
UDP_PORT = 50157  # => a different port from TCP_PORT -- the two protocols don't share a namespace
 
 
def tcp_server(ready: threading.Event) -> None:  # => co-07: connection-oriented, byte-stream  # fmt: skip
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:  # => same triple as Ex 29+  # fmt: skip
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)  # => same boilerplate as before  # fmt: skip
        sock.bind((HOST, TCP_PORT))  # => claims the TCP port -- UDP claims its OWN port below  # fmt: skip
        sock.listen(1)  # => TCP-only: listen() marks the socket passive
        ready.set()
        conn, _ = sock.accept()  # => TCP-only: an explicit three-way-handshake accept step  # fmt: skip
        with conn:  # => TCP-only: accept() hands back a SEPARATE connected socket from "sock"  # fmt: skip
            for _ in range(3):  # => the client sends exactly three messages over one connection  # fmt: skip
                data = conn.recv(64)  # => reads from the connected "conn", not the listening "sock"  # fmt: skip
                conn.sendall(data)  # => echoes on the same persistent connection, no re-addressing  # fmt: skip
 
 
def udp_server(ready: threading.Event) -> None:  # => co-08: connectionless, message-oriented  # fmt: skip
    with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:  # => SOCK_DGRAM, not SOCK_STREAM  # fmt: skip
        sock.bind((HOST, UDP_PORT))  # => UDP: binds, but never listen()s or accept()s at all  # fmt: skip
        ready.set()
        for _ in range(3):  # => the client sends exactly three independent datagrams
            data, addr = sock.recvfrom(64)  # => each call surfaces the sender's address directly  # fmt: skip
            sock.sendto(data, addr)  # => UDP-only: every reply re-addresses explicitly, no "conn"  # fmt: skip
 
 
tcp_ready = threading.Event()  # => a separate ready-signal per server, since two servers now race  # fmt: skip
udp_ready = threading.Event()  # => set independently once udp_server's own bind() completes  # fmt: skip
threading.Thread(target=tcp_server, args=(tcp_ready,), daemon=True).start()
# => starts the TCP server concurrently with the UDP server line right below
threading.Thread(target=udp_server, args=(udp_ready,), daemon=True).start()
# => both servers now run side by side -- neither blocks the other from starting
tcp_ready.wait(timeout=5)  # => blocks until the TCP server's own bind()+listen() truly completed  # fmt: skip
udp_ready.wait(timeout=5)  # => blocks until the UDP server's own bind() truly completed
 
# TCP: ONE connect() call, then a persistent stream -- order is guaranteed (co-07).
tcp_replies: list[bytes] = []
with socket.create_connection((HOST, TCP_PORT), timeout=5) as sock:  # => the explicit handshake  # fmt: skip
    for msg in (b"one", b"two", b"three"):  # => all three ride the SAME already-open connection  # fmt: skip
        sock.sendall(msg)  # => no destination argument needed -- connect() already fixed the peer  # fmt: skip
        tcp_replies.append(sock.recv(64))  # => recv() carries no sender info -- there's one peer  # fmt: skip
print(f"TCP replies (order guaranteed): {tcp_replies}")
 
# UDP: NO connect() call -- every send is independently addressed (co-08).
udp_replies: list[bytes] = []
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:  # => a plain, unconnected UDP socket  # fmt: skip
    sock.settimeout(5)  # => co-08: no delivery guarantee, so a timeout avoids hanging forever  # fmt: skip
    for msg in (b"one", b"two", b"three"):  # => three separate, individually-addressed datagrams  # fmt: skip
        sock.sendto(msg, (HOST, UDP_PORT))  # => every datagram names its destination explicitly  # fmt: skip
        reply, _ = sock.recvfrom(64)  # => recvfrom() DOES surface sender info -- UDP has no "conn"  # fmt: skip
        udp_replies.append(reply)  # => appended in receive order, which UDP never itself promises  # fmt: skip
print(f"UDP replies (order NOT guaranteed by the protocol): {udp_replies}")
 
assert tcp_replies == [b"one", b"two", b"three"]  # => TCP: exact order, every time
assert set(udp_replies) == {b"one", b"two", b"three"}  # => UDP: all arrived, order not promised  # fmt: skip
# TCP asserts an exact LIST (order matters); UDP asserts only a SET (order does not) -- that
# single difference in the assertion itself is the API contrast this whole example demonstrates.
# On this machine's loopback interface both happen to arrive in order -- the API difference
# (connect/accept vs. bind-only, recv vs. recvfrom) is real regardless of what one run shows;
# Example 75 demonstrates UDP's lack of a delivery guarantee concretely, with real loss.
print("ex-57 OK")  # => confirms both API shapes completed their three-message exchange

Run: python3 example.py

Output:

TCP replies (order guaranteed): [b'one', b'two', b'three']
UDP replies (order NOT guaranteed by the protocol): [b'one', b'two', b'three']
ex-57 OK

Key takeaway: On this machine's loopback interface, both arrived in order this run -- but only TCP's guarantee is a PROTOCOL promise; UDP's matching order here is circumstance, not a contract, which Example 75's genuine packet loss demonstrates concretely.

Why it matters: The API shape difference (connect/accept + recv vs. bind-only + recvfrom) is a direct, mechanical reflection of the underlying protocol difference -- TCP's extra steps exist specifically to establish and maintain the ordering/reliability guarantee UDP deliberately omits. Production engineers choosing between TCP and UDP for a new service -- a chat protocol, a game server, a metrics pipeline -- are really choosing between these exact API shapes and the guarantees behind them, not just a checkbox in a config file. Seeing both side by side, sending the identical messages, makes that tradeoff concrete rather than abstract.


Example 58: Measure a TCP Round Trip in Python

ex-58 · exercises co-07, co-10

time.perf_counter() is Python's high-resolution clock, appropriate for measuring sub-millisecond durations -- this example measures 5 real round trips over a loopback TCP connection and reports each one plus an average.

# learning/code/ex-58-measure-latency-socket/example.py
"""Example 58: Measure a TCP Round Trip in Python."""
 
import socket  # => stdlib sockets -- the round trip being timed is a plain send/recv pair
import threading  # => only the ready-signal + background thread, not real concurrency
import time  # => perf_counter() is the actual measurement instrument this example turns on
 
HOST = "127.0.0.1"  # => loopback -- gives a latency FLOOR, not a real-network figure
PORT = 50058  # => co-05: a fresh ephemeral port, unique to this example
# a real network hop would add real transit time on top of this floor -- this measures pure
# socket + OS overhead, the minimum any TCP round trip on this machine could possibly cost.
 
 
def server(ready: threading.Event) -> None:  # => backgrounded so the client below can run inline  # fmt: skip
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
        # => IPv4 + TCP, scoped to this "with" block so the fd always closes on exit
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        # => set BEFORE bind() -- lets an immediate re-run reuse a TIME_WAIT'd port
        sock.bind((HOST, PORT))
        # => claims (HOST, PORT) for this process -- must happen before listen()
        sock.listen(1)
        # => flips the socket passive, ready to queue one pending connection
        ready.set()
        # => unblocks the main thread's wait() below -- no guessed sleep() needed
        conn, _ = sock.accept()
        # => BLOCKS until the client's connect() completes the TCP handshake
        with conn:  # => a context manager -- conn's fd closes automatically when this block exits
            for _ in range(5):  # => this demo measures 5 separate round trips
                data = conn.recv(64)
                if not data:
                    break
                conn.sendall(data)  # => echoes back as fast as possible -- no artificial delay  # fmt: skip
 
 
ready_event = threading.Event()  # => the same ready-signal pattern used since Example 34  # fmt: skip
thread = threading.Thread(target=server, args=(ready_event,), daemon=True)
# => daemon=True: this thread never blocks process exit if something above hangs
thread.start()
# => runs the server concurrently with the timed client loop that follows below
ready_event.wait(timeout=5)
# => blocks here until bind()+listen() genuinely completed, avoiding a race with connect()
 
latencies_ms: list[float] = []  # => co-01: measured, not assumed, exactly like Example 24 did  # fmt: skip
# perf_counter() (not time.time()) is used because it's monotonic and immune to clock adjustments.
with socket.create_connection((HOST, PORT), timeout=5) as sock:  # => one connection, reused below  # fmt: skip
    for i in range(5):  # => five independent measurements smooth out one-off scheduling jitter  # fmt: skip
        start = time.perf_counter()  # => a high-resolution clock, appropriate for sub-ms timing  # fmt: skip
        sock.sendall(b"ping")  # => a tiny 4-byte payload -- measures overhead, not bandwidth  # fmt: skip
        sock.recv(64)  # => blocks until the echoed reply arrives
        # 64 bytes comfortably exceeds the 4-byte payload -- no partial-read handling needed here.
        elapsed_ms = (time.perf_counter() - start) * 1000  # => convert seconds to milliseconds  # fmt: skip
        latencies_ms.append(elapsed_ms)  # => accumulated across all 5 iterations for the average  # fmt: skip
        print(f"round trip {i + 1}: {elapsed_ms:.3f} ms")  # => per-iteration figure shows jitter  # fmt: skip
 
thread.join(timeout=5)
# => waits for the server thread to finish handling all five round trips before exiting
 
average_ms = sum(latencies_ms) / len(latencies_ms)  # => a simple mean across the 5 measurements  # fmt: skip
# a genuine average of REAL measurements, not a hand-picked "looks about right" number.
print(f"average: {average_ms:.3f} ms")  # => the single headline number this example exists to produce  # fmt: skip
 
assert all(ms >= 0 for ms in latencies_ms)  # => sanity check: time never runs backward
assert average_ms < 50  # => loopback round trips are consistently well under 50ms
print("ex-58 OK")  # => confirms all five round trips were measured and stayed within bounds  # fmt: skip

Run: python3 example.py

Output:

round trip 1: 0.081 ms
round trip 2: 0.062 ms
round trip 3: 0.058 ms
round trip 4: 0.061 ms
round trip 5: 0.083 ms
average: 0.069 ms
ex-58 OK

Key takeaway: All five loopback round trips finished in well under a tenth of a millisecond -- a concrete baseline for "how fast can a TCP round trip possibly be," useful for recognizing when a REAL network's latency (Examples 9, 25, 70, 79) is dominated by something other than the socket API's own overhead.

Why it matters: Loopback latency is close to a hardware/OS floor -- any latency ABOVE this floor, when measured against a real remote host, is attributable to actual network transit, DNS, or TLS, not to Python's socket calls themselves. Production latency dashboards and SLOs are meaningless without a baseline like this one -- without knowing the floor, it is impossible to tell whether a slow request is genuinely network-bound or simply application logic taking its time. Measuring this floor once on real hardware is far more useful than trusting a textbook number.


Example 59: connect() to an Open Port vs. a Closed Port

ex-59 · exercises co-05, co-10

connect() to a genuinely open port (a real, real remote HTTPS server) succeeds; connect() to a closed local port raises ConnectionRefusedError almost instantly -- the concrete gap between "the host is reachable" (Example 9's ping) and "this specific port has something listening."

# learning/code/ex-59-port-scan-connect/example.py
"""Example 59: connect() to an Open Port vs. a Closed Port."""
 
import socket  # => stdlib sockets -- connect() itself is the whole demonstration here
 
OPEN_HOST = "example.com"  # => co-05: a real host with a real listener on port 443 (HTTPS)  # fmt: skip
OPEN_PORT = 443  # => HTTPS's well-known port -- genuinely has something listening
CLOSED_HOST = "127.0.0.1"  # => loopback -- guaranteed nothing is listening on this port
CLOSED_PORT = 50059  # => deliberately unused in this entire topic's port range
 
 
def try_connect(host: str, port: int) -> str:  # => co-10: connect() either succeeds or raises  # fmt: skip
    try:  # => wrapped so a refused connection doesn't crash the script -- it raises instead
        with socket.create_connection((host, port), timeout=5):  # => the actual connect() attempt  # fmt: skip
            return "connected successfully"  # => reached only if the handshake truly completed
    except ConnectionRefusedError as err:  # => co-10: the OS actively rejected the SYN
        return f"ConnectionRefusedError: {err}"  # => the exact exception text, so both cases print
 
 
open_result = try_connect(OPEN_HOST, OPEN_PORT)  # => a genuinely open, real remote port
closed_result = try_connect(CLOSED_HOST, CLOSED_PORT)  # => a genuinely closed local port  # fmt: skip
 
print(f"{OPEN_HOST}:{OPEN_PORT} -> {open_result}")  # => expect "connected successfully"
print(f"{CLOSED_HOST}:{CLOSED_PORT} -> {closed_result}")  # => expect "ConnectionRefusedError: ..."  # fmt: skip
 
assert open_result == "connected successfully"  # => confirms the genuinely open port succeeds  # fmt: skip
assert closed_result.startswith("ConnectionRefusedError")  # => confirms the closed port raises, not hangs  # fmt: skip
print("ex-59 OK")  # => confirms both the success case and the failure case were reproduced  # fmt: skip

Run: python3 example.py

Output:

example.com:443 -> connected successfully
127.0.0.1:50059 -> ConnectionRefusedError: [Errno 61] Connection refused
ex-59 OK

Key takeaway: The exact same try_connect() function succeeded against a real open port and raised a real, specific exception against a closed one -- ConnectionRefusedError is a checkable, catchable fact, not a hang or a silent failure.

Why it matters: A host answering ping (Example 9) says nothing about whether any given PORT is open -- this is the concrete gap between co-06's reachability and co-10's "is this specific service listening," and the exact reason a "the server is up but the site is down" bug report is a real, distinct category from "the server is unreachable."


Example 60: Resolve a Hostname to an IP Address in Python

ex-60 · exercises co-03, co-10

socket.gethostbyname() is the simple, IPv4-only resolver; socket.getaddrinfo() is the modern, protocol-agnostic resolver that returns EVERY matching address (IPv4 and IPv6 alike) plus the connection parameters needed for each.

%% Color Palette: Blue #0173B2, Orange #DE8F05, Teal #029E73
graph LR
    A["example.com"]:::blue --> B["gethostbyname#40;#41;"]:::orange
    A --> C["getaddrinfo#40;#41;"]:::orange
    B --> D["1 IPv4 address"]:::teal
    C --> E["N results:<br/>IPv4 + IPv6"]:::teal
 
    classDef blue fill:#0173B2,stroke:#000000,color:#FFFFFF,stroke-width:2px
    classDef orange fill:#DE8F05,stroke:#000000,color:#FFFFFF,stroke-width:2px
    classDef teal fill:#029E73,stroke:#000000,color:#FFFFFF,stroke-width:2px
# learning/code/ex-60-resolve-in-python/example.py
"""Example 60: Resolve a Hostname to an IP Address in Python."""
 
import socket  # => stdlib sockets -- both resolver functions below live in this one module
 
HOST = "example.com"  # => co-03: the same demo host every dig-based example resolved
 
# gethostbyname is the simple, IPv4-only resolver call -- co-03, co-10.
ipv4_address = socket.gethostbyname(HOST)  # => a single blocking DNS lookup, returns one IPv4  # fmt: skip
print(f"gethostbyname: {ipv4_address}")  # => one address, no family/port metadata attached  # fmt: skip
 
# getaddrinfo is the modern, protocol-agnostic resolver -- returns EVERY matching address,
# IPv4 and IPv6 alike, plus the socket parameters needed to connect to each one (co-03).
results = socket.getaddrinfo(HOST, 80, proto=socket.IPPROTO_TCP)  # => a richer, multi-result lookup  # fmt: skip
print(f"getaddrinfo returned {len(results)} result(s)")  # => count varies by host's IPv4/IPv6 records  # fmt: skip
# each getaddrinfo() result is a 5-tuple: family, socktype, proto, canonical name, sockaddr.
for family, _socktype, _proto, _canonname, sockaddr in results:
    family_name = "IPv4" if family == socket.AF_INET else "IPv6"  # => classifies THIS one result  # fmt: skip
    print(f"  {family_name}: {sockaddr[0]}")  # => sockaddr[0] is the address; [1] is the port  # fmt: skip
 
assert ipv4_address.count(".") == 3  # => confirms a dotted-quad IPv4 address came back
assert len(results) >= 1  # => confirms getaddrinfo found at least one real address
print("ex-60 OK")  # => confirms both resolver functions genuinely resolved the same host  # fmt: skip

Run: python3 example.py

Output:

gethostbyname: 104.20.23.154
getaddrinfo returned 4 result(s)
  IPv4: 172.66.147.243
  IPv4: 104.20.23.154
  IPv6: 2606:4700:10::ac42:93f3
  IPv6: 2606:4700:10::6814:179a
ex-60 OK

Key takeaway: getaddrinfo returned FOUR results -- 2 IPv4, 2 IPv6 -- for the exact same hostname gethostbyname resolved to just one IPv4 address; this matches Example 11's dig A and Example 13's dig AAAA results exactly, confirming Python's own resolver sees the same DNS records dig does.

Why it matters: getaddrinfo is what production code should actually use -- it's dual-stack-aware (IPv4 and IPv6), protocol-agnostic, and is the same call socket.create_connection() uses internally to resolve a hostname before connecting; gethostbyname is the simpler, IPv4-only tool this topic used earlier for readability. Production services that only call gethostbyname() silently ignore any IPv6-only host and miss out on getaddrinfo()'s ability to try multiple addresses in order when the first one fails. Defaulting to getaddrinfo() in new code avoids both of those real, production-relevant limitations.


← Previous: Beginner Examples · Next: Advanced Examples

Last updated July 13, 2026

Command Palette

Search for a command to run...