Shipping It
Get what you built in front of real people, and keep it alive.
- Level
- Some background helps
- Lessons
- 74
- Reading time
- 658 min
- Price
- Free, no sign-up to read
Most of what stops people shipping is not code. It is a domain that will not resolve, a key committed by accident, a bill that arrives three weeks later, and a backup nobody has ever restored. This course covers the boring machinery between "it works on my laptop" and "strangers are using it" — servers, DNS, HTTP, secrets, deploys, logs, cost, abuse and recovery — with real numbers and real failure modes. You need to have built something small already. You do not need to have deployed anything.
Opens after the Agents and Automation exam
Sign in, finish that course, and pass its exam. You can read this syllabus meanwhile.
Go to Agents and AutomationModule 1
How the internet reaches your process
Before you can fix a live site you need a picture of what happens between somebody typing your name and your code running. Six layers, each with its own way of failing. This comes first because every later module assumes you can say which layer you are looking at.
By the end you can
Trace one request from a typed URL to your running process and back, and name which layer is at fault when it fails
- 1What a server actually isA server is just a process holding a port open; the cloud only changes who owns the machine.
- 2The first ten minutes on a new machineA fresh machine is scanned within minutes, so keys-only login, a non-root user, a closed firewall and automatic security updates are not hardening you do later — they are the machine's first ten minutes.
- 3What keeps your process running when you close the laptopSomething other than your terminal has to own the process: a supervisor that starts it at boot and restarts it on exit, plus the journal that says why it exited — and a process that keeps vanishing without a log line was probably killed by the kernel for using too much memory.
- 4Why your site is not loadingNothing propagates; DNS answers are cached for their TTL, so lower the TTL before you change anything.
- 5What a status code is telling youThe first digit says who has to fix it; a status code that lies makes every layer above you lie too.
- 6The padlock, and what it does not proveA certificate proves control of a domain name and nothing else, and the way it kills your site is not being broken but quietly expiring.
- 7What sits in front of your appA reverse proxy takes the port, the certificate and the client's IP address away from your app, and every one of those has a default that will surprise you.
- 8One request, with a stopwatchMost of a first request is round trips you cannot code your way out of, so measure where the time actually goes before optimising the part you wrote.
- 9The handshake you keep paying forMost of a small request is handshake, so the wins are reuse — keep-alive in, a pooled client out, and an app idle timeout longer than the proxy's, or you will serve 502s nobody can explain.
Module 2
Making a deploy boring
A deploy should be a non-event: one command, the same artifact, and an undo you have already practised. This module builds that, including the two things that make deploys frightening — configuration you cannot see and schema changes under live traffic.
By the end you can
Ship a code change and a schema change to a live site without dropping a request, and undo either one in under two minutes
- 10The key you must not commitA leaked key is only fixed by rotating it; scrubbing git history does not un-leak anything.
- 11The same build, everywhereBuild one artifact from a locked dependency tree and move that exact thing between environments, because anything rebuilt per environment is a different program.
- 12Which commit is live right nowStamp the commit SHA into the artifact and serve it from a /version route, because "which code is running on the machine that answered this request" is the first question in every incident and the only one you can make free to answer.
- 13The checks that run without youA pipeline is worth having only if it is fast enough that people wait for it and honest enough that a red build always means something is broken.
- 14The console clicks nobody wrote downInfrastructure created by clicking is a design nobody wrote down; put it in files so you get a diff, a rebuild and a history, and treat the plan output as the review step where a replace means a new empty database.
- 15Deploying for nearly nothingBoth versions run during a deploy, so schema changes take three deploys, not one.
- 16Ending a process without dropping a requestA deploy stops a process as well as starting one, so catch SIGTERM, fail readiness first, drain with a deadline shorter than the platform's grace period, and check that your container is not swallowing the signal at PID 1.
- 17The copy you are allowed to breakA staging environment is worth having and will still lie to you, so make it cheap, make it disposable, and put a wall between it and anything that can reach a real person.
- 18Changing the schema under a running siteA schema change is safe when it never takes a long lock and never assumes only one version of the code is running, and both properties are things you arrange rather than hope for.
- 19Undo, in under two minutesCode rolls back, data does not, so keep them separate and make sure the undo you rely on is one you have actually performed while nothing was wrong.
Module 3
Knowing what it is doing
A live site is opaque unless you make it otherwise. Logs, metrics, errors, health checks, alerts and traces are five different answers to five different questions, and people conflate them. Here you learn which question each one answers and how much of it you actually need.
By the end you can
Answer "is it broken, for whom, and since when" in under five minutes using data you are already collecting
- 20Reading logs at 2amGive every request an ID and log it everywhere; that one habit turns guessing into searching.
- 21Where the logs go, and what they costLogs are a pipeline with a price per gigabyte, so log structured JSON to stdout, index only low-cardinality labels, sample the successful noise, and set a retention number on purpose rather than discovering one on an invoice.
- 22Four numbers that describe any serviceAverages hide exactly the requests you care about, so measure latency as percentiles from histograms and keep the number of distinct label values small.
- 23One screen that answers the questionA dashboard is a saved answer to a question somebody asks, so keep three screens — service health, dependencies, business counts — with thresholds drawn on and percentages rather than counts, and never mistake a screen for an alert.
- 24Errors that come to youAn error nobody sees is an error nobody fixes, so send exceptions somewhere that groups them, counts them and tells you which release they started in.
- 25The health check that liesA health check that always returns 200 tests only that your web server can serve a constant, and the check you rely on must run from outside the thing it is checking.
- 26Alerts you would get out of bed forEvery alert must pass two tests — a human is needed, and needed now — and an alert that fails either one is training you to ignore the ones that do not.
- 27Deciding how reliable it has to bePick a reliability target you can defend, express it as good events over valid events, and treat the remaining failure allowance as a budget you spend on shipping — with burn-rate alerts so one rule catches both the outage and the slow bleed.
- 28Finding the slow bitSlowness is nearly always waiting rather than computing, and a trace or a few timed log lines will show you what the request was waiting for.
Module 4
Load, cost and the traffic that is not customers
Everything in this module is the same problem seen from three sides: work arriving faster than you can do it. Caching removes work, capacity planning finds the ceiling, queues move work out of the request, and rate limits keep strangers from spending your money.
By the end you can
Find your first real ceiling with a load test and raise it using caching, pooling or a queue, without raising the bill
- 29The bill, and the traffic that runs it upAlerts tell you the money is gone; only caps, limits and caching stop it leaving.
- 30The request you never serveCaching converts traffic into nothing at all, and the two ways it goes wrong are serving one person's data to another and every copy expiring at the same instant.
- 31Serving from near the user, and what bandwidth costsA CDN wins twice — cached bytes never cross the ocean, and even uncacheable requests get a nearby handshake — and the number to watch is cache hit ratio, because egress metered at cents per gigabyte is what turns one large image into a bill.
- 32Finding your ceiling before your users doCapacity is measured, not guessed, and past the knee of the curve latency rises while throughput does not, so shedding load beats queueing it.
- 33The day you run two copiesRunning two copies breaks everything held inside one process — sessions, counters, uploads, caches and cron — so externalise that state rather than pinning users to an instance, and test it by restarting the process while logged in.
- 34It is almost always the databaseThe first ceiling is nearly always connections and the second is one missing index, and both are found with two queries you can run today.
- 35Work that does not belong in a requestMove anything slow or externally dependent out of the request into a queue, and design every job to be safe to run twice, because at some point it will be.
- 36The traffic that is not customersLimit per account and per endpoint rather than only per IP address, and put the tightest limits on whatever spends money per request.
- 37Files people send youUploads belong in object storage, written directly by the browser through a short-lived presigned URL with the key, type and size constrained — and everything the client tells you about the file, including its name and content type, is attacker-controlled until you check the bytes.
- 38Why your email went to spamTransactional email fails silently, and the three DNS records that stop it failing are the ones the major providers now check on every message.
Module 5
Staying alive: the bad day and the day after
The last module is about time. Things expire, data accumulates obligations, and eventually something breaks while people are using it. These are the habits that decide whether that is forty minutes or a weekend, and what is left behind for whoever runs this next.
By the end you can
Run the first thirty minutes of a real outage from a written runbook, and leave behind a postmortem and a handover that change what happens next time
- 39The restore you have never testedAn untested backup is a guess; the drill is the only thing that turns it into a plan.
- 40The short list that actually mattersSmall sites are compromised through leaked credentials, exposed services, missing authorisation checks and unpatched dependencies, so spend your effort there before anywhere else.
- 41Data you must keep, and data you must deleteEvery store of personal data is a liability with a lifetime, so decide the number for each one and enforce it with a job rather than a policy document.
- 42Everything expires, and nothing tells youMost preventable outages are something with a date on it that nobody owned, so keep one file listing what expires, when, and on whose card.
- 43Timeouts, retries, and the failure you made worseOne dependency without a timeout can take down every page you serve, and retries without backoff, jitter and a breaker turn a small fault into a storm you generated yourself.
- 44The outage that is not yoursWhen the broken thing belongs to somebody else, your work is to prove it quickly, degrade or serve stale rather than wait, and communicate from infrastructure that does not share the failure — and most small projects should buy cheap redundancy for stateless providers rather than an architecture they cannot operate.
- 45The first thirty minutes of an outageStop the bleeding before you understand the cause, because diagnosis works fine from logs and a rolled-back artifact while the fire does not need to keep burning.
- 46Writing down what happenedA postmortem is worth writing only if it changes something, so measure detection time, name systemic causes rather than people, and leave with fewer than four owned actions.
- 47Breaking it on purpose, on a TuesdayEvery recovery mechanism is a belief until somebody has used it deliberately while nothing was wrong, so rehearse restores, rollbacks, instance kills and dependency failures on a calm Tuesday and record how long each really took.
- 48Writing it down for whoever is nextFour short documents — how to run it, how to operate it, what expires, and why it is like this — are what turn a running site into something somebody else can keep alive.
Module 6
Shipping a model, not just a web app
A prediction endpoint is a web service with four extra problems: the weights are large, the latency is dominated by arithmetic rather than waiting, the cost is per token or per GPU-second, and the thing can get worse without anybody deploying anything. This module is the operations of machine learning — serving, cost, versioning, skew, drift and the fallback for when the model is not there.
By the end you can
Put a model behind an HTTP endpoint with a measured latency budget and a stated cost per request, ship a new version by shadow then canary rather than by hope, and detect the week its inputs stop resembling the data it was trained on
- 49A prediction endpoint is a web service with a large secretA model endpoint is an ordinary web service whose memory, start-up time, concurrency and per-request cost are one to three orders of magnitude larger, so load weights once, size workers against cores rather than against waiting, and push anything slow into a queue.
- 50Where inference time actually goesLanguage-model latency splits into a parallel prefill that sets time-to-first-token and a memory-bandwidth-bound decode that sets tokens per second, which is why smaller and quantised models are faster, why batching raises throughput almost for free, and why output length is the lever that matters most.
- 51What a prediction costs, worked outCompute cost per request from token prices or from GPU-hours divided by realistic utilisation, and the comparison settles itself: hosted APIs win while you are idle, your own hardware wins only at sustained volume, and caching plus a per-user quota is the lever you control either way.
- 52Which model, which prompt, which versionA model system has four independently moving versions — model snapshot, prompt, decoding parameters and retrieval index — so pin each one and record all four with every response, or no complaint about last Tuesday can be investigated.
- 53The feature computed two different waysMost deployed models fail because the features at serving time are computed by different code, from different data, at a different moment than the features they were trained on — so share one implementation and diff the two paths on real rows before anybody trusts the score.
- 54Shadow, then canary, then everybodyShip a model in three stages — shadow to find what breaks, canary on a user-hashed slice with automatic rollback, then a properly sized experiment — and keep a small holdback afterwards, because the artifact rolls back but the data it wrote does not.
- 55The model that quietly stopped fittingA model degrades without any deploy, so watch what you can see immediately — input distributions, missingness and the prediction distribution — collect a small stream of real labels every week, and make retraining a triggered decision that passes the same gate as any other model change.
- 56The model that trains on its own decisionsA deployed model selects, censors and generates the data it will later be trained on, so log impressions and interventions, mark machine-generated content at write time, keep a small randomised slice and an untouched human-labelled anchor set — or your metrics will measure agreement with your own past decisions.
- 57What the page says when the model is not thereModel calls fail slowly, partially and expensively, so set idle timeouts on streams, validate the shape of every answer, define one cheaper fallback per feature and write down in advance whether each gate fails open or closed.
Module 7
Security after the short list
The short list earlier in this course covers what compromises most small sites. This module is what comes after it, and it is mostly about authority: who may do what, which key proves it, what happens when a session or a key must be taken back, and what you do on the day something is genuinely wrong. It ends with the two situations nobody plans for — a breach, and a legal demand about one of your users.
By the end you can
Enforce authorisation per object rather than per page, revoke a session or rotate a key without an outage, and run the first day of a breach or an abuse report with evidence preserved and users told the truth
- 58Authentication is not authorisationBeing logged in is not permission to touch a particular row, so scope the query by the owner rather than checking the route, name each rule as a testable function, fail closed, and remember that an unguessable ID is a secret rather than an authorisation check.
- 59Sessions, and taking one backDesign sessions around revocation — an opaque ID looked up server-side revokes instantly, while a stateless token cannot until it expires — and rotate the ID on every privilege change, hash it at rest, and re-authenticate in front of dangerous actions.
- 60The code you did not writeYour dependencies run with your application's permissions, so install from a committed lockfile, delay adoption of brand-new versions, pin CI actions by commit hash and triage scanner output by whether the vulnerable path is reachable — and accept that a patient social attack defeats all of it.
- 61Changing a key while everything is using itRotate by overlapping — both keys valid, deploy the new one, watch the old one go quiet, disable before deleting — and give keys an identifier so you can tell who is still using the old one instead of guessing.
- 62Bots, scrapers and the flood you cannot absorbVolumetric floods are handled upstream by a CDN you did not write, but application-layer abuse is yours: cache the public pages, put quotas and challenges in front of the expensive ones, limit by account rather than address, and log every block because the people you wrongly refuse cannot tell you.
- 63Taking money without holding card numbersLet the processor collect the card so raw numbers never reach you, grant access on a signature-verified webhook rather than a redirect, deduplicate by event ID, and send an idempotency key on every charge so a dropped connection cannot bill somebody twice.
- 64The tools that can do anythingThe admin panel is unlimited authority with the fewest tests, so gate it with an explicit allowlist that answers 404, require a reason on every action, write an append-only audit log in the same transaction, and hold destructive powers as break-glass rather than by default.
- 65The first day of a security incidentIn a security incident, preserve evidence before you clean anything, contain narrowly and rotate credentials, establish scope from logs rather than assumptions, and find out which notification clock applies to you before the day you need to know.
- 66The email about one of your usersHave one monitored address, preserve the evidence before you act on any serious report, verify who is really asking and give only what is demanded, and learn which copyright and disclosure regime applies to you before the first notice arrives rather than during it.
Module 8
Running it for years
Everything so far gets a thing launched and keeps it alive through a bad week. This module is the longer horizon: what you chose to run yourself and what that costs in attention, the upgrades that arrive whether or not you want them, what a user costs you, how to move house, what the law expects of a public site, and how one person keeps this going without burning out.
By the end you can
Decide what to run yourself and what to buy using a cost per user and a maintenance budget you have actually calculated, keep a system upgraded and legally presentable over several years, and move it to another provider without losing a URL
- 67What to run yourself, and what to pay forCompare money against hours of attention, not money against money: buy the stateful things whose backup, restore and upgrade obligations would eat your weekends, run the stateless ones that recover by restarting, and check how you would leave before you join.
- 68The upgrade you keep postponingEnd-of-life dates are published years ahead and arrive anyway, so keep dependencies moving weekly, take one major upgrade a quarter in small steps, rehearse database upgrades because they do not roll back, and write down any version you are deliberately staying on.
- 69What one user costs youDivide the bill by active users, split it into fixed and variable, and price the one action that dominates the variable half — then test whether ten times the users would still be affordable, because the answer is nearly always one line item rather than the architecture.
- 70Moving to another provider without losing a URLMigrate in parallel and in order — lower the DNS TTL days ahead, replicate the data continuously, cut over at the quiet hour with the old system still alive, and serve 301s for every old URL — because the one cost you cannot recover is a link that stops working.
- 71The hour a week that keeps it aliveEverything that keeps a system healthy is recurring and forgettable, so put it in a calendar with intervals, one owner per item and a written note per run — and when you fall behind, restore a backup and review access before anything else.
- 72Removing a feature, and retiring a siteRemoval is a shipping skill with a sequence: count real usage, announce with a date, offer an export, dark-launch the switch-off, remove code before data, and answer old URLs with a 301 or an honest 410 rather than a redirect to the homepage.
- 73The pages every public site needsA public site needs terms, a truthful privacy policy written from your own schema, a reachable contact, visible rules and an honest tracking position — and the cleanest answer to consent banners is to set nothing that needs consent.
- 74Keeping it going without burning outAttention is the scarce resource in a small project, so decide in advance what is worth waking for, automate the frequent and write runbooks for the rare, refuse features you cannot maintain, and judge the system by how long it runs without you and how quickly you can understand it when you return.