How Software Development Changed
Writing code got cheap. Knowing what to build did not.
- Level
- Some background helps
- Lessons
- 87
- Reading time
- 789 min
- Price
- Free, no sign-up to read
A plain account of what actually shifted in how software gets built, for people who write code or manage people who do. Where the bottleneck moved, why plausible-but-wrong code is a new kind of failure, how review and testing have to change, why specifications got more valuable, and which parts of the job got harder rather than easier. Vendor-neutral, with the contested parts marked as contested.
Opens after the AI at Work exam
Sign in, finish that course, and pass its exam. You can read this syllabus meanwhile.
Go to AI at WorkModule 1
Where the cost actually went
The productivity claim, taken apart. What got cheaper, by how much, measured how, and why a faster writing stage does not make a faster team when the queue is somewhere else.
By the end you can
Account for a productivity claim in numbers — name which stage of the delivery pipeline got faster, compute what that does to end-to-end throughput when the constraint sits elsewhere, and read a study or a vendor figure well enough to say precisely what it does and does not establish
- 1The cost curve moved, but only part of itGenerating code got cheap; deciding what to build and confirming it is right did not.
- 2The fifth time code got cheapEvery wave of cheaper code has attacked the difficulty of expressing intent and left the difficulty of deciding what should happen untouched, and the current wave differs mainly in that it fails confidently rather than loudly.
- 3Cheaper software means more softwareMaking software cheap to produce expands what gets built rather than shrinking the work, but it expands the maintenance surface at the same rate, and maintenance is made of the activities that did not get cheaper.
- 4How to read a claim about developer productivityMatch the claim to the study design that produced it: a fixed-task trial can only speak about fixed tasks, self-reported savings measure belief rather than time, and a speed result with no defect measure is half a finding.
- 5Why it feels faster than it measuresFelt speed and measured speed diverge because waiting is not remembered as effort and debugging is charged to a different account, so decisions about how a team works need a clock and a defect count rather than an impression.
- 6Queues, not speedsThroughput is set by the slowest stage, waiting time explodes non-linearly as that stage approaches full utilisation, so a faster writing stage in front of a busy review stage lengthens delivery rather than shortening it.
- 7Where the gain is large, small, and negativeThe payoff from generated code tracks how cheap verification is, not how easy the code is, so the strongest use is work with a perfect oracle such as a refactor and the weakest is work whose difficulty was deciding what should happen.
- 8What the loop costs, and the free pathToken spend is only the visible cost of the loop, latency and attention are usually larger, and a free local model plus a strict type checker beats an expensive model with no oracles.
- 9Measuring the effect on yourselfA two-week self-experiment with coin-flip allocation, a merged-and-deployed end point and a thirty-day defect column will tell you more about your own practice than any published study, provided you write the decision rule before you see the data.
Module 2
Reading what you did not write
Review became the constraint and the most valuable skill on a team. How to orient in an unfamiliar system, size and triage a diff, read generated tests, and find the safety that quietly went missing.
By the end you can
Review a change whose author may not have read it closely — orient in an unfamiliar system fast, triage a diff by risk and cap it at a size where defect detection still works, judge tests and dependency changes on their own terms, and check the change against the stated intent rather than against its own internal consistency
- 10Reviewing code you did not writeYou own every line you submit, whoever typed it; if you cannot explain it in review, it is not ready.
- 11The new failure mode: plausible and wrongCode that looks right is no longer evidence that it is right; polish and correctness came apart.
- 12Orienting in a system you have never seenUnderstand an unfamiliar system by reading its schema, tracing one request end to end and asking git which files really change, because the data model and the commit history are the two artefacts that cannot lie about what the system does.
- 13Spending a review budgetReview is a budget, so allocate it by what each file can cost rather than evenly across the diff, and record what you actually read so an approval carries a specific meaning rather than an implied one.
- 14Why the diff has to be smallDefect detection per line falls as a review grows, because past a certain size a reviewer silently switches from comparing before and after states to pattern-matching on how the code looks, which is exactly the check generated code passes trivially.
- 15Reading a test suite you did not writeA test's strength lies in where its expected value came from, so a suite whose assertions restate the implementation will pass through every bug and break on every refactor, and the cheapest way to find out is to break the code on purpose and watch.
- 16Safety that quietly went missingRemoved checks are the highest-risk lines in a diff and the ones human attention least reliably reaches, so the fix is a mechanical grep over deletions plus a norm that every removed guard is explained in the description.
- 17Reviewing a dependency changeA one-line dependency addition is an unbounded commitment to run other people's code with your permissions, so it earns the two-minute check — existence, publisher, transitive count, install scripts, licence, maintenance — that no other line in a diff needs.
- 18The second job of review, and how it brokeReview transferred understanding only because both people had thought hard about the change, so when the author has not, the knowledge-transfer half of review disappears silently and reappears as a system nobody can explain.
- 19Letting a machine help you reviewDelegate review questions you can verify in under a minute and refuse the ones you cannot, and configure any review bot for precision over coverage, because a checker that is usually wrong trains people to dismiss the one time it is right.
Module 3
Building oracles that stay ahead
If producing code is cheap, the scarce asset is anything that can tell you the code is wrong. Properties, differential runs, strict types, database constraints, mutation checks and staged release — the machinery of knowing.
By the end you can
Assemble a layered set of oracles for a change — properties and differential runs where behaviour must be preserved, types and database constraints that run for ever at no cost, mutation checks to prove the suite bites, and a staged release that can stop itself — and say for any change what would have to be true for it to be wrong and which layer would catch it
- 20Testing when generating is freeA test only counts if you have seen it fail for the right reason.
- 21Properties, not examplesA property states what must be true of every input and lets the machine hunt for a counterexample, which is cheap for you to write and expensive for pattern-matched code to satisfy by accident.
- 22The old version is your best oracleWhen the requirement is that behaviour must not change, the previous version is a perfect oracle, so refactors and migrations should be verified by running both implementations over real traffic rather than by reading the diff.
- 23The type checker got more valuableA type checker is the only reviewer whose capacity scales with the amount of code, and it earns most when types carry meaning rather than shape, so identifiers, units and currencies become mistakes the compiler refuses.
- 24The database is the last honest gateA database constraint enforces a rule against every writer that will ever exist, so it converts a class of bug from unlikely to impossible for the price of one line of DDL and a validation scan.
- 25Proving the suite bitesMutation testing measures whether a suite would notice if the code were wrong, which is the question coverage cannot answer, and the informative comparison is a module with high coverage and a low mutation score.
- 26Inputs designed to hurtKeep a committed corpus of hostile inputs and fuzz anything that parses, remembering that a fuzzer finds crashes rather than wrong answers, so it layers with properties and differential runs rather than replacing them.
- 27A suite that lies intermittentlyFlake probability compounds across a suite, so a small per-test failure rate makes a red build meaningless, and the fix is automatic detection, a capped quarantine with a deletion deadline, and never a blanket retry.
- 28Designing a pipeline that is actually a gateA pipeline is only a gate if it is fast enough that nobody routes around it and tests the merge result rather than the branch, so blocking checks must be limited to what is deterministic, quick and unambiguously wrong.
- 29Verification that continues after mergeProduction is the last and most informative oracle, so design releases to be reversible and pick the halt metric and canary duration from your actual traffic rate rather than by feel.
Module 4
Saying what you want
Ambiguity used to stop you while typing. Now it gets resolved silently, in some direction. Writing the decision down became a technical skill, and the same page can be the prompt, the review criterion and the test names.
By the end you can
Turn a vague request into a written specification that serves as prompt, review criterion and test names at once — enumerate the ambiguities a competent stranger could not resolve, state the out-of-scope boundary, record the decisions that have no technically correct answer, and keep the whole thing in the repository so it cannot drift from the code
- 30Why writing it down matters more nowAmbiguity used to stop you while typing; now it gets resolved for you, silently, in some direction.
- 31The ambiguity auditRun a request against a fixed list of ambiguity categories — matching, ordering, emptiness, permission, time, language, failure — because the questions you do not ask get answered anyway, silently, by whatever produces the code.
- 32Criteria that become test namesReplace every adjective in a requirement with an observation somebody could make, and write criteria as worked examples with real values, because a specification's expected values are the only ones that come from outside the code.
- 33Writing down the whyA decision record earns its place through the alternatives it rejects and the costs it admits, and a one-line comment linking code to its record is what stops a deliberate oddity from being helpfully corrected later.
- 34What the tool can actually know about your codebaseA model reasons only over the text in its window, so it invents helpers it cannot see, and the fix is naming specific files plus a short honest conventions file rather than expecting retrieval to find the constraint that is worded differently.
- 35Design the seam, generate the fillingInterfaces are commitments and implementations are disposable, so spend your thinking on the signature, the types and the documented contract, then let the body be generated against tests that came from the contract rather than from the code.
- 36Bounding what a change is allowed to touchA change with no stated boundary expands to the complete version of what the tool recognises, so name what is out of scope, forbid new dependencies by default, and instruct it to stop and ask rather than route around an obstacle.
- 37Examples beat adjectivesAn example fixes the properties you would not have thought to name, so replace adjectives with worked cases and domain vocabulary — while varying the examples along dimensions that do not matter, or they will be copied too.
- 38Keeping the written thing trueWritten specifications stay true through two mechanisms only — living in the same commit as the code and being checked by something that fails — so make what you can executable, keep decisions immutable, and delete prose you are not maintaining.
- 39When writing it down is the wrong moveWrite down what would be expensive or invisible to get wrong, and nothing else — a specification's value lies entirely in the decisions a human made in it, so a long generated document with no choices in it is not a specification at all.
Module 5
The working loop
What the day actually looks like: small reversible steps, git as a safety net, sandboxed agents whose diffs you read, debugging that still works the old way, and knowing when to stop arguing and start again.
By the end you can
Run a change through a loop that keeps the cost of being wrong at fifteen minutes — steps small enough to throw away, git used so that reversal is instant, an agent confined so a mistake cannot reach production or your credentials, and an explicit rule for abandoning a conversation that has gone wrong twice
- 40Pairing with a model on real workWork in steps you can run and throw away; speed only helps when reversing it is cheap.
- 41Git is what makes speed safeSmall reversible steps are only cheap if reversal is instant, so commit constantly, stage with `git add -p`, keep a clean tree before letting any tool edit files, and remember that reflog holds almost everything you think you destroyed.
- 42Confining an agentAn unattended agent inherits everything you can read and run, and text it reads can act as instruction, so confinement must be a real boundary — a container without your credentials — with a permission list narrow enough that a prompt still means something.
- 43Debugging still works the old wayDebugging held its value because the evidence lives in the running system rather than in any text, so the method stays reproduce, minimise, bisect, hypothesise and instrument, and a suggested cause is a hypothesis to test rather than a diagnosis.
- 44The task it is best atMechanical change is the safest use of generation because the old version is a perfect oracle, so keep behaviour-preserving work in its own verified commit and prefer a deterministic pattern tool wherever the change is a pattern.
- 45Shipping in a language you do not knowGeneration lets you ship in an unfamiliar language but not maintain or debug in one, and what still bites is the language's error-handling norms, concurrency model and tooling — so turn the strictest linter on first and get one fluent human to read the first thing you ship.
- 46When to put the tool downGeneration serves output and undermines understanding, so name which one you are after before you start — and schedule unassisted practice deliberately, because the fast review judgement everything now depends on is built only by having written the code yourself.
- 47Knowing when to start againA conversation that has been wrong twice is a context containing the wrong approach three times, so restart with what the failure taught you rather than making a third correction, and time-box the decision because judgement is compromised exactly when it is needed.
- 48A complete free workflowA completely free stack — strict type checker, tests, property testing, git, a quantised local coding model and a free pipeline — covers most of the work, and the tasks where a small local model falls short are largely the tasks you should be doing yourself anyway.
- 49Signs the loop has gone badThe dangerous session is the one that feels productive for two hours, so watch for the diff growing unread, nothing being run, tests moving to match code, and suppressions appearing — and go back to the last commit that worked rather than pushing on.
Module 6
Teams, process and delivery
What a manager or a tech lead has to change. Measuring delivery rather than activity, planning verification as work, capping the queue, defending architectural coherence, and writing a tool policy people will follow.
By the end you can
Change a team's process to match the new constraint — measure delivery with the four keys rather than activity counts, cap work in progress at the bottleneck, put verification in the plan with a named owner and real hours, schedule consolidation, defend coherence with automated fitness functions, and write a one-page tool policy that survives a deadline
- 50The parts that got harderMore code with unchanged verification capacity is not more throughput; it is a growing queue.
- 51Four numbers that resist gamingMeasure at the end of the pipeline with deployment frequency, lead time, change failure rate and recovery time, because those four interlock so that cheating on one degrades another, while every activity count rises with tool use regardless of what is delivered.
- 52Why everything looks nearly doneThe demo now arrives on day one and completion takes as long as it ever did, so estimate by counting unresolved decisions against a reference class of past work, state the estimate to deployed rather than to demo, and put the edge cases on the board before anybody sees it working.
- 53Capping the queueCapping work in progress moves effort to the constraint automatically and cuts time in system without changing throughput, and the argument that persuades managers is Little's Law computed from the team's own numbers.
- 54Putting verification in the planVerification is work, so it belongs in the plan with an oracle, a named person and hours rather than being the residue of whatever time is left, and the cheapest version is one sentence naming who confirms what by which method.
- 55Scheduling the deletionAdding code is easy and locally justified while deleting requires understanding the whole system, so consolidation must be scheduled rather than hoped for, and the argument that works with management is incident arithmetic rather than elegance.
- 56Coherence has to be defended by somethingNo individual diff is where architectural drift gets decided, so coherence needs a written page of how this codebase does recurring things plus an automated fitness function that fails the build, while the genuinely architectural bets stay with people.
- 57Onboarding into code nobody wroteOnboarding used to run on an author's memory, so where that memory does not exist it must be replaced by verifiable exercises — a request trace, a schema drawn from scratch, a reverse review — and by tracking how many people can explain each service.
- 58Operating a system nobody wrote by handWhen nobody remembers writing the code, the first question at 3am becomes what changed in the last day, so build the change timeline and stamp every log line with the deploy version — and convert every incident into one permanent test in the same week.
- 59A policy people will actually followA workable tool policy is one page with an approved list, a specific data rule that names a permitted alternative, the author owning every line, review scrutiny tiered by risk, and an explicit ban on measuring individuals by activity counts.
Module 7
Skill, hiring and the missing rung
How competence actually forms, why generated code short-circuits the loop that forms it, what a team can do to rebuild the first rung of the ladder on purpose, and how to hire when every candidate has the same tools open.
By the end you can
Diagnose where skill formation has been short-circuited and design around it — separate fluency from understanding with a test you can run in five minutes, name the parts of the craft whose feedback loop a generator cannot close, construct an apprenticeship task that is still expensive to generate, and run a hiring loop that discriminates between candidates who all have the same tools open
- 60Juniors, and the rung that went missingThe work that used to make people senior is the work that got cheapest, so the rung has to be rebuilt on purpose.
- 61Why the hard part was the part that taught youLearning comes from retrieval under difficulty, and reading generated code substitutes recognition for retrieval while producing the identical feeling of understanding, so the internal signal that used to indicate learning now tracks fluency instead.
- 62A five-minute test for whether you understood itConfidence tracks how fluently something reads rather than whether you could reproduce it, so the only reliable check is to close the file and attempt the explanation, and the cheapest version of that is predicting the branches before you generate.
- 63The layer that does not get cheaperInvest study time where a wrong answer fails silently or late, because loud immediate failures are corrected by the loop itself while silent ones need a person who already knows.
- 64Practising when the answer is one keystroke awayDesign practice that pays off immediately as well as eventually — predicting the branches before generating catches bugs today and forces retrieval at the same time, which is why it survives a deadline when abstinence does not.
- 65Tasks that are still expensive to generateApprenticeship work now has to be selected for tasks where deciding what to do is the expensive part — diagnosis, migration, incident write-ups, ambiguous requests — because the tasks that were cheap to get wrong are exactly the ones generation made free.
- 66Hiring when everyone has the same tools openRun the exercise with the tools on and score the process around them — whether the candidate runs it, reads the error, rejects an output, checks the documentation — because what became scarce is reading and judgment, not recall of syntax.
- 67What a repository proves, and what it stopped provingThe finished artefact stopped carrying information, so the signal moved to things somebody else gated or that record the process — a merged pull request, a minimal reproduction, a review you left, a post-mortem, a commit history that shows a reversal.
- 68Where the work moved, and what is genuinely unknownWork sold as transcription lost its price while work that decides what should exist and confirms it is right did not, and the employment data from 2023 onward is too confounded by interest rates and post-2021 over-hiring to attribute the junior-hiring fall to any single cause.
Module 8
Provenance, security and the unsettled law
The attack surface that opened when code started arriving faster than anybody could read it — invented package names, insecure defaults inherited from old public code, secrets leaving in context, instructions hidden in data an agent reads — and the licensing and copyright questions that remain genuinely unresolved.
By the end you can
Secure a change whose author may have been a model — verify a dependency before installation rather than after, name the vulnerability classes that arrive by default and the one class no scanner can find, break the lethal trifecta so an injected instruction cannot both reach secrets and send them anywhere, and state the shape of the copyright disagreement across jurisdictions without pretending it is settled
- 69The package that did not exist until an attacker made itInvented package names repeat across runs, which turns a hallucination into a predictable target an attacker can pre-register, so the defence is verifying a package exists and has history before installing rather than trusting that a failed install will catch it.
- 70Insecure defaults, inheritedGenerated code inherits the insecure defaults of old public code, and the one class that no scanner can catch is missing authorisation, because who may touch which object is a business rule that exists in nobody's training corpus.
- 71What leaves the building when you typeRetention and training are separate questions answered differently by consumer and business tiers, and since you cannot verify either claim, the only control you actually hold is minimising what goes into context in the first place.
- 72Instructions hidden in the things an agent readsA model cannot distinguish instructions from data, so safety comes from architecture rather than wording: break the combination of private data access, untrusted content and an outbound channel, because any two of the three are survivable and all three are an exfiltration path.
- 73Scanners, and the trick that makes them usableScan for the mechanical vulnerability classes and block only on findings the change introduced, because a baseline ratchet is the difference between a tool people act on and a wall of pre-existing findings that trains everyone to ignore alerts.
- 74Four questions before you mergeFour questions per diff — where untrusted input enters, what privilege the code holds, which line checks permission, and what an attacker gains if it is wrong — direct attention at exactly the classes no scanner can reach.
- 75Knowing what you actually shippedComponent inventory and pinning are solvable and pay off on the morning of an advisory, while identifying which lines were generated is not solvable at all — watermarks do not survive code's constraints and detectors misfire on clean, conventional writing.
- 76When the output looks like something elseVerbatim reproduction concentrates in distinctive, heavily duplicated fragments rather than ordinary application code, so the practical controls are duplication filters, similarity scanning before release, and never asking a model to reimplement a specific named project.
- 77The disagreement, laid out without a verdictTraining and output raise separate unresolved questions decided differently under US fair use, the EU's opt-out TDM exception, the UK's research-only exception and regimes with no exception at all, so the only sound planning rule is to avoid depending on a particular ruling.
- 78The commons under a flood of free submissionsProducing a plausible report costs seconds while disproving one costs a maintainer half an hour, so sustainable policy filters on evidence — a reproduction, passing tests, the ability to answer a question — rather than on guessing how a submission was written.
Module 9
What did not change, and what to do now
The constraints that survived every previous cost collapse and survived this one — essential complexity, irreversible decisions, domain knowledge, the ten-year cost of code — what history says about automations that grew a job and ones that ended it, an honest list of the questions still open, and a plan for the next month.
By the end you can
Separate what got cheaper from what did not — identify which decisions in a design are one-way doors worth slowing down for, estimate a change by its ten-year cost rather than its writing cost, state what it would cost to leave a tool before adopting it, and name which questions in this field are genuinely open along with the evidence that would settle each one
- 79The distinction that keeps being rightGeneration attacks the accidental difficulty of expressing a decision and leaves the essential difficulty of making one, which is why identical tools produce large gains on representation-dominated work and none where requirements, domain or organisational agreement are the constraint.
- 80Decisions that do not swing backReversible decisions got much cheaper and irreversible ones did not move at all, so deliberation should now concentrate on the few choices — interface shape, key scheme, time representation, tenancy, what you collect — whose reversal costs weeks rather than an afternoon.
- 81The facts that exist in no corpusThe rules that decide whether software is correct live in regulations, in practitioners' heads and in the behaviour of the system being replaced, so acquiring them deliberately is what converts fast generation from plausible output into correct output.
- 82Writing cost, and the cost that follows itGeneration cut the cost of writing code and not the cost of keeping it, while plausibly raising the second through volume, duplicated concepts and absent reasoning — so review and design should optimise for how few distinct concepts a system contains rather than for how fast each part arrived.
- 83Ask the exit question before you enterLock-in accumulates in prompts, evaluation suites and habits rather than in the model itself, so keeping instructions and evaluations as ordinary files in your repository and building against an interface several providers implement keeps switching to a configuration change.
- 84Doing this work on the machine you ownA phone with a terminal plus a borrowed machine covers nearly all of this work, and the honest hardware floor is a 4-bit 7B model at roughly 4 to 5 GB of memory, with training, large models and Apple-platform development being the things that genuinely require hardware you may not have.
- 85What happened the last few timesAutomations have both grown and destroyed occupations, and the outcome turned on whether cheaper supply expanded demand and whether the remaining tasks still needed those workers — so the actionable question is which of your own tasks are the automated part and which are the residue.
- 86What this course could not tell youThe practical recommendations in this field rest on a small set of established mechanisms, while the headline empirical questions — productivity, skill, employment, law, maintainability — remain genuinely open, so advice that depends on those is advice that expires.
- 87The next month, in orderMake reversal instant first, add the always-on checkers second, and only then change how you work — because every practice worth adopting assumes that throwing away fifteen minutes of work is free and that something mechanical is watching while you read.