Agents and Automation
What an agent actually is, and when you should build a script instead.
- Level
- Some background helps
- Lessons
- 72
- Reading time
- 633 min
- Price
- Free, no sign-up to read
Past the hype: what an agent actually is, how the loop works, why most agent demos collapse in production, and how to build automation that survives contact with real data. Covers no-code tools (n8n, Zapier, Make) and code equally, and is honest about when a plain script is the better answer.
Opens after the Context Engineering exam
Sign in, finish that course, and pass its exam. You can read this syllabus meanwhile.
Go to Context EngineeringModule 1
Before you build: script, workflow, or agent
Most requests that arrive labelled 'we need an AI agent' are a scheduler and thirty lines of code. This block teaches you to take the request apart, price it, and pick the cheapest machine that does the job. It comes first because the most expensive mistake in this subject is building the wrong shape.
By the end you can
Given any automation request, choose between a script, a fixed workflow and an agent, and justify the choice with that shape's per-run cost, failure rate and recovery story.
- 1Three machines, one wordAn agent is a program where the model, not the author, decides what happens next.
- 2The boring stack that already does thisMost automation requests are a scheduler and thirty lines of code; put a model only in the one step where the input is genuinely unstructured.
- 3Watch the process before you automate itThe process on paper is not the one that runs, and the ceiling on any automation is the share of elapsed time held by the steps it actually replaces.
- 4Which work is worth automatingAutomate high-volume, low-variance work whose errors are cheap and visible; the rest costs more in maintenance and supervision than it returns.
- 5What a wrong answer costs, and who finds itClassify work by the cost of an error and by whether anyone would notice it; the expensive-and-invisible quadrant needs a detector built before the automation, not after.
- 6Why the demo worked and production did notPer-step accuracy compounds, so shortening the chain usually beats upgrading the model.
- 7What one run costs, and what it is worthPrice one run in tokens, seconds and the human minutes it replaces before you build it: the transcript is re-sent every step, so the fifth step is cheap and the twentieth is not.
- 8Buy it, build it, or wait for itSearch the tools you already pay for before designing anything, and remember that an API call is about five per cent of the work of an integration.
- 9n8n, Zapier, Make, or a file of PythonVisual tools win on connectors and time to first run; code wins the moment you need a loop with a condition, a code review, or a rollback.
Module 2
The machinery: transcript, tools, loop
Open the box. An agent is a list of messages, a set of tool definitions and a while loop, and everything that makes one good or bad happens in code you own. This block comes second because you cannot debug, budget or secure a machine whose moving parts you have never seen.
By the end you can
Trace any agent run from its message list — tool definitions going in, tool results coming back — and explain from the transcript alone why the model chose each step.
- 10A tool call is a request, not an actionA tool call is a request your code fulfils, and the result string is the model's only feedback.
- 11A schema constrains the shape, not the truthConstrained decoding guarantees the output parses, and nothing else — a required field the model has no evidence for will be invented, because the sampler is not allowed to leave it out.
- 12What the model actually sees each turnAn agent's entire state is a list of messages you own and can rewrite; there is no session and no hidden memory between calls.
- 13The instruction block that has to survive forty turnsA system prompt is text competing with a growing transcript, so anything that must hold on turn forty belongs in the harness or in the tool description, not in a rule at the top.
- 14Plan, act, observe, repeatThe loop is trivial; deciding what goes back in as the observation is the entire craft.
- 15Plan first, or just start?A plan written into the transcript becomes something the model has to stay consistent with, so keep the plan as harness-owned state that is re-derived when evidence contradicts it.
- 16Tools that survive being called by a machineDesign tools for a caller that retries, invents arguments and never reads the documentation: idempotent, paginated, timed out, enumerated, and safe to dry run.
- 17The observation is where runs dieShrink the observation at its source, and when you do cut, say in the text that you cut — silent truncation makes the model confident about a subset.
- 18What to throw away, and whenCost grows with the square of the run length because the whole transcript is re-sent every turn, so the harness must keep a working set rather than a history — and never mutate the cached prefix.
- 19Memory is a directory, not a mysteryMemory that works is a file you can open and a table you can edit; a vector store of everything the agent ever saw is unauditable, undeletable and usually unhelpful.
- 20When a second agent helps, and when it just costs moreA subagent buys you a separate context window and nothing else; pay for it only when isolation or genuine parallelism is the problem you have.
Module 3
The data underneath: sources, documents, records
Every automation that survives contact with a real business spends most of its code on data rather than on models: a supplier who changed their invoice layout, three spellings of the same customer, a date that means two different days depending on who typed it. This block is the plumbing — reading real sources, extracting fields you can defend, matching records, giving the agent something searchable, and writing back into a system other people depend on.
By the end you can
Take a messy real source — scanned invoices, an inbox, a spreadsheet a team edits by hand — and build the layer beneath the agent: parsed documents, field extraction with checkable provenance, normalised values, matched records, retrieval the agent can search, and writes that cannot duplicate a row or silently overwrite a person’s edit.
- 21Files, inboxes, sheets and APIsLand the raw source untouched before you transform anything, and remember that an incremental sync on a timestamp will never show you a deletion.
- 22PDFs, scans and the layout problemA PDF has no tables and no reading order, only positioned glyphs — so check first whether there is a text layer at all, because that single fact decides your entire pipeline.
- 23Turning a document into a row you can defendMake the model quote its evidence and check the quote appears verbatim in the source, which converts the dangerous failure — a confident wrong value — into the safe one, a blank field.
- 24The boring bugs that eat automationsMoney in floating point, a date without a locale and a name split into first and last are the three defects that survive every review and corrupt data quietly for months.
- 25The same customer, three timesMost duplicate problems are solved by normalising a key rather than by fuzzy matching, and a wrong merge is far more expensive than a surviving duplicate because merges are hard to undo.
- 26Grep first, embeddings laterAn agent that can search repeatedly needs a good search tool more than it needs a clever index, and lexical search beats vector search on exactly the identifiers that matter most.
- 27What an embedding actually isAn embedding is a list of numbers whose only meaning is distance to other numbers from the same model, so a similarity score is not comparable across models and changing model means re-encoding everything.
- 28Writing into a system somebody depends onUniqueness must be enforced by a database constraint rather than by checking first and inserting after, because between the check and the insert another attempt can succeed.
Module 4
Surviving real data
Everything so far works on the happy path. This block is about the other path: how runs actually fail, how to see it in a trace, and the four fixes that turn a demo into something that can be left alone overnight. It is the longest block for a reason — reliability is where the work lives.
By the end you can
Diagnose a failing agent run from its trace and apply the specific fix it needs: a retry policy, a validator, a shorter chain, or no loop at all.
- 29The six ways a run goes wrongEvery agent failure mode is invisible in the final message, which is why the final message is the last thing you should judge a run by.
- 30You cannot debug what you did not recordA histogram of steps per run is the most informative agent metric there is: a spike at the step limit means a slice of your runs were cut off, and their final messages will look perfectly normal.
- 31Reading a run, step by stepAsk three questions of every trace, in order: did it see the right thing, was the next step reasonable given only that, and did the harness do what was asked.
- 32Budgets belong in the harness, not the promptAnything you need guaranteed goes in the harness; a prompt can only ask.
- 33When the loop should be a graphSplitting an agent into named states lets each state carry only the tools it needs, which is real containment rather than an instruction not to use the dangerous one.
- 34Retries, timeouts and doing it twiceRetry only what is transient, back off with jitter, and generate the idempotency key once before the first attempt — a new key per attempt is simply a new request.
- 35The validator is the productCheck the output in code before anyone sees it: schema, then arithmetic, then every number and identifier traced back to a tool result in the same run.
- 36Testing something that is not deterministicRecord tool results once and replay them: with the tools held fixed, almost the whole harness becomes an ordinary deterministic test.
- 37Thirty tasks you run before every changeScore an agent on the state of the world after the run rather than on what it said, and compare per-task results, because a three-point move on forty tasks is about one task.
Module 5
Permissions, injection and blast radius
An agent can do exactly what its credentials allow, for whoever can write into its inputs. This block draws the boundary: what it may touch, what a stranger's text could make it do, what runs in a sandbox, and which actions stop for a person. It comes after reliability because you cannot contain a system you cannot yet observe.
By the end you can
Draw the permission boundary for an automation: name what it may touch, what somebody writing into its inputs could make it do, and which actions must stop for a person.
- 38The email that tells your agent what to doAn agent with private data, untrusted input and an outbound channel will eventually do what a stranger wrote; remove one of the three rather than adding a rule to the prompt.
- 39The web is the most hostile input there isA browser agent reads the DOM or the accessibility tree, so text a human cannot see is text the agent reads as plainly as the headline.
- 40Give it the smallest key that worksAn agent can do exactly what its credentials allow; scope the database role, the token, the channel and the network egress before you touch the prompt.
- 41What leaves the buildingSending fewer fields is a control you can verify; stripping names out of free text is a reduction in exposure that should never be described as anonymisation.
- 42Running code the model wroteThere is no in-process Python sandbox; isolate at the container or virtual-machine boundary, and turn the network off by default.
- 43The agent gets its own accountA tool that takes an arbitrary URL and headers hands the model your credentials, because anything that can shape a request can send the key somewhere else.
- 44Where the person goesGate on what cannot be undone, never on how confident the model sounds.
- 45Money, messages and deletionsAnything you cannot undo gets a dry run, a hard numeric limit in code, and a way to stop the whole thing inside thirty seconds.
- 46Telling people, and keeping the receiptsRecord the run, the prompt version and the pinned model id with every action, and name the person accountable — 'the system decided' is not an answer anyone accepts.
Module 6
Eight patterns that keep working
Almost all automation that survives in production is one of a small number of shapes. Each has known parts, a characteristic way of failing and a mechanical check that catches it. Learning the shapes saves you from designing every system from first principles, and more usefully it stops you building an agent where a classifier and a queue would have done the job at a fiftieth of the cost.
By the end you can
Identify which recurring shape an automation request is, assemble it from that shape’s known parts, and state in advance how it will fail and which mechanical check catches that failure before a person sees the output.
- 47Pattern: one call, one label, then codeTwo people labelling the same hundred items set the ceiling for any classifier, so measure human agreement before you measure the model.
- 48Pattern: it writes, a person sendsA fluent draft raises the chance a reviewer approves something they would not have written, so the interface must make the reviewer do a task rather than a glance.
- 49Pattern: sort, enrich, escalateAttaching the relevant evidence to an item is more valuable and far safer than deciding the item, and the human’s final disposition is a free training label you should be capturing.
- 50Pattern: search, read, cite — and verify the citationA research agent’s output is prose with real-looking sources, so the only defence that scales is checking mechanically that every quoted span exists in the page it is attributed to.
- 51Pattern: watch something, say when it changesA monitor that dies is indistinguishable from a world where nothing is happening, so the first thing to monitor is that the monitor ran.
- 52Pattern: forty thousand rows, onceSample randomly and stratify before a backfill, because the first two hundred rows are almost always the oldest or the tidiest, and they will tell you the job is safer than it is.
- 53Pattern: the agent that writes and runs codeCoding agents work because the feedback signal is objective and cheap — and they fail when the agent can edit the thing producing that signal.
- 54Joining two automations without joining their failuresThe interface between two stages should be a queue — even if the queue is just a status column — so that a slow or broken stage produces a visible backlog instead of failures attributed to the wrong place.
Module 7
Shipping it, and keeping it alive
The last block is the one most courses skip: how the thing starts, where it runs, who hears about it when it breaks, and what it costs to still be working in a year. It ends with building one real automation end to end, because everything before this was preparation for that.
By the end you can
Ship one automation end to end — trigger, harness, tools, validator, alert — and write the runbook that lets somebody else fix it at 3am.
- 55MCP, and why a standard for tools mattersA tool standard removes integration work; it does nothing about whether the model chooses tools well.
- 56Writing a small MCP serverAn MCP server is a small process answering JSON-RPC over a pipe; the docstring is the interface, and anything printed to standard output breaks the protocol.
- 57How the thing startsAcknowledge webhooks fast and deduplicate on the provider's event id; when you poll, advance a cursor over the data, never over the clock.
- 58Where it runs when your laptop is shutPick the runner by run duration, state and who else can restart it — and remember cron gets almost no environment, so a variable exported in your shell profile is not there.
- 59Rate limits, workers and the thundering herdFor agents the tokens-per-minute limit usually binds long before the requests-per-minute one, because a twenty-turn transcript is enormous compared with a single chat message.
- 60The model you tested is not the model you deployedLog the model id the provider returns rather than the one you sent, because an alias resolves to a different version over time and that difference is the explanation you will be looking for.
- 61Shadow, canary, and the off switchAssign a canary by hashing a stable identifier rather than rolling a die per request, or the same item will be handled by both versions and the comparison becomes meaningless.
- 62The 3am versionAlert on the job not running and on outputs that look wrong, not on exceptions — and keep a six-line runbook next to the code.
- 63What it costs to keep it aliveAutomation is a subscription paid in attention: review volume, error rate, spend and readership quarterly, and delete what nobody reads.
- 64Build the boring version firstMost automation that survives is a deterministic pipeline with one model call and a real validator.
Module 8
The year after it ships
Every course on this subject ends at the deploy. The year afterwards is where automations are actually won and lost: who owns it, whether it delivered what was claimed, what it costs, whether the people it was built for use it or quietly work around it, what you owe them in disclosure, what the rules require in a field where they differ by country and are still moving, what the system kept and must forget, and how to switch it off when its time has come.
By the end you can
Take responsibility for a live automation over twelve months — name its owner and on-call path, prove its value against an honest counterfactual, hold its cost to a stated budget, detect when people are routing around it, meet disclosure and record-keeping duties without giving or taking legal advice, and decommission it cleanly when it is no longer worth running.
- 65Whose is it, at 3am, in eight monthsAn automation with an active credential and no named owner is the worst state a system can be in: it still acts, nobody watches, and the first sign of trouble is the damage.
- 66Did it actually help, and how would you knowHours saved is the weakest claim available, because a saved hour only becomes value if you can say what it turned into.
- 67What it costs, and where the cost hidesTranscript growth dominates the bill, so the same task done in twice as many steps costs about four times as much — which is why shortening the chain beats every other optimisation.
- 68When people quietly work around itRemoving a manual step also removes whatever incidental checking that step provided, and the errors it used to catch reappear somewhere nobody is looking.
- 69Saying so, and the tools that cannot prove itText detectors are unreliable and flag non-native writers disproportionately, so disclosure has to be a record you keep rather than a property anyone can test afterwards.
- 70The rules, where they agree and where they do notBuild the same four capabilities regardless of jurisdiction — records, human override, an explanation, and deletion — because they satisfy most regimes and nothing else you build transfers as well.
- 71Retention: the duty to keep and the duty to deleteKeep a decision record rather than the full payload, so the duty to explain what happened stops competing with the duty to delete personal data.
- 72Switching it off, on purposeRevoke the credentials before you delete the code, because once the code is gone nobody remembers the key existed and it stays valid for years.