Context Engineering
The window is a budget. Learn to spend it.
- Level
- Assumes you have built something
- Lessons
- 87
- Reading time
- 757 min
- Price
- Free, no sign-up to read
For people who have already written the clever prompt and hit its ceiling. This course treats the context window as an engineering surface: what you put in it, in what order, from where, at what cost, and how you prove a change was an improvement. Nine lessons covering token budgets, the system prompt boundary, forced schemas, retrieval against long context, chunking, prefix caching, positional attention, untrusted input, and shipping prompts you can version and roll back. Vendor-neutral, with real numbers.
Opens after the Data, SQL and Getting to the Answer exam
Sign in, finish that course, and pass its exam. You can read this syllabus meanwhile.
Go to Data, SQL and Getting to the AnswerModule 1
The window as an engineering surface
What context engineering is, what is actually inside a request, how to count it, and what happens when it does not fit.
By the end you can
Account for every token in a request — system prompt, tool definitions, history, retrieved material and output reserve — count them with the tokeniser the provider actually uses, and predict from that budget what the call will cost, how long its first token will take, and which part gets dropped when it overflows
- 1The window is a budget, not a containerEvery token costs money, latency and attention, so treat the window as spend, not storage.
- 2Nobody typed thisContext engineering starts the moment a program, not a person, assembles what the model reads — and the questions that matter become selection, order, cost, provenance and proof rather than wording.
- 3Four kinds of context, four different problemsSort every token in the window into instructions, exemplars, knowledge or state — the four differ in half-life, trust, cache behaviour and failure mode, and almost every layout decision follows from which kind a block is.
- 4What the model literally receivesYour message list is rendered into one token sequence by a chat template that adds delimiters, role labels and serialised tool schemas — you pay for all of it, and the system role's authority is training, not a permission bit.
- 5What belongs in the system promptStable and yours goes on top; variable or untrusted goes below, always.
- 6Counting tokens without lying to yourselfCount with the tokeniser the model actually uses, log the billed count on every response, and remember that non-Latin scripts can cost two to four times more tokens for the same sentence.
- 7Writing the budget downWrite the token allocation as code with an assertion, subtract the output reserve first, and decide the cut order before an incident decides it for you.
- 8The advertised window and the usable oneThe advertised context length is an API limit, not a quality guarantee — measure the accuracy curve on your own material with real distractors, and budget below the point where it bends.
- 9Making the assembler testableMake context assembly a pure function of injected inputs — no clock, no network, no hidden randomness — so the request can be snapshot-tested, reproduced from a log and versioned by a hash of its configuration.
- 10What actually happens when it does not fitOverflow arrives as a rejected request, a silent truncation in your own code, or an upstream cut you never see — so cap unbounded sources at their boundary, never trim instructions automatically, and alert on the 99th percentile of budget usage rather than on failures.
Module 2
Selection: what earns a place
You always have more candidate material than budget. This module is how to choose, drop, compress and pack it.
By the end you can
Given far more candidate material than the budget allows, choose what goes in — filtering by permission and freshness before ranking, removing near-duplicates and boilerplate, deciding between a source and its summary, packing the remainder to a token budget — and state afterwards what was left out and why
- 11Retrieval versus long context: the real tradeCorpus size, permissions and freshness pick the architecture; window size only changes the budget.
- 12Similar is not the same as relevantVector similarity measures whether two texts keep similar company, not whether one answers the other — so negations, on-topic non-answers and rare identifiers all rank high, and a raw score means nothing until you have calibrated it on your own judged pairs.
- 13Filter before you rank, not afterApply correctness constraints inside the retrieval query rather than to its results, because filtering after top-k silently shrinks the evidence set — and remember that a highly selective filter can wreck approximate-search recall, at which point exact search over the small set is the better tool.
- 14How many passages is the right numberSweep k and measure accuracy, retrieval recall and tokens together — answer quality peaks and then falls because passages ranked just below the answer are on-topic distractors, and the flat left edge of the curve is cheaper than the peak.
- 15The same passage three timesDuplicate and near-duplicate passages spend budget twice and manufacture false consensus in the window, so deduplicate at ingestion and again before packing, detect boilerplate statistically rather than by rule, and resolve version conflicts before assembly rather than hoping the model will.
- 16Telling the model when things were trueInject today's date at the stable–variable boundary, attach each passage's date to the passage itself rather than a distant header, treat recency as a score adjustment rather than a filter, and monitor how stale your index is as a separate number from how old your documents are.
- 17The header on every chunkA per-chunk header is paid k times on every request, so keep the fields that change what the model does — a short citation handle, a date when versions conflict, and the section path that makes a mid-document chunk intelligible — and leave ids, scores and filter fields in the index.
- 18The source or a summary of itSummaries preserve coverage and lose precision — especially negations, qualifiers and exact figures — so compress orientation material at ingestion time, keep verbatim the text an answer must be grounded in, and always store a pointer back to the source.
- 19Packing the budgetPack the retrieved budget in tokens rather than in chunk counts, never include a partial chunk, pin the material the user explicitly referred to before anything competes for space, and keep packing separate from ordering so each can be changed alone.
- 20Admitting the set is partialTell the model its evidence set is partial, abstain in code when the retrieval signal is too weak to be worth a call, show the user what was searched and what was excluded, and record a manifest of what actually entered the window so a disputed answer can be explained.
Module 3
Retrieval as a context pipeline
From a document that fights back to a passage the model can cite: parsing, chunk context, query rewriting, hybrid search, reranking and measurement.
By the end you can
Build the pipeline that turns a question into the passages that fill the window — parse awkward sources, situate each chunk in its document, rewrite the query, run keyword and vector retrieval together and fuse the results, rerank to a budget, and measure retrieval on its own so a bad answer can be blamed on the right stage
- 21Getting text out of documents that resistExtraction is reconstruction of structure the file threw away, so expect column interleaving, orphaned table values, boilerplate in the body and silently empty scans — assert on characters per page, measure OCR error on your own scripts, and serialise tables one self-contained line per row.
- 22Chunking that does not destroy meaningRetrieve the unit a human would quote, and carry down the context that made it mean what it means.
- 23Giving each chunk its own contextA chunk cut from a long document loses the referents that made it meaningful, so restore them at ingestion — a free heading-path header first, then a generated situating sentence indexed alongside the chunk in both vector and keyword indexes — and keep the original text as the thing you actually show the model.
- 24Rewriting the query before you searchThe user's message is rarely a good search query — resolve references from the conversation, split compound questions, expand paraphrases and optionally search with a hypothetical answer, but always fuse the original query back in so a bad rewrite cannot silently replace the real one.
- 25Keywords still win, so run bothDense and sparse retrieval fail in opposite directions, so run both and fuse by reciprocal rank rather than by normalising incomparable scores — and check that your keyword analyser actually tokenises the languages you serve, because that failure is completely silent.
- 26Reranking, and why it reads better than the retrieverA cross-encoder scores query and passage together, so it can see negation and specificity a bi-encoder's separate vectors cannot — retrieve broadly, rerank a shortlist, and check the reranker's input limit before blaming it for long chunks it never fully read.
- 27Rows, records and schemas in the windowStructured data reaches the window as records, as a schema the model writes a query against, or as the result of a function you wrote — choose by whether the answer is a lookup, an aggregate or a known operation, and never let generated SQL run with more than read permission on named views.
- 28Questions no single passage answersTop-k retrieval assumes the answer sits in one passage, so route aggregates to a database, comparisons to one query per side, and only genuine chains to an iterative loop with a hop cap — because each hop multiplies its error rate and permanently adds its wreckage to the window.
- 29Citations you can actually verifyGive every passage a short stable handle, keep the mapping outside the window, and verify after generation that each cited handle exists and each quoted span really appears — because citation presence is easy and citation correctness is not, and only the second is worth showing a user.
- 30Measuring retrieval without the modelScore retrieval separately with recall@k on a gold set — it costs no model calls, it caps everything downstream, and it splits failures into missing evidence and unused evidence — but discount recall measured on model-generated questions, which reuse the chunk's own words.
Module 4
Position, order and what the model actually reads
Where a fact sits in the window changes whether it is used. How to measure that on your own material and lay out the request accordingly.
By the end you can
Predict and measure how position changes whether material in the window gets used — run a position probe on your own model and corpus, place instructions, examples and evidence deliberately, mark boundaries the model cannot confuse, and choose an evidence order that matches the task rather than the retriever's ranking
- 31The experiment that named the problemHolding content constant and moving only the position of the answer document produces a U-shaped accuracy curve — high at the ends, lowest in the middle, in the worst case below answering with no documents at all — and the size of that dip is a property of your model and task that has to be measured, not assumed.
- 32Ordering, and the fact that models skimPosition changes whether a fact gets used, so measure the position curve instead of trusting rank order.
- 33Measuring your own position curveMeasure your own position curve by holding the question, the distractors and the sampling fixed and moving only the gold passage — then check the differences against a bootstrap interval before treating the shape as real.
- 34Top, bottom, or bothCaching pulls instructions to the top and attention pulls them towards the question, so keep the large stable block cached at the front and pay for a short uncached reminder of the format and the one critical constraint next to the question.
- 35Saying it twice without saying two thingsRestate only the gating constraints — output contract, grounding, refusal — next to the question, render both copies from one source of truth so they cannot drift apart, and remember that a validator or a constrained decoder is a guarantee where a repeated instruction is only a nudge.
- 36Marking where one thing endsUse paired tag-style delimiters rather than markdown headings the content can imitate, guard against a document closing your block by escaping or by a per-request nonce, apply the same shape to every block, and explain in one sentence what the tagged regions mean.
- 37Ranked order is not always reading orderRetrieval rank decides what enters the window; reading order is a separate decision — relevance order for finding a fact, natural order whenever a human reading the passages out of sequence would be misled — and either way state the order in one line so the model does not infer a different one.
- 38What examples cost, and what they buyExamples are the most expensive way to give an instruction, so choose two to four that differ from each other, include the refusal case, put the most representative one last, balance the labels, and re-test periodically whether they still beat the instruction that could replace them.
- 39Why the first tokens are specialSoftmax forces attention to sum to one, so heads with nothing to attend to dump mass onto the earliest tokens, making them structural attention sinks — which is why evicting the first tokens from a KV cache destroys generation, and why the start of a request should hold your most stable, most authoritative text.
Module 5
Cost, caching and latency
The window is billed and it is timed. Where the money and the milliseconds go, and how to get them back without changing the output.
By the end you can
Cut the price and the time-to-first-token of a call without changing what it returns — order the prefix by stability so the cache actually hits, measure the hit rate rather than assuming it, move non-interactive work to a batch lane, route by context size, and cap the paths through which a single user can spend an unbounded amount
- 40The arithmetic of a requestInput, output, cached input and cache writes are four different prices, output is the expensive one, and a conversation bills quadratically because every turn resends the whole history — so measure cost per resolved outcome, not per call.
- 41Where the milliseconds goPrefill is compute-bound and scales with input, decode is bandwidth-bound and scales with output, so input length sets time-to-first-token while output length sets the rest — and streaming only hides the second of the two.
- 42Caching a stable prefixCaching matches from the first token forward, so anything variable near the top throws the whole discount away.
- 43Ordering the request so the cache can hitLay the request out strictly from most stable to most variable, put the cache breakpoint at the end of the shared block and a second after append-only history, and hash the rendered prefix in your logs so an accidental timestamp or a reordered dictionary cannot silently destroy the discount.
- 44Measuring the cache instead of believing in itCompute the hit rate from the cached and uncached token counts on every response, compare it against the ceiling implied by your own layout rather than someone else's number, alert on drops after deploys and per endpoint — and never refuse a quality improvement to protect a cache entry.
- 45The other two caches, and the dangerous oneCache embeddings by text and model id, cache responses by an exact hash of the assembled request, and treat semantic caching as a wrong-answer generator until you have measured its false-hit rate — while the underrated safe win is caching the retrieval result and still generating fresh.
- 46Work that can waitBatch endpoints trade latency for roughly half price, so route everything with no human waiting — ingestion, evaluation, backfills, scheduled work — through the same assembler into a batch file with a joinable id, and size fast-lane concurrency against the tokens-per-minute limit rather than the request count.
- 47Not every request needs the big modelMatch the model to the request — by context size, task type, or a measured slice where a small model matches a large one — and remember that a cascade only saves money while the escalation rate stays low, so measure that rate on real traffic before building one.
- 48How one user spends your whole budgetCost is requests times tokens times price, and a user can inflate all three without any exploit — so cap input at every entry point, cap output on every call, bound agent loops and tool results in code, and enforce a per-user daily token budget counted from billed usage rather than a request rate limit.
Module 6
Conversation, memory and compaction
A conversation outgrows any window. What to keep word for word, what to compress, what to store as structure, and what to delete.
By the end you can
Keep a long-running conversation inside a fixed window — decide what to keep verbatim, trigger compaction on tokens rather than turns, hold the facts that must not be lost as structured state rather than prose, carry memory across sessions with provenance and a deletion path, and tell the user what was forgotten instead of pretending
- 49Every turn carries all the previous onesA conversation is your code resending the whole transcript every turn, which makes the bill quadratic, dilutes the original instruction, and turns an early model error into a confident fact the later turns build on.
- 50Which turns are worth their tokensValue in a transcript is not correlated with recency, so pin the task statement and the constraints verbatim at the head, keep the last handful of turns verbatim at the tail, compress the middle, and stub out spent tool results — while keeping the full transcript in storage regardless of what the window holds.
- 51Compacting without losing the threadTrigger compaction on a fraction of the token budget rather than on turn count, ask for structured fields — especially decisions with rejections and approaches already tried — re-derive each compaction from the raw transcript rather than from the previous summary, and check that every id and number in the record appears in the source.
- 52State as fields, not as proseHold the facts that must not be lost — ids, amounts, constraints, decisions and rejections — as named fields with provenance and supersession rather than as sentences in a summary, render them at a fixed position with an explicit precedence rule, and show them to the user so a bad extraction can be corrected before it compounds.
- 53Memory between conversationsSeparate stated preferences from derived inferences, write sparingly and only from user-authored turns, store provenance and a date so conflicts resolve by recency and explicitness — and make the whole store visible, correctable and deletable, because a wrong memory is a wrong answer that repeats forever.
- 54Searching the transcript instead of carrying itIndexing the transcript keeps context bounded however long a conversation runs, but conversational turns retrieve badly until they are segmented by topic and rewritten into standalone form at write time — and retrieved history must supplement the verbatim recent turns rather than replace them.
- 55Passing context to whoever comes nextA handover note is the same artefact whether it crosses sessions, agents or into human hands — goal, status, next action, decisions with reasons, what failed, artefacts and open questions — and the section listing what was already tried is the one that saves the next reader from repeating it.
- 56Deleting what you keptUser text ends up in ten stores, so keep an inventory of every one of them as code, give each a deliberate retention period, and make erasure a tested function that asserts zero remaining rows — while treating embeddings and derived summaries as personal data and leaving the legal questions, which differ by country and remain unsettled, to lawyers.
- 57Being honest about a bounded windowA product may only claim to remember what it actually stores, so show the context indicator, mark where compaction happened, list stored memories with delete controls, and tell the model precisely what it does and does not have so it asks instead of confabulating.
Module 7
Tools, schemas and the agent's window
Once a program calls tools, the window fills with things you did not write: schemas, results, errors and the wreckage of earlier steps. How to bound all four.
By the end you can
Design the context of a tool-using system — account for the standing token cost of a toolset and measure its effect on routing, enforce a schema while understanding that enforcement guarantees shape and never truth, bound tool results and error text at the boundary in code, and choose between stubbing an agent's spent observations, delegating to a sub-agent and moving the payload to disk, justifying the choice by cost, cache behaviour and what each one permanently loses
- 58Structured output, and what a schema does not buy youA schema constrains the shape of an answer, never its truth, and its field order is a prompt.
- 59How a schema is actually enforcedStructured output is implemented by masking the model's logits at each step so only tokens that continue a valid parse survive, which guarantees a document that parses and guarantees nothing about whether it is true, complete, or an honest answer to the question.
- 60Every tool costs tokens before anyone calls itTool schemas are serialised into every request whether or not a tool is used, so a large toolset is a fixed tax on every call and a measurable drag on the model's ability to pick the right one.
- 61The description is the prompt that picks the toolA tool description is not documentation for a developer but the entire basis on which the model decides to call it, so it must state the boundaries, the limits and the cases where the tool is the wrong choice.
- 62The biggest thing in the window is a tool resultTool output is the one input to the window whose size you do not control and cannot predict, so it must be converted, then truncated in code at the boundary, with a marker that tells the model how to retrieve the rest.
- 63An error message is context tooError text written for a human log tells a model nothing it can act on, so every tool error should state what failed, whether retrying can possibly help, and what to do instead — and the retry cap belongs in code regardless.
- 64When the toolset outgrows the windowPast a few dozen tools the answer is to retrieve a subset per request rather than to list them all, which trades a fixed token tax and a routing problem for a recall problem whose failures are invisible.
- 65Why a twenty-step agent run costs what it doesAn agent loop resends its entire accumulated transcript on every step, so cost grows with the square of the number of steps and most of what is being resent is observations whose purpose has already been served.
- 66Sub-agents buy you a window and charge you a boundaryThe reason to spawn a sub-agent is that it reads a great deal and returns a little, so the parent never pays for the reading — and the price is that everything not in the return value is permanently gone, which makes the return contract the real design decision.
- 67Put the payload on disk and the pointer in the windowAny material the model needs to consult rather than read can live in a file with a query tool over it, which turns a fixed window into unbounded working memory — at the price of stale reads, forgotten artefacts and a new surface for untrusted text.
Module 8
Untrusted text and the trust boundary
Most of what fills a window was written by somebody else. What that lets an attacker do, why no prompt can stop it, and which controls actually hold.
By the end you can
Assess and bound the risk of a context pipeline that reads material your organisation did not write — explain from the token sequence why no instruction can guarantee the model ignores an injected one, enumerate the channels by which untrusted content enters and by which data can leave, carry a provenance label through the assembler and narrow the available tools from it, apply spotlighting while stating its measured limits, treat model output as input to whatever consumes it next, and report an attack success rate from a corpus delivered through the real entry points
- 68When untrusted text enters the contextYou cannot stop a model from reading an instruction, so limit what it is able to do about one.
- 69Why this one has no fixSQL injection was solved by a protocol that separates code from data, and no equivalent exists for a language model because instructions and data arrive as one undifferentiated token sequence in which the system role is a learned prior rather than an enforced boundary.
- 70The attacker is not the one typingThe dangerous payload does not arrive in the chat box but inside content your system fetched on the user's behalf, which means every source your assembler reads is an input channel from whoever controls that source.
- 71Three ingredients, and the recipe only works with all threeSerious exfiltration requires access to private data, exposure to untrusted content, and some way to send data outwards, so removing any single one of the three makes the attack fail regardless of how well written the payload is.
- 72The outward channels you did not know you hadAny mechanism that causes a request to a URL an attacker influences is an exfiltration channel, which makes rendered images, clickable links, fetch tools and even a search query into data egress paths that no security review labelled as such.
- 73Labelling every block with where it came fromTrust is a property of a block's origin rather than of its content, so the assembler should carry a provenance label on every block it packs and expose whether the finished window contains anything untrusted, which turns an unanswerable question into a boolean your code can act on.
- 74Limit what it can do, not what it can readSince you cannot prevent a model reading an instruction, the defence has to be that the instruction is not worth giving — which means the set of actions available in a turn is narrowed by code according to what is in the window, not by a request in the prompt.
- 75Marking untrusted text, and how far that gets youDelimiting, datamarking and encoding all make untrusted text more visibly distinct to the model and measurably reduce attack success, but each reduces rather than eliminates and each carries a cost in tokens or capability that has to be weighed.
- 76What the model writes is input to something elseModel output is partly shaped by whatever was in the window, so it must be validated before it reaches a browser, a database, a shell or a filesystem, exactly as any other input from outside would be.
- 77Making attack success a number you trackInjection defences are worthless unless measured, so build a corpus of payloads delivered through the channels they would really arrive by, score attack success as a rate, and run it on every change the way you run any other regression test.
Module 9
Shipping context, and proving it got better
A prompt is a production change with no compiler behind it. The evaluation set, the cheap offline checks, the honest statistics, the rollout and the diagnosis when it goes wrong.
By the end you can
Take a change to a context pipeline from idea to production and defend it afterwards — build a stratified golden set from real traffic and split it so the reported number still means something, assert on the assembled request in continuous integration, establish by ablation which blocks earn their tokens, decide with a paired test whether a difference is larger than the measurement's own noise, calibrate a model judge before trusting it, log enough to replay any request, ship behind a version with a revert that needs no deploy, and diagnose a bad answer down to the stage that produced it
- 78A prompt you can version, test and roll backVersion the whole call and log its hash, or you will not be able to explain your own regressions.
- 79The set of examples you are allowed to trustA golden set must be sampled from real traffic, labelled with required behaviour rather than expected wording, stratified to include abstentions and known failures, and split so that the portion you tune against is not the portion you report from.
- 80The tests that need no model at allMost context bugs are properties of the assembled request rather than of the model's answer, so they can be caught by assertions and a snapshot diff that run in milliseconds, cost nothing, and belong in continuous integration.
- 81Take it out and see what happensThe only honest way to know whether a block of context earns its tokens is to remove it and re-measure, which routinely reveals paragraphs that have been paid for on every request for a year and change nothing.
- 82Three points better, or three points of noiseTwo variants must be compared on the same inputs so that per-item differences can be counted, because a paired design detects far smaller effects than independent samples and because most reported prompt improvements are within the noise of the evaluation that produced them.
- 83Using a model to mark the homeworkA model judge is the only practical way to score open-ended answers at volume, and it carries position, verbosity and self-preference biases that must be controlled for and an agreement rate with humans that must be measured before any of its numbers mean anything.
- 84Logging enough to rebuild the requestLog the inputs, the configuration version and the prefix hash rather than the rendered prompt, because a pure assembler can reconstruct the exact request from those three while storing the prompt itself is expensive, full of personal data, and still not enough to explain a bill.
- 85Rolling out a prompt like a deploymentA prompt change is a production change with no type system and no compiler, so it needs the same machinery any risky deployment gets: one versioned configuration object, a shadow run on real traffic, a percentage rollout, and a revert that does not require a deploy.
- 86The dependency you do not controlA prompt is tuned against a specific model and that model is a hosted dependency that can move beneath you, so pin dated versions, re-run the golden set on a schedule to see drift as a trend, and budget re-tuning time into every migration.
- 87From a complaint to the stage that caused itA bad answer is produced by one identifiable stage of the pipeline, and with a logged manifest and a replay command the diagnosis is a short sequence of lookups rather than a guess — which is the whole reason for the instrumentation this course has been building.