Cutting a job into steps you can inspect

Why the enormous prompt fails

Somebody in your office has written a prompt like this:

Read these 30 customer emails, work out the main themes, rank them by how often they come up, pick the best quote for each, write a two-page report for the operations director with recommendations, keep it under 800 words, use British spelling, and don't mention the refund policy.

It will return something. The something will have four themes where there were six, one quote that does not appear in any email, 1,100 words, and the refund policy in paragraph three.

The mechanism is worth being precise about. The model is producing one continuous stretch of text, and every constraint you gave is a soft influence on that production, not a rule enforced by anything. Stack ten influences and each is diluted. There is no component that goes back at the end and checks the word count, because there is no end-of-generation check anywhere in the system.

One step, one call, and you hold the state

The alternative is to do what a competent analyst does: break the job into steps, run one step at a time, and keep the intermediate results yourself.

Here is the same job, decomposed.

Step 1 — extract. Five emails at a time:

For each email below, output one line: date | product | the customer's problem in under twelve words | what they asked us to do. If a field is not stated, write NOT STATED. Do not summarise across emails.

Step 2 — group. Paste your 30 one-line problems back in:

Group these 30 problems into between four and eight categories. Give each category a name and list the line numbers in it. Do not merge two problems that have different causes.

Then read the groups. This is the step where your judgement earns its keep, and it takes four minutes. Rename a category, split one, move two lines.

Step 3 — evidence. For each approved category:

From the emails in category 3, quote the two sentences that best show the problem. Quote exactly; do not paraphrase.

Step 4 — draft. Now, with the structure fixed and the quotes chosen:

Write the "Delivery delays" section: two sentences describing the problem, the two quotes, one sentence on how many of the 30 cases it covers. 120 words.

Four calls instead of one. Pennies rather than a penny. And you will finish sooner, because you are not rewriting a report whose structure was wrong.

What you gain by holding the middle

The decisive advantage is not quality. It is diagnosis.

With the mega-prompt, you get one artefact. If it is wrong, you do not know whether the extraction missed emails, the grouping was crude, or the drafting drifted. You can only try again and hope.

With four steps, every intermediate result is a thing you can look at. Thirty extracted lines: count them, and if there are 28 you have found a silently dropped pair before it becomes a wrong number in a report. Six categories: read them, and if "billing" and "payment" are separate you know it now rather than after the operations director asks.

This is the same reason a spreadsheet shows working. The intermediate values are where errors are cheap to catch.

Batch or split?

Within a step, you still choose how many items go in one call.

Batch when items are short, independent, and the task is mechanical — extraction from 30 similar emails, tagging, reformatting. It is cheaper and faster.

Split when each item needs care, when the items are long, or when quality on item 25 matters as much as on item 2. Attention is not uniform across a long list, and the middle of a batch of fifty is where quiet omissions live.

A practical rule: batch in fives or tens, not fifties, and always ask for a count in the output so you can check that everything that went in came out.

Where to stop cutting

Decomposition has a cost — your attention, once per step. Three or four steps is usually the sweet spot for a working task. Twelve steps for a one-page memo means you have built a factory to make one thing.

The signal that a task needs decomposing is not its length. It is the number of different kinds of judgement in it. The example above contains four: what happened, how things group, what is representative, and how to argue. That is why one prompt could not hold it. A 3,000-word draft from a structure you have already decided is one kind of judgement, and one call is fine.

When it becomes routine

If you run the same decomposition weekly, write the steps down as a numbered procedure with the exact prompt text for each, and put it where colleagues can find it. A later lesson covers turning that into a shared asset properly.

If you run it daily, it is worth a small script — a loop that reads a folder of emails, calls step 1 for each, and writes the results to a CSV. That is twenty lines of Python, and the free path is complete: Python itself, the provider's own client library or a plain HTTP request, and a local model through Ollama if the material must not leave the building. You do not need to be a programmer to copy a twenty-line loop and change the filename in it.