Teaching it a boundary it cannot guess

Sorting is a different job from writing

Half the AI work in an office is not drafting. It is sorting: which of these 400 tickets are about billing, which of these invoices are for capital items, which of these applications meet the criteria, which of these survey answers are complaints.

This is where a small amount of technique produces a very large difference, and where a vague prompt produces confident nonsense that looks like a finished spreadsheet.

Your boundary is not the obvious one

Ask for tickets to be sorted into "billing, technical, account" and you will get an answer. What you will not get is your answer, because the interesting boundary is not the one in the category names.

A customer writes: "I was charged twice because your app logged me out mid-payment." Billing? Technical? In your team that is technical, because the fix belongs to engineering and the refund is automatic. Nothing in the words "billing" and "technical" says so. The model will pick one, plausibly, consistently, and wrongly.

Category names carry almost no information. Examples carry it all.

Label the edges, not the middle

The instinct is to give clear examples. Clear examples are the ones the model already gets right, and they teach nothing.

Choose eight to twelve examples that sit at the boundaries, including at least two that look like one category and belong in another. One clause of reason each, because the reason generalises where the label alone does not:

"Charged twice after the app crashed" → technical. (Cause is ours, refund is automatic; billing means the customer disputes the amount owed.)

"I want to change the card on my account" → account. (No money is in dispute; a self-service change.)

"Your invoice shows 12 licences, we have 9" → billing. (The amount owed is disputed.)

Three lines have now defined a boundary that a paragraph of description could not.

Always allow "none of these"

This one matters more than anything else in the lesson, and it follows directly from the mechanism.

The model produces a continuation. If the only continuations you have made available are three labels, you will receive one of three labels — for every item, including the one that is a supplier's marketing email that arrived in the queue by mistake.

So the category list always ends with an escape:

If the item does not clearly fit one of the three categories, output UNCLEAR and quote the phrase that made it ambiguous.

You will typically see five to fifteen per cent land in UNCLEAR. Read those. They are the most informative items in the batch: they contain either a missing category or a boundary you never defined. Both are things you needed to know, and a forced-choice run hides both.

Measure the agreement, and measure yourself

Before running 400 items, label 40 by hand. Then run those same 40 and compare.

Two numbers come out, and the second one is the one nobody computes.

The first is agreement between you and the model: say 34 of 40, 85 per cent.

The second is your agreement with yourself. Re-label the same 40 items a fortnight later without looking at your first answers. Most people, on a genuinely ambiguous category set, agree with their own earlier judgement 85 to 95 per cent of the time. If your self-agreement is 88 per cent and the model's agreement with you is 85 per cent, the model is essentially at the noise floor of the task, and further prompt engineering is polishing something that has no measurable target.

That is a liberating result when you get it, and an honest one. It also tells you when to stop.

Where it cannot work

A category that depends on information not in the text is not learnable from the text. "Priority customer" is not in the email if the contract tier lives in your CRM. "Duplicate of an existing ticket" requires the other tickets. "Fraudulent" usually requires the account history.

You can supply that information — join the contract tier onto each row before you send it — and then it becomes learnable. What you cannot do is expect the model to know the thing you did not give it. When a sorting task performs badly, the first question is not "is my prompt good enough" but "is the answer present in what I sent".

The free path is a spreadsheet

Nothing here needs a machine-learning platform. Two columns in LibreOffice Calc, Google Sheets or Excel: item text, and label. Paste ten labelled examples into your prompt, run the rest in batches, paste the answers back into a third column, and add a fourth for the ones you corrected.

That fourth column is the valuable artefact. It is your growing set of hard cases, and every one of them is a candidate example for the next run. The system improves by accumulating boundary cases, which is exactly how you would train a new colleague.