Triage: sorting risk by what it costs

A tool you will use for the rest of the course

Everything that follows — bias, privacy, deepfakes, automated decisions — needs a way of deciding how much care a given use deserves. Uniform caution is not achievable; nobody verifies every sentence of a brainstorm, and nobody should. What works is a triage with four questions, applied in about thirty seconds.

1. If this is wrong, can it be undone?

Reversibility is the single most useful axis, and it is chronically underweighted. A wrong sentence in a draft is fixed by editing. A wrong figure in a filed tax return is fixed with penalties. A wrong dose is not fixed. A message sent, a file deleted, a payment made, a public post — these are one-way doors, and one-way doors deserve a pause that two-way doors do not.

2. Who bears the cost, and did they choose it?

If the person harmed is you, you can accept the risk. If it is a client, a patient, a tenant, a job applicant, a child — someone who did not choose the tool, cannot see it, and has no route to complain — the standard rises sharply. This asymmetry is the moral core of most of the harms in this course. The person best placed to catch the error is rarely the person who pays for it.

3. How many times will this happen?

One wrong answer is an incident. The same wrong answer applied to forty thousand applications is a policy. Automation changes the arithmetic of error: a human recruiter with a prejudice affects the people they meet; a screening model with the same prejudice affects everyone, identically, at once, and does it invisibly because there is no variation to notice.

Scale also removes the accidental safety of inconsistency. Humans are unreliable in different directions, which spreads harm around. A model is reliable in one direction, which concentrates it.

4. Is there a route to appeal, and does anyone answer it?

A system with a real appeal route can be wrong and recoverable. A system with none is a final judgment made by something that cannot explain itself. When you are on the receiving end, the presence of a named human who can look again is the difference between a mistake and a wall.

Three tiers, and what each requires

Run the four questions and most uses land in one of three tiers.

Low. Reversible, affects only you, one instance, easily corrected. Drafting, brainstorming, explaining a concept, formatting, first-pass translation of something you will read. Verification: skim. Do not spend more than this; you will exhaust the attention you need for the other tiers.

Medium. Reversible with effort, or affects colleagues, or repeats. External emails, reports someone will act on, code going into a system, analysis feeding a decision. Verification: check every specific claim, number and name. Have your own view of what the answer should be before you read the output.

High. Irreversible, or affects someone who did not choose this, or operates at scale, or has no appeal route. Anything medical, legal, financial or safety-related. Anything that decides about a person. Anything published under your name to an audience. Verification: check against a primary source you open yourself, have a second person look, and keep the record of what was checked.

There is a fourth category that is not a tier: uses where the honest answer is not to. A crisis conversation, a decision you are professionally licensed to make personally, a document you are legally required to have read. Knowing this category exists is what keeps the other three from expanding.

Write it down once

The reason to formalise this is that risk assessment made in the moment tracks convenience. At the end of a long day, everything looks low-risk. A rule written on a calm afternoon — anything that goes to a client gets checked against the source; anything about a person gets a second reader — survives the day the rule is inconvenient, which is the only day it matters.

Organisations should do the same thing in one page: which tasks are approved for which tools, what verification each tier requires, and who to tell when something goes wrong. Most organisations have instead a policy that says "use AI responsibly", which delegates the entire question back to the person least equipped to answer it at 6pm.

The rest of this course is, in a sense, nine modules of detail about how to fill in that page.