If you work in law or compliance

The most documented professional failure of the decade

In 2023, in a personal injury case in the Southern District of New York, lawyers filed a brief citing several judicial decisions that did not exist. The citations were well-formed, the case names were plausible, and the quotations were fluent. Sanctions followed, and the matter became the standard reference.

It has happened repeatedly since, in multiple countries. In 2025 the Divisional Court in England and Wales dealt with two referred cases involving fictitious or inaccurate citations and set out the court's powers and the professional obligations engaged. Courts and regulators in several jurisdictions have issued guidance or practice directions; some now require certification about the use of AI in filings.

Why citations specifically

The mechanism explains why this failure, rather than some other, is the one that keeps happening.

A legal citation is one of the most regular text patterns in existence: party names, year, court abbreviation, report series, page. Generating a well-formed one is the easiest thing in the world for a system that models text patterns. The formatting carries no information about existence.

And it lands where verification is expensive: checking whether a case exists takes a minute, and checking whether it says what has been claimed takes considerably longer — which is exactly the second check people skip.

So the rule is two-stage and neither stage is optional. Does it exist, in a real database. Does it say that, by reading the passage. A real case cited for a proposition it does not support is the more insidious error, and it survives a citation check.

Where it genuinely helps

The good list is substantial, and every item shares a property: you supply the material.

• First-pass contract review against a checklist you wrote. Change of control, indemnity caps, notice periods, governing law, assignment. It finds candidates; you decide.
• Chronologies from documents you provide, with every entry citing the document and page.
• Plain-English explanation of a clause for a client, drafted from the clause itself.
• Finding inconsistencies between a contract, its schedules and a side letter. Dull, mechanical, and genuinely good.
• Standard correspondence and the administrative drafting that fills a practice.
• Orientation in a bundle you are about to read properly — never as a substitute for reading it.

Where it is dangerous

Anything where the answer is the law rather than your documents. Models are trained overwhelmingly on material from a few large jurisdictions, and will answer a question about English, Indian, Nigerian or Kenyan law in a framework borrowed from elsewhere without mentioning that it has done so. The vocabulary will be right. The doctrine underneath may not be.

The failure is invisible precisely because the register is confident and the terminology is correct.

Privilege and confidentiality

Client material pasted into a consumer service is disclosed to a third party. Depending on jurisdiction and circumstances, that may engage confidentiality duties, may create arguments about privilege, and will certainly engage your regulator's rules on client confidentiality — the solicitors' regulator, the bar council, or the law society that admits you.

Most legal regulators have now published guidance on AI use. Read the actual document rather than a summary of it: the requirements tend to be about competence, supervision, confidentiality and client communication, and they are usually short.

In compliance and anti-money-laundering work, add the tipping-off point: restrictions on who may be told about a suspicion apply to third-party services as much as to people.

The free path, and it is the right one for verification

Verification should happen in a free, authoritative database, not in the tool that produced the citation:

• legislation.gov.uk for UK statute, EUR-Lex for EU law, India Code for Indian central acts.
• BAILII, CanLII, AustLII and CourtListener for case law across several jurisdictions, all free.
• Your own jurisdiction's official court and gazette sites, which are usually free and are authoritative in a way that no aggregator is.

For privileged material, a local model through Ollama does contract review, chronology building and inconsistency-finding on your own machine. It is weaker on subtle drafting and perfectly adequate at "list every date mentioned in these documents with the page it appears on" — which is most of the volume work.

The line to hold

You sign it. The certification is yours, the duty to the court is yours, and every authority in the document is one you have read. A model can help you find things faster and can help you explain them more clearly. It cannot verify anything, and the moment you allow it to appear to, you have joined a list of cases that is still growing.