Where the data came from

Somebody chose

"Trained on the internet" is a phrase that sounds like a natural fact, the way rain is a natural fact. It is not. Every large model's training set is the result of a long series of human decisions, most of them undocumented, each of which shows up later in the output.

The raw material for most text models starts with web crawls. Common Crawl is the best-known: a non-profit that has been archiving pages since 2008, now petabytes of them, freely downloadable. Nobody trains on it directly. They filter it — and the filters are where the decisions live.

A typical pipeline drops pages by language, removes duplicates, strips boilerplate navigation, and then applies a quality classifier. One widely copied approach trained a classifier to recognise pages that resemble Wikipedia references and Reddit-linked pages with a decent score, then kept pages the classifier liked. Read that again: "good writing" was operationally defined as "resembles the pages a particular English-speaking community links to". Everything downstream inherits that definition.

Then come the deliberate additions, because crawled web text alone produces a mediocre model. Books, code repositories, scientific papers, licensed news archives, question-and-answer sites, and increasingly text generated by other models. The mix is a competitive secret at most labs, which is one reason independent scrutiny of these systems is so hard.

What the mix does to you

The consequences are concrete.

Language. English is enormously over-represented, and the gap is not proportional to speakers. A model that answers well in English and clumsily in Marathi, Yoruba or Tagalog is not making a mistake — it saw far less of those languages, and much of what it saw was translated rather than native. This is why an answer in your own language may be both less accurate and oddly phrased, as though translated from English, because in a sense it was.

Time. Every model has a training cutoff. Ask about something after it and you get either a refusal, a search result, or a confident answer assembled from the shape of older events. The third is the dangerous one.

Who writes on the internet. Web text over-represents people with time, connectivity and confidence to publish. That skews young, male, urban, and towards the Global North. The model's sense of what is normal, what is worth mentioning, and what a doctor or an engineer sounds like comes from that population.

Its own descendants. As more of the web becomes model output, later crawls contain it. Training heavily on synthetic text degrades models in measurable ways — variety collapses first, and rare cases disappear before common ones. Labs now spend real effort filtering their own kind out of the data.

The part that is not automated

Between the crawl and the assistant sits a large amount of human labour, and it is worth knowing about because it is usually invisible.

People write example answers. People rank pairs of model outputs to produce the preference data used in alignment training. People label content as harmful so that classifiers can be trained to filter it. This work is largely outsourced, often to Kenya, the Philippines, India and Venezuela, at wages that have been reported in the low single digits of dollars per hour. The content-moderation portion involves reading, in volume, the material you never want the model to produce. Reporting in 2023 documented workers in Nairobi doing exactly this for an AI supply chain, and the psychological cost of that work is well documented in the older social-media moderation literature.

This is not a footnote to the technology. The politeness and safety of a commercial assistant is, in significant part, a product of that labour.

What you can actually check

For open datasets you can look. The Pile, C4, RedPajama, Dolma and FineWeb all publish their composition, and several have search interfaces so you can ask whether a document is in them. For commercial models you generally cannot, though the EU AI Act now requires providers to publish a sufficiently detailed summary of training content, which is the first regulatory crack in that wall.

When you read a claim that a model is "unbiased" or "trained on all of human knowledge", the useful reply is a question: which crawl, filtered how, in what proportions, up to when? Nobody who has actually built one of these will find that question rude.