Crowds that are not there

The cheapest thing to fake is a person

A fake video is expensive to make well and easy to check by provenance. A fake person — an account with a history, a photograph, opinions, a plausible posting rhythm — is now nearly free, and there is no provenance to check.

This matters because a great deal of public reasoning runs on apparent consensus. What are people saying about this product, this policy, this candidate, this employer? The answer is assembled from a stream of accounts, and the stream can be manufactured.

The forms it takes

Astroturfing. Manufactured grassroots support or opposition. Regulatory consultations are a documented target: in the 2017 US net neutrality proceeding, millions of comments were submitted using real people's identities without their knowledge, and a state investigation found large-scale fabrication funded by industry groups. That happened before cheap text generation. The cost has fallen since.

Fake reviews. Regulators have moved on this specifically. The US Federal Trade Commission's rule effective in 2024 prohibits fake and AI-generated reviews and testimonials, with civil penalties. The rule exists because the practice is widespread and the arithmetic is compelling — reviews drive purchases and a review costs almost nothing to write.

Sockpuppets in discussion. Multiple accounts held by one operator, agreeing with each other. The manufactured impression is not that an argument is correct but that it is normal, which is a much easier thing to fake and a much more effective one.

Engagement farming. Accounts generating volume to build audiences that are later sold or redirected. The content is often incoherent on inspection and is not meant to be inspected.

Research contamination. Online survey panels and paid task platforms now contain participants who route questions through a model. This corrupts the datasets a great deal of social science and market research rests on, and researchers have begun publishing methods to detect it.

Why proving humanity is hard

The obvious answer — make people prove they are human — runs into a wall in every direction.

CAPTCHAs are largely solved by machines and remain an obstacle mainly for humans with disabilities. Their remaining value is behavioural signals gathered while you interact, not the puzzle.

Phone verification raises cost and does not stop a determined operator, since numbers are purchasable in bulk in many markets.

Government identity works and destroys anonymity, which is a genuine loss: anonymity protects dissidents, whistleblowers, people asking about their health, abuse survivors, and anyone in a place where an opinion is dangerous.

Proof-of-personhood schemes, including biometric ones, attempt to give each human exactly one credential without revealing who they are. The cryptography is real; the deployments have raised serious concerns about biometric collection, about incentivising the poorest to sell an iris scan, and about who holds the registry. Several regulators have restricted them.

There is no solution here that is simultaneously cheap, anonymity-preserving and effective. Anyone offering all three is worth reading carefully.

What to do as a reader

Weight identifiable people. A named person with a professional reputation attached to a claim is a different quality of evidence from an account with a plausible name and no history. Not infallible, but it is the axis that still carries information.

Discount volume entirely. The number of accounts saying a thing is now uninformative. This is a large adjustment for most people and it is the correct one.

Look for costly signals. Content that took effort — original data, a long-standing body of work, a reputation that would be damaged by being wrong — is expensive to fake, which is precisely why it still means something.

Check whether the crowd is corroborating or copying. Fifty accounts repeating one claim is one source. Fifty accounts reporting the same event from different angles is fifty.

The direction of travel

Provenance standards such as C2PA attach signed information about origin to media, and adoption is growing among camera makers and platforms. They are real progress with a hard limit already noted in this course: absence of a signature proves nothing, since most genuine content has none.

The likely medium-term equilibrium is not verified content but verified sources — accounts, publishers and institutions that carry accumulated, costly reputation. That is how the pre-internet information system worked, and it may be where this returns, with the loss of openness that implies.