The bank had six hundred circulars going back eleven years. Every one of them told a loan officer what a scheme allowed, what it required and when it changed. They lived in a shared folder, named by date, and finding the right one took a phone call to head office. In the sowing season that call took two days.

Two people in the IT cell built a search. They split the circulars into about seven thousand passages, embedded each with a free 384-dimensional model that runs on a laptop, and stored the vectors. The arithmetic said the scale was never going to be the problem: seven thousand passages at 384 numbers of four bytes each is eleven megabytes, and a brute-force comparison against all of them takes under a millisecond. Nothing here needed a vector database, a server, or a subscription.

The first version worked well on questions and badly on facts. "What is the margin requirement for a dairy loan?" returned the right passage. "KCC-2019/07" returned nothing useful at all. The embedding model had never seen that string as a token, so it had no row to look up and no direction to point in; the circular that carried the code was ranked forty-first.

They also found their threshold was meaningless. They had copied a rule from a tutorial: accept a match above cosine 0.8. On their model almost nothing scored above 0.8, and the search returned an empty list for most queries. So they labelled two hundred query-passage pairs by hand, matching and not matching, and plotted the scores. Unrelated pairs averaged 0.61. Matching pairs averaged 0.78. The whole signal lived in a band about fifteen points wide, sitting well above zero, because the model pushed all its vectors into a narrow cone. The right cut-off for their model and their documents was 0.71, and it could not have been guessed.

That left the real decision. The dense search alone would ship in a week. It would answer questions about schemes and it would fail, quietly and confidently, on scheme codes, account formats, circular numbers and the surnames of the officers who signed them. A hybrid search, running a BM25 index alongside the embeddings and merging the two rankings by position, would take about three more weeks: building the index was an afternoon, but the merge, the evaluation set and the retraining of forty branch managers were not.

The cost of waiting was concrete. The sowing season started in six weeks and the counter queues would double. Every week without search was another week of two-day phone calls, and the branch managers had been promised something before the season, not after it.

The cost of shipping the dense version alone was harder to see and worse. A loan officer who searches a scheme code, gets a confident-looking passage from a superseded 2017 circular, and quotes it at the counter has been actively misled by the tool. Nothing in the interface would say the code had not matched; the score would look ordinary. The IT cell had seen that failure in testing and could not think of a way to warn a user about it, because a dense model does not know that it has never seen a token.

They put both options to the general manager with the numbers attached: eleven megabytes, one millisecond, a measured threshold of 0.71, and a list of twelve real queries from the branches on which the dense search returned the wrong circular.

The product was narrow and useful: municipal notices, tenancy orders and consumer-court summonses translated from English into Marathi in a register that lawyers would accept. A general model got the words right and the register wrong, rendering "the respondent is hereby directed" into everyday Marathi that read like a text message. Their customers were law clerks, and to a law clerk that is a wrong translation.

They had 9,400 aligned pairs, collected over a year, and one graphics card with 16 GB of memory.

The first question was whether they could train at all. The base model has 1.3 billion parameters. Full fine-tuning holds about sixteen bytes per parameter once the optimiser's two moments and the float32 master copy are counted, which is 20.8 GB before a single activation is stored. Their card could not hold it. Renting two data-centre cards for thirty hours would cost roughly what they spent on rent in a month, and would have to be repeated every time they changed anything.

The alternative was a low-rank adapter. Freeze the base model at two bytes a parameter, 2.6 GB, and train only two thin matrices inserted at each attention projection. At rank 8 across twenty-four layers and four projections of width 2,048, that is about 3.1 million trainable parameters: a quarter of one per cent of the model. Sixteen bytes applies only to those. The whole run fits with room to spare, and takes four hours rather than thirty.

Before committing they ran one check that took ten minutes. They took the singular values of a query projection matrix from the middle of the model and looked at where the squared magnitude accumulated. Ninety per cent of it sat in the first 180 of 2,048 directions. That did not prove a rank-8 adapter would work — the singular values describe the weights, not the change the task needs — but it told them the matrices were nowhere near using their full width, which made the low-rank story plausible enough to try first.

The rank-8 adapter trained cleanly and the result was disappointing. Everyday sentences improved. The legal register did not: "hereby directed" still came out casual, and a set of twenty fixed formulae that appear in almost every notice were translated inconsistently, sometimes correctly and sometimes not, within the same document.

Here the decision had a cost on both sides, and it is the decision the module exists to prepare you for. The obvious hypothesis, and the one both of them wanted to believe, was that the learning rate was wrong. Adapters start from zero and need a larger rate than a full fine-tune; theirs might have been too small. Testing that meant a sweep, which meant a day, and if it was right they would have saved the memory.

The other hypothesis was less comfortable: that the change the task required simply was not rank 8. Legal register is not a small correction to general translation; it is a systematic shift in word choice across the whole vocabulary. Raising the rank to 64 costs eight times the adapter parameters, which is still only 25 million and still fits, and costs one more four-hour run.

One of them wanted the sweep, because a smaller adapter would keep their deployment cheap. The other wanted the rank raised, because the failure looked systematic rather than under-trained: everyday sentences had improved, which is not what an under-trained model does.

The department had one graphics card and six research students. The rota gave each of them one night a week, ten hours, from eight in the evening. Miss the slot and the next one is seven days away, which in a project with a June deadline is a real loss.

The assistant had 11,000 photographs across seven categories — yellow rust, brown rust, loose smut, karnal bunt, aphid damage, nitrogen deficiency, and healthy — collected by extension officers over two seasons. The plan for the night was to fine-tune an image model that had already been pre-trained on general photographs.

The first attempt died in forty steps. The loss fell from 1.94 to 1.6, then printed as NaN and stayed there. The learning rate was 3e-4, taken from a widely shared post about training transformers.

He restarted with the rate at 1e-4. The loss went to NaN in about ninety steps. He restarted at 5e-5, and it survived, and after two hours the loss was 1.71 and falling so slowly that it would clearly not reach anything useful by morning.

At that point three hours of a ten-hour slot were gone and the decision arrived. He could start a full run at 5e-5 and let it go overnight, and have something to show his supervisor in the morning even if it was poor. Or he could spend twenty minutes on a learning-rate range test — start absurdly low, multiply the rate by 1.1 after each of two hundred steps, and record the loss — which would leave under seven hours for the run and might still produce nothing.

His supervisor's meeting was at eleven the next morning. A run that finished and was mediocre could be discussed. A slot spent on diagnostics with no model at the end could not, and the next slot was a week away.

He also had two numbers he had not looked at. He had never logged the gradient norm, so he could not tell whether the NaN was an explosion or an overflow somewhere in the forward pass. And he had not counted dead units, so he did not know what the two failed runs had done to the network before he restarted them.

He spent five minutes adding both. The gradient norm at 3e-4 rose from 2.1 to 340 over the forty steps before the NaN, which is an explosion and not an overflow. And when he loaded the checkpoint from the failed run and counted, 41 per cent of the units in the third block were producing zero for every image in a batch: a large early step had pushed their inputs negative for the whole dataset, and a unit in that state has no gradient and never comes back.

That second number changed the argument. It meant the run at 5e-5, started from a model already damaged in the first minutes, was not a slow model but a smaller one than its parameter count suggested. Continuing it overnight would produce a number that told him nothing about his data.

The campaign was planned around a mobile van, three technicians and two radiographers. Twenty thousand people would be screened in ten working days. Every image the tool flagged would be read by a radiographer before anybody was contacted.

The vendor's numbers were good and honestly stated. The tool flags 96 per cent of the images a radiologist would call abnormal, and it wrongly flags 8 per cent of the images a radiologist would call normal. Those figures came from a study of thirty thousand images and there was no reason to doubt them.

The health officer running the campaign did the arithmetic before the contract was signed, and she did it by counting a population rather than by using a formula.

In the group being screened, the expected rate of abnormal images was about 0.8 per cent. Out of twenty thousand people that is 160 abnormal images and 19,840 normal ones. The tool would flag 154 of the 160 and miss 6. It would also flag 8 per cent of 19,840, which is 1,587. Total flags: 1,741. Of those, 154 are real. Precision is 8.8 per cent.

Two radiographers reading forty confirmatory images a day each can get through eighty a day, or eight hundred over the campaign. Seventeen hundred flags is twenty-two days of reading for a ten-day campaign. The plan did not fit, and nothing about the vendor's numbers was wrong; the base rate did that.

There were three ways out and each cost something.

Raise the threshold. Tuning the tool so that it wrongly flags 3 per cent rather than 8 brings false alarms down to 595. Recall falls too, to about 88 per cent, so the tool now flags 141 of the 160 and misses 19. Total flags 736, precision 19 per cent, nine days of reading. The plan fits. Thirteen more people are missed by the tool than before.

Add readers. Two more radiographers for ten days would clear seventeen hundred flags. There were not two more radiographers in the district who could be released for ten days, and hiring from outside had a cost the campaign budget did not carry.

Screen fewer people, more selectively. Restricting the campaign to wards with a higher prior rate would raise the base rate and therefore the precision, because precision depends on the ratio of the two groups. Doubling the rate to 1.6 per cent takes precision from 8.8 to about 16 per cent for the same tool. It also means not screening the people who were excluded.

A fourth option was raised and rejected quickly: contact people directly on a flag, with a radiographer reading only the ones who came in. At 8.8 per cent precision that meant telling roughly sixteen hundred people that an automated system had found something in their chest X-ray when it had not. The officer would not do it, and the arithmetic is why: a flag at that precision is a reason to look, not a finding, and an interface or a phone script that presents it as a finding is wrong nine times in ten.

The last complication was that the 0.8 per cent was itself an estimate, taken from the previous year's campaign in a neighbouring district. She ran the numbers again at 0.5 and at 1.5 per cent. At 0.5 the precision falls to 5.7 and the flag count rises; at 1.5 it rises to 15.5. The decision came out the same across that range, which is the useful thing to know about a prior you are not sure of.

The support team handled about four hundred tickets a day in a mixture of English, Hindi and Telugu, and routed them by hand into eleven queues. Routing took roughly three people-hours a day, and the point of the model was to give those three hours back.

An engineer trained a classifier on 38,000 labelled tickets from the previous eighteen months. It trained without any error. The loss fell from 2.44 to 2.34 over the first epoch and then sat there for four more, moving in the third decimal place. Accuracy on a held-out set was 34 per cent.

Thirty-four per cent across eleven categories sounds like learning. Eleven categories at random would be nine per cent, and the model was nearly four times that, so the team's reading was that the model had learned something real and needed more data or a bigger architecture. The plan agreed on Tuesday was to label another twenty thousand tickets, which would take the support team about three weeks of evenings.

One person asked what the loss would be for a model that had learned nothing at all, and computed it before the meeting ended.

The eleven categories were not balanced. Deliveries were 31 per cent of tickets, warranty claims 18, returns 12, payments 9, installation 7, missing parts 6, transit damage 5, cancellations 4, exchanges 3.5, spare parts 2.5, and everything else 2. The entropy of that distribution is the average surprise of the label column, and it came to 2.06 nats. That is the cross-entropy of a model that predicts the base rates every time and knows nothing else.

Their model's loss was 2.34. It was worse than the base rates.

The accuracy figure had hidden it, because 34 per cent looks respectable next to nine and nobody had computed the other comparison: always answering "deliveries" scores 31 per cent, and the model was three points above that.

The second number pointed at the cause. The natural log of eleven is 2.398, and 2.34 is just under it. A loss stalling immediately below the log of the number of classes is the signature of a model whose outputs have been flattened almost to uniform before they reach the loss. The engineer looked, and the training loop applied a softmax to the logits and then passed the probabilities into a cross-entropy loss that expects raw logits. The loss applied a second softmax. Probabilities all lie between zero and one, so treated as logits they differ by at most one, and a softmax over such numbers is close to flat: three classes with 0.9, 0.05 and 0.05 come out as 0.53, 0.23 and 0.23. The model was training, honestly and slowly, towards a ceiling it could not pass.

That was Wednesday. The decision was whether to fix it and retrain, or to demonstrate on Friday with the model they had.

Retraining was four hours, so time was not the constraint. The constraint was that nobody could promise what the fixed model would score, and the board had already been told a number. Showing a working demonstration with a model that routes worse than a rule saying "send everything to deliveries" was, in the engineer's phrase, a thing that would be true for exactly as long as nobody checked.

The vendor's claim was specific and testable: on fifty scripts marked by both the system and a senior examiner, the two agreed on the grade 47 times. Ninety-four per cent. There were also no cases in the fifty where the system was more than one grade away from the examiner.

The examination cell had four weeks to decide. Marking 1.9 million scripts by hand takes six weeks and about eleven thousand examiner-days, and the results date is fixed by the academic calendar. The system, if it worked, would take four days and let examiners handle only the scripts it was unsure of.

An officer in the cell took the vendor's fifty-script table and did three pieces of arithmetic on it.

First, the interval. Forty-seven out of fifty is a proportion near the top of the range, where the textbook recipe of the estimate plus or minus 1.96 standard errors misbehaves. The Wilson interval, which solves for the true rates under which forty-seven of fifty would be unsurprising, gives 84 to 98 per cent. That is the honest reading of the vendor's evidence: somewhere between eighty-four and ninety-eight. At the bottom of that range, one script in six gets the wrong grade, which across 1.9 million scripts is three hundred thousand students.

Second, the zero. "No script more than one grade wrong in fifty" bounds that failure rate at roughly three divided by fifty, which is six per cent, with 95 per cent confidence. Six per cent of 1.9 million is 114,000 scripts. The zero was not evidence that the failure does not happen; it was evidence that it happens in under about one script in seventeen, which was not a reassuring sentence when written out in full.

Third, where the fifty came from. They had all been drawn from one school, marked by one examiner. Scripts from a single school share a teacher, a syllabus emphasis and a house style of answering, so they are not fifty independent observations of the system's behaviour. The officer could not compute the design effect without more data, but the direction was certain: the effective sample size was below fifty, so the true interval was wider than 84 to 98, not narrower.

There was a fourth thing, which the vendor volunteered when asked. Twelve configurations had been tried on the fifty scripts, and the one reported was the best of the twelve. The best of twelve draws from a distribution sits, on average, about one and a half standard errors above the truth, and the standard error here is about three and a half points. So the 94 was selected partly for having been lucky, and would be expected to fall back by four or five points when measured on scripts it had not been chosen on.

The decision had a cost on both sides and both were large.

A proper pilot meant drawing four thousand scripts across sixty districts, marking each twice by hand as well as by machine, and comparing. That is about eight hundred examiner-days and six weeks, and six weeks past the results date means 1.9 million students waiting, admissions calendars slipping and questions in the assembly.

Going ahead on the vendor's number meant accepting a system whose true agreement rate the cell did not know within fourteen points, evaluated on one school, tuned on the same fifty scripts it was measured on, and applied to every child in the state.

The question the business asked was ordinary. A buyer sends a technical sheet; a merchandiser needs to know within an hour whether the factory has made something close enough to quote from. Fourteen years of catalogues, spreadsheets and order files held about eight million descriptions, and the only way to search them was a name-based lookup that failed on anything phrased differently.

A consultant proposed a search system and quoted for a graphics-card server, about four and a quarter lakh rupees, plus setup. The owner asked her nephew, who was in the second year of a computer science degree, to check the number before she signed.

He did four multiplications and then a fifth.

Deduplication first, because half the eight million were near-copies of each other and the consultant's plan compared every description with every other one to find them. Eight million items make about thirty-two trillion pairs. At a microsecond a pair that is three hundred and seventy days. It was not a slow step in the plan; it was a step that would never finish, and nothing in the proposal said so. The standard replacement hashes each description into a short signature so that similar ones land in the same bucket, and only compares within buckets, which turns the work from the square of the item count into something close to linear. It runs in an afternoon on a laptop.

Embedding next. Eight million descriptions of about sixty tokens each is 480 million tokens. A small free embedding model of 33 million parameters costs about two FLOPs per parameter per token, so 66 million FLOPs per token, and the whole corpus is about 3.2 times ten to the sixteenth FLOPs. His laptop manages around a hundred billion FLOPs a second, which is 3.7 days of continuous running. A rented graphics card, at a hundred times that in practice, does it in under an hour for a few hundred rupees.

Memory. Eight million vectors of 384 numbers at four bytes each is 12.3 GB, which does not fit in the 8 GB laptop the office had. Stored as one-byte integers instead, it is 3.1 GB, which does.

Query time. Comparing a query against all eight million vectors is about six billion FLOPs, or sixty milliseconds on that laptop. For one merchandiser at a desk that is instant. For the catalogue website the owner also wanted to connect, at forty queries a second, it is not.

The fifth calculation was the one with a real decision in it. Partitioning the vectors into about two thousand eight hundred clusters and searching only the sixteen nearest clusters brings a query from eight million comparisons down to about forty-eight thousand: a hundred and sixty times faster, comfortably fast enough for the website. The cost is recall. The true nearest description sometimes sits in a cluster the search did not open, and at that setting it will be missed roughly six times in a hundred.

Six in a hundred is not an abstract number in a garment factory. A missed match means the merchandiser is told the style is new when it is not, and the factory re-develops a sample it already has: about eleven thousand rupees and nine days, each time it happens.

The model identified fourteen cotton pests and disorders from a photograph of a leaf or a boll. It had 300 million parameters and worked well: 0.91 accuracy overall on a held-out set of nine thousand field photographs, and 0.81 on the four rare classes that mattered most, because those were the ones an extension officer would not recognise on sight.

In half precision the model is 600 megabytes. The company's user research said 62 per cent of their farmers had phones with three gigabytes of memory, on which the practical budget for an app of this kind is about 350 megabytes. At int8 the model is 300 megabytes and fits; at int4 it is 150 and fits comfortably.

The alternative was to keep the model on a server and send photographs to it. That is the easy engineering and it fails on the ground: the photograph has to be taken at the plant, and 44 per cent of the villages the company works in have no usable data connection in the field. A model that needs the network is a model that works in the office.

So the decision was between shipping a quantised model to the phone and shipping a better model that most users could not reach.

The first int4 attempt was a failure of a specific and instructive kind. Overall accuracy fell from 0.91 to 0.86, which the product manager could live with. Accuracy on the four rare classes fell from 0.81 to 0.42, which she could not, because those four are the reason the app exists.

The engineer looked at the tensors rather than at the accuracy, and found the cause in two channels. Quantisation stores each weight as a rounded multiple of a scale, and the scale is set by the largest value in the group so that the largest value lands on the top integer. Two channels in the later blocks carried activations about ninety times larger than the rest of the tensor. With one scale for the whole tensor, those two channels set a step so coarse that almost every other weight in the tensor rounded to zero or to a single step. One number had flattened a million.

The fix was arithmetic rather than retraining: a separate scale for each block of 128 weights, so that the outlier sets the step only for its own block of 128. That costs sixteen extra bits per block, which is an eighth of a bit per weight, taking the model from 150 to about 155 megabytes. Rare-class accuracy came back to 0.78.

During the calibration pass the engineer also hit a NaN, and the trace is worth recording. A normalisation statistic was computed as the mean of the squares minus the square of the mean, on activations whose mean was around three thousand. Both terms were near nine million, both were known to about seven significant digits, and their difference was supposed to be a variance of about two — smaller than the error in either term. In one batch it came out negative, the square root returned NaN, and the NaN propagated through every later layer of the calibration. Subtracting the mean before squaring, so that the numbers being squared were near one rather than near three thousand, removed it.

That left the last decision, which was not technical. At int4 with block scales the model is 0.78 on the four rare classes against 0.81 for the full-precision model on a server. Three points, on the classes that matter most, in exchange for reaching every farmer rather than 56 per cent of them.