Integers, overflow, and the token id that would not fit

Integers are exact, until they are not

An integer type stores a whole number exactly, with no rounding, within a fixed range set by its width:

uint8    0 to 255
int8     −128 to 127
int16    −32,768 to 32,767
int32    −2,147,483,647 to 2,147,483,647   (about ±2.1 billion)
int64    about ±9.2 × 10^18

Inside the range, every operation is exact, which is why counters, indices, ids and pixel values are integers. Outside it, the behaviour is not an error. In NumPy and most compiled code, an int32 that passes 2,147,483,647 wraps to −2,147,483,648, and the program continues with a negative number where a large positive one should be. Python's own int grows without limit, so this never happens in plain Python; it happens the moment data enters a NumPy array, a PyTorch tensor, a database column or a file format, all of which have fixed widths.

Where 2.1 billion arrives

Two billion sounds large until you count tokens. A training corpus of a trillion tokens has positions up to 10^12, and an int32 offset into it wraps at the second billion. A dataset index built with np.arange(len(corpus)) on a platform whose default integer is 32 bits, which Windows NumPy was until recently, produced negative positions past 2.1 billion tokens, and negative indices in NumPy are legal: they count from the end. The bug did not crash; it read the wrong tokens.

The same limit appears in file sizes (a 32-bit length field caps at 2 GB, the reason older formats could not store a file larger than that), in a sum over an int32 column of byte counts, and in any count × size multiplication where both are int32: 50,000 × 50,000 = 2.5 × 10^9 overflows before it is assigned to the int64 you were about to store it in.

import numpy as np
a = np.int32(50_000)
print(a * a)         # -1794967296, with at most a warning

The fix is a word: int64 for anything that counts tokens, bytes, rows or products of sizes. Vocabulary ids are safe in int32, since no vocabulary approaches two billion; positions, offsets and totals are not.

Small integers are where the memory goes

At the other end, small integer types are how large data fits. An image is stored as uint8: 256 levels per channel, one byte per value, and a 224 × 224 colour image is 150 KB rather than the 600 KB of float32. A million such images are 150 GB, which is the figure module 7 used. Token ids fit in uint16 when the vocabulary is under 65,536 and in int32 otherwise, and a billion tokens are 2 or 4 GB accordingly. Labels for a thousand classes fit in uint16.

The habit that goes with this: convert to float only at the point of computation, not at the point of loading. A pipeline that decodes images to float32 on disk has quadrupled its storage for nothing.

uint8 arithmetic is a trap of its own

Subtracting two uint8 pixels, 100 − 150, does not give −50. It wraps to 206. Adding two, 200 + 100, gives 44. Image code that computes a difference between frames in uint8 produces bright artefacts where it should show darkness, and the code looks correct. Cast to int16 or float32 before any subtraction, and back to uint8, with clipping, only for storage.

Products of small integers

Quantised inference, the next lesson, multiplies int8 weights by int8 activations. Each product can be as large as 127 × 127 = 16,129, which does not fit in int8 or int16 once you add four thousand of them together. The accumulation is therefore done in int32, and the result rescaled. If you ever write such a loop yourself, the accumulator's width is the first thing to get right; the hardware's integer matrix units get it right for you.

Integers inside floats

The previous lessons showed that float32 represents integers exactly only up to 2^24 = 16.8 million, and float64 up to 2^53 = 9 × 10^15. A record id above 16.8 million stored in a float32 column is rounded to an even number, and two distinct customers become one. This happens when a CSV with a missing value in an id column is loaded, because a column with a NaN cannot be integer, so the loader silently promotes it to float. Check dtype on id columns after loading, and store ids as strings if the values can be large.

Booleans and bits

A boolean is one byte in NumPy, not one bit; a mask over a billion tokens is a gigabyte. Packed bit arrays, np.packbits, are eight times smaller and are what Bloom filters and large masks use. And a sum over a boolean array counts the Trues, which is the fastest way to count anything in NumPy, provided the accumulator is not a uint8: np.sum(mask, dtype=np.int64) if there are more than 255 of them.

The rule

Count in int64. Store in the smallest type the values fit, and know what that type's range is. Never subtract in uint8. And when a number is exactly 2,147,483,647, −2,147,483,648, 65,535, 255 or 16,777,216, you are looking at a limit, not a measurement.