Addaly is in open beta. Things will change, and AI answers can be wrong — check anything that matters.

Making Things With AI

Images, video, voice and music — how they work, where they break, who owns them.

Lesson 3 of 849 min

The small space the picture is built in

A 512-pixel image is not built at 512 pixels

Running the denoising loop directly on pixels was tried, and it was ruinous. A 512 by 512 colour image is 786,432 numbers. Thirty passes of a large network over that many numbers, for every image, is why the first generation of these systems needed a data centre.

The fix that made image generation ordinary was to do the work somewhere smaller. Before the diffusion model ever sees your request, a separate network called a variational autoencoder — the VAE — has been trained to squash images into a compact grid and to restore them afterwards. In the widely used Stable Diffusion family, the squash factor is eight in each direction, with four channels instead of three:

image   512 x 512 x 3  =  786,432 numbers
latent   64 x  64 x 4  =   16,384 numbers   (48x fewer)

The diffusion loop runs entirely on that 64 by 64 grid. Only at the very end does the VAE's decoder expand the finished latent back into pixels. Everything the model reasons about — composition, colour, the shape of a face — happens in a representation where one cell stands for an eight-by-eight block of the final picture.

Where the denoising actually happens1024 by 1024 by 3 — the picture3,145,728 numbers. Thirty passes of a large networkover that many is why the first systems needed adata centre.The autoencoder's encoderEight times smaller in each direction, and fourchannels instead of three.128 by 128 by 4 — the latent grid65,536 numbers, forty-eight times fewer. One cellstands for an eight-by-eight block of the finalpicture.Every step, every mask, every conditioning mapAll of it happens here. Composition, colour and theshape of a face are decided at this resolution.The decoder, once, at the endSmall text, thin wires and distant faces were neverrepresented finely enough to survive the trip back.Push a photograph through the encoder and straight back out with no diffusion at all. What returns isthe best possible output of that model: no prompt, fine-tune or sampler beats it, because everythingleaves through this door.
Where the denoising actually happens1024 by 1024 by 3 — the picture3,145,728 numbers. Thirty passes of a largenetwork over that many is why the first systemsneeded a data centre.The autoencoder's encoderEight times smaller in each direction, and fourchannels instead of three.128 by 128 by 4 — the latent grid65,536 numbers, forty-eight times fewer. Onecell stands for an eight-by-eight block of thefinal picture.Every step, every mask, every conditioningmapAll of it happens here. Composition, colour andthe shape of a face are decided at thisresolution.The decoder, once, at the endSmall text, thin wires and distant faces werenever represented finely enough to survive thetrip back.Push a photograph through the encoder and straightback out with no diffusion at all. What returns isthe best possible output of that model: no prompt,fine-tune or sampler beats it, because everythingleaves through this door.

What that buys, and what it costs

It buys the whole field. It is the difference between an image taking nine seconds on a laptop graphics card and ninety on a rented server.

The cost is that the VAE is lossy, and it is lossy in a patterned way rather than a random one. It was trained to reconstruct the kinds of images in its training set with the least average error, which means it spends its capacity on what is common and starves what is rare. Three consequences you will meet:

  • Small text is destroyed even when the model got it right. The diffusion model can place letter shapes correctly in the latent; if each letter is two latent cells wide, the decoder has no room to render it and returns a smear. Text at large sizes survives. Text on a signpost in the distance never will.
  • Fine regular patterns wobble. Grille work, chain-link fence, printed circuit boards, distant window rows. The decoder reconstructs texture statistically, so a repeating grid comes back subtly irregular.
  • Faces at small scale degrade first. A face occupying 40 pixels is five latent cells across. There is not enough there to hold two eyes, a nose and a mouth in the right relation, which is why crowd scenes look correct until you zoom in.

None of these are prompt problems. No adjective adds latent resolution. The fixes are structural: generate larger, or generate the small thing separately at full size and composite it, or use a model whose latent uses more channels. Newer architectures moved from four latent channels to sixteen precisely because four could not carry fine detail, and the improvement in small text and skin texture is visible without measuring anything.

The seam you can see

Because the decoder works over the whole latent at once but has a limited receptive field, generating an image much larger than the model's native size produces a specific artefact: repeated subjects. Ask a model trained at 512 for a 1024-wide image and you frequently get two heads, or a body with an extra torso. The model is not confused about anatomy. Each region of the oversized latent independently looks like a plausible start of a subject, and nothing coordinates them.

This is why the standard workflow is generate at native size, then upscale, rather than generating large in one pass. Native size is stated on every model card: 512 for the older Stable Diffusion checkpoints, 1024 for SDXL and most current models. Working at native size and enlarging afterwards costs less and fails less.

Checking the ceiling for yourself

There is a five-minute experiment that teaches this better than any explanation. Take any photograph. Push it through the VAE's encoder and straight back out through the decoder, with no diffusion at all. Most local tools expose this; in ComfyUI it is a two-node graph, and it is free.

What comes back is the best possible output of that model. No prompt, no fine-tune, no sampler can beat it, because everything passes through this decoder on the way out. Compare the original and the round trip at 100% zoom and you will see exactly which details the pipeline cannot hold: the small type, the eyelashes, the moiré on a shirt. After that, you stop trying to fix those with words.

The one thing to keep

Diffusion happens in a compressed latent grid roughly sixty-four times smaller than the image, and the autoencoder that compresses and restores it is the source of a specific family of defects.

Before you move on

A model generates a street scene at its native 1024 pixels. The shop sign in the middle distance is a convincing smear of letter-like marks rather than readable words. Which explanation fits the mechanism?

Pick the one you would defend. Nobody sees your answer.

No ads. No data sale. No public scores on people. Ever.

© 2026 Addaly