How your words become a steering signal

The prompt is not read, it is attached

There is a mental model most people arrive with: the model reads the sentence, understands it, then draws. That is not the shape of the machinery, and the difference explains most prompt behaviour.

Your text goes first to a separate, already-trained text encoder — a language model that never generates anything. It converts the prompt into a sequence of vectors, one per token, each a few hundred to a few thousand numbers long. That sequence is then handed to the denoising network as a fixed side input. At every one of the thirty steps, and at every position in the latent grid, the network looks at that sequence and asks which parts of it are relevant here.

The mechanism for looking is cross-attention. For each cell of the latent, the network computes a similarity between what it is currently building there and each token vector, then blends the token vectors in proportion. A cell that is turning into fur attends strongly to "cat"; a cell in the background attends to "kitchen". Nothing assigns regions to words in advance. The assignment emerges, step by step, and it can go wrong.

Which encoder, and why it matters

The choice of text encoder is one of the largest differences between generations of image models, and it is not cosmetic.

• Early Stable Diffusion used CLIP, a model trained to match images with their captions. CLIP is excellent at the gist of a caption and poor at grammar. It was trained on alt text, where word order carries little meaning, and it inherits that.
• SDXL used two CLIP encoders together, concatenating their outputs, which improved reliability without changing the character.
• Current models add or replace these with T5, a general text model trained on prose. T5 tracks clauses, negation and relations far better, which is exactly why newer models can handle "the red mug is to the left of the blue one" when older ones could not.

This is the mechanism behind advice that seems contradictory across tools. On a CLIP-only model, comma-separated tags outperform sentences, because CLIP is a bag-of-concepts machine and the sentence structure is wasted. On a T5-based model, a plain sentence outperforms tags, because the encoder can use the syntax and the tag soup looks nothing like its training text. Neither piece of advice is wrong; each is about a different encoder.

The binding problem, at its source

Cross-attention explains the most common failure in prompting. Ask for a red cube and a blue sphere and you will sometimes get a blue cube and a red sphere, or two purple objects.

The token vectors for "red" and "cube" are separate. Nothing in the architecture ties an adjective to the noun that follows it. Both are available at every cell, and the cell picks by similarity. Regions building a cube attend to "cube" strongly and to both colour words weakly, so the colour that wins is close to arbitrary. Encoders that read syntax reduce the leak because "red" and "cube" end up with more similar representations; they do not eliminate it.

The practical consequence is that attribute binding gets more reliable when you separate the objects in the request — into different clauses, different regions specified explicitly, or different generation passes composited afterwards. It does not get more reliable when you shout.

The number nobody mentions

CLIP text encoders have a context of 77 tokens, including two markers, so roughly 75 tokens of prompt. A token is about three-quarters of an English word. Past that limit, the rest is silently dropped or, in some tools, split into a second chunk and averaged with the first — which is not the same as being read.

Interfaces rarely warn you. A 200-word prompt on a CLIP-based model means about 55 words steer and the rest do nothing at all, and because the output still looks like the beginning of the prompt, people conclude their late clauses were ignored for stylistic reasons. They were not read. T5-based pipelines allow 256 or 512 tokens, which is why long prompts suddenly started working on newer models.

If your tool shows a token counter, watch it. If it does not, the test is simple: append something outrageous — "and a giraffe" — to the end of a long prompt. If no giraffe appears and no other change occurs, you are past the limit.