Teaching it something it does not know

When a prompt cannot get there

The previous module drew a line between "underspecified" and "not there". Everything on the not-there side of that line needs training, not wording: your face, your product, a specific character, a regional garment the caption corpus never described, a house illustration style.

Training a whole model is out of reach. Training a small correction to one is not, and this is the single largest capability most people are unaware they have.

What a LoRA is

Full fine-tuning updates every weight in the model — billions of numbers, enormous memory, and a separate multi-gigabyte file per subject.

A LoRA — low-rank adaptation — instead learns a small pair of matrices whose product is added to the original weights. The mathematics is that most useful adjustments are low-rank: they can be expressed as a small correction rather than a full rewrite. In practice the trained file is 10 to 200 megabytes rather than 6 gigabytes, several can be loaded at once, and each can be applied at an adjustable strength.

The related methods are worth distinguishing:

• Textual inversion. Learns a new word rather than new weights — a few kilobytes, very fast, limited to concepts the model can already almost express.
• DreamBooth. Full fine-tuning on a small set with a preservation term. Higher fidelity, large files, more compute.
• LoRA. The middle, and where nearly everyone lands.

What training actually requires

The numbers, so you can judge feasibility:

• A subject (a face, a product, an object): 15 to 30 images.
• A style: 30 to 100 images.
• Time: 10 to 40 minutes on a mid-range consumer GPU; a free Colab session is enough for a small one.
• Cost: nothing, if you have a suitable computer or use a free tier.

The images matter far more than the settings, and this is where almost every failed attempt goes wrong.

Vary everything except the subject. Different backgrounds, lighting, distances, angles, clothing. If every photograph was taken in the same room, the model learns the room. This is the classic failure: a LoRA that reproduces a face beautifully and always with the same bookshelf behind it.

Crop and resize to the training resolution — square or the model's bucket sizes. Sloppy crops teach sloppy framing.

Caption honestly and describe the variable parts. Caption what changes between images (the background, the pose, the clothing) and not the thing you are teaching. Anything you leave uncaptioned is absorbed into the trigger word. Caption "a photograph of sks person in a garden" and the garden is described, so the garden stays variable; leave the garden out and the model may bind it to the subject.

Twenty good images beat two hundred mediocre ones. Duplicates and near-duplicates cause the memorisation effect from module one, at small scale and quickly.

Over-training, and how to see it

The most common outcome of a first attempt is a LoRA that has learned too hard. Symptoms: the subject appears correctly and everything else in the image degrades; prompts stop working; every output has the same composition; the style bleeds onto objects it should not touch.

The fix is to save checkpoints during training — most trainers do this by default, every few hundred steps — and test several. The best one is usually not the last. Generate the same three prompts with each checkpoint at the same seed and pick by eye. This takes ten minutes and is the difference between a usable LoRA and a broken one.

Applying strength below 1.0 at generation time is the other half of the control. A slightly over-trained LoRA at 0.7 is often perfect.

The part that is not technical

Training on somebody's face requires their consent, and the same standard applies as for voice, which the voice module covers in full: informed, specific, written, and revocable. Training on a living artist's work to reproduce their style is legal in some places, contested in others, and the subject of the final module. What is not contested anywhere is that publishing the result as though it were theirs is passing off.

Training on your own photographs, your own products, your own drawings and openly licensed material is uncomplicated, and for most commercial work it is also what you actually need. A LoRA of your client's product photographed from twenty angles solves a problem no prompt can, and nobody has any claim on it.