Taking something out of a moving shot

The job

A microphone in frame. A light stand in the corner. A logo you do not have the rights to. A tracking marker you put there deliberately. A modern car in a period piece. A person who withdrew their consent after filming.

The mechanism in every case is the same: cover the unwanted thing with pixels that plausibly belong, in a way that holds up while the shot moves.

The order of solutions, cheapest first

1. Reframe. If the object is near an edge and your source is larger than your delivery, scale and reposition. This takes ten seconds and solves a large fraction of real cases. Always try it first.

2. Cut around it. Use less of the shot, or a different take. Also ten seconds.

3. Cover it with something. A graphic, a lower third, a split screen. Sometimes legitimate, often a fudge.

4. Patch from another part of the same frame. If the background behind the object is uniform — a wall, a sky, a road — copy a clean region over it with a soft-edged mask. Works immediately on a locked-off shot.

5. Patch from another frame. If the camera moves, the area behind the object is visible at some other point in the shot. Take that frame, track it into position, and composite it.

6. Clean plate. Film the scene without the object and composite it in. Requires planning at the shoot and is the reliable professional answer.

7. Paint it out frame by frame. The last resort, and it costs what rotoscoping costs.

The locked-off case

When the camera does not move, removal is nearly trivial: copy a clean region of the same frame over the object, soft-mask it, and it holds for the whole shot because nothing moves.

This is why locking the camera off is the single most useful thing you can do for a shot you know needs work.

The moving case

When the camera moves, a static patch slides off. The patch has to move with the shot, which means:

1. Track the surface the object sits on — planar track if it is flat, point track if it is simple.
2. Attach the patch to that track.
3. Mask the patch to only where it is needed, with a soft edge.
4. Match the grain, brightness and blur of the patch to the surrounding frame. A clean patch on grainy footage is visible precisely because it is too clean.

Step 4 is where these fail. Add grain to the patch.

Temporal median

A technique worth knowing for a specific case: something moves through an otherwise static shot and you want it gone.

If the camera is locked off and the unwanted thing moves, then for every pixel, the median value across many frames is the background — because the object only covers each pixel for a fraction of the time. Computing the per-pixel median over the shot gives you a clean plate with the moving object removed.

This is how "remove the tourists from my photo" features work. FFmpeg can do a rough version with tmix; Fusion and most compositors have a temporal median.

It requires a locked-off camera and an object that keeps moving. A person standing still is background as far as the median is concerned.

Machine inpainting

Generative models will now fill a masked region with plausible content, including across a moving shot with reasonable temporal consistency. Several free and paid implementations exist and the capability is improving quickly.

Three honest points.

It is generating content, not recovering it. The pixels behind the object were never recorded. What comes back is a plausible invention, and on anything structured — text, a face, a pattern, architecture — the invention is often visibly wrong in a way that is hard to unsee once noticed.

In factual work this matters. Removing a light stand from a documentary shot is uncontroversial. Removing a person, an object or a piece of text from a record of a real event alters the record, and the fact that the tool made it easy does not change that. Several jurisdictions are moving toward labelling requirements for synthetically altered media, and they differ and are changing.

Temporal consistency is the weak point. Frame-by-frame generation produces flicker, and the tools that address it do so by constraining the generation, which reduces quality. Check the result at speed, not frame by frame.

What is actually hard

• Removing something that crosses in front of a moving subject, because the patch has to respect the subject's edge, which is rotoscoping.
• Removing something reflected in a surface, which usually requires removing it twice.
• Removing something that casts a shadow, because the shadow stays.
• Anything against a detailed, structured background — brickwork, text, a crowd.

Today

Take a locked-off shot with something unwanted in it, copy a clean region over it with a feathered mask, and add a little grain to the patch. Five minutes, and it is most of the technique.