Retiming, and the two ways of inventing frames

The arithmetic

A clip at 50% speed has to fill twice as many timeline frames as it has source frames. A clip at 200% has to discard half of them. Discarding is easy. Filling is where every artefact comes from.

Three methods, in increasing order of quality and of failure severity.

Nearest frame

Each missing frame is a duplicate of the nearest real one. At 50% every frame plays twice.

The result stutters — it is literally a lower frame rate — but nothing is invented, so nothing is wrong. On material with little movement this is often the cleanest option, and it is what editors reach for when the alternatives look worse.

Frame blending

Each missing frame is a cross-fade between the two real frames either side.

Smoother than duplication, and it produces a visible ghost: two positions of a moving object, each at half opacity. On small movements this reads as motion blur and is acceptable. On a moving hand or a walking person it reads as a double exposure.

Optical flow

The software estimates how every pixel moved between two frames — a motion vector field — and warps the image along those vectors to construct a genuinely new intermediate frame.

When it works, the result is indistinguishable from footage shot at the higher frame rate. It is remarkable.

When it fails, it fails in a specific and recognisable way: the estimate is wrong somewhere, and the image tears, smears or melts in that region. Faces distort, fingers stretch, edges wobble.

It fails predictably when:

• Something enters or leaves frame, so there is no correspondence to find.
• Objects cross each other, and the algorithm cannot tell which is in front.
• Motion is faster than the search window, typically a fast pan or a whip.
• There is little texture, so there is nothing to match — a plain wall, a sky, fog.
• The footage is noisy, so the noise itself generates spurious vectors.
• Repeating patterns — a striped shirt, a railing, a brick wall — give the matcher many equally good answers and it picks wrong.

The practical procedure

1. Try optical flow first.
2. Play the shot at full speed and look specifically at hands, faces and edges.
3. If a region melts, try frame blending on that shot.
4. If blending ghosts unacceptably, use nearest frame and accept the stutter.
5. If none is acceptable, the shot cannot be slowed and you need a different shot.

Step 5 is a real outcome and it is worth saying to a client early.

In Resolve, retime processing is set per clip in the Inspector, so you can use optical flow on most shots and drop one problem shot back to nearest frame. There is also a Speed Warp mode, which uses a machine-learning estimator and is generally better — and is a Studio feature on most versions, so check yours.

Shoot the frames instead

The answer that always works. Film at 50 or 60 fps and place the clip on a 25 or 30 fps timeline at 50%. Every frame is real and there is nothing to invent.

This costs nothing but light — a higher frame rate needs a faster shutter and therefore more exposure — and it is why any camera's high-frame-rate mode is worth using for anything you might slow down.

If you shoot at 120 fps, you get clean quarter speed on a 30 fps timeline.

Speed ramps

A change of speed within a shot. Slow into a moment, fast out of it; or a fast whip that decelerates into a reveal.

The mechanics are in the earlier keyframe lesson: use the retime frame graph and ease the keyframes, so the rate of change itself is gradual rather than stepped.

Two things that make a ramp work:

• Put the speed change on a movement, not on a still moment. The viewer's eye is busy and the transition hides.
• Add motion blur if your tool has it. Real fast motion is blurred; a sped-up shot made of sharp frames looks like a fast slideshow.

Audio during a ramp

You cannot ramp speech and keep it intelligible — the pitch shifts and the words smear. The standard handling: unlink the audio, let it play at normal speed over the ramped picture, or drop the sync audio entirely and carry the moment on music or ambience.

Some tools offer pitch-corrected time stretching, which works for modest speed changes on music and poorly on speech beyond about ±10%.

Free tools

Resolve's optical flow retime is in the free version. Kdenlive and Shotcut offer speed changes with frame blending. FFmpeg's minterpolate filter does motion-compensated interpolation and is slow but free:

ffmpeg -i in.mp4 -vf "minterpolate=fps=60:mi_mode=mci:mc_mode=aobmc" out.mp4

Today

Take a clip of somebody moving, slow it to 40% with optical flow, and look closely at the hands. Then try frame blending. You will see both failure modes in one minute.