Motion, Sound and Music
Why nothing in the world starts at full speed, and why a blanket beats a better microphone.
- Level
- Some background helps
- Lessons
- 72
- Reading time
- 713 min
- Price
- Free, no sign-up to read
Motion first: what a keyframe stores, why easing fixes most bad animation, and how timing, anticipation and overshoot give a logo weight. Then lower-thirds, Lottie handover, green-screen keying and exporting video that is not 40MB. Then sound: recording a voice in an untreated room, compression explained properly, EQ in three moves, music licensing and Content ID, and the consent rules for a cloned voice.
Opens after the Video With AI exam
Sign in, finish that course, and pass its exam. You can read this syllabus meanwhile.
Go to Video With AIModule 1
How motion reads as real
Before any software, the part that decides whether animation looks expensive or amateur: what a frame rate actually commits you to, how to read a speed graph, why the default ease is a compromise nobody chose, what a spring buys and what it costs, and which of the old animation principles survive on a screen somebody looks at four hundred times.
By the end you can
Diagnose a piece of flat or floaty animation by reading its timing and spacing — naming whether the fault is the easing curve's influence, the spacing between frames, a missing arc, or an absent anticipation — and correct it by editing the value graph rather than adding an effect on top
- 1Nothing in the world starts at full speedEase-out for anything arriving or responding, ease-in for anything leaving, and linear only for motion that never starts or stops.
- 2Motion blur will not fix a floaty logoWeight is uneven distance-per-frame plus a small anticipation and settle; effects added afterwards cannot supply it.
- 3Frames, not secondsFrame rate is a commitment made before the first keyframe, because a duration in milliseconds only becomes a real animation once it has been rounded to whole frames — and changing the rate afterwards silently resamples everything you spaced by hand.
- 4The panel most people never openThe speed graph is the derivative of the value graph, so the shape of one tells you what the motion is doing between keyframes — and a rectangular speed graph is the signature of the linear default that makes animation read as wrong.
- 5Why pressing Easy Ease still looks wrongA keyframe handle has an angle and a length, and the one-key ease sets both to a symmetric default nobody chose — dragging the outgoing handle's influence to roughly three-quarters is what turns a mushy move into one that commits and then settles.
- 6Springs, and what you give up for themA spring's duration is an output of stiffness, damping and mass rather than something you set, which is why it can be redirected mid-flight without a visible restart — and why it is the wrong tool for anything that must land on a beat or must never exceed its target value.
- 7Straight lines and the things that lag behindArcs and small two-to-four-frame offsets between parts are what separate animated motion from interpolated motion, but the element under the user's finger must never lag, because perceived responsiveness is measured from input to first visible change.
- 8The old principles, filtered for a screen seen four hundred timesFilm motion is watched once and interface motion is watched hundreds of times, so anticipation, exaggeration and squash-and-stretch — the principles that make a single viewing memorable — are precisely the ones that must be reduced or dropped on a screen somebody uses every day.
- 9Where the eye already isA viewer needs roughly 150–250ms to decide to look somewhere and another 20–40ms to get there, so a new element that appears far from where the last one left is missed for its first several frames — place the next point of interest where the eye already is, or hold a beat.
Module 2
Motion that carries meaning
Animation in an interface as a functional layer rather than decoration: what a transition claims, how long it should take, which properties the browser can animate cheaply, and what to do for the substantial number of people who are made ill by large-field motion.
By the end you can
Specify the motion in an interface as something a second developer could build from a written brief — naming which element moves, for how many milliseconds, on which curve, in what order relative to its neighbours, what it does when the viewer has asked for reduced motion, and which CSS properties the browser can animate without re-running layout — and justify each number from the perceptual or rendering limit it comes from
- 10The three jobs an animation can doInterface motion earns its place only by answering what just happened, where a thing came from, or whether the system is alive — and the causal reading depends on the movement starting within roughly a tenth of a second of the action that caused it.
- 11How long an animation should takeDurations come from a small named set rather than from feel, exits run at roughly three-quarters of entrance time, and distance scales duration sub-linearly — but perceived length is set by the shape of the curve's tail more than by the number.
- 12The transition is the explanationA shared-element transition measures an element before and after, inverts the difference with a transform and plays it off — which is why it can only describe a relationship that genuinely exists between the two states.
- 13Staggering without making people waitA stagger works by overlapping heavily — 20 to 60ms of offset per item with a capped total window — and starting the next element when the previous is roughly halfway, because the eye reads intent long before a movement settles.
- 14Why your lower-third looks like a templateAnimation time is not reading time — budget two and a half to three seconds of still text first, then add the entrance and exit on top.
- 15Naming the motion so it stops driftingMotion tokens are named by role rather than by value so the number can change without the name becoming a lie, and redefining them inside a reduced-motion query is the cheapest way to make every future component inherit the right behaviour.
- 16Spinners, skeletons and honest waitingDelay the indicator by 200 to 300ms so fast responses never flash a spinner, use an indeterminate indicator whenever you do not genuinely know the progress, and treat a skeleton as a promise about layout that will cost you twice if it is wrong.
- 17Motion that makes people illLarge-area, parallax and zoom motion cause a visual-vestibular conflict that produces real nausea in a large minority of adults, so the media query should substitute a cross-fade rather than switch animation off — and the effects most likely to trigger it should not be in the default experience at all.
- 18Why transform is cheap and width is notTransform and opacity describe how an already-painted layer is placed, so animating them skips layout and paint and can run on the compositor thread even while the main thread is blocked — which is why they survive a heavy script task and a width animation does not.
Module 3
Shipping the motion
Getting an animation out of the tool it was made in and onto somebody else's screen: which of the five delivery technologies fits, what Lottie and SVG cannot carry, why a GIF costs twenty times what a video costs, how transparency actually works on the web, and what a developer needs in writing.
By the end you can
Choose a delivery format for a given piece of motion by reasoning from whether it must respond to state, whether its content is vector or photographic, and who will edit it next — then produce that file with the right encoder settings, verify it on the target platform rather than in the authoring tool, and hand it over with a written specification covering trigger, interrupt, exit, responsive behaviour and reduced motion
- 19Five ways to put motion on a screenThe delivery format follows from three questions — does it respond to state, is the content vector or photographic, and how often will it change — and the first of those eliminates more options than the other two combined.
- 20What Lottie carries and what it dropsA Lottie file is a list of shapes with keyframed properties, so anything that is not expressible that way — effects, most expressions, motion blur, cameras — does not survive the export, and imported raster images are base64-inflated into the JSON rather than compressed.
- 21Animating a drawing that is also a documentSVG elements sit in the DOM, so CSS and JavaScript animate them directly — and the two techniques worth memorising are stroke-dashoffset for a self-drawing line, normalised with pathLength, and transform-box: fill-box to stop a rotation swinging across the viewBox.
- 22Why a GIF costs twenty times what a video costsGIF stores each frame as a palette of 256 colours with one bit of transparency and no motion compensation between frames, so a clip that a video codec describes as block movement has to be stored almost in full every frame.
- 23Transparent video, and why it is two filesNo single video file carries alpha everywhere: VP9 in WebM covers Chrome and Firefox, HEVC with the hvc1 tag covers Safari, and the luma-matte workaround doubles the frame width and requires WebGL — so the first question is whether the background is genuinely arbitrary.
- 24Frames in a grid, and the memory they costA sprite sheet gives instant random access to any frame with steps() timing, but its cost is decoded memory — width times height times four bytes, regardless of how well the PNG compressed — which is why a 4096-square sheet can cost 67MB and evict everything else on the page.
- 25Motion that responds to stateInteractive motion is a state machine, and the hard part is interruption — a tool like Rive blends from the current position automatically, CSS transitions get this free from the browser, and a play-a-clip approach gets it not at all.
- 26Send the developer a file, not a videoIf a developer cannot recolour it, pause it, or place it on a dark background, you exported the wrong format.
- 27The specification a developer can build fromA recording shows one playthrough of the happy path, so the specification has to supply what it cannot — trigger, interrupt behaviour, exit, loop count, responsive numbers and reduced-motion substitution — and it should be reviewed at quarter speed on the oldest device available.
Module 4
Putting two images together
Compositing from the arithmetic up: what a matte is and where one comes from, why a dark fringe means the alpha convention is wrong, what blend modes really compute, how a key is rescued by despill and light wrap rather than by better settings, and what a tracker and a camera solver can and cannot know.
By the end you can
Combine two pieces of footage that were never filmed together so the seam does not show — building the matte by the cheapest route the shot allows, diagnosing an edge fringe from its colour, repairing spill and edge interaction rather than the matte's interior, attaching the insert with a track or solve whose error you have read, and matching black level, grain, focus and lens distortion in a defensible order
- 28The greyscale image that decides everythingA composite is one weighted mix per pixel, so the interesting work is in the grey edge values — and a difficult key is solved by splitting the frame into a garbage matte, a solid core and a narrow edge band rather than by finding better keyer settings.
- 29The dark fringe and what causes itPremultiplied RGB has already been multiplied by alpha, so compositing it with the straight formula multiplies the edges twice — a dark fringe means premultiplied treated as straight, a bright halo means the reverse, and colour correction belongs between an unpremultiply and a re-premultiply.
- 30Every blend mode is one line of arithmeticMultiply can only darken and drops white, Screen can only lighten and drops black, Add is physically correct for light and clips in 8-bit, and Difference is a measuring instrument rather than a look — which is why stock light elements are filmed on black and fail over a bright sky.
- 31A green screen is fixed on set, not in the editA keyer builds a matte out of colour information, so anything that degrades colour on set — uneven light, spill, subsampling — cannot be recovered afterwards.
- 32Spill is real light, and light wrap is an honest fakeGreen bounce lands on the subject where alpha is one, so the matte cannot remove it — despill clamps green to the average of red and blue and therefore desaturates anything genuinely green, and light wrap replaces the missing interaction between subject and background by pushing a blurred copy of the background onto the edge band.
- 33Drawing the matte by handRoto is done by halving intervals rather than working forward frame by frame, with the subject split into simply-moving parts — and machine segmentation is best used to generate the garbage and core mattes in seconds while the hand work is spent only on the edge band it gets wrong.
- 34What a tracker can and cannot seeA tracker correlates a patch of pixels against a search region, so a feature on a straight edge is unsolvable along that edge — the aperture problem — and every good feature is one with contrast in two directions.
- 35Solving for a camera, and when you cannotA camera solve triangulates depth from parallax, so a purely rotating camera carries no depth information at all and can only be solved as a nodal pan — and reprojection error in pixels, not the look of the point cloud, is the number that says whether the solve will hold.
- 36Black level, grain, focus, and the order to do them inA composite usually fails on black level first and grain second, both of which are measurable rather than matters of taste — and grain and lens distortion go last in the chain because they are properties of the camera, so anything applied after them is physically happening inside the lens.
Module 5
Codecs, colour and delivery
What a compressor throws away and why, the difference between a frame you can seek to and one you cannot, how to hit a quality target or a size target on purpose, and the four colour and timing decisions behind most files that look washed out, banded or juddery when they arrive.
By the end you can
Export a piece of video against a stated constraint — a size budget, a platform's requirements, an archival master, an editable intermediate — by choosing the codec class, rate-control mode, bit depth and colour tags from what the file is for; and diagnose a delivered file that looks milky, crushed, banded, juddery or twenty times too large by naming which of those decisions produced it
- 37What a codec throws awayCompression exploits specific limits of vision — colour resolution first, then fine detail within blocks — so the bitrate a clip needs is set by how unpredictable its content is rather than by its resolution, and noise is expensive precisely because it cannot be predicted.
- 38Frames that stand alone, and frames that do notA P- or B-frame is defined in terms of its neighbours, so a decoder cannot show frame 43 without decoding from the last keyframe forward — which is why long-GOP material scrubs badly, why all-intra intermediates exist, and why a proxy beats a hardware upgrade.
- 39Quality target or size targetCRF asks for a quality and lets the size fall where it may, two-pass asks for a size and distributes the bits; the preset changes how hard the encoder searches rather than the quality it aims at, and a change of about six in CRF roughly halves or doubles the file.
- 40Why your export looks washed outLimited range puts black at 16 and white at 235 while full range uses the whole 0 to 255 — so a milky, low-contrast export is limited data read as full, a crushed and over-contrasty one is full data read as limited, and untagged files leave every player to guess differently.
- 41Counting the steps in a gradientAn 8-bit channel has 256 levels, so a fifty-level sky across a 1080-pixel frame has steps 21 pixels wide and bands visibly — and dither works by turning a structured quantisation edge into random error the eye integrates as smooth.
- 42Log footage and what a LUT is notLog encoding allocates code values roughly evenly per stop so a wide range survives a limited file, and a LUT is a fixed lookup table with a defined input domain — which is why a conversion LUT is decoding rather than grading, and why applying a look before exposure is corrected destroys detail permanently.
- 43Frame rates, shutter and judderMotion reads as continuous because each frame carries blur covering roughly half the interval to the next — the 180-degree rule — and judder comes from showing frames for unequal lengths of time, as 3:2 pulldown does when 24 is forced onto a 60Hz display.
- 44Your hero video is why the page feels slowFile size follows how much changes between frames, not how many pixels are in one — so bitrate and content, not resolution, are the levers.
- 45The tool under most of the othersOptions before -i apply to reading and options after apply to writing, -c copy trims and joins with no decode and no loss, and -movflags +faststart moves the MP4 index to the front so a web video can start before it has finished downloading.
Module 6
Capturing sound
Recording a voice in a room you cannot rebuild: what the two microphone constructions can and cannot do, why distance beats equipment, where the level should sit and why the two ends of that range are not symmetrical, what the noise floor costs you three steps later, and how to get two devices into sync and into phase.
By the end you can
Record a usable voice in an ordinary room — choosing pattern and distance from what the room is doing rather than from a specification sheet, setting gain so peaks land with headroom you can justify, capturing room tone as a matter of routine, and bringing a second microphone or a second device in without comb filtering or drift — and say for any given capture fault whether it can be repaired afterwards or not
- 46Pressure, gradient, and everything that followsA sealed capsule measures pressure and is omnidirectional with no proximity effect, while an open one measures the front-to-back difference and gains both directionality and a close-up bass lift — and a shotgun's interference tube only works where the tube length is comparable to the wavelength.
- 47People forgive a soft picture and leave over bad soundDistance and soft furnishings set the ratio of voice to room; the microphone's price is not on that list.
- 48Direct sound, reverberant sound, and the distance between themDirect sound falls 6dB per doubling of distance while the reverberant field stays roughly constant, so the ratio that makes a recording sound professional is set by distance — and a porous absorber only works for frequencies whose quarter-wavelength is comparable to its thickness.
- 49Where the level should sitClipping replaces the waveform's peak with a flat line carrying harmonics that were never in the source and cannot be recovered, while recording quietly only moves you closer to a noise floor that 24-bit has room to spare above — so the two ends of the meter are not symmetrical and headroom is cheap.
- 50Two numbers doing two different jobsSample rate sets the frequency ceiling and anything above it folds back as an inharmonic alias, while bit depth sets dynamic range at about 6dB per bit — which is why 24-bit makes headroom cheap and why dither belongs only at the final reduction.
- 51What the noise floor costs you laterRoom noise is diffuse and roughly constant while the voice obeys the inverse square law, so every halving of distance buys 6dB of signal-to-noise for nothing — and spectral subtraction leaves a residue of isolated tonal bursts because the noise it subtracts is an average of something random.
- 52Two microphones and a comb filterA path difference of d metres puts the first cancellation at 343/(2d) hertz and repeats at odd multiples, so a 34cm difference notches the speech band at 504Hz upward — and a polarity flip, being frequency-independent, cannot fix a difference caused by time.
- 53Latency, and why a speaker starts stumblingBuffer latency is buffer size divided by sample rate and is paid both ways, so a 512-sample buffer at 48kHz is over 20ms round trip — past the point where hearing your own voice late disrupts speech — and direct monitoring in hardware sidesteps the whole chain.
- 54Two devices, one eventOffset is a constant to be slid and drift is a rate error to be stretched, and the commonest reported drift is not a clock problem at all but a 1000/1001 frame-rate mismatch worth 3.6 seconds an hour — while timecode fixes the offset and does nothing about the drift.
Module 7
Shaping and mixing sound
From a raw voice to a delivered mix: the three EQ moves that cover most speech, why compression makes sibilance worse, what pre-delay is really telling a listener, how a music bed and a voice stop competing, and the loudness and true-peak numbers a platform will measure you against.
By the end you can
Take a raw voice recording to a delivered mix — filtering and cutting before boosting, ordering the chain so each stage is not re-creating work for the next, placing the voice in a space without losing intelligibility, sitting music under it by arrangement rather than by ducking alone, and hitting a named integrated loudness and true-peak ceiling with a gain change rather than a limiter — and predict what the mix will do on a phone speaker before hearing it there
- 55Three EQ moves that cover most voicesHigh-pass below the speaker's fundamental, cut two to four decibels of mud between 200 and 400Hz, add a broad presence lift around 2 to 6kHz — and cut narrow while boosting wide, because problems are resonances and pleasantness is a shape.
- 56Compression does not make anything louderA compressor only turns loud parts down; the make-up gain afterwards is what raises everything, including whatever noise sits in the gaps.
- 57Wind, hiss, and why both are placement problemsA plosive is a jet of air rather than a loud sound, so angling the microphone 15 to 30 degrees off axis prevents what no filter fully repairs — and a de-esser is a compressor with a filtered sidechain, which belongs after compression because compression is what raised the sibilance.
- 58Reverb is information about wherePre-delay corresponds to the distance of the first reflecting surface at about 34cm per millisecond, and increasing it separates the voice from its own tail — which is why a muddy dialogue reverb is fixed with pre-delay before it is fixed with level.
- 59Music under speechA ducker can only react while a human automating the level can anticipate, so manual riding beats sidechain compression on edited material — and cutting a two to four decibel hole around 1.5 to 3kHz in the music costs less than the level drop it replaces.
- 60Three boring sounds beat any filterRoom tone plus one or two ordinary foley layers raises perceived quality more than any effect, and a Content ID claim is a match, not a verdict.
- 61The number every platform measuresLUFS is a K-weighted, gated loudness measure, so platforms normalise playback and a master squashed 6dB louder than the target simply gets turned down 6dB — and the true-peak ceiling of −1 dBTP exists because the reconstructed waveform can exceed any stored sample.
- 62What survives a phone speakerA phone speaker produces little below 500 to 700Hz, so bass is inferred from its harmonics through the missing-fundamental effect and a pure sine sub disappears entirely — which is why the 1 to 4kHz band, where intelligibility lives, is the one that has to be right.
- 63The chain, and why the order is not arbitraryEach stage changes what the next one sees, so repair precedes processing, subtractive EQ precedes compression, de-essing follows it, and the master carries only loudness and safety — anything more there is a decision that belonged further up.
Module 8
Machines that make sound and motion
Generative and analytic models in a motion and sound pipeline: what a text-to-speech system cannot read off the page, what a voice clone's embedding does and does not capture, why a five per cent word error rate lands on the words that matter, what a separator is estimating, and where copyright, licensing and disclosure are genuinely unresolved.
By the end you can
Use a generative or analytic model inside a motion and sound pipeline — synthesised speech, a cloned voice, automatic captions, source separation, generated music, interpolated or upscaled frames — and state for each what it is estimating, which characteristic failure to look for and where, what consent or licence the use requires, and which parts of the surrounding law are unsettled rather than merely complicated
- 64What a text-to-speech system cannot readWritten text does not encode stress, and stress is meaning — so a synthesiser guesses prosody from statistical regularities and produces a convincing reading of the wrong sentence, which is why a pronunciation lexicon and SSML are quality control rather than decoration.
- 65The vector that is supposed to be youA speaker embedding captures timbre — the resonances and average pitch that make a voice recognisable in a word — and not idiolect, the timing and stress habits of how a particular person speaks, which is why a clone convinces for three seconds and not for thirty.
- 66A cloned voice needs a yes you can point toConsent for a voice is scoped — to a project, a set of uses, a period, and whether the model itself is kept — and disclosure to the audience is a separate duty on top of it.
- 67Five per cent, and which five per centWord error rate is a uniform average over errors that are not uniformly distributed — they cluster on names, numbers and technical terms — and a generative recogniser can produce fluent sentences during silence, so no automatic transcript is publishable unread.
- 68Taking a mixture apartSeparation estimates a mask per source per time-frequency bin, so where two sources had energy in the same bin the split is a learned guess rather than a recovery — and because the masks partition the mixture, the stems sum back cleanly, which is what makes separate-fix-recombine a gentler repair than processing the whole mix.
- 69Generated music, and three separate legal questionsWhether training was lawful, whether an output infringes, and whether an output is owned are three different questions with different answers in different jurisdictions — and the practical response is records, terms, indemnities and checking the output, because the law is genuinely unsettled rather than merely complicated.
- 70What a music licence actually coversA recording needs both a sync licence for the composition and a master licence for the performance, royalty-free means no recurring royalty rather than no conditions, and a Creative Commons attribution requirement is a legal condition whose omission leaves you unlicensed.
- 71Frames that were never photographedInterpolation assumes every pixel has a correspondence in the neighbouring frame, so it tears at occlusion boundaries and fragments thin structures — and an upscaler does not reveal detail but fabricates it, which makes upscaled output unusable as a record of anything.
- 72Provenance, watermarks and what to tell peopleProvenance is a signed record of what cooperating tools reported doing and can be stripped by an ordinary re-encode, while detection is an inference that always trails the generators — so an absent credential or an undetected watermark is evidence of nothing, and disclosure is a separate question from either.