AI vehicle video pipeline

The problem

A dealership has hundreds or thousands of cars and nobody with time to film each one. A short narrated video per vehicle sells better than photos alone, but only if the narration sounds right, including model names like “2500HD” or “4MATIC”.

My part

A colleague built the first scaffold, and the renderer is owned by another engineer. I own pronunciation, speech quality checks, background removal, the concurrency and retry machinery, model migrations, and the production cost and model study.

How it works

Image classification, then vehicle classification, script, pronunciation, audio, feature timing, render and post-processing. About 22 Lambda functions run under Step Functions, and a failed run can resume from the stage where it stopped, reusing cached images, script and audio.

What’s unusual

• A pronunciation compiler, not a prompt. Prioritised rules and an alias dictionary built from real inventory produce SSML, so “2500HD” becomes “twenty-five hundred H D”. The model is called only for real ambiguity, such as “SEAT” the brand versus “driver seat”.
• Customers can record how their name is said. The recording is converted to phonetic spelling (IPA), and candidates that sound wrong are rejected.
• Closed-loop speech QA. Whisper transcribes every generated clip, which is aligned to the expected text and regenerated if it fails. Whisper has a hard time budget so the check never times out the function.
• Audio cache keyed by content. When a pronunciation rule changes, only the affected audio is regenerated.
• Model choices settled by measurement. Cheap models are checked against a rule oracle on adversarial cases before any migration, then run in shadow before the switch.