The challenges
Many Output Formats from One Input
A single video file can produce hundreds of files — five video resolutions, five audio bitrates per audio track, plus thumbnails and previews. We had to organize all of that cleanly and upload it fast.
Multiple Audio Streams per Video File
Some videos come with several audio tracks in different languages. The system had to detect them all automatically, convert each one separately, and build a master playlist that references every track with the right language tag.
Queue Reliability and Retry Logic
With SQS you need to handle duplicates, visibility timeouts, and failed jobs properly. If a job fails, the message should come back to the queue for retry. If the same file gets queued twice, we should not process it again.
Management on Kubernetes
Each conversion job uses a lot of CPU and memory. We had to set up ephemeral volumes, node affinity rules, and proper cleanup so jobs do not pile up or block each other.
Multi-Project Isolation on S3
The service runs for several projects at once. Each one needs its own output path on S3, and we had to support both publisher-based and series-based folder structures without breaking anything.
Automatic Codec Selection
Not every machine has GPU acceleration. The system needed to detect what codecs are available — NVIDIA, Intel Quick Sync, AMD, or just software — and pick the best one automatically.