Video transcoding and streaming
Video transcoding
When videos are uploaded to Tator, they must be transcoded to new files which serve one of two roles: streaming or optimized downloads. Videos transcoded for streaming enable all the features in our video player, including frame scrubbing, frame stepping, rewind, variable rate playback, fast seeking, etc. This format also enables rapid access to specific frames for extract, transform, load (ETL) workflows such as frame extraction, localization image extraction, track animations, and video clip extraction. Videos transcoded for optimized downloads are typically encoded using a higher quality codec such as h265 or AV1, and may be stored in a separate user-defined bucket.
Transcodes can take a significant amount of time, and Tator provides flexibility in where transcodes are processed. Tator deployments by default will initiate transcodes as Argo Workflows on the same Kubernetes cluster where Tator web services are running. When coupled with Cluster Autoscaler and Horizontal Pod Autoscaler, this enables massive scalability of transcode workloads. Deployment administrators may also specify a separate Kubernetes cluster to use for transcodes, which is useful in a variety of situations, such as when users want to use local resources for web hosting but cloud for transcoding, or to use a separate cloud provider for transcodes, or a different region. For users with adequate local processing resources, local transcodes are an option using the Tator python client. This may also be desirable for very large files, such as those recorded using ProRes or BlackMagic RAW formats to avoid having to upload the files to the cloud prior to initiating transcodes.
Streaming
Tator's video player supports a configurable number of streaming resolutions for each project and user-defined media type, allowing users to balance load speed with streaming quality for their connection speed. By default, Tator will encode 144p, 360p, 480p, 720p, and 1080p resolutions (up to the native resolution of the media). Different resolutions are used for different purposes within the player. For example, the scrub buffer uses a lower resolution to allow scrubbing shortly after the video is loaded, whereas the play buffer is higher quality, but may not be the highest quality available depending on the user's internet connection speed. On pause, the player will retrieve the frame corresponding to the highest quality available to ensure the best possible accuracy of annotations. The resolution used for each streaming task is configurable via the player's advanced settings. Tator also supports simultanous streaming and playback of synchronized videos, enabling recording configurations such as stereo video or multi-camera monitoring/surveillance.