Skip to content
BilbycastBilbycast

Codec Matrix

The release binary bundles every video encoder + decoder backend the edge knows about — libx264, libx265, NVIDIA NVENC + NVDEC, Intel QSV (x86_64 only), VAAPI, and Rockchip RKMPP (h264_rkmpp / hevc_rkmpp encode + RKMPP decode, shipped in the aarch64-linux-rockchip artefact). At startup the hardware probe walks the active set and opens a minimal session against each; that result reaches the manager on resource_budget.hw_encoders / hw_encoder_chroma. HealthPayload.capabilities is a different question — the decoder, display and sdi-decklink bits there are probe-gated, but the video-encoder-* bits only say what was compiled in (see Capability strings below). The manager UI keys per-output codec dropdowns off both.

This page covers the static support matrix, the *_auto resolver that does the right thing without operator hand-holding, and the verification commands for confirming what activated on a given host.

Pick *_auto unless you have a reason not to

Section titled “Pick *_auto unless you have a reason not to”

For most operators the right answer on video_encode.codec is h264_auto or hevc_auto. The edge resolves Auto on flow start against the host’s probed capabilities and the requested chroma + bit-depth, picking the cheapest backend that handles the combination:

Family + chroma + bit-depth Resolver chain (head → tail)
H.264 + 4:2:0 + 8-bit (the dominant distribution path) RKMPP (Rockchip) ≻ NVENC ≻ QSV ≻ VAAPI ≻ libx264
H.264 + anything else libx264 (no HW H.264 backend supports 4:2:2 / 10-bit / 4:4:4)
HEVC + 4:2:0 + 8-bit RKMPP (Rockchip) ≻ NVENC ≻ QSV ≻ VAAPI ≻ libx265
HEVC + 4:2:0 + 10-bit NVENC ≻ VAAPI ≻ QSV ≻ libx265
HEVC + 4:2:2 + 8-bit VAAPI (Intel iHD) ≻ libx265
HEVC + 4:2:2 + 10-bit VAAPI (Intel iHD) ≻ libx265
HEVC + 4:4:4 (any) libx265 (HW backends reject NV24 today)

Operators who need wire-level reproducibility (compliance deployments, vendor-specific bitstream behaviour) pick an explicit backend — the manager UI cross-validates the chroma + bit-depth combo before flow start, so an invalid combination fails at save time, not at flow bring-up.

Video encode — pixel-format / bit-depth matrix

Section titled “Video encode — pixel-format / bit-depth matrix”
Backend 4:2:0 / 8 4:2:2 / 8 4:2:0 / 10 4:2:2 / 10 4:4:4
libx264 (CPU, video-encoder-x264) ✓ ✓ ✓ ✓ ✓
libx265 (CPU, video-encoder-x265) ✓ ✓ ✓ ✓ ✓
h264_nvenc (NVIDIA, video-encoder-nvenc) ✓ ✗ ✗ ✗ ✗
hevc_nvenc (NVIDIA, video-encoder-nvenc) ✓ ✗ ✓ ✗ ✗
h264_qsv (Intel, video-encoder-qsv) ✓ ✗ ✗ ✗ ✗
hevc_qsv (Intel, video-encoder-qsv) ✓ ✗ ✓ ✗ ✗
h264_vaapi (Linux, video-encoder-vaapi) ✓ ✗ ✗ ✗ ✗
hevc_vaapi (Intel iHD Tiger Lake+) ✓ ✓ ✓ ✓ ✗ (NV24 deferred)
hevc_vaapi (AMD radeonsi) ✓ usually ✗ ✓ usually ✗ ✗
h264_rkmpp (ARM Rockchip RK3568 / RK3588, video-encoder-rkmpp) ✓ ✗ ✗ ✗ ✗
hevc_rkmpp (ARM Rockchip RK3568 / RK3588) ✓ ✗ ✗ ✗ ✗

Implication for broadcast 4:2:2 contribution: on NVIDIA-only and AMD-only hosts there is no GPU path for 4:2:2 10-bit; libx265 is the only option. Intel Tiger Lake+ iGPUs have the full broadcast HEVC matrix through VAAPI.

Static rejection happens at config validation and at encoder open, so an operator who bypasses the manager UI still gets a precise error before any frame is encoded.

Video decode — coverage by path

Section titled “Video decode — coverage by path”
Backend Display output (KMS) TS → TS transcode Notes
libavcodec (CPU) ✓ (fallback) ✓ (universal) Every format, every bit-depth.
NVDEC (video-decoder-nvdec) ✓ zero-copy scanout ✓ x86_64; 4:2:0 only.
QSV decode (video-decoder-qsv) ✓ ✓ x86_64; 4:2:0 only.
VAAPI decode (video-decoder-vaapi) ✓ zero-copy DMA-BUF ✓ Linux; Intel iHD does 4:2:2 + Main10.
RKMPP decode (video-decoder-rkmpp) ✓ (sysmem NV12) ✓ aarch64 Rockchip RK3568 / RK3588.

Auto resolution priority — both the display path and the transcode input-decode path share the same priority:

VAAPI ≻ NVDEC ≻ QSV ≻ RKMPP ≻ CPU

RKMPP only ever compiles into the aarch64-linux-rockchip artefact, where none of VAAPI / NVDEC / QSV can resolve — so on that platform it is the first choice, ahead of the CPU last resort.

The display path also relies on KMS atomic-commit zero-copy via DRM PRIME (see Display Output); the transcode path doesn’t get the same fast scanout but still benefits from offloading decode to dedicated silicon.

Audio codec matrix

Section titled “Audio codec matrix”
Codec Encode Decode Sample rates Channels Notes
AAC-LC FDK-AAC in-process (fdk-aac) FDK-AAC 8 k – 48 k 1 – 8 Production AAC, lowest latency.
HE-AAC v1 FDK-AAC FDK-AAC 8 k – 48 k 1 – 2 SBR adds chroma-bandwidth dependency.
HE-AAC v2 FDK-AAC FDK-AAC 8 k – 48 k 2 (stereo only) Parametric Stereo — manager UI hard-filters mono.
Opus libopus via libavcodec libopus via libavcodec 48 k 1 – 2 Decode wired; transcode-from-Opus-on-TS works.
MP2 libavcodec libavcodec 32 k / 48 k 1 – 2 DVB-T / SD broadcast.
AC-3 libavcodec libavcodec 32 / 44.1 / 48 k 1 – 6 (5.1) ATSC / Blu-ray.
E-AC-3 (passthrough only) libavcodec — 1 – 7.1 UHD ATSC 3.0.
PCM (ST 2110-30 / -31) passthrough passthrough 48 k / 96 k 1 – 16 Hardcoded sample-rate in config.
SMPTE 302M inline mux inline demux matches input matches input transport_mode: audio_302m on SRT / UDP / rtp_audio.

Capability strings on HealthPayload.capabilities

Section titled “Capability strings on HealthPayload.capabilities”

Two classes of bit sit in this table, and the difference matters. The video-encoder-* bits — and mv-compositor — say only that the backend is compiled into this binary: no probe is consulted, so every node running the *-x86_64-linux-full artefact advertises video-encoder-nvenc, -qsv and -vaapi whether or not it has silicon that can open one of them. The video-decoder-*, display and sdi-decklink bits additionally require the boot probe to find the driver / GPU / card usable. Anything missing is disabled in the manager UI dropdowns with a tooltip.

The table below is the codec and media subset — an edge advertises considerably more (replay, clocking, network, bonding and tunnel surfaces among them). bilbycast-edge --print-capabilities lists what a given binary carries without starting it, though probe-gated bits are absent from that cold listing by design.

Capability Source Meaning
video-encode any of x264 / x265 / nvenc / qsv / rkmpp Edge can re-encode video at all.
video-encoder-x264 video-encoder-x264 libx264 available.
video-encoder-x265 video-encoder-x265 libx265 available.
video-encoder-nvenc video-encoder-nvenc — compiled in, no probe h264_nvenc / hevc_nvenc are in the binary. Whether this host can open them is on resource_budget.hw_encoders, not here.
video-encoder-qsv video-encoder-qsv — compiled in, no probe h264_qsv / hevc_qsv are in the binary; runtime answer on resource_budget.hw_encoders.
video-encoder-vaapi video-encoder-vaapi — compiled in, no probe h264_vaapi / hevc_vaapi are in the binary; runtime answer on resource_budget.hw_encoders.
video-encoder-rkmpp video-encoder-rkmpp — compiled in, no probe h264_rkmpp / hevc_rkmpp are in the binary (Rockchip RK3568 / RK3588); runtime answer on resource_budget.hw_encoders.
video-decoder-nvdec video-decoder-nvdec + probe NVDEC available (transcode + display).
video-decoder-qsv video-decoder-qsv + probe QSV decode available.
video-decoder-vaapi video-decoder-vaapi + probe VAAPI decode available.
video-decoder-rkmpp video-decoder-rkmpp + probe RKMPP decode available (Rockchip RK3568 / RK3588).
display display + ≥ 1 KMS connector enumerated Local-display output usable.
mv-compositor multiviewer + one of video-encoder-x264 / -x265 / -nvenc / -qsv Mosaic compositor (multiviewer wall) usable. Both halves are required: the flow bus carries MPEG-TS, so a canvas reaches an output only by being encoded and muxed. A build with the feature and no such encoder advertises nothing and the manager hides the wall surfaces entirely — nothing in the UI explains the absence, so check bilbycast-edge --print-capabilities on the node. Only those four encoder features satisfy the gate: a self-built binary whose only encoder is VAAPI or RKMPP does not advertise the capability today. All three published release artefacts carry video-encoder-x264, so any node on a published binary advertises it. The wall’s canvas is encoded with the best of those the host can open, hardware first — so QuickSync, NVENC, VAAPI or RKMPP carries it and libx264 is the floor, not the default, whatever the host’s GPU or VPU could do. Releases up to and including v0.105.0 encoded every wall on CPU libx264 regardless of the host (edge #129) — budget CPU on those builds; from v0.106.0 the canvas codec resolves through the probe hardware-first, so the RK3588 VPU carries the wall on the Rockchip artefact and QuickSync / NVENC / VAAPI carry it elsewhere. Ordinary transcoding outputs are unaffected and resolve encoders normally. See Multiviewer.
sdi-decklink sdi-decklink + Desktop Video reachable + ≥ 1 card present now SDI capture / playout usable. Driven by the status poller, so it tracks card hot-plug in both directions rather than freezing the boot enumeration. Shipped in the two *-linux-full artefacts from v0.103.0; never in *-rockchip. See SDI.
fdk-aac fdk-aac In-process AAC family.
media-codecs media-codecs (default on) libavcodec video decode + Opus / MP2 / AC-3.
webrtc webrtc (default on) WHIP / WHEP supported.
tls tls (default on) HTTPS + RTMPS.
replay replay (default on) Recording + clip playback.

The resource_budget.hw_encoders and resource_budget.hw_encoder_chroma blocks on the same payload carry the per-backend runtime answer and the per-(codec, chroma, bit-depth) matrix, one boolean per cell — that pair, not the capability strings above, is the source of truth the manager UI reads when it grays out an encoder this host cannot open, or 4:2:2 chroma against NVENC.

What *_auto activates on each host class

Section titled “What *_auto activates on each host class”

Static support is one thing; what your host actually opens at runtime is another. Common combinations:

Host class Auto resolves to (H.264 4:2:0 8-bit) Auto resolves to (HEVC 4:2:2 10-bit)
x86_64 NVIDIA NVENC libx265 (NVENC rejects 4:2:2)
x86_64 Intel Tiger Lake+ iGPU QSV (or VAAPI if explicitly preferred) VAAPI (iHD does 4:2:2 + Main10)
x86_64 AMD VAAPI libx265 (AMD radeonsi rejects 4:2:2)
aarch64 NVIDIA Jetson NVENC libx265
aarch64 AMD APU SBC VAAPI libx265
aarch64 Rockchip RK3568 / RK3588 RKMPP libx265 (RKMPP is 8-bit 4:2:0 only)
Headless / no GPU libx264 libx265

Verification — confirm Auto activates the right backend

Section titled “Verification — confirm Auto activates the right backend”

  1. Inspect the host capability matrix. Start the edge and curl /api/v1/stats/health (or open the manager Node detail → Resources card):

    Terminal window
    curl https://edge:8443/api/v1/stats/health \
         -H "Authorization: Bearer $TOKEN" | \
      jq '.resource_budget.hw_encoder_chroma'

    On Tiger Lake+ Intel iHD: hevc_vaapi_yuv422_10bit = true. On AMD radeonsi: hevc_vaapi_yuv422_10bit = false. On any host with NVENC: no hevc_nvenc_yuv422_* key is present at all — NVENC has no 4:2:2 path on any GPU generation, so the matrix carries no such cell rather than a false one. The only NVENC cell in the block is hevc_nvenc_yuv420_10bit.

  2. Confirm Auto resolution lands on the expected backend. Create a flow with video_encode.codec = "h264_auto", chroma yuv420p, 8-bit. Watch the edge logs for:

    video_encode auto-resolved 'h264_auto' → <backend>

    On an NVIDIA host this prints → nvenc_h264; on Intel Tiger Lake+ → qsv_h264 (or → vaapi_h264 with explicit preference); on AMD → vaapi_h264; on a host without any compiled-in HW backend that the probe could open → libx264.

  3. Confirm an invalid combination is rejected up front. Create a flow with video_encode.codec = "h264_nvenc", chroma yuv422p. The manager preflight returns HTTP 422 + error_code: encoder_chroma_not_supported before the WS round-trip — the flow never reaches the edge.

  4. Confirm HW transcode decode activates. Set video_encode.hw_decode = "auto" (the default) on a flow that decodes a source video. Edge logs:

    ts_video_replace: opened HW decoder (backend=Vaapi)

    (or Nvdec / Qsv). Running perf top against the edge process should show av_hwframe_transfer_data in the hot path (sysmem download) but no libavcodec_decode_h264 — the decode itself ran on the GPU.

Cost model — pixel-rate aware

Section titled “Cost model — pixel-rate aware”

Per-flow cost units scale with pixel rate. The HW vs SW base is the dominant axis: HW backends start at base 100, software at base 500.

units = base × (width × height × fps) / (1920 × 1080 × 30)
        × 1.5    if bit_depth == 10
        × 1.33   if chroma == yuv422p
        × 2.0    if chroma == yuv444p

The result is rounded, then clamped to [base, 100 000] — so any HW encode costs at least 100 units and any SW encode at least 500, however small the raster.

Examples:

Profile Approximate units
1080p25 H.264 4:2:0 8-bit on NVENC 100 (83 computed, floored at the HW base)
1080p50 H.264 4:2:0 8-bit on NVENC (3G-SDI tier-1) 167
1080p59.94 H.264 4:2:0 8-bit on NVENC 200
1080p50 HEVC 4:2:2 10-bit on NVENC (broadcast contribution) 333
4K30 H.264 4:2:0 8-bit on NVENC 400
1080p50 H.264 4:2:0 8-bit on libx264 833
4K50 HEVC 4:2:2 10-bit on libx265 (broadcast contribution) ~6 650
4K59.94 HEVC 4:2:2 10-bit on libx265 (broadcast contribution) ~7 980
4K59.94 HEVC 4:2:0 8-bit on libx265 ~4 000

The per-host budget is 1000 + 200 × physical_cores, so a 4-core edge gets 1 800 units, a 32-core EPYC gets 7 400. See Resources & Capacity for the per-family HW session caps and how oversubscription is surfaced.

See also

Section titled “See also”