Skip to content

Events & Alarms

bilbycast-relay generates operational events and forwards them to bilbycast-manager via WebSocket. Events provide real-time visibility into edge connections, tunnel state changes, and manager connectivity.

Events are sent as WebSocket messages with type "event":

{
"type": "event",
"timestamp": "2026-04-02T12:00:00Z",
"payload": {
"severity": "warning",
"category": "tunnel",
"message": "Tunnel bind rejected: invalid token",
"flow_id": "550e8400-e29b-41d4-a716-446655440000"
}
}
Severity Meaning Action
critical Service-impacting failure Operator should investigate immediately
warning Degradation or potential issue Operator should investigate when possible
info Notable state change No action required, operational awareness
Field Type Required Description
severity string yes "info", "warning", or "critical"
category string yes Event category (see tables below)
message string yes Human-readable description
flow_id string no Tunnel UUID (for tunnel-scoped events)
details object no Structured context

Events are queued in an unbounded in-memory channel. When the relay is not connected to the manager, events accumulate and are delivered once the connection is re-established.


Severity Message Trigger Details
info Edge connected from {addr} New QUIC connection accepted from an edge node { remote_addr }
info Edge disconnected from {addr} Edge QUIC connection closed { remote_addr }
warning Edge connection failed: control stream error from {addr} Failed to accept bidirectional control stream { remote_addr }
warning Connection rejected: per-IP cap exceeded ({n} active from {ip}) New QUIC connection dropped at handshake — per-IP connection cap (max_connections_per_ip, default 64) reached. DoS mitigation { error_code: "relay_dos_suspect", remote_addr, remote_ip, active_connections, cap }
warning Protocol version mismatch (edge={v}, relay={v}) Edge Hello message version differs from relay { edge_version, relay_version }
warning QUIC connection accept failed: {error} QUIC/TLS handshake failure at the server level { error }

Source: src/session.rs, src/server.rs


Severity Message Trigger Details
info Tunnel active (both sides bound) Both ingress and egress edges have bound { direction }
info Tunnel waiting: {direction} side bound Only one side has bound, waiting for peer { direction }
info Tunnel unbound by edge Edge sent TunnelUnbind
warning Tunnel bind rejected: invalid token HMAC-SHA256 bind token verification failed { remote_addr }
warning Tunnel bind rejected: per-connection cap exceeded TunnelBind rejected — per-connection tunnel-bind cap (max_tunnels_per_connection, default 100) reached on this connection. DoS mitigation; relay replies TunnelDown { reason: "per-connection tunnel limit exceeded" } { error_code: "relay_dos_suspect", remote_addr, active_binds, cap }
warning Tunnel down: edge disconnected Edge QUIC connection lost, affecting bound tunnel
warning Native-UDP register rejected: invalid token Register on the native plain-UDP plane (:4434) failed bind-token verification { remote_addr, transport: "udp" }
warning Native-UDP register rejected: per-IP session cap exceeded Register on the native plain-UDP plane dropped — per-IP session cap reached. DoS mitigation { error_code: "relay_dos_suspect", remote_addr, remote_ip, transport: "udp" }
warning Native-UDP register rejected: implausible source address Register on the native plain-UDP plane arrived from an address that can never be a legitimate slot occupant. reason: "unroutable_source" — an address no host can own (unspecified / multicast / broadcast / port 0); reason: "both_slots_same_address" — the tunnel’s opposite slot already holds exactly this address, which would make the session forward to its own source (an unbounded loop if that address routes back to the relay). Not latchable — a slot is a send target { error_code: "relay_invalid_source_addr", reason, remote_addr, remote_ip, transport: "udp" }
warning Native-UDP register refused: slot held by a live peer Register tried to move a slot that is currently carrying media from a different source IP. Refused for 12 s after that slot’s last media datagram, so a flowing contribution feed cannot be redirected by one unauthenticated datagram. A same-IP port rebind is always allowed; a genuine WAN-IP change recovers once the incumbent goes quiet { error_code: "relay_slot_takeover_refused", remote_addr, remote_ip, transport: "udp" }
warning Native-UDP plane accepts unauthenticated registers Emitted once at startup when require_bind_auth: false and udp_relay_enabled: true. For any tunnel the manager has not pre-authorised, a Register on the native plane can move a slot (media hijack), not merely join a tunnel as on QUIC. See Security — the permissive default { error_code: "relay_native_plane_unauthenticated", require_bind_auth: false, udp_relay_enabled: true }

The flow_id field contains the tunnel UUID for all tunnel events.

Source: src/session.rs, src/udp_relay.rs


Severity Message Trigger
info Connected to manager WebSocket auth succeeded (auth_ok or register_ack)
warning Manager connection lost, rotating to next URL WebSocket closed normally
warning Manager connection lost, rotating to next URL: {error} WebSocket errored
critical Manager authentication failed: {reason} Auth rejected by manager

Source: src/manager/client.rs


Severity Message Trigger
info Secret rotated successfully rotate_secret command completed
warning Credential persistence failed: {error} Failed to write credentials to config file after rotation

Source: src/manager/client.rs


Emitted only by a relay built with the optional viewer-distribution role (default-off Cargo feature; the shipped distribution artefacts use viewer-distribution-vendored). A plain opaque-forwarder relay never emits these; they appear only on a distribution-role build that runs the WHEP SFU + LL-HLS/CMAF origin.

Severity Message Trigger Details
info WHIP ingest opened for stream ‘{stream}’ A WHIP-in ingest began feeding a stream’s hub { stream }
info distribution ingest opened for stream ‘{stream}’ A QUIC ES ingest began feeding a stream’s hub { stream, has_audio }
info distribution ingest closed for stream ‘{stream}’ A QUIC ES ingest stopped; the stream tears down { stream }
warning per-IP viewer cap ({cap}) reached from {ip} A WHEP viewer request was rejected — per-source-IP concurrent-viewer cap (max_viewers_per_ip) reached { ip, cap }
warning WHEP viewer: off-path datagram dropped by the media source pin A WHEP session received a datagram from an address other than the one that completed the DTLS handshake, and dropped it before the WebRTC stack saw it. Emitted once per session. Either a spoofed ICE nomination trying to redirect the SRTP flow at a third party, or — more often — a legitimate viewer whose public IP changed mid-session (Wi-Fi to cellular, CGNAT rotation), who will see the player go black until they reload { error_code: "webrtc_offpath_source", stream, session_id, pinned_ip, source_addr, source_ip }

Source: src/distribution/


In addition to events sent by the relay, the manager itself generates these events:

Severity Category Message Trigger
info connection Node connected to manager Relay successfully authenticates
warning compatibility Node WS protocol version differs Protocol version mismatch during auth
critical connection Node disconnected from manager Relay WebSocket closes

These are generated server-side in bilbycast-manager/crates/manager-server/src/ws/node_hub.rs.


Category Count Description
edge 6 Edge QUIC connection lifecycle (with structured details), incl. per-IP DoS cap and QUIC accept failures
tunnel 11 Tunnel state changes, authentication, lifecycle (waiting, unbound), per-connection DoS cap, native plain-UDP register rejections (invalid token + per-IP session cap + implausible source address), live-slot takeover refusals, and the startup posture warning
manager 6 Manager connection and credential management
distribution 5 Viewer-distribution ingest lifecycle, per-IP viewer cap, and the WHEP media source-pin drop (only on viewer-distribution builds)
Total 28

The always-compiled edge, tunnel, and manager categories account for 23 events. The distribution category (5 events) is present only when the relay is built with the optional viewer-distribution role.

Three of these — the per-IP connection cap (edge), the per-connection tunnel-bind cap (tunnel), and the native-UDP per-IP session cap (tunnel) — carry the structured error_code: "relay_dos_suspect" in their details. This is the identifier the security-hardening guidance tells operators to monitor for DoS-cap rejections.

Severity Count Description
critical 1 Manager authentication failure
warning 17 Disconnects, bind rejections, protocol mismatches, QUIC accept failures, persistence failures, DoS-cap rejections (per-IP + per-connection + native-UDP session cap), native-UDP token rejections, implausible source addresses, live-slot takeover refusals, the unauthenticated-native-plane posture warning, per-IP viewer cap, WHEP source-pin drops
info 10 Connections, tunnel activation/waiting/unbound, secret rotation, distribution ingest lifecycle

The always-compiled event surface (excluding the optional distribution category) tallies 23: critical=1, warning=15, info=7.