Events & Alarms
bilbycast-relay generates operational events and forwards them to bilbycast-manager via WebSocket. Events provide real-time visibility into edge connections, tunnel state changes, and manager connectivity.
Event Protocol
Section titled “Event Protocol”Events are sent as WebSocket messages with type "event":
{ "type": "event", "timestamp": "2026-04-02T12:00:00Z", "payload": { "severity": "warning", "category": "tunnel", "message": "Tunnel bind rejected: invalid token", "flow_id": "550e8400-e29b-41d4-a716-446655440000" }}Severity Levels
Section titled “Severity Levels”| Severity | Meaning | Action |
|---|---|---|
critical |
Service-impacting failure | Operator should investigate immediately |
warning |
Degradation or potential issue | Operator should investigate when possible |
info |
Notable state change | No action required, operational awareness |
Event Fields
Section titled “Event Fields”| Field | Type | Required | Description |
|---|---|---|---|
severity |
string | yes | "info", "warning", or "critical" |
category |
string | yes | Event category (see tables below) |
message |
string | yes | Human-readable description |
flow_id |
string | no | Tunnel UUID (for tunnel-scoped events) |
details |
object | no | Structured context |
Buffering
Section titled “Buffering”Events are queued in an unbounded in-memory channel. When the relay is not connected to the manager, events accumulate and are delivered once the connection is re-established.
Event Reference
Section titled “Event Reference”Edge Connections (edge)
Section titled “Edge Connections (edge)”| Severity | Message | Trigger | Details |
|---|---|---|---|
| info | Edge connected from {addr} | New QUIC connection accepted from an edge node | { remote_addr } |
| info | Edge disconnected from {addr} | Edge QUIC connection closed | { remote_addr } |
| warning | Edge connection failed: control stream error from {addr} | Failed to accept bidirectional control stream | { remote_addr } |
| warning | Connection rejected: per-IP cap exceeded ({n} active from {ip}) | New QUIC connection dropped at handshake — per-IP connection cap (max_connections_per_ip, default 64) reached. DoS mitigation |
{ error_code: "relay_dos_suspect", remote_addr, remote_ip, active_connections, cap } |
| warning | Protocol version mismatch (edge={v}, relay={v}) | Edge Hello message version differs from relay | { edge_version, relay_version } |
| warning | QUIC connection accept failed: {error} | QUIC/TLS handshake failure at the server level | { error } |
Source: src/session.rs, src/server.rs
Tunnels (tunnel)
Section titled “Tunnels (tunnel)”| Severity | Message | Trigger | Details |
|---|---|---|---|
| info | Tunnel active (both sides bound) | Both ingress and egress edges have bound | { direction } |
| info | Tunnel waiting: {direction} side bound | Only one side has bound, waiting for peer | { direction } |
| info | Tunnel unbound by edge | Edge sent TunnelUnbind | |
| warning | Tunnel bind rejected: invalid token | HMAC-SHA256 bind token verification failed | { remote_addr } |
| warning | Tunnel bind rejected: per-connection cap exceeded | TunnelBind rejected — per-connection tunnel-bind cap (max_tunnels_per_connection, default 100) reached on this connection. DoS mitigation; relay replies TunnelDown { reason: "per-connection tunnel limit exceeded" } |
{ error_code: "relay_dos_suspect", remote_addr, active_binds, cap } |
| warning | Tunnel down: edge disconnected | Edge QUIC connection lost, affecting bound tunnel | |
| warning | Native-UDP register rejected: invalid token | Register on the native plain-UDP plane (:4434) failed bind-token verification |
{ remote_addr, transport: "udp" } |
| warning | Native-UDP register rejected: per-IP session cap exceeded | Register on the native plain-UDP plane dropped — per-IP session cap reached. DoS mitigation |
{ error_code: "relay_dos_suspect", remote_addr, remote_ip, transport: "udp" } |
| warning | Native-UDP register rejected: implausible source address | Register on the native plain-UDP plane arrived from an address that can never be a legitimate slot occupant. reason: "unroutable_source" — an address no host can own (unspecified / multicast / broadcast / port 0); reason: "both_slots_same_address" — the tunnel’s opposite slot already holds exactly this address, which would make the session forward to its own source (an unbounded loop if that address routes back to the relay). Not latchable — a slot is a send target |
{ error_code: "relay_invalid_source_addr", reason, remote_addr, remote_ip, transport: "udp" } |
| warning | Native-UDP register refused: slot held by a live peer | Register tried to move a slot that is currently carrying media from a different source IP. Refused for 12 s after that slot’s last media datagram, so a flowing contribution feed cannot be redirected by one unauthenticated datagram. A same-IP port rebind is always allowed; a genuine WAN-IP change recovers once the incumbent goes quiet |
{ error_code: "relay_slot_takeover_refused", remote_addr, remote_ip, transport: "udp" } |
| warning | Native-UDP plane accepts unauthenticated registers | Emitted once at startup when require_bind_auth: false and udp_relay_enabled: true. For any tunnel the manager has not pre-authorised, a Register on the native plane can move a slot (media hijack), not merely join a tunnel as on QUIC. See Security — the permissive default |
{ error_code: "relay_native_plane_unauthenticated", require_bind_auth: false, udp_relay_enabled: true } |
The flow_id field contains the tunnel UUID for all tunnel events.
Source: src/session.rs, src/udp_relay.rs
Manager Connection (manager)
Section titled “Manager Connection (manager)”| Severity | Message | Trigger |
|---|---|---|
| info | Connected to manager | WebSocket auth succeeded (auth_ok or register_ack) |
| warning | Manager connection lost, rotating to next URL | WebSocket closed normally |
| warning | Manager connection lost, rotating to next URL: {error} | WebSocket errored |
| critical | Manager authentication failed: {reason} | Auth rejected by manager |
Source: src/manager/client.rs
Configuration / Security (manager)
Section titled “Configuration / Security (manager)”| Severity | Message | Trigger |
|---|---|---|
| info | Secret rotated successfully | rotate_secret command completed |
| warning | Credential persistence failed: {error} | Failed to write credentials to config file after rotation |
Source: src/manager/client.rs
Viewer Distribution (distribution)
Section titled “Viewer Distribution (distribution)”Emitted only by a relay built with the optional viewer-distribution role (default-off Cargo feature; the shipped distribution artefacts use viewer-distribution-vendored). A plain opaque-forwarder relay never emits these; they appear only on a distribution-role build that runs the WHEP SFU + LL-HLS/CMAF origin.
| Severity | Message | Trigger | Details |
|---|---|---|---|
| info | WHIP ingest opened for stream ‘{stream}’ | A WHIP-in ingest began feeding a stream’s hub | { stream } |
| info | distribution ingest opened for stream ‘{stream}’ | A QUIC ES ingest began feeding a stream’s hub | { stream, has_audio } |
| info | distribution ingest closed for stream ‘{stream}’ | A QUIC ES ingest stopped; the stream tears down | { stream } |
| warning | per-IP viewer cap ({cap}) reached from {ip} | A WHEP viewer request was rejected — per-source-IP concurrent-viewer cap (max_viewers_per_ip) reached |
{ ip, cap } |
| warning | WHEP viewer: off-path datagram dropped by the media source pin | A WHEP session received a datagram from an address other than the one that completed the DTLS handshake, and dropped it before the WebRTC stack saw it. Emitted once per session. Either a spoofed ICE nomination trying to redirect the SRTP flow at a third party, or — more often — a legitimate viewer whose public IP changed mid-session (Wi-Fi to cellular, CGNAT rotation), who will see the player go black until they reload | { error_code: "webrtc_offpath_source", stream, session_id, pinned_ip, source_addr, source_ip } |
Source: src/distribution/
Manager-Generated Events
Section titled “Manager-Generated Events”In addition to events sent by the relay, the manager itself generates these events:
| Severity | Category | Message | Trigger |
|---|---|---|---|
| info | connection | Node connected to manager | Relay successfully authenticates |
| warning | compatibility | Node WS protocol version differs | Protocol version mismatch during auth |
| critical | connection | Node disconnected from manager | Relay WebSocket closes |
These are generated server-side in bilbycast-manager/crates/manager-server/src/ws/node_hub.rs.
Event Categories Summary
Section titled “Event Categories Summary”| Category | Count | Description |
|---|---|---|
edge |
6 | Edge QUIC connection lifecycle (with structured details), incl. per-IP DoS cap and QUIC accept failures |
tunnel |
11 | Tunnel state changes, authentication, lifecycle (waiting, unbound), per-connection DoS cap, native plain-UDP register rejections (invalid token + per-IP session cap + implausible source address), live-slot takeover refusals, and the startup posture warning |
manager |
6 | Manager connection and credential management |
distribution |
5 | Viewer-distribution ingest lifecycle, per-IP viewer cap, and the WHEP media source-pin drop (only on viewer-distribution builds) |
| Total | 28 |
The always-compiled edge, tunnel, and manager categories account for 23 events. The distribution category (5 events) is present only when the relay is built with the optional viewer-distribution role.
Three of these — the per-IP connection cap (edge), the per-connection tunnel-bind cap (tunnel), and the native-UDP per-IP session cap (tunnel) — carry the structured error_code: "relay_dos_suspect" in their details. This is the identifier the security-hardening guidance tells operators to monitor for DoS-cap rejections.
By Severity
Section titled “By Severity”| Severity | Count | Description |
|---|---|---|
| critical | 1 | Manager authentication failure |
| warning | 17 | Disconnects, bind rejections, protocol mismatches, QUIC accept failures, persistence failures, DoS-cap rejections (per-IP + per-connection + native-UDP session cap), native-UDP token rejections, implausible source addresses, live-slot takeover refusals, the unauthenticated-native-plane posture warning, per-IP viewer cap, WHEP source-pin drops |
| info | 10 | Connections, tunnel activation/waiting/unbound, secret rotation, distribution ingest lifecycle |
The always-compiled event surface (excluding the optional distribution category) tallies 23: critical=1, warning=15, info=7.