feat: redesign audio networking for real-network resilience

Replaces the fire-and-forget datagram path with sequenced packets, a
per-peer jitter buffer, and persistent per-peer send tasks. Together these
fix three intertwined weaknesses that only showed up off localhost.

Packet format: every audio frame now carries a 4-byte little-endian
sequence number header ([seq][opus payload]), the basis for reordering and
loss detection.

Jitter buffer (core/jitter.rs): incoming packets are reordered by sequence
behind a fixed ~60ms playout delay. Missing sequences with later packets
already buffered are concealed via Opus PLC (decode(None)) -- a path the
decoder supported but nothing ever invoked. Underruns go idle and re-buffer
rather than concealing indefinitely. Covered by unit tests using real
encoded frames (reorder, gap-conceal, prime, late-drop).

Transport (network/iroh_impl.rs): each peer gets one long-lived send task
fed by a shallow bounded channel (drop-oldest on backpressure), instead of
spawning a throwaway task per peer per 20ms frame. Connections are now
established reactively on peer-join and torn down on peer-leave; the
lexicographically-lower EndpointId dials so a full-mesh pair forms exactly
one shared bidirectional connection instead of two racing ones. This also
removes the previous lock-held-across-connect().await serialization.

Opus decoder: PLC output is now sized to one 20ms frame, so concealment
synthesizes 20ms instead of a 120ms burst from the oversized max buffer.

Known follow-up (Tier 2): no reconnect on transient connection loss; a
send error currently retires the peer until they rejoin.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-05-29 15:48:43 -04:00
co-authored by Claude Opus 4.8
parent 7af0235736
commit 875e6e124c
5 changed files with 437 additions and 176 deletions
+28 -20
View File
@@ -31,44 +31,52 @@ impl AudioEncoder for OpusEncoder {
pub struct OpusDecoder {
decoder: Decoder,
channels: Channels,
/// Samples-per-channel of the frames we transmit (20ms @ 48kHz = 960).
/// Used to size the Packet Loss Concealment output, since libopus conceals
/// `frame_size` samples when given no input — passing the full max buffer
/// would synthesize a 120ms burst instead of a single 20ms frame.
frame_samples: usize,
}
impl OpusDecoder {
/// Creates a new Opus decoder.
/// Standard voice parameters: sample_rate = 48000, channels = Channels::Mono
pub fn new(sample_rate: u32, channels: Channels) -> Result<Self, CodecError> {
/// Standard voice parameters: sample_rate = 48000, channels = Channels::Mono.
/// `frame_samples` is the per-channel length of one transmitted frame (e.g. 960).
pub fn new(sample_rate: u32, channels: Channels, frame_samples: usize) -> Result<Self, CodecError> {
let decoder = Decoder::new(sample_rate, channels)
.map_err(|e| CodecError::Init(format!("Failed to create Opus decoder: {}", e)))?;
Ok(Self { decoder, channels })
Ok(Self { decoder, channels, frame_samples })
}
fn channels_count(&self) -> usize {
match self.channels {
Channels::Mono => 1,
Channels::Stereo => 2,
}
}
}
impl AudioDecoder for OpusDecoder {
fn decode(&mut self, compressed: Option<&[u8]>) -> Result<Vec<i16>, CodecError> {
// Maximum Opus frame size is 120ms. At 48kHz, this is 5760 samples per channel.
let channels_count = match self.channels {
Channels::Mono => 1,
Channels::Stereo => 2,
};
let max_samples = 5760 * channels_count;
let mut pcm = vec![0i16; max_samples];
let channels_count = self.channels_count();
let decoded_samples_per_channel = match compressed {
let (mut pcm, input): (Vec<i16>, &[u8]) = match compressed {
Some(data) if !data.is_empty() => {
// Normal decode
self.decoder.decode(data, &mut pcm, false)
.map_err(|e| CodecError::Decode(format!("Opus decoding failed: {}", e)))?
// Normal decode. Size the buffer to the maximum Opus frame (120ms =
// 5760 samples/channel); libopus decodes the packet's true duration.
(vec![0i16; 5760 * channels_count], data)
}
_ => {
// Packet Loss Concealment (PLC)
// In opus-rs, passing an empty slice triggers PLC.
self.decoder.decode(&[], &mut pcm, false)
.map_err(|e| CodecError::Decode(format!("Opus PLC decoding failed: {}", e)))?
// Packet Loss Concealment: an empty input makes libopus synthesize
// exactly `frame_samples` of concealment, so size the buffer to match.
(vec![0i16; self.frame_samples * channels_count], &[])
}
};
let total_samples = decoded_samples_per_channel * channels_count;
pcm.truncate(total_samples);
let decoded_per_channel = self.decoder.decode(input, &mut pcm, false)
.map_err(|e| CodecError::Decode(format!("Opus decoding failed: {}", e)))?;
pcm.truncate(decoded_per_channel * channels_count);
Ok(pcm)
}
}