feat: redesign audio networking for real-network resilience
Replaces the fire-and-forget datagram path with sequenced packets, a per-peer jitter buffer, and persistent per-peer send tasks. Together these fix three intertwined weaknesses that only showed up off localhost. Packet format: every audio frame now carries a 4-byte little-endian sequence number header ([seq][opus payload]), the basis for reordering and loss detection. Jitter buffer (core/jitter.rs): incoming packets are reordered by sequence behind a fixed ~60ms playout delay. Missing sequences with later packets already buffered are concealed via Opus PLC (decode(None)) -- a path the decoder supported but nothing ever invoked. Underruns go idle and re-buffer rather than concealing indefinitely. Covered by unit tests using real encoded frames (reorder, gap-conceal, prime, late-drop). Transport (network/iroh_impl.rs): each peer gets one long-lived send task fed by a shallow bounded channel (drop-oldest on backpressure), instead of spawning a throwaway task per peer per 20ms frame. Connections are now established reactively on peer-join and torn down on peer-leave; the lexicographically-lower EndpointId dials so a full-mesh pair forms exactly one shared bidirectional connection instead of two racing ones. This also removes the previous lock-held-across-connect().await serialization. Opus decoder: PLC output is now sized to one 20ms frame, so concealment synthesizes 20ms instead of a 120ms burst from the oversized max buffer. Known follow-up (Tier 2): no reconnect on transient connection loss; a send error currently retires the peer until they rejoin. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
+28
-20
@@ -31,44 +31,52 @@ impl AudioEncoder for OpusEncoder {
|
||||
pub struct OpusDecoder {
|
||||
decoder: Decoder,
|
||||
channels: Channels,
|
||||
/// Samples-per-channel of the frames we transmit (20ms @ 48kHz = 960).
|
||||
/// Used to size the Packet Loss Concealment output, since libopus conceals
|
||||
/// `frame_size` samples when given no input — passing the full max buffer
|
||||
/// would synthesize a 120ms burst instead of a single 20ms frame.
|
||||
frame_samples: usize,
|
||||
}
|
||||
|
||||
impl OpusDecoder {
|
||||
/// Creates a new Opus decoder.
|
||||
/// Standard voice parameters: sample_rate = 48000, channels = Channels::Mono
|
||||
pub fn new(sample_rate: u32, channels: Channels) -> Result<Self, CodecError> {
|
||||
/// Standard voice parameters: sample_rate = 48000, channels = Channels::Mono.
|
||||
/// `frame_samples` is the per-channel length of one transmitted frame (e.g. 960).
|
||||
pub fn new(sample_rate: u32, channels: Channels, frame_samples: usize) -> Result<Self, CodecError> {
|
||||
let decoder = Decoder::new(sample_rate, channels)
|
||||
.map_err(|e| CodecError::Init(format!("Failed to create Opus decoder: {}", e)))?;
|
||||
Ok(Self { decoder, channels })
|
||||
Ok(Self { decoder, channels, frame_samples })
|
||||
}
|
||||
|
||||
fn channels_count(&self) -> usize {
|
||||
match self.channels {
|
||||
Channels::Mono => 1,
|
||||
Channels::Stereo => 2,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl AudioDecoder for OpusDecoder {
|
||||
fn decode(&mut self, compressed: Option<&[u8]>) -> Result<Vec<i16>, CodecError> {
|
||||
// Maximum Opus frame size is 120ms. At 48kHz, this is 5760 samples per channel.
|
||||
let channels_count = match self.channels {
|
||||
Channels::Mono => 1,
|
||||
Channels::Stereo => 2,
|
||||
};
|
||||
let max_samples = 5760 * channels_count;
|
||||
let mut pcm = vec![0i16; max_samples];
|
||||
let channels_count = self.channels_count();
|
||||
|
||||
let decoded_samples_per_channel = match compressed {
|
||||
let (mut pcm, input): (Vec<i16>, &[u8]) = match compressed {
|
||||
Some(data) if !data.is_empty() => {
|
||||
// Normal decode
|
||||
self.decoder.decode(data, &mut pcm, false)
|
||||
.map_err(|e| CodecError::Decode(format!("Opus decoding failed: {}", e)))?
|
||||
// Normal decode. Size the buffer to the maximum Opus frame (120ms =
|
||||
// 5760 samples/channel); libopus decodes the packet's true duration.
|
||||
(vec![0i16; 5760 * channels_count], data)
|
||||
}
|
||||
_ => {
|
||||
// Packet Loss Concealment (PLC)
|
||||
// In opus-rs, passing an empty slice triggers PLC.
|
||||
self.decoder.decode(&[], &mut pcm, false)
|
||||
.map_err(|e| CodecError::Decode(format!("Opus PLC decoding failed: {}", e)))?
|
||||
// Packet Loss Concealment: an empty input makes libopus synthesize
|
||||
// exactly `frame_samples` of concealment, so size the buffer to match.
|
||||
(vec![0i16; self.frame_samples * channels_count], &[])
|
||||
}
|
||||
};
|
||||
|
||||
let total_samples = decoded_samples_per_channel * channels_count;
|
||||
pcm.truncate(total_samples);
|
||||
let decoded_per_channel = self.decoder.decode(input, &mut pcm, false)
|
||||
.map_err(|e| CodecError::Decode(format!("Opus decoding failed: {}", e)))?;
|
||||
|
||||
pcm.truncate(decoded_per_channel * channels_count);
|
||||
Ok(pcm)
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user