Public Access

Files

T

molluskandClaude Opus 4.8 fde6f8a680 feat(audio): multitrack stem recording — Stage 1 (pure core + plan)

Scopes the differentiating "record every peer to their own synced track"
feature and lands its pure, isolated core (no live-audio wiring yet).

- docs/multitrack-recording-plan.md: scope contract + locked decisions
  (raw stems pre-volume/mute, stems + a mixed track, silence-pad late joiners).
- src/audio/multitrack.rs: MultitrackRecorder over the existing WavWriter.
  One master clock = the mixer cycle; every end_cycle() appends exactly
  FRAME_SAMPLES to every track (silence where idle) so all stems stay
  sample-aligned. add_peer back-pads a late joiner to cycle 0; track_filename
  gives fs-safe `<slug>-<shortid>.wav` (reuses sanitize_name). Optional mix
  track for "Both" mode.
- +5 unit tests: equal length across tracks, late-joiner leading silence,
  stems-only omits mix, fit() pad/truncate, filename slugging/disambiguation.

Stage 2 (wire into the mixer) is next, behind a checkpoint. 168 lib tests,
clippy --all-targets clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-06-13 23:48:07 -04:00

4.3 KiB

Raw Blame History

Multitrack (stem) recording — plan / scope contract

Status: Stage 1 in progress (2026-06-13). This doc is the scope contract; update it as stages land.

Goal & differentiation

Record each call participant to their own synced WAV track (stems) — podcast/streamer-grade source for post-production — natively, offline, with no server or bot. This is possible because PeerSpeak is full-mesh P2P: every client already decodes each peer into its own buffer and sums them locally (the playout mixer in src/core/mod.rs). Server-mixed tools (Discord/Zoom) only get one pre-blended stream, so they can't do this without a bot (Craig) or paid tier. Holding the unmixed stems on every machine is the moat.

Locked design decisions (user, 2026-06-13)

Raw stems — record each peer's decoded audio before local volume/mute/limiter, so the stems are pristine source (your local listening tweaks don't bake in).
Output = stems + a mixed track ("Both"): per-peer stems + your mic stem + a convenience mixed WAV, all in one session folder.
Silence-pad late joiners from session start — a peer who joins mid-recording gets leading silence so every track lines up at sample 0 (drop them all on a timeline → synced).

Architecture / tap point

The mixer loop (src/core/mod.rs ~L761–788) already pops a decoded frame per peer with peer_id in scope, then discards the association into peer_frames for summing. We tap the raw frame there (before apply_volume/mute/limiter) for stems. Mic arrives separately via the capture task (push_mic, ~L663). Alignment uses the existing per-mixer-cycle clock: every cycle, write exactly FRAME_SAMPLES (960) to every track — real audio for peers who produced a frame this cycle, silence for those idle — so all tracks stay sample-aligned by construction (same principle as today's mic FIFO in recorder.rs).

Stages

Stage 1 — Pure `MultitrackRecorder` core (no wiring) — in progress

src/audio/multitrack.rs. Owns a per-peer HashMap<EndpointId, Track> + a mic track + an optional mix track (Both mode), each a reused recorder::WavWriter, plus a cycles counter (the master clock). API:

add_peer(id, name) — create the track, pre-pad with cycles * FRAME_SAMPLES silence (late-join alignment); idempotent.
write_peer(id, &[i16]) / write_mic(&[i16]) / write_mix(&[i16]) — append one cycle's frame (fit to FRAME_SAMPLES: zero-pad if short), mark the track written-this-cycle.
end_cycle() — pad every track NOT written this cycle with FRAME_SAMPLES silence, reset flags, cycles += 1. Guarantees equal lengths.
finalize(self) — finalize all writers. Pure/testable: tests assert "after N cycles every track is exactly N×FRAME_SAMPLES" and "a peer added at cycle k carries k×FRAME_SAMPLES leading silence." Plus a filesystem-safe track-naming helper (me.wav, <slug>-<shortid>.wav, mix.wav) reusing sanitize::sanitize_name.

Stage 2 — Wire into the mixer + mic tasks

Retain peer_id alongside each decoded frame in the mixer; tap the raw frame for stems; per cycle call write_peer for present peers, write_mix with the finished mix, write_mic from the mic FIFO, then end_cycle. add_peer on join (RoomEvent::PeerJoined/PeerUpdated carries the name), keep the track on leave (it just goes silent). Keep the existing single-file mixed Recorder path for Mixed mode. Gate by is_recording like today.

Stage 3 — Config + UI

AppConfig.recording_mode: Mixed | Multitrack | Both (serde-default Mixed, back-compat). New "Recording" category in Settings (fits the category-header layout) with the mode picker + an output-dir note. Output to a per-session dir ~/peerspeak-recordings/<timestamp>/ (Multitrack/Both); Mixed keeps today's single-file behaviour.

Stage 4 — Field-test (dopedart)

Solo first (mic + silence stems, inspect WAVs), then 2-machine desktop↔dopedart: each voice isolated on its own track, tracks sample-aligned + in sync, late-joiner leading-silence correct, peer-leave handled.

Notes / out of scope

Disk: mono 48kHz/16-bit ≈ 5.5 MB/min/track. A 5-person 1-hr call ≈ ~1.6 GB. Acceptable; a FLAC option is a future wishlist item.
Stems are mono (matches the whole pipeline). Spatial/stereo is a separate wishlist item (W1).
No editing/mixing UI — files only; users post-produce in their DAW/editor.

4.3 KiB Raw Blame History Unescape Escape