molluskandClaude Opus 4.8
d3e831d5e7
Adds the "Recording" category to the Settings page so the recording mode is selectable from the UI (config + core wiring already existed from Stage 2). - AppMessage::RecordingModeSelected → persists config + sends SetRecordingMode (takes effect on the next recording start). recording_mode_hint copy. - iced 0.14 pick_list can't host per-option tooltips, so the three modes are RADIO BUTTONS each wrapped in a `tooltip` (hover explains that mode) — chosen over a dropdown so each option is self-documenting. Placed after Microphone, using the shared section_header; output-dir note below. - +1 config test (recording_mode round-trip + is_multitrack classification). User-verified live (Both mode writes the expected WAVs; radios + tooltips approved). Senior-written — Gemini's unsanctioned Stage 3 attempt was discarded. 179 tests (169 lib +1 ign, 6 reconnect, 4 transport), clippy --all-targets clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
42 lines
5.2 KiB
Markdown
42 lines
5.2 KiB
Markdown
# Multitrack (stem) recording — plan / scope contract
|
||
|
||
**Status:** Stages 1, 2 + 3 DONE (Stage 3 on 2026-06-14, all by the senior). Stage 4 (2-machine field-test on dopedart) is the only remaining step. This doc is the scope contract; update it as stages land.
|
||
|
||
## Goal & differentiation
|
||
|
||
Record each call participant to their **own synced WAV track** (stems) — podcast/streamer-grade source for post-production — natively, offline, with no server or bot. This is possible because PeerSpeak is **full-mesh P2P**: every client already decodes each peer into its own buffer and sums them locally (the playout mixer in `src/core/mod.rs`). Server-mixed tools (Discord/Zoom) only get one pre-blended stream, so they can't do this without a bot (Craig) or paid tier. Holding the unmixed stems on every machine is the moat.
|
||
|
||
## Locked design decisions (user, 2026-06-13)
|
||
|
||
1. **Raw stems** — record each peer's decoded audio **before** local volume/mute/limiter, so the stems are pristine source (your local listening tweaks don't bake in).
|
||
2. **Output = stems + a mixed track** ("Both"): per-peer stems + your mic stem + a convenience mixed WAV, all in one session folder.
|
||
3. **Silence-pad late joiners from session start** — a peer who joins mid-recording gets leading silence so every track lines up at sample 0 (drop them all on a timeline → synced).
|
||
|
||
## Architecture / tap point
|
||
|
||
The mixer loop (`src/core/mod.rs` ~L761–788) already pops a decoded `frame` per peer with `peer_id` in scope, then discards the association into `peer_frames` for summing. We tap the **raw** frame there (before `apply_volume`/mute/limiter) for stems. Mic arrives separately via the capture task (`push_mic`, ~L663). Alignment uses the existing per-mixer-cycle clock: **every cycle, write exactly `FRAME_SAMPLES` (960) to every track** — real audio for peers who produced a frame this cycle, silence for those idle — so all tracks stay sample-aligned by construction (same principle as today's mic FIFO in `recorder.rs`).
|
||
|
||
## Stages
|
||
|
||
### Stage 1 — Pure `MultitrackRecorder` core (no wiring) — DONE (`fde6f8a`)
|
||
`src/audio/multitrack.rs`. Owns a per-peer `HashMap<EndpointId, Track>` + a mic track + an optional mix track (Both mode), each a reused `recorder::WavWriter`, plus a `cycles` counter (the master clock). API:
|
||
- `add_peer(id, name)` — create the track, pre-pad with `cycles * FRAME_SAMPLES` silence (late-join alignment); idempotent.
|
||
- `write_peer(id, &[i16])` / `write_mic(&[i16])` / `write_mix(&[i16])` — append one cycle's frame (fit to `FRAME_SAMPLES`: zero-pad if short), mark the track written-this-cycle.
|
||
- `end_cycle()` — pad every track NOT written this cycle with `FRAME_SAMPLES` silence, reset flags, `cycles += 1`. Guarantees equal lengths.
|
||
- `finalize(self)` — finalize all writers.
|
||
Pure/testable: tests assert "after N cycles every track is exactly N×FRAME_SAMPLES" and "a peer added at cycle k carries k×FRAME_SAMPLES leading silence." Plus a filesystem-safe track-naming helper (`me.wav`, `<slug>-<shortid>.wav`, `mix.wav`) reusing `sanitize::sanitize_name`.
|
||
|
||
### Stage 2 — Wire into the mixer + mic tasks — DONE (`f6520b7`)
|
||
Done: mixer taps each peer's raw frame into `stems`, writes peer stems + mix (Both) + `end_cycle` per cycle; mic pushed to the recorder's FIFO from the capture thread; `add_peer` on `PeerJoined` (named) + for everyone present at recording start; `Mixed` mode keeps the single-file `Recorder`; `RecordingMode` config + `SetRecordingMode` command (sent at startup) select the path; `is_multitrack` is the fast-path gate; `stop_recording` finalizes both. Note: mic uses an internal FIFO drained per cycle (not a per-cycle `write_mic`), matching `recorder.rs`. No UI yet → defaults to `Mixed`; set `recording_mode` in config.json to exercise stems until Stage 3.
|
||
|
||
### Stage 3 — Config + UI — DONE (2026-06-14, senior-written)
|
||
`AppConfig.recording_mode` was added in Stage 2; Stage 3 added the **"Recording"** Settings category (after Microphone): `AppMessage::RecordingModeSelected` (persists + sends `SetRecordingMode`), `recording_mode_hint`, and the picker. **Note:** iced 0.14 `pick_list` can't host per-option tooltips, so the modes are rendered as **radio buttons each wrapped in a `tooltip`** (hover shows the per-mode explanation) — user-chosen over a dropdown. Output dir note included. Output to a per-session dir `~/peerspeak-recordings/<timestamp>/` (Multitrack/Both); `Mixed` keeps the single-file behaviour. +1 config test (`test_recording_mode_field`). User-verified live (2 WAVs created as expected in Both mode; radios + tooltips approved). (Gemini's unsanctioned Stage 3 attempt was discarded; this is the senior's implementation.)
|
||
|
||
### Stage 4 — Field-test (dopedart)
|
||
Solo first (mic + silence stems, inspect WAVs), then 2-machine desktop↔dopedart: each voice isolated on its own track, tracks sample-aligned + in sync, late-joiner leading-silence correct, peer-leave handled.
|
||
|
||
## Notes / out of scope
|
||
- **Disk:** mono 48kHz/16-bit ≈ 5.5 MB/min/track. A 5-person 1-hr call ≈ ~1.6 GB. Acceptable; a FLAC option is a future wishlist item.
|
||
- Stems are mono (matches the whole pipeline). Spatial/stereo is a separate wishlist item (W1).
|
||
- No editing/mixing UI — files only; users post-produce in their DAW/editor.
|