Symptom: cross-machine streams drifted ~6-9 seconds per minute. mpv
showed continuous "Audio device underrun detected" with video packets
piling up in its demuxer queue while audio queue stayed at 0.
Root causes:
- pipewiresrc captured at the host monitor's refresh rate (180Hz),
not at the configured framerate. The encoder + mux produced 6x more
frames per wallclock second than mpv could consume at realtime,
burying audio packet density in mpv's demuxer queue.
- `--profile=low-latency` sets `audio-buffer=0`, which is too
aggressive — any sub-millisecond network jitter immediately
starves the audio device. Underruns slowed mpv's audio clock, and
with `video-sync=audio` (also from low-latency profile) video
followed.
Fixes:
- `videorate ! video/x-raw,framerate=<fps>/1` after pipewiresrc to
cap input to the requested rate deterministically.
- mpv command grows `--audio-buffer=0.2 --demuxer-max-bytes=2M
--demuxer-readahead-secs=0.5`: small audio buffer to absorb network
jitter, demuxer cap to prevent runaway buildup.
A leaky-queues attempt landed and was reverted in the same commit —
it removed backpressure without addressing the root cause and made
things worse.
Verified cross-machine 6m51s: drift held at ~1s floor, zero audio
underruns, perfect A-V sync.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The old tokio::select! tore down the whole bridge when EITHER
direction's io::copy finished. For a one-way streaming workload the
reverse direction carries only the initial HTTP GET; once mpv stops
sending and the read half EOFs, the data direction got killed mid-
stream and the host logged "bridge closed cleanly" while the user's
video disappeared.
Spawn the reverse direction as a detached task and `.await` only
the data direction. When the data direction ends naturally, abort
the reverse task. The function gains Send + 'static bounds on T,
which TcpStream satisfies.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The previous ffmpeg-as-HTTP-server pipeline shape held back two
improvements at once. ffmpeg as the runtime server lost a one-shot
`-listen 1` accept to a probe-and-discard health check, and forced
us to size analyze/probe budgets carefully so ffmpeg would serve
before our deadline. Replacing it with a small tokio task that
accepts once, drains the HTTP request, writes a fixed 200 OK, then
`tokio::io::copy`s gst stdout to the socket removes all of that.
VAAPI H.264 (vah264enc) drops CPU encode from ~50% of a core to
single-digit %. An earlier attempt at vaav1enc had to be abandoned:
libavformat cannot demux AV1-in-MPEG-TS with the custom mapping
even with a 20MB probe budget — mpv reports video=eof. H.264 keeps
the hardware win on the well-trodden demuxer path.
scripts/smoke-pipeline.sh mirrors the runtime pipeline with
videotestsrc/audiotestsrc into a file and asserts that mpv reports
`video=playing` (not video=eof). The naive --frames=10 check was
a false positive when no video stream is recognized; the verbose
grep is the real gate.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Moves the full capture+encode+mux pipeline into gst-launch, leaving
ffmpeg as a thin HTTP server. Verified end-to-end on KDE Plasma 6
Wayland: screencast portal → mpv mirror-tunnel rendering in real time.
Pipeline:
pipewiresrc(do-timestamp) → videoconvert → x264enc (zerolatency
ultrafast) → h264parse(config-interval=-1) → byte-stream caps →
mpegtsmux ← (aacparse ← avenc_aac ← audioconvert ← pulsesrc) →
fdsink fd=1
ffmpeg -fflags nobuffer+discardcorrupt+genpts -flags low_delay
-analyzeduration 0 -probesize 32 -f mpegts -i pipe:0 -c copy
-f mpegts -listen 1 http://127.0.0.1:N
Why each piece is load-bearing (do not relitigate without cause):
- x264enc + h264parse + byte-stream caps: raw video over a pipe hits
stride/format negotiation problems (green screens with mis-aligned
rows). Encoding inside gst sidesteps that entirely.
- mpegtsmux inside gst: H.264 Annex B carries no timestamps. Without
a container, ffmpeg sees "Timestamps are unset" and downstream
muxing breaks. mpegts in gst preserves pipewiresrc's clock.
- byte-stream + alignment=au caps: h264parse defaults to AVC format
(length-prefixed NALUs) for some downstreams; ffmpeg's mpegts
demuxer needs Annex B start codes.
- audio in gst (pulsesrc + avenc_aac): keeping ffmpeg as a pure
passthrough (`-c copy`) avoids ffmpeg's audio-input dependency
delaying HTTP serving until both inputs are ready.
- `-analyzeduration 0 -probesize 32`: stop ffmpeg from buffering 5MB
/ 5s of input before deciding it understands the stream.
- Also fixes a separate one-shot bug from earlier: the previous
health-probe in wait_for_listener consumed ffmpeg's single
`-listen 1` accept slot, so the actual bridge connect hit
Connection refused. Replaced with connect_to_capture which
returns the bridge socket.
Adds dep checks for pipewiresrc, x264enc, h264parse, mpegtsmux,
pulsesrc, avenc_aac, aacparse with per-distro install hints.
Known gap: VLC currently shows a green screen against the stream
even though mpv works fine. Likely VLC-specific demuxer/latency
settings, not a pipeline correctness issue — to investigate as a
follow-up. mpv is the recommended client either way.
`gst-launch-1.0` and `pipewiresrc` ship in separate packages on most
distros (Arch: `gst-plugin-pipewire`, Debian: `gstreamer1.0-pipewire`,
Fedora/openSUSE: `pipewire-gstreamer`). Having the gst binary present
was no guarantee the Wayland capture pipeline would actually work —
without the plugin gst would bail at runtime with `no element
"pipewiresrc"`, which then cascaded into ffmpeg seeing EOF on its
stdin and exiting before its HTTP listener bound, then the host
hitting "Connection refused" against its own port. Confusing.
Now `deps::check_host_binaries` probes `gst-inspect-1.0 --exists
pipewiresrc` on Wayland and fails early with a per-distro install
hint pointing at the right package.
Implements the Wayland host pipeline from plan §4.5:
ashpd ScreenCast portal
-> CreateSession + SelectSources + Start + OpenPipeWireRemote
-> (pipewire fd, node_id, width, height)
gst-launch-1.0 pipewiresrc fd=N path=NODE_ID ! videoconvert
! video/x-raw,format=NV12 ! fdsink fd=1
ffmpeg
-f rawvideo -pix_fmt nv12 -video_size WxH -i pipe:0
-f pulse -i default
-c:v libx264 -preset ultrafast -tune zerolatency
-c:a aac -f mpegts -listen 1 http://127.0.0.1:<rand>
Phase 1 ships software x264 per plan §7; VAAPI is Phase 2.
src/host/wayland.rs is the new module. capture.rs becomes a thin
dispatcher with a CaptureHandle enum (Wayland today, X11 next).
host/mod.rs swaps the 150ms sleep for a poll-until-listener-ready
helper, and calls handle.shutdown().await for an orderly SIGTERM /
1s grace / SIGKILL teardown. The Drop impl is the panic backstop.
The pipewire fd handoff clears CLOEXEC before gst-launch spawn and
closes the parent's copy of the raw fd after the child has it.
Also deletes the empty src/host/tunnel.rs and src/viewer/tunnel.rs
placeholder files — the generic bridge in common/tunnel.rs is doing
the work, and there's no host- or viewer-specific tunnel concern
worth a module yet.
mollusk