Recover a guessed audio stream's rate by correlating hook vs loopback (step a)
When the host attaches to an already-running game it never saw the stream's Initialize, so the render-hook assumes the device mix format and measures only the sample rate from the render cadence -- which a jittery game can make wrong (intermittent pitch shift). But during the measurement window the game is still audible, so we have the same audio twice: the hook (pre-mix, unknown format) and a process-loopback (post-mix, the known device format). Cross-correlating them pins the true rate from ground truth. - common/include/coop/audio_correlate.hpp: the pure correlator. Resample the hook by each candidate standard rate up to the device rate and score how well it aligns with the loopback across the window (drift-detecting). audio_correlation_test recovers every rate (score ~1.0 vs ~0.01 for wrong ones), incl. 44100-vs-48000, and rejects unrelated signals. - Hook measurement tap: a host-set verify_capture ring flag makes the hook push a still-being-measured (guessed) stream's raw pre-mix bytes WITHOUT silencing, so the host can co-capture both signals (a silenced game's loopback is silent). Inert by default -- the shipping no-echo path is untouched. - host/src/audio/audio_format_verifier: co-captures hook + loopback and correlates, feeding a correction into the existing override channel. Wired into AudioMirror's measurement window (hidden in the gap loopback already covers, so exact streams pay nothing). audio_verify_test drives it end-to-end against coop_mock_game. Rate vs layout are coupled (correlating the waveform needs the right channel de-interleaving), so this step assumes the hook layout matches the device (the common stereo-on-stereo case); recovering a different channel count / bit depth is step b. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
34
README.md
34
README.md
@@ -228,6 +228,22 @@ ctest --test-dir build -C Debug --output-on-failure
|
||||
reader/writer. Synthesizes a clean tone, a wrong-rate (pitch-shifted) tone, a tone with
|
||||
injected clicks, and one with silence gaps, and asserts each metric matches what was injected
|
||||
(e.g. 44100 played as 48000 → +147 cents). Pure header logic, no device.
|
||||
- **`audio_correlation_test`** — unit test of the two-path audio-format correlator
|
||||
([`common/include/coop/audio_correlate.hpp`](common/include/coop/audio_correlate.hpp)), which
|
||||
recovers a guessed stream's true sample rate from *ground truth* instead of cadence. Synthesizes
|
||||
one continuous signal sampled at two rates (the hook's true rate + the device rate, with capture
|
||||
skew + noise — exactly the hook-vs-loopback situation) and asserts `correlate_rate()` recovers the
|
||||
true rate, scoring the right candidate ≈1.0 and the wrong ones ≈0 (incl. the hard 44100-vs-48000
|
||||
case the cadence method can misread), and that unrelated signals are *not* confidently matched.
|
||||
Pure header logic, no device.
|
||||
- **`audio_verify_test`** — integration test of the host's two-path verifier
|
||||
([`host/src/audio/audio_format_verifier.cpp`](host/src/audio/audio_format_verifier.cpp)). Launches
|
||||
`coop_mock_game` rendering a tone at a non-device rate (matching the device's *channel* count, so
|
||||
this rate test isn't perturbed by a channel mismatch — that's the next task), injects the hook
|
||||
late (a guessed stream), and runs the real `verify_stream_format()`: it co-captures the hook
|
||||
(pre-mix, via the ring's `verify_capture` tap) and a parallel process-loopback (post-mix) of the
|
||||
same audio and correlates them. Asserts it recovers the game's true rate, not the device guess.
|
||||
Skips cleanly without an audio endpoint.
|
||||
- **`render_pacer_test`** — unit test of the mirror's render-feed pacing policy
|
||||
(`host/src/audio/render_pacer.hpp`). Simulates a producer/consumer device timeline and asserts
|
||||
the shipping `RenderPacer` rides producer jitter that makes the old re-prime-on-partial-fill
|
||||
@@ -518,6 +534,24 @@ Non-obvious things that cost time and constrain the design:
|
||||
estimate as explicitly *low-confidence* (shown red). The operator can also re-measure
|
||||
or override the format via a per-stream `AudioRingHeader` op channel; the host rebuilds
|
||||
its render client when `format_generation` bumps, so it takes effect live.
|
||||
- **Two capture paths beat one guess: correlate the hook against the loopback.** Cadence
|
||||
measurement *rejects* a bad rate reading but is still a guess from one signal, and it can't recover
|
||||
channels/bit-depth at all. But during the measurement window the game is still audible, so we have
|
||||
the *same* audio twice: the **hook** (pre-mix, unknown format) and a **process-loopback** (post-mix,
|
||||
the *known* device format — it's the hook signal resampled by WASAPI's AUTOCONVERTPCM). Resampling
|
||||
the hook by each candidate rate and cross-correlating against the loopback pins the true rate from
|
||||
ground truth: the right rate holds alignment across the whole window (score ≈1.0); a wrong rate
|
||||
time-warps the hook so a single alignment can't hold and the correlation collapses (≈0). The catch
|
||||
that makes this need a measurement *tap*: the no-echo path **silences** the game, so a loopback of a
|
||||
silenced game is silent — the co-capture must happen while the stream is still being measured (not
|
||||
yet published, so not yet silenced). A host-set `verify_capture` ring flag makes the hook push the
|
||||
guessed stream's raw pre-mix bytes (no silence) during that window; the host
|
||||
(`audio_format_verifier`) co-captures both, correlates (`coop/audio_correlate.hpp`), and feeds a
|
||||
correction into the existing override channel. **Rate vs layout are coupled, though**: correlating
|
||||
the *waveform* needs the hook bytes de-interleaved at the right channel count, so the rate step
|
||||
assumes the hook layout matches the device (true for the common stereo-on-stereo case); recovering a
|
||||
*different* channel count / bit depth is the layout step, which tries candidate de-interleavings and
|
||||
keeps whichever correlates.
|
||||
- **Re-priming the render feed on a *partial* fill manufactures the gap it's avoiding.** The
|
||||
mirror re-renders the captured ring to the output device. The original feed loop re-primed
|
||||
(withheld the feed until ~30 ms had rebuffered) whenever it couldn't completely fill the free
|
||||
|
||||
Reference in New Issue
Block a user