Audio: operator re-measure + format override (host<->hook op channel)

Add a per-stream op channel in AudioRingHeader (op_seq + op_* fields, version
2): the host posts re-measure / override commands, the hook applies them and
re-publishes (bumping format_generation), and the host rebuilds its render
client live on the change. The Audio panel (under Debug details) gains a
"Re-measure rate" button and a rate/channels/bit-depth/format override -- for
when detection is wrong or the channels/bit-depth were unrecoverable.

Also add a debug-only IPC test harness (-DCOOP_TEST_HARNESS, off by default,
absent from the shipped host): a file-based command channel that drives the
host's real UI code paths (inject / audio / re-measure / override / screenshot
/ status) for scripted validation, instead of unreliable synthetic mouse input.
Used to validate live: late-attach to coop_tone@44100 -> measured 44100,
promoted to hooked; override -> 2ch state, re-measure -> reconverge.

Trim the README roadmap to what's left; document the harness + rate_estimator_test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-22 01:46:59 +02:00
parent c24bfdd64f
commit 90f40ae479
13 changed files with 607 additions and 111 deletions

105
README.md
View File

@@ -77,8 +77,10 @@ and covers anything the hooked path doesn't (Vulkan, D3D9 — see Roadmap).
device would be mirrored with the wrong layout (garbled audio) on the hooked path —
but never an over-read/crash (a `VirtualQuery` clamp guards the copy), and the
loopback fallback is always format-correct. The Audio panel shows each stream's
format provenance (*known* / *measuring* / *measured rate (ch/bits assumed)*) so the
assumption is visible. Streams created *after* injection are captured exactly.
format provenance (*known* / *measuring* / *measured rate* / *low-confidence* /
*override*) so the assumption is visible, and (under Debug details) lets the operator
**re-measure** the rate or **override** the format when the guess is wrong. Streams
created *after* injection are captured exactly.
- **Debug-oriented UI:** the ImGui overlay is laid out for diagnosing the
pipeline, not for end use. F1 hides it entirely so the window is a clean mirror
for RPT; F2 frees the operator cursor; **F10 saves a PNG screenshot** (back buffer,
@@ -88,71 +90,30 @@ and covers anything the hooked path doesn't (Vulkan, D3D9 — see Roadmap).
### Planned (next up)
**Audio format reliability.** The hooked audio path can mis-detect a pre-existing
stream's sample rate, or fall back to loopback with no explanation. Two root causes:
the rate measurement is fragile, and late injection forces the guess path in the
first place.
- *Why the rate is sometimes wrong.* The measurement window is only ~200 ms. WASAPI
delivers audio in quantized ~10 ms buffers, so one extra buffer at a window edge is
a ~5% error (e.g. 44100 → ~46205 Hz). The snap-to-standard tolerance is ±2%, so a
~5%-off value snaps to *nothing* and is published verbatim instead of rejected, and
it's single-shot (no averaging). Fix: **measure over a longer window (~1–1.5 s),
sample the (frames, QPC) endpoints atomically, require consensus across a few
windows, and refuse to publish a rate that doesn't land near a standard rate** —
keep measuring (or flag low-confidence) rather than committing a bogus value.
- *Why it falls back to loopback silently.* The host waits a fixed 1 s for the hook
to publish a format; the guess+measure path needs ~400–700 ms of *continuous* audio
after the ring attaches, which a momentarily-quiet game can miss. Once on loopback
it never retries, and the UI shows no reason. Fix: **keep the rings live and
auto-promote to the hooked path whenever the hook later publishes a format, surface
the concrete fallback reason** (no format in time / not renderable / hook off) in the
Audio panel + log, and **flag a low-confidence rate (non-standard, didn't snap) in
red** so a bad measurement is obvious at a glance.
- **Re-measure button.** A debug action (Audio panel) to re-run a stream's rate
measurement on demand. Needs a small host→hook control surface: add operator-control
fields to the reserved area of `AudioRingHeader` (per-stream, already shared both
ways) — a bumped `op_seq` plus a "re-measure" request. The hook resets that stream's
measurement window, returns it to *Measuring*, clears `format_valid`, and republishes
once it reconverges; the host re-inits its render client when `format_generation`
bumps (so the corrected rate takes effect live).
- **Per-game format overrides (persisted), incl. the unmeasurable channels/bit-depth.**
A dropdown/inputs in the Audio panel — **shown only under the Debug details flag**,
per stream — to set sample rate, channels, bit-depth, and PCM/float when detection is
wrong or unrecoverable, **keyed by game image name and persisted** to a small store
next to the exe, so a known-bad game is auto-corrected on its next launch. A
**rate-only** override is host-side only (re-init
the render loop at the operator's rate — the bytes are already framed correctly); a
**channels/bits** override goes through the same `AudioRingHeader` op channel so the
*hook* re-frames its capture copy (`block_align`) and drops the over-read clamp, then
the host re-inits on the `format_generation` bump. **A format caught exactly at
`IAudioClient::Initialize` is itself saved as that game's override** (ground truth),
so a later late-attach to the same game is corrected automatically; if that overwrites
a previously-stored override that *differs*, log a warning.
- **Per-game persisted audio overrides + auto-learn.** The Audio panel already lets
the operator re-measure or override a stream's format (Debug details → *Fix the
primary stream*), but the override is session-only. Persist overrides **keyed by game
image name** to a small store next to the exe so a known-bad game is auto-corrected on
its next launch, and **auto-save a format caught exactly at `IAudioClient::Initialize`
as that game's override** (ground truth) so a later late-attach is corrected
automatically. If a caught format overwrites a stored override that *differs*, log a
warning.
- **Auto re-attach the same game on relaunch (session-only).** A checkbox on the
attached (or just-terminated) target, default off, *not* persisted, and **always
visible — not gated behind Debug details** (it's the recommended recovery path, not a
diagnostic). While it's on and the target has terminated, the host watches the process
list for a process of the same image name and injects automatically the moment it
reappears — built on the existing re-attach-by-image-name path, so there's no target
to type (it's the same game that was just attached). The workflow this enables: if
audio came out wrong (late attach forced the guess), tick the box, kill and relaunch
the game, and the host re-attaches *early* — early enough to catch
`IAudioClient::Initialize` and read the exact format, no guessing. This is a
**dependable fallback, not the intended primary workflow**: hardened first-launch
detection (above) should get it right without a relaunch; auto-re-attach is the
reliable recovery when it doesn't, and never asks the operator to guess a format.
- **Color-coded log levels.** The Log window should color warnings and errors (amber /
red) so they stand out from routine lines. `LogRecord` already carries a
(currently-unused) `level` field; add severity variants to the hook's `logf` and have
the host color by level. Used by the override-overwrite warning and the fallback
reasons above.
Ordering: measurement hardening + visible/red fallback reason + log levels first
(fixes the bug outright for most cases, no protocol change) → the `AudioRingHeader` op
channel (re-measure + channels/bits override; bumps `kAudioRingVersion`) → per-game
override persistence + auto-saving caught formats → session auto-re-attach (host-only,
extends the existing re-attach path).
attached (or just-terminated) target, default off, *not* persisted, **always visible**
(it's the recommended recovery path, not a diagnostic). While on and the target has
terminated, the host watches the process list for the same image name and injects the
moment it reappears — built on the existing re-attach-by-image-name path, so there's no
target to type. Workflow: if audio came out wrong (late attach forced the guess), tick
the box, kill and relaunch the game, and the host re-attaches *early* — early enough to
catch `IAudioClient::Initialize` and read the exact format. A dependable fallback, not
the intended primary path (hardened first-launch detection should get it right anyway).
- **Mock game + stress-test suite.** A bare-bones test game that renders an animated
(non-static) pattern — so dropped/duplicated/torn frames are obvious — with a
selectable graphics backend (DX11/DX12 now; structured to add OpenGL/Vulkan) and a
configurable audio output (rate/channels/bit-depth/format, like `coop_tone`). Then a
comprehensive suite that drives it through the real capture + audio + hook paths:
verify both capture backends, every audio variant, hook/unhook cycles, and that
nothing races or crashes under stress.
### Future work
@@ -225,6 +186,11 @@ ctest --test-dir build -C Debug --output-on-failure
push/pop, wrap-around, format handshake, overrun/drop). No device needed.
- **`audio_mix_test`** — unit test of the multi-stream mixer math (decode / sum /
soft-clip / encode for float32 + int16). No device needed.
- **`rate_estimator_test`** — unit test of the robust sample-rate estimator (the fix for
the wrong-rate bug). Feeds synthetic, adversarial render cadences and asserts it
converges to the right standard rate, rejects burst windows (never commits to a wrong
neighbour, incl. the real 46205 misread), flags a genuinely non-standard rate
low-confidence instead of spinning, and ignores idle windows. Pure logic, no device.
- **`audio_hook_test`** — in-process self-test of the WASAPI render-hook's **format
detection**, the part that gets pitch right. Using a shared configurable
`ToneSource` (the same render helper `coop_tone` uses), it renders tones at a matrix
@@ -283,6 +249,13 @@ artifacts; tests build into `bin/<config>/tests/`) and resolve `coop_hook.dll` f
the root one level up, so run them from there. **Kill the game between runs** — the
loaded DLL locks `coop_hook.dll` against the next rebuild.
A debug-only **test harness** drives the host overlay's own code paths (inject / enable
audio / re-measure / override / screenshot / read state) without simulating mouse input,
for scripted UI validation. Build it with `-DCOOP_TEST_HARNESS=ON` (off by default, so
the shipped host never contains it); the host then reads one command line from
`%TEMP%\coop_test_cmd.txt` and replies in `%TEMP%\coop_test_resp.txt`. See
[`host/src/test_harness.hpp`](host/src/test_harness.hpp).
## Running the tool (manual, end-to-end)
This needs Steam, a donor game that supports Remote Play Together, and a second