Build coop_audio_validate, a tool that turns "the mirror audio sounds off"
into numbers. It plays a known sine (coop_tone, 44.1 kHz on a 48 kHz
endpoint -- the Godot/Brotato case), injects the hook as the host does, and
runs a fidelity analyzer (coop/tone_analysis.hpp: pitch error in cents,
SNR/THD, click + dropout counts), dumping a .wav to listen to. Modes:
--render drives the real AudioMirror and measures its rendered output;
--baseline/--selfcheck give the measurement floor; --listen <pid> records a
live coop_host's output; --wav analyzes a recording. Analyzer + WAV I/O are
unit-tested (tone_analysis_test) against synthesized defects.
Using it, the capture ring measures pristine (~68 dB, 0 gaps) while the
render path dropped to ~18 dB with gaps -- localizing a real defect in
AudioMirror::run_hooked: it re-primed (withheld the feed until ~30 ms had
rebuffered) on any partial fill (to_write < avail). A partial fill is normal
producer jitter, and withholding the feed drains the device, so a one-frame
ring dip became a full ~30 ms drop-out; on a jittery game it fired
constantly. Fix: feed whatever is available each tick and re-prime only on a
genuine starvation (device empty AND ring empty). The policy is factored
into a pure RenderPacer reused by run_hooked + run_loopback and proven by
render_pacer_test (the old policy withholds available data ~168x and drains
to one period from silence on a jittery schedule; the new one never
withholds).
ctest 17/17.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Since guessed streams are now captured-but-not-silenced (the over-write fix),
the hooked path is only no-echo for an exact/override format. The panel
inferred "Hooked (no echo)" unconditionally, which was misleading. Show
"Hooked (echo -- guessed format)" (amber) for a guessed primary stream and
"Hooked (no echo)" (green) only for exact/override; drop the unconditional
"no echo" from source_name() and the mirror status string.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Persist audio overrides keyed by game image name (coop_audio_overrides.ini
next to the exe) so a known-bad game is auto-corrected on its next launch:
on attach to a guessed stream the host applies any saved override, and a
manual Override now saves too. A format the hook catches exactly at
IAudioClient::Initialize is auto-saved as that game's override (ground truth);
if it overwrites a differing stored value, a warning is logged. Host-originated
log lines now reach the Log window via IpcServer::host_log (color-coded).
Validated live (harness): a pre-seeded override for coop_tone is auto-applied
over the guess (state -> manual override, 48000/2/16). audio_overrides_test
covers persist/reload/case-insensitive lookup/differing-overwrite. Trim README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a per-stream op channel in AudioRingHeader (op_seq + op_* fields, version
2): the host posts re-measure / override commands, the hook applies them and
re-publishes (bumping format_generation), and the host rebuilds its render
client live on the change. The Audio panel (under Debug details) gains a
"Re-measure rate" button and a rate/channels/bit-depth/format override -- for
when detection is wrong or the channels/bit-depth were unrecoverable.
Also add a debug-only IPC test harness (-DCOOP_TEST_HARNESS, off by default,
absent from the shipped host): a file-based command channel that drives the
host's real UI code paths (inject / audio / re-measure / override / screenshot
/ status) for scripted validation, instead of unreliable synthetic mouse input.
Used to validate live: late-attach to coop_tone@44100 -> measured 44100,
promoted to hooked; override -> 2ch state, re-measure -> reconverge.
Trim the README roadmap to what's left; document the harness + rate_estimator_test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The host used to wait a fixed 1 s for the hook to publish a format and, on
timeout, fall to loopback permanently with no explanation -- which the now
slower (consensus) measurement made common. Restructure the audio thread to
keep the rings live the whole session and alternate: prefer hooked, and while
it isn't ready run loopback (echo) so guests still hear audio, watching the
ring to promote to the no-echo hooked path the instant the hook publishes a
format. Surface the concrete reason loopback is active (no format yet / not
renderable / no ring) in the Audio panel, in amber.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hooked audio mirroring played back pitch-shifted on games we inject into
that render at a non-device sample rate (e.g. Godot/Brotato render 44100 Hz
on a 48000 Hz endpoint via WASAPI AUTOCONVERTPCM). We attach to an
already-running game, so the render-hook never saw its IAudioClient::
Initialize and assumed the device mix format -- right channels/bits, wrong
rate -- so 44100 audio was rendered as 48000 (+~1.5 semitones).
Fix: treat a pre-existing client's format as a guess and measure its true
sample rate from the render cadence (frames/sec over a steady-state window,
snapped to the nearest standard rate) before publishing it, deferring
capture until verified. Discard the first measurement window so the
buffer-fill burst at attach time doesn't over-count. Streams created after
we inject still carry their exact Initialize format.
Channels/bit-depth genuinely can't be recovered for a pre-existing client:
AUTOCONVERTPCM hands GetBuffer a fixed staging buffer (no buffer stride to
measure -- confirmed empirically) and WASAPI exposes no API for the format.
They stay the device-mix guess, which is correct for the common case
(engines render stereo float, matching the endpoint). To keep a wrong guess
safe, a VirtualQuery clamp stops the capture copy from ever over-reading the
source buffer when the guessed bytes/frame is too large.
Surface all of this: a per-stream AudioFormatState (known / measuring /
measured rate (ch/bits assumed)) in HookStatus, shown in the Audio panel for
the hooked path and as "device endpoint (known)" for loopback; clear hook
logs; and enriched mirror status strings. Documented in README (Limitations
+ Lessons learned). The loopback fallback was always correct (post-mix at
the device format).
Tests: extract a shared, configurable ToneSource (used by coop_tone and the
hook self-test); coop_tone takes rate/channels/bits/format args. Rewrite
audio_hook_test to a format matrix x both code paths -- see-init (exact) and
guess (rate measured) -- plus a byte-incompatible guess that asserts the
clamp keeps capture safe. The matrix caught the attach-burst over-count.
audio_loopback_test now spawns coop_tone at several source formats to
confirm loopback is format-agnostic. 11/11 x64 + 3/3 x86 pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Games with several concurrent WASAPI render streams (e.g. Spider-Man: Miles
Morales) only had their first ("primary") stream mirrored; the rest kept playing
locally and never reached the guest. Now the render-hook captures + silences EVERY
tracked stream into its own ring (coop_audio_<pid>[_<index>]), each published with
that stream's own detected format (Initialize when caught, else GetMixFormat -- the
per-stream format detection, now actually used per ring rather than only for the
primary). The host creates a ring per stream and mixes the same-format streams with
a soft clip (host/src/audio/audio_mix.hpp); streams whose format differs from the
primary are still silenced (no echo) but skipped from the mix (would need
resampling).
The single-stream case is byte-for-byte unchanged: when only one stream is active
the host passes it through without the mixer, so the common path has no overhead or
fidelity change.
Verified: new audio_mix_test covers the decode/sum/soft-clip/encode math (float32 +
int16); audio_hook_test (x64 + x86) still passes, guarding the primary
capture+silence path against regression; full build x64 + x86 clean; ctest x64
11/11, x86 3/3. Multi-stream mixing against a real multi-stream game needs a live
session to fully confirm.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rework the spike-era debug readouts into general-purpose status, with the
verbose diagnostics gated behind the menu bar's "Debug details" switch:
- Controllers (was "Phase 0 spike"): always shows slot/source + live
buttons; the raw stick/trigger numbers are debug-only.
- Injection hook status: general view is attached + focus spoof + a single
"Game reading controller: N polls/s" summary; the per-slot poll table,
focus-API counts, and input-path detection are debug-only.
- Audio: general view adds a "Buffered: N ms" health/latency proxy
(AudioMirror now tracks buffered audio in both render paths) and keeps the
render-stream count; the per-stream table is debug-only.
Default overlay is now clean general status; flip Debug details for the full
diagnostics. Completes the "generalize the UI" roadmap task. All tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
End-to-end plumbing of the render-hook audio path.
Hook (coop_hook.dll):
- CMake: build audio_hook.cpp, link ole32/mmdevapi, NTDDI_WIN10_CO.
- dllmain: CoInitializeEx(MTA) on the worker thread; install the audio hooks
even before the ring exists (so streams are counted); open the host's
coop_audio_<pid> ring when it appears and attach it (enabling capture+silence);
remove_audio_hooks on clean detach.
Host (coop_host.exe):
- AudioMirror now creates the shared audio ring (owns capture_enabled) and tries
the Hooked path first: waits ~1s for the hook to publish a format, then
re-renders the game's frames from the ring with AUTOCONVERTPCM (no echo, since
the hook silences the game locally).
- Automatic fallback: if the ring can't be created, no format arrives in time,
or the render client won't initialize, it disables capture (so the game stays
audible) and reverts to the existing process-loopback path (echo, no regress).
- Exposes Source (Hooked/Loopback/None) for the upcoming panel indicator.
Loopback render loop kept intact as run_loopback. Manual end-to-end (M5) and the
panel UI (M4) are next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mirror the real game's audio so Steam Remote Play Together (which streams the
host's own audio session) carries it to the guest. The host captures the game
by PID via WASAPI process loopback and re-renders it on the default endpoint;
the game still plays locally too (accepted "double audio" for now).
- ProcessLoopbackCapture: process-loopback capture client, frame-sink + stats.
The completion handler must be agile (IAgileObject) or
ActivateAudioInterfaceAsync rejects every call with E_ILLEGAL_METHOD_CALL.
- AudioMirror: wraps capture with an event-driven render client and a primed
ring buffer; AudioPanel drives it from the injected game's window/PID.
- coop_tone: standalone WASAPI sine-wave process used as a known audio source.
- audio_loopback_test (CTest): captures coop_tone by PID and asserts non-silent
audio arrives, so the path is verifiable without a second Steam account.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
