From 90f40ae47968cdaecc48703a78c00533a5fb32eb Mon Sep 17 00:00:00 2001 From: BlackMark Date: Mon, 22 Jun 2026 01:46:59 +0200 Subject: [PATCH] Audio: operator re-measure + format override (host<->hook op channel) Add a per-stream op channel in AudioRingHeader (op_seq + op_* fields, version 2): the host posts re-measure / override commands, the hook applies them and re-publishes (bumping format_generation), and the host rebuilds its render client live on the change. The Audio panel (under Debug details) gains a "Re-measure rate" button and a rate/channels/bit-depth/format override -- for when detection is wrong or the channels/bit-depth were unrecoverable. Also add a debug-only IPC test harness (-DCOOP_TEST_HARNESS, off by default, absent from the shipped host): a file-based command channel that drives the host's real UI code paths (inject / audio / re-measure / override / screenshot / status) for scripted validation, instead of unreliable synthetic mouse input. Used to validate live: late-attach to coop_tone@44100 -> measured 44100, promoted to hooked; override -> 2ch state, re-measure -> reconverge. Trim the README roadmap to what's left; document the harness + rate_estimator_test. Co-Authored-By: Claude Opus 4.8 --- README.md | 105 ++++++++++----------------- common/include/coop/audio_ring.hpp | 75 ++++++++++++++++++- hook/src/audio_hook.cpp | 83 +++++++++++++++++---- host/CMakeLists.txt | 13 +++- host/src/audio/audio_loopback.cpp | 86 ++++++++++++++++------ host/src/audio/audio_loopback.hpp | 34 ++++++++- host/src/audio_panel.cpp | 46 +++++++++++- host/src/audio_panel.hpp | 45 ++++++++++++ host/src/injection_panel.cpp | 18 +++++ host/src/injection_panel.hpp | 6 ++ host/src/main.cpp | 113 +++++++++++++++++++++++++++++ host/src/test_harness.cpp | 51 +++++++++++++ host/src/test_harness.hpp | 43 +++++++++++ 13 files changed, 607 insertions(+), 111 deletions(-) create mode 100644 host/src/test_harness.cpp create mode 100644 host/src/test_harness.hpp diff --git a/README.md b/README.md index 8084049..e328c8f 100644 --- a/README.md +++ b/README.md @@ -77,8 +77,10 @@ and covers anything the hooked path doesn't (Vulkan, D3D9 — see Roadmap). device would be mirrored with the wrong layout (garbled audio) on the hooked path — but never an over-read/crash (a `VirtualQuery` clamp guards the copy), and the loopback fallback is always format-correct. The Audio panel shows each stream's - format provenance (*known* / *measuring* / *measured rate (ch/bits assumed)*) so the - assumption is visible. Streams created *after* injection are captured exactly. + format provenance (*known* / *measuring* / *measured rate* / *low-confidence* / + *override*) so the assumption is visible, and (under Debug details) lets the operator + **re-measure** the rate or **override** the format when the guess is wrong. Streams + created *after* injection are captured exactly. - **Debug-oriented UI:** the ImGui overlay is laid out for diagnosing the pipeline, not for end use. F1 hides it entirely so the window is a clean mirror for RPT; F2 frees the operator cursor; **F10 saves a PNG screenshot** (back buffer, @@ -88,71 +90,30 @@ and covers anything the hooked path doesn't (Vulkan, D3D9 — see Roadmap). ### Planned (next up) -**Audio format reliability.** The hooked audio path can mis-detect a pre-existing -stream's sample rate, or fall back to loopback with no explanation. Two root causes: -the rate measurement is fragile, and late injection forces the guess path in the -first place. - -- *Why the rate is sometimes wrong.* The measurement window is only ~200 ms. WASAPI - delivers audio in quantized ~10 ms buffers, so one extra buffer at a window edge is - a ~5% error (e.g. 44100 → ~46205 Hz). The snap-to-standard tolerance is ±2%, so a - ~5%-off value snaps to *nothing* and is published verbatim instead of rejected, and - it's single-shot (no averaging). Fix: **measure over a longer window (~1–1.5 s), - sample the (frames, QPC) endpoints atomically, require consensus across a few - windows, and refuse to publish a rate that doesn't land near a standard rate** — - keep measuring (or flag low-confidence) rather than committing a bogus value. -- *Why it falls back to loopback silently.* The host waits a fixed 1 s for the hook - to publish a format; the guess+measure path needs ~400–700 ms of *continuous* audio - after the ring attaches, which a momentarily-quiet game can miss. Once on loopback - it never retries, and the UI shows no reason. Fix: **keep the rings live and - auto-promote to the hooked path whenever the hook later publishes a format, surface - the concrete fallback reason** (no format in time / not renderable / hook off) in the - Audio panel + log, and **flag a low-confidence rate (non-standard, didn't snap) in - red** so a bad measurement is obvious at a glance. -- **Re-measure button.** A debug action (Audio panel) to re-run a stream's rate - measurement on demand. Needs a small host→hook control surface: add operator-control - fields to the reserved area of `AudioRingHeader` (per-stream, already shared both - ways) — a bumped `op_seq` plus a "re-measure" request. The hook resets that stream's - measurement window, returns it to *Measuring*, clears `format_valid`, and republishes - once it reconverges; the host re-inits its render client when `format_generation` - bumps (so the corrected rate takes effect live). -- **Per-game format overrides (persisted), incl. the unmeasurable channels/bit-depth.** - A dropdown/inputs in the Audio panel — **shown only under the Debug details flag**, - per stream — to set sample rate, channels, bit-depth, and PCM/float when detection is - wrong or unrecoverable, **keyed by game image name and persisted** to a small store - next to the exe, so a known-bad game is auto-corrected on its next launch. A - **rate-only** override is host-side only (re-init - the render loop at the operator's rate — the bytes are already framed correctly); a - **channels/bits** override goes through the same `AudioRingHeader` op channel so the - *hook* re-frames its capture copy (`block_align`) and drops the over-read clamp, then - the host re-inits on the `format_generation` bump. **A format caught exactly at - `IAudioClient::Initialize` is itself saved as that game's override** (ground truth), - so a later late-attach to the same game is corrected automatically; if that overwrites - a previously-stored override that *differs*, log a warning. +- **Per-game persisted audio overrides + auto-learn.** The Audio panel already lets + the operator re-measure or override a stream's format (Debug details → *Fix the + primary stream*), but the override is session-only. Persist overrides **keyed by game + image name** to a small store next to the exe so a known-bad game is auto-corrected on + its next launch, and **auto-save a format caught exactly at `IAudioClient::Initialize` + as that game's override** (ground truth) so a later late-attach is corrected + automatically. If a caught format overwrites a stored override that *differs*, log a + warning. - **Auto re-attach the same game on relaunch (session-only).** A checkbox on the - attached (or just-terminated) target, default off, *not* persisted, and **always - visible — not gated behind Debug details** (it's the recommended recovery path, not a - diagnostic). While it's on and the target has terminated, the host watches the process - list for a process of the same image name and injects automatically the moment it - reappears — built on the existing re-attach-by-image-name path, so there's no target - to type (it's the same game that was just attached). The workflow this enables: if - audio came out wrong (late attach forced the guess), tick the box, kill and relaunch - the game, and the host re-attaches *early* — early enough to catch - `IAudioClient::Initialize` and read the exact format, no guessing. This is a - **dependable fallback, not the intended primary workflow**: hardened first-launch - detection (above) should get it right without a relaunch; auto-re-attach is the - reliable recovery when it doesn't, and never asks the operator to guess a format. -- **Color-coded log levels.** The Log window should color warnings and errors (amber / - red) so they stand out from routine lines. `LogRecord` already carries a - (currently-unused) `level` field; add severity variants to the hook's `logf` and have - the host color by level. Used by the override-overwrite warning and the fallback - reasons above. - -Ordering: measurement hardening + visible/red fallback reason + log levels first -(fixes the bug outright for most cases, no protocol change) → the `AudioRingHeader` op -channel (re-measure + channels/bits override; bumps `kAudioRingVersion`) → per-game -override persistence + auto-saving caught formats → session auto-re-attach (host-only, -extends the existing re-attach path). + attached (or just-terminated) target, default off, *not* persisted, **always visible** + (it's the recommended recovery path, not a diagnostic). While on and the target has + terminated, the host watches the process list for the same image name and injects the + moment it reappears — built on the existing re-attach-by-image-name path, so there's no + target to type. Workflow: if audio came out wrong (late attach forced the guess), tick + the box, kill and relaunch the game, and the host re-attaches *early* — early enough to + catch `IAudioClient::Initialize` and read the exact format. A dependable fallback, not + the intended primary path (hardened first-launch detection should get it right anyway). +- **Mock game + stress-test suite.** A bare-bones test game that renders an animated + (non-static) pattern — so dropped/duplicated/torn frames are obvious — with a + selectable graphics backend (DX11/DX12 now; structured to add OpenGL/Vulkan) and a + configurable audio output (rate/channels/bit-depth/format, like `coop_tone`). Then a + comprehensive suite that drives it through the real capture + audio + hook paths: + verify both capture backends, every audio variant, hook/unhook cycles, and that + nothing races or crashes under stress. ### Future work @@ -225,6 +186,11 @@ ctest --test-dir build -C Debug --output-on-failure push/pop, wrap-around, format handshake, overrun/drop). No device needed. - **`audio_mix_test`** — unit test of the multi-stream mixer math (decode / sum / soft-clip / encode for float32 + int16). No device needed. +- **`rate_estimator_test`** — unit test of the robust sample-rate estimator (the fix for + the wrong-rate bug). Feeds synthetic, adversarial render cadences and asserts it + converges to the right standard rate, rejects burst windows (never commits to a wrong + neighbour, incl. the real 46205 misread), flags a genuinely non-standard rate + low-confidence instead of spinning, and ignores idle windows. Pure logic, no device. - **`audio_hook_test`** — in-process self-test of the WASAPI render-hook's **format detection**, the part that gets pitch right. Using a shared configurable `ToneSource` (the same render helper `coop_tone` uses), it renders tones at a matrix @@ -283,6 +249,13 @@ artifacts; tests build into `bin//tests/`) and resolve `coop_hook.dll` f the root one level up, so run them from there. **Kill the game between runs** — the loaded DLL locks `coop_hook.dll` against the next rebuild. +A debug-only **test harness** drives the host overlay's own code paths (inject / enable +audio / re-measure / override / screenshot / read state) without simulating mouse input, +for scripted UI validation. Build it with `-DCOOP_TEST_HARNESS=ON` (off by default, so +the shipped host never contains it); the host then reads one command line from +`%TEMP%\coop_test_cmd.txt` and replies in `%TEMP%\coop_test_resp.txt`. See +[`host/src/test_harness.hpp`](host/src/test_harness.hpp). + ## Running the tool (manual, end-to-end) This needs Steam, a donor game that supports Remote Play Together, and a second diff --git a/common/include/coop/audio_ring.hpp b/common/include/coop/audio_ring.hpp index bbc8a60..6337f02 100644 --- a/common/include/coop/audio_ring.hpp +++ b/common/include/coop/audio_ring.hpp @@ -24,7 +24,18 @@ namespace coop inline constexpr std::uint32_t kAudioRingMagic = 0x47525541u; // Bump whenever AudioRingHeader's layout changes. -inline constexpr std::uint32_t kAudioRingVersion = 1; +inline constexpr std::uint32_t kAudioRingVersion = 2; + +// Operator commands the host issues per stream (host -> hook), applied via the op_* +// fields in the header. The host writes the fields then bumps op_seq; the hook applies +// the command once per new op_seq. Lets the Audio panel re-measure a stream's rate or +// override its format when detection is wrong/unrecoverable. +enum AudioRingOp : std::uint32_t +{ + AudioRingOp_None = 0, + AudioRingOp_Remeasure = 1, // re-run the sample-rate measurement for this stream + AudioRingOp_Override = 2, // adopt the op_rate/channels/bits/format_tag verbatim +}; // Per-pid mapping name, mirroring kSharedMemoryPrefix: coop_audio_. inline constexpr wchar_t kAudioRingPrefix[] = L"Local\\coop_audio_"; @@ -66,7 +77,17 @@ struct AudioRingHeader std::atomic frames_produced; // cumulative frames pushed std::atomic overruns; // packets dropped on a full ring - std::uint8_t reserved[64]; + // --- Operator control (host -> hook) --------------------------------------- + // The host writes op_kind + the op_* fields, then bumps op_seq (release); the hook + // applies the command once per new op_seq (acquire). See AudioRingOp. + std::atomic op_seq; // bumped by the host on each new command + std::uint32_t op_kind; // AudioRingOp + std::uint32_t op_rate; // override: sample rate + std::uint32_t op_channels; // override: channel count + std::uint32_t op_bits; // override: bits per sample + std::uint32_t op_format_tag; // override: WAVE_FORMAT_PCM / _IEEE_FLOAT + + std::uint8_t reserved[40]; // std::uint8_t data[capacity] follows immediately in the mapping. }; @@ -104,6 +125,12 @@ inline void audio_ring_init(AudioRingHeader& h, std::uint32_t capacity) h.read_pos.store(0, std::memory_order_relaxed); h.frames_produced.store(0, std::memory_order_relaxed); h.overruns.store(0, std::memory_order_relaxed); + h.op_seq.store(0, std::memory_order_relaxed); + h.op_kind = 0; + h.op_rate = 0; + h.op_channels = 0; + h.op_bits = 0; + h.op_format_tag = 0; std::memset(h.reserved, 0, sizeof(h.reserved)); } @@ -185,6 +212,50 @@ inline std::uint32_t audio_ring_pop(AudioRingHeader& h, void* dst, std::uint32_t return bytes; } +// Host: post an operator command to the hook for this stream. Writes the fields, then +// bumps op_seq (release) so the hook applies it exactly once. For a re-measure the +// rate/channels/bits are ignored. +inline void audio_ring_post_op(AudioRingHeader& h, std::uint32_t kind, std::uint32_t rate = 0, + std::uint32_t channels = 0, std::uint32_t bits = 0, + std::uint32_t format_tag = 0) +{ + h.op_kind = kind; + h.op_rate = rate; + h.op_channels = channels; + h.op_bits = bits; + h.op_format_tag = format_tag; + h.op_seq.fetch_add(1, std::memory_order_release); +} + +// One operator command read back by the hook. +struct AudioRingOpCmd +{ + std::uint32_t kind = AudioRingOp_None; + std::uint32_t rate = 0; + std::uint32_t channels = 0; + std::uint32_t bits = 0; + std::uint32_t format_tag = 0; +}; + +// Hook: if a new op was posted since `last_seq`, read it into `out`, advance `last_seq`, +// and return its kind; otherwise returns AudioRingOp_None. Robust to ring re-creation +// (op_seq resets to 0 -> a stale higher last_seq just reads the zeroed None command). +inline std::uint32_t audio_ring_poll_op(AudioRingHeader& h, std::uint32_t& last_seq, AudioRingOpCmd& out) +{ + const std::uint32_t seq = h.op_seq.load(std::memory_order_acquire); + if (seq == last_seq) + { + return AudioRingOp_None; + } + last_seq = seq; + out.kind = h.op_kind; + out.rate = h.op_rate; + out.channels = h.op_channels; + out.bits = h.op_bits; + out.format_tag = h.op_format_tag; + return out.kind; +} + // Build the per-pid audio ring name both sides agree on. Stream 0 keeps the bare // coop_audio_ name (backward compatible / the single-stream case); additional // streams append _ (coop_audio__1, _2, ...). The host captures every diff --git a/hook/src/audio_hook.cpp b/hook/src/audio_hook.cpp index 2762e7d..f59d840 100644 --- a/hook/src/audio_hook.cpp +++ b/hook/src/audio_hook.cpp @@ -199,6 +199,10 @@ RateEstimator g_rate_estimator[kMaxAudioStreams] = {}; // Per-stream AudioFormatState (how its format was determined), mirrored to the host UI. std::uint32_t g_stream_format_state[kMaxAudioStreams] = {}; +// Last operator-op sequence applied per stream (host posts re-measure / override via the +// ring's op channel; we apply each new op once). Guarded by g_setup_mutex. +std::uint32_t g_last_op_seq[kMaxAudioStreams] = {}; + // GetBuffer/ReleaseBuffer are paired on one thread, never nested: stash the // pointer the game just got so ReleaseBuffer can copy it before releasing. thread_local IAudioRenderClient* t_gb_client = nullptr; @@ -404,6 +408,58 @@ bool publish_stream_format_locked(std::uint32_t slot) return true; } +// Apply an operator command (host -> hook via the ring op channel) to stream `slot`: +// re-run the rate measurement, or override the format. Both clear the ring's published +// format so publish_stream_format_locked re-publishes (bumping format_generation, which +// makes the host rebuild its render client at the new format). Caller holds g_setup_mutex. +void apply_audio_op_locked(std::uint32_t slot, const AudioRingOpCmd& cmd) +{ + AudioRingHeader* ring = g_rings[slot].load(std::memory_order_acquire); + if (ring == nullptr || g_stream_formats[slot].rate == 0) + { + return; // no ring / no stream in this slot + } + if (cmd.kind == AudioRingOp_Remeasure) + { + logw("audio stream %u: operator requested re-measure", slot); + g_stream_rate_guess[slot] = true; + g_rate_estimator[slot] = RateEstimator{}; + g_stream_formats[slot].rate = g_mix_format.rate; // back to the device-mix guess while measuring + g_stream_format_state[slot] = AudioFormat_Measuring; + g_streams[slot].assumed_format.store(1, std::memory_order_relaxed); + ring->format_valid.store(0, std::memory_order_release); // force re-publish after measuring + publish_stream_info_locked(slot, g_stream_formats[slot], AudioFormat_Measuring, + g_streams[slot].frames.load(std::memory_order_relaxed)); + } + else if (cmd.kind == AudioRingOp_Override) + { + CapturedFormat cf; + cf.rate = cmd.rate; + cf.channels = cmd.channels; + cf.bits = cmd.bits; + cf.tag = cmd.format_tag ? cmd.format_tag : WAVE_FORMAT_PCM; + cf.block_align = cmd.channels * (cmd.bits / 8); + if (cf.rate == 0 || cf.channels == 0 || cf.block_align == 0) + { + logw("audio stream %u: ignoring invalid override %uHz/%uch/%ubit", slot, cf.rate, cf.channels, + cf.bits); + return; + } + logw("audio stream %u: operator override -> %uHz/%uch/%ubit tag=%u", slot, cf.rate, cf.channels, + cf.bits, cf.tag); + g_stream_formats[slot] = cf; + g_stream_rate_guess[slot] = false; + g_stream_format_state[slot] = AudioFormat_Override; + // Keep the over-read clamp on: a too-large operator block is capped to the real + // buffer (garbled but safe); a correct override makes the clamp a no-op. + g_streams[slot].assumed_format.store(1, std::memory_order_relaxed); + g_streams[slot].block_align.store(cf.block_align, std::memory_order_relaxed); + ring->format_valid.store(0, std::memory_order_release); // re-publish at the new format + publish_stream_info_locked(slot, cf, AudioFormat_Override, + g_streams[slot].frames.load(std::memory_order_relaxed)); + } +} + // Registers a newly created render client: assigns it a debug slot, marks the // first as primary (the one we capture), publishes it to HookStatus, and hooks // the render-client vtable on first sight. `rate_is_guess` is true when `cf` is the @@ -700,24 +756,20 @@ bool install_audio_hooks(IpcClient& ipc, AudioRingHeader* ring) void republish_audio_format() { - // Nothing to do if no ring needs a format yet (cheap pre-check, no lock). - bool any_pending = false; - for (std::uint32_t i = 0; i < kMaxAudioStreams; ++i) - { - AudioRingHeader* ring = g_rings[i].load(std::memory_order_acquire); - if (ring != nullptr && !audio_ring_format_ready(*ring)) - { - any_pending = true; - break; - } - } - if (!any_pending) - { - return; - } std::scoped_lock lock(g_setup_mutex); for (std::uint32_t i = 0; i < kMaxAudioStreams; ++i) { + AudioRingHeader* ring = g_rings[i].load(std::memory_order_acquire); + if (ring == nullptr) + { + continue; + } + // Apply any operator command (re-measure / override) the host posted on this ring. + AudioRingOpCmd cmd; + if (audio_ring_poll_op(*ring, g_last_op_seq[i], cmd) != AudioRingOp_None) + { + apply_audio_op_locked(i, cmd); + } // Publishes an exact format immediately; a guessed rate is measured first and // published once a measurement window completes (retried on the next tick). publish_stream_format_locked(i); @@ -784,6 +836,7 @@ void remove_audio_hooks() g_stream_rate_guess[i] = false; g_stream_format_state[i] = AudioFormat_Unknown; g_rate_estimator[i] = RateEstimator{}; + g_last_op_seq[i] = 0; g_rings[i].store(nullptr, std::memory_order_release); } g_client_formats.clear(); diff --git a/host/CMakeLists.txt b/host/CMakeLists.txt index 545c2af..6fefa05 100644 --- a/host/CMakeLists.txt +++ b/host/CMakeLists.txt @@ -19,10 +19,21 @@ add_executable(coop_host WIN32 src/capture/window_capture.cpp src/capture/shared_texture.cpp src/audio/audio_loopback.cpp - src/audio/process_loopback_capture.cpp) + src/audio/process_loopback_capture.cpp + src/test_harness.cpp) target_include_directories(coop_host PRIVATE src) +# Debug-only IPC test harness: lets a script drive the real UI code paths (inject / +# audio / override / ...) and read state back, instead of simulating mouse input. OFF +# by default, so the shipped host never contains it (the source compiles to nothing +# without the macro). Enable with -DCOOP_TEST_HARNESS=ON for a debuggable build. +option(COOP_TEST_HARNESS "Build the host with the debug IPC test harness (dev only)" OFF) +if(COOP_TEST_HARNESS) + target_compile_definitions(coop_host PRIVATE COOP_TEST_HARNESS) + message(STATUS "Host test harness ENABLED (debug IPC driver)") +endif() + # Process-loopback capture (AUDIOCLIENT_ACTIVATION_TYPE_PROCESS_LOOPBACK) needs the # Windows 10 20H1 (NTDDI_WIN10_CO) headers; raise the target SDK version for it. target_compile_definitions(coop_host PRIVATE NTDDI_VERSION=0x0A00000B) diff --git a/host/src/audio/audio_loopback.cpp b/host/src/audio/audio_loopback.cpp index 0e7c117..3d14ade 100644 --- a/host/src/audio/audio_loopback.cpp +++ b/host/src/audio/audio_loopback.cpp @@ -140,6 +140,34 @@ void AudioMirror::enable_capture(AudioRingHeader* const* rings, bool on) } } +void AudioMirror::request_op(unsigned slot, std::uint32_t kind, std::uint32_t rate, std::uint32_t channels, + std::uint32_t bits, std::uint32_t format_tag) +{ + if (slot >= kMaxAudioStreams) + { + return; + } + std::lock_guard lock(ops_mutex_); + pending_ops_.push_back({slot, kind, rate, channels, bits, format_tag}); +} + +void AudioMirror::drain_ops() +{ + std::vector ops; + { + std::lock_guard lock(ops_mutex_); + ops.swap(pending_ops_); + } + for (const PendingOp& op : ops) + { + AudioRingHeader* ring = (op.slot < kMaxAudioStreams) ? session_rings_[op.slot] : nullptr; + if (ring != nullptr) + { + audio_ring_post_op(*ring, op.kind, op.rate, op.channels, op.bits, op.format_tag); + } + } +} + bool AudioMirror::start(DWORD pid) { stop(); @@ -228,6 +256,7 @@ void AudioMirror::thread_main(DWORD pid) audio_ring_init(*rings[i], kAudioRingCapacity); created_primary = created_primary || (i == 0); } + session_rings_[i] = rings[i]; // visible to drain_ops on this (audio) thread } if (!created_primary) @@ -259,10 +288,15 @@ void AudioMirror::thread_main(DWORD pid) if (wait_for_format(rings[0], kHookWaitMs)) { set_fallback_reason({}); // hooked path is taking over - if (run_hooked(rings)) + const HookedResult r = run_hooked(rings); + if (r == HookedResult::Stopped) { break; // ran to a clean stop } + if (r == HookedResult::Reinit) + { + continue; // hook re-published (re-measure / override) -> re-read the new format + } if (stop_requested()) { break; @@ -289,6 +323,10 @@ void AudioMirror::thread_main(DWORD pid) } enable_capture(rings, false); + for (unsigned i = 0; i < kMaxAudioStreams; ++i) + { + session_rings_[i] = nullptr; // audio thread owns this; cleared before unmapping + } for (auto& shm : audio_ring_shm_) { shm.reset(); @@ -310,23 +348,15 @@ void AudioMirror::thread_main(DWORD pid) // Consume the render-hook's shared ring and re-render the game's frames. The // game is silenced locally by the hook, so the operator hears no echo. Returns // true if it ran to a clean stop; false on setup failure (caller falls back). -bool AudioMirror::run_hooked(AudioRingHeader* const* rings) +AudioMirror::HookedResult AudioMirror::run_hooked(AudioRingHeader* const* rings) { AudioRingHeader* primary = rings[0]; enable_capture(rings, true); // hook silences the game + pushes frames into the rings - auto disable_all = [&] { - for (unsigned i = 0; i < kMaxAudioStreams; ++i) - { - if (rings[i] != nullptr) - { - rings[i]->capture_enabled.store(0, std::memory_order_release); - } - } - }; - auto fail_to_loopback = [&] { - disable_all(); // let the game play locally again - return false; - }; + + // Snapshot the format generation up front; if the hook re-publishes (operator + // re-measure / override) it bumps, and we tear down + return Reinit so the caller + // re-reads the new format and rebuilds the render client. + const std::uint32_t start_gen = primary->format_generation.load(std::memory_order_acquire); const unsigned rate = primary->sample_rate; const unsigned channels = primary->channels; @@ -335,7 +365,8 @@ bool AudioMirror::run_hooked(AudioRingHeader* const* rings) const unsigned block_align = primary->block_align ? primary->block_align : channels * (bits / 8); if (rate == 0 || channels == 0 || block_align == 0) { - return fail_to_loopback(); + enable_capture(rings, false); // let the game play locally again + return HookedResult::Failed; } // Reconstruct the game's WAVEFORMATEX and let shared-mode WASAPI convert it @@ -379,6 +410,7 @@ bool AudioMirror::run_hooked(AudioRingHeader* const* rings) IAudioRenderClient* render = nullptr; HANDLE render_event = nullptr; bool started = false; + HookedResult result = HookedResult::Stopped; auto fail = [&](const char* msg, HRESULT hr) { char buf[160]; @@ -477,6 +509,12 @@ bool AudioMirror::run_hooked(AudioRingHeader* const* rings) { break; // stop requested } + drain_ops(); // post any queued operator ops (re-measure / override) to the hook + if (primary->format_generation.load(std::memory_order_acquire) != start_gen) + { + result = HookedResult::Reinit; // hook re-published -> re-read the new format + break; + } UINT32 padding = 0; if (FAILED(render_client->GetCurrentPadding(&padding))) @@ -552,7 +590,12 @@ bool AudioMirror::run_hooked(AudioRingHeader* const* rings) render_client->Stop(); } while (false); - disable_all(); // game audible again on stop + // On a re-init (format changed) keep capturing so the rebuilt render client picks up + // seamlessly; otherwise free the game's local playback (stop / fall back to loopback). + if (result != HookedResult::Reinit) + { + enable_capture(rings, false); + } if (render) { @@ -577,11 +620,11 @@ bool AudioMirror::run_hooked(AudioRingHeader* const* rings) if (!started) { - // Never got a working render client; let the caller try loopback. Leave - // capture disabled (already cleared above) so loopback hears the game. - return false; + // Never got a working render client; let the caller try loopback. Capture is + // already disabled above so loopback hears the game. + return HookedResult::Failed; } - return true; + return result; } bool AudioMirror::run_loopback(DWORD pid, AudioRingHeader* promote_ring) @@ -713,6 +756,7 @@ bool AudioMirror::run_loopback(DWORD pid, AudioRingHeader* promote_ring) set_status(capture.status()); break; } + drain_ops(); // operator ops (re-measure / override) reach the hook even on loopback // Auto-promote: the hook published a format -> hand back so the caller switches // to the no-echo hooked path (the rings stayed live the whole time). if (promote_ring != nullptr && audio_ring_format_ready(*promote_ring)) diff --git a/host/src/audio/audio_loopback.hpp b/host/src/audio/audio_loopback.hpp index ba2d873..d181256 100644 --- a/host/src/audio/audio_loopback.hpp +++ b/host/src/audio/audio_loopback.hpp @@ -14,6 +14,7 @@ #include #include #include +#include #include @@ -99,18 +100,32 @@ public: // panel. Empty when on the hooked path or before any fallback decision. [[nodiscard]] std::string fallback_reason() const; + // Post an operator command (AudioRingOp) to the hook for stream `slot` -- re-measure + // the rate or override the format. Thread-safe; queued and applied to the ring on the + // audio thread (which owns the ring mappings). For a re-measure the format args are 0. + void request_op(unsigned slot, std::uint32_t kind, std::uint32_t rate = 0, std::uint32_t channels = 0, + std::uint32_t bits = 0, std::uint32_t format_tag = 0); + private: void thread_main(DWORD pid); - // Returns true if it owned the session to a clean stop; false if setup failed - // and the caller should fall back to the loopback path. `rings[0]` is the primary - // stream; additional non-null rings are mixed in. - bool run_hooked(AudioRingHeader* const* rings); + // Outcome of a hooked render session. + enum class HookedResult + { + Stopped, // clean stop (mirror stopping) -> done + Failed, // setup failed (format not renderable) -> caller falls back to loopback + Reinit, // the hook re-published the format (re-measure/override) -> re-read and retry + }; + // Runs the hooked (no-echo) render path until stop, a setup failure, or a format change + // (re-measure/override). `rings[0]` is the primary stream; additional non-null rings are + // mixed in. + HookedResult run_hooked(AudioRingHeader* const* rings); // Loopback (echo) capture. If `promote_ring` is non-null, returns true the moment // that ring's format becomes ready (the hook caught up -> caller promotes to hooked); // returns false when stopped. With a null ring it only returns false (on stop). bool run_loopback(DWORD pid, AudioRingHeader* promote_ring); bool wait_for_format(AudioRingHeader* ring, DWORD timeout_ms); static void enable_capture(AudioRingHeader* const* rings, bool on); + void drain_ops(); // audio thread: post queued operator ops to the session rings bool stop_requested() const; void set_status(std::string s); void set_fallback_reason(std::string s); @@ -120,6 +135,17 @@ private: DWORD pid_ = 0; SharedMemory audio_ring_shm_[kMaxAudioStreams]; // per-stream rings (coop_audio_[_]) + AudioRingHeader* session_rings_[kMaxAudioStreams] = {}; // set on the audio thread for the session + + // Operator ops queued by request_op (any thread) and applied to the rings on the + // audio thread (which owns the mappings). Guarded by ops_mutex_. + struct PendingOp + { + unsigned slot; + std::uint32_t kind, rate, channels, bits, format_tag; + }; + std::mutex ops_mutex_; + std::vector pending_ops_; std::atomic running_{false}; std::atomic source_{Source::None}; diff --git a/host/src/audio_panel.cpp b/host/src/audio_panel.cpp index 78c3591..41f4465 100644 --- a/host/src/audio_panel.cpp +++ b/host/src/audio_panel.cpp @@ -72,12 +72,16 @@ ImVec4 audio_format_state_color(std::uint32_t state) void AudioPanel::draw_ui(const HookStatusView& status, bool debug_details) { - const bool have_target = target_ != nullptr && IsWindow(target_); DWORD pid = 0; - if (have_target) + if (target_ != nullptr && IsWindow(target_)) { GetWindowThreadProcessId(target_, &pid); } + if (dev_pid_ != 0) + { + pid = dev_pid_; // test harness: a windowless target (e.g. coop_tone) has no HWND + } + const bool have_target = pid != 0; apply_panel_layout(Panel::Audio); ImGui::Begin("Audio mirror"); @@ -256,6 +260,44 @@ void AudioPanel::draw_ui(const HookStatusView& status, bool debug_details) rate_base_time_ = now; } + // --- Operator controls: re-measure / override the primary stream's format ----- + // For when detection is wrong (re-measure) or unrecoverable (override the channels/ + // bit-depth the hook had to assume). Only meaningful while mirroring is active. + if (mirror_.running()) + { + ImGui::SeparatorText("Fix the primary stream (debug)"); + if (ImGui::Button("Re-measure rate")) + { + mirror_.request_op(0, AudioRingOp_Remeasure); + } + ImGui::SameLine(); + ImGui::TextDisabled("re-run the rate measurement"); + + ImGui::SetNextItemWidth(110.0f); + ImGui::InputInt("Hz", &ov_rate_, 0, 0); + ImGui::SameLine(); + ImGui::SetNextItemWidth(70.0f); + ImGui::InputInt("ch", &ov_channels_, 0, 0); + ImGui::SameLine(); + ImGui::SetNextItemWidth(90.0f); + ImGui::Combo("##ovbits", &ov_bits_idx_, "16-bit\0" "32-bit\0"); + ImGui::SameLine(); + ImGui::SetNextItemWidth(80.0f); + ImGui::Combo("##ovfmt", &ov_fmt_idx_, "PCM\0" "float\0"); + ImGui::SameLine(); + if (ImGui::Button("Override")) + { + ov_rate_ = std::clamp(ov_rate_, 8000, 384000); + ov_channels_ = std::clamp(ov_channels_, 1, 8); + const std::uint32_t bits = ov_bits_idx_ == 0 ? 16u : 32u; + const std::uint32_t tag = + ov_fmt_idx_ == 1 ? static_cast(WAVE_FORMAT_IEEE_FLOAT) + : static_cast(WAVE_FORMAT_PCM); + mirror_.request_op(0, AudioRingOp_Override, static_cast(ov_rate_), + static_cast(ov_channels_), bits, tag); + } + } + ImGui::End(); } diff --git a/host/src/audio_panel.hpp b/host/src/audio_panel.hpp index 6373b91..5d82bc0 100644 --- a/host/src/audio_panel.hpp +++ b/host/src/audio_panel.hpp @@ -27,8 +27,47 @@ public: // `debug_details` on, the per-stream table is shown. void draw_ui(const HookStatusView& status, bool debug_details); +#ifdef COOP_TEST_HARNESS + // Test-harness hooks (debug builds only): drive the real audio code paths and read + // state back, incl. targeting a windowless process by pid (coop_tone has no window). + void dev_set_enabled(bool on) + { + enabled_ = on; + } + void dev_set_pid(DWORD pid) + { + dev_pid_ = pid; + } + void dev_request_op(unsigned slot, std::uint32_t kind, std::uint32_t rate, std::uint32_t ch, + std::uint32_t bits, std::uint32_t tag) + { + mirror_.request_op(slot, kind, rate, ch, bits, tag); + } + [[nodiscard]] bool dev_running() const + { + return mirror_.running(); + } + [[nodiscard]] unsigned dev_rate() const + { + return mirror_.sample_rate(); + } + [[nodiscard]] unsigned dev_channels() const + { + return mirror_.channels(); + } + [[nodiscard]] std::string dev_source() const + { + return mirror_.source_name(); + } + [[nodiscard]] std::string dev_reason() const + { + return mirror_.fallback_reason(); + } +#endif + private: HWND target_ = nullptr; + DWORD dev_pid_ = 0; // test harness only: force a (windowless) target pid; 0 in production bool enabled_ = false; AudioMirror mirror_; @@ -41,6 +80,12 @@ private: std::uint64_t rate_base_frames_[kMaxAudioStreams] = {}; double frames_per_s_[kMaxAudioStreams] = {}; double rate_base_time_ = 0.0; + + // Operator format-override editor (debug details). Applies to the primary stream. + int ov_rate_ = 48000; + int ov_channels_ = 2; + int ov_bits_idx_ = 1; // 0 = 16-bit, 1 = 32-bit + int ov_fmt_idx_ = 1; // 0 = PCM, 1 = float }; } // namespace coop diff --git a/host/src/injection_panel.cpp b/host/src/injection_panel.cpp index e2d6895..5990601 100644 --- a/host/src/injection_panel.cpp +++ b/host/src/injection_panel.cpp @@ -254,6 +254,24 @@ void InjectionPanel::reattach() inject_selected(); } +#ifdef COOP_TEST_HARNESS +unsigned long InjectionPanel::dev_inject_by_name(const std::wstring& image_name) +{ + refresh_processes(); + for (const ProcessEntry& e : processes_) + { + if (iequals_name(e.exe_name, image_name)) + { + selected_pid_ = e.pid; + selected_name_ = e.exe_name; + inject_selected(); + return injected_ ? selected_pid_ : 0; + } + } + return 0; +} +#endif + void InjectionPanel::publish(const std::array& pads) { if (!test_input_.load(std::memory_order_relaxed)) diff --git a/host/src/injection_panel.hpp b/host/src/injection_panel.hpp index b41505f..204f041 100644 --- a/host/src/injection_panel.hpp +++ b/host/src/injection_panel.hpp @@ -45,6 +45,12 @@ public: // focus-API counts, input-path detection); off shows a general summary. void draw(bool debug_details); +#ifdef COOP_TEST_HARNESS + // Test harness (debug builds only): inject into the first running process whose image + // name matches. Returns the pid on success, 0 otherwise. Same path as the UI button. + unsigned long dev_inject_by_name(const std::wstring& image_name); +#endif + // Forward the latest pad snapshot to the injected hook (if connected). When // test-input mode is on, a synthetic pattern is sent instead of `pads`. void publish(const std::array& pads); diff --git a/host/src/main.cpp b/host/src/main.cpp index 106bb96..b4d038d 100644 --- a/host/src/main.cpp +++ b/host/src/main.cpp @@ -11,8 +11,14 @@ #include +#include #include +#ifdef COOP_TEST_HARNESS +#include +#include +#endif + #include #include "imgui.h" @@ -27,6 +33,7 @@ #include "injection_panel.hpp" #include "input/input_worker.hpp" #include "log_panel.hpp" +#include "test_harness.hpp" #include "ui/app_chrome.hpp" namespace @@ -82,6 +89,102 @@ void draw_screenshot_toast(double seconds_since, const std::string& name) ImGui::End(); } +#ifdef COOP_TEST_HARNESS +std::wstring widen(const std::string& s) +{ + if (s.empty()) + { + return {}; + } + const int n = MultiByteToWideChar(CP_UTF8, 0, s.c_str(), static_cast(s.size()), nullptr, 0); + std::wstring w(static_cast(n), L'\0'); + MultiByteToWideChar(CP_UTF8, 0, s.c_str(), static_cast(s.size()), w.data(), n); + return w; +} + +// Run a test-harness command on the main thread, hitting the same code the UI buttons do. +// Returns a one-line response the driver reads back. +std::string apply_test_command(const std::string& cmd, coop::UiState& ui, coop::InjectionPanel& injection, + coop::AudioPanel& audio, coop::D3D11Window& window) +{ + std::vector tok; + { + std::istringstream is(cmd); + std::string t; + while (is >> t) + { + tok.push_back(t); + } + } + if (tok.empty()) + { + return "empty"; + } + const std::string& v = tok[0]; + auto arg = [&](std::size_t i) -> std::string { return i < tok.size() ? tok[i] : std::string(); }; + auto num = [&](std::size_t i) -> unsigned { return static_cast(std::strtoul(arg(i).c_str(), nullptr, 10)); }; + + if (v == "inject") + { + const unsigned long pid = injection.dev_inject_by_name(widen(arg(1))); + return pid != 0 ? ("ok pid " + std::to_string(pid)) : "fail no-process-or-inject-failed"; + } + if (v == "audio") + { + const bool on = arg(1) == "on"; + if (on) + { + audio.dev_set_pid(injection.target_pid()); + } + audio.dev_set_enabled(on); + return "ok"; + } + if (v == "debug") + { + ui.debug_details = (arg(1) == "on"); + return "ok"; + } + if (v == "remeasure") + { + audio.dev_request_op(num(1), coop::AudioRingOp_Remeasure, 0, 0, 0, 0); + return "ok"; + } + if (v == "override") + { + const std::uint32_t tag = arg(5) == "float" ? static_cast(WAVE_FORMAT_IEEE_FLOAT) + : static_cast(WAVE_FORMAT_PCM); + audio.dev_request_op(num(1), coop::AudioRingOp_Override, num(2), num(3), num(4), tag); + return "ok"; + } + if (v == "screenshot") + { + const std::wstring p = screenshot_path(); + window.request_screenshot(p); + return "ok"; + } + if (v == "quit") + { + ui.request_quit = true; + return "ok"; + } + if (v == "status") + { + const coop::HookStatusView st = injection.hook_status(); + const std::string reason = audio.dev_reason(); + char buf[512]; + std::snprintf(buf, sizeof(buf), + "audio_running=%d source=%s rate=%u ch=%u state=%u streams=%u inj_pid=%lu inj_state=%d " + "reason=%s", + audio.dev_running() ? 1 : 0, audio.dev_source().c_str(), audio.dev_rate(), + audio.dev_channels(), st.audio_streams[0].format_state, st.audio_streams_seen, + injection.target_pid(), static_cast(injection.target_state()), + reason.empty() ? "-" : reason.c_str()); + return buf; + } + return "unknown-command"; +} +#endif // COOP_TEST_HARNESS + #ifdef COOP_WITH_STEAM // Absolute path to the bundled Steam Input action manifest (next to the exe). std::string steam_manifest_path() @@ -208,6 +311,9 @@ int run() coop::register_ui_settings(ui); coop::FrameStats stats; + coop::TestHarness harness; // debug builds only; a no-op shim otherwise + harness.init(); + // Frame-sync: the hook generation we last presented (so we wait for the next one). std::uint32_t last_synced_gen = 0; @@ -249,6 +355,13 @@ int run() stats.tick(ImGui::GetIO().DeltaTime * 1000.0f); log.pull(injection); // drain hook log lines even while the Log window is hidden +#ifdef COOP_TEST_HARNESS + if (std::string tcmd = harness.poll_command(); !tcmd.empty()) + { + harness.write_response(apply_test_command(tcmd, ui, injection, audio, window)); + } +#endif + if (ImGui::IsKeyPressed(ImGuiKey_F1, false)) { show_overlay = !show_overlay; diff --git a/host/src/test_harness.cpp b/host/src/test_harness.cpp new file mode 100644 index 0000000..32f5ead --- /dev/null +++ b/host/src/test_harness.cpp @@ -0,0 +1,51 @@ +#include "test_harness.hpp" + +#ifdef COOP_TEST_HARNESS + +#include + +#include + +namespace coop +{ +namespace +{ +std::wstring temp_file(const wchar_t* name) +{ + wchar_t dir[MAX_PATH] = {}; + const DWORD n = GetTempPathW(MAX_PATH, dir); + return (n != 0 && n < MAX_PATH) ? std::wstring(dir) + name : std::wstring(name); +} +} // namespace + +void TestHarness::init() +{ + cmd_path_ = temp_file(L"coop_test_cmd.txt"); + resp_path_ = temp_file(L"coop_test_resp.txt"); + DeleteFileW(cmd_path_.c_str()); // drop any stale command from a previous run + DeleteFileW(resp_path_.c_str()); +} + +std::string TestHarness::poll_command() +{ + std::ifstream f(cmd_path_.c_str()); // MSVC accepts a wide path + if (!f) + { + return {}; + } + std::string line; + std::getline(f, line); + f.close(); + DeleteFileW(cmd_path_.c_str()); // ack: the command has been taken + return line; +} + +void TestHarness::write_response(const std::string& resp) +{ + std::ofstream f(resp_path_.c_str(), std::ios::trunc); + f << resp; +} + +} // namespace coop + +#endif // COOP_TEST_HARNESS diff --git a/host/src/test_harness.hpp b/host/src/test_harness.hpp new file mode 100644 index 0000000..af5ec52 --- /dev/null +++ b/host/src/test_harness.hpp @@ -0,0 +1,43 @@ +// Debug-only IPC test harness (compiled only when COOP_TEST_HARNESS is defined; the +// shipped product never contains it). Lets a test script drive the real overlay code +// paths -- inject, enable audio, re-measure, override, etc. -- by writing a command to +// %TEMP%\coop_test_cmd.txt and reading the reply from %TEMP%\coop_test_resp.txt, instead +// of simulating mouse/keyboard input (which ImGui doesn't accept reliably via PostMessage). +// +// Protocol: the driver writes one command line to the cmd file; the host consumes it +// (deleting the cmd file to ack), runs it on the main thread (so it hits the same code +// the UI buttons do), and writes a single response line to the resp file. One command at +// a time. See tools/test_drive notes / the per-phase validation scripts. +#pragma once + +#include + +namespace coop +{ + +class TestHarness +{ +public: +#ifdef COOP_TEST_HARNESS + void init(); // resolve the %TEMP% file paths and clear any stale command + // Main thread: returns the next pending command line (acking by deleting the cmd + // file), or an empty string if none is waiting. + std::string poll_command(); + // Main thread: write the response for the command just handled. + void write_response(const std::string& resp); + +private: + std::wstring cmd_path_; + std::wstring resp_path_; +#else + // No-op shims so call sites don't need their own #ifdef. + void init() {} + std::string poll_command() + { + return {}; + } + void write_response(const std::string&) {} +#endif +}; + +} // namespace coop