diff --git a/README.md b/README.md index fb5ffab..929ef21 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ XInput game becomes Remote-Play-Together-able. | Forward input to game | DLL injection + XInput hook (SafetyHook) — game sees *only* our pad | `coop_hook.dll` | done | | Keep game running unfocused | Hook spoofs focus so the game polls while the host holds OS focus | `coop_hook.dll` | done | | Mirror video | Windows Graphics Capture of the game window, letterboxed into the host window | `coop_host.exe` | done | -| Mirror audio | WASAPI process-loopback capture of the game, re-rendered on the host | `coop_host.exe` | done | +| Mirror audio | Injected render-hook copies the game's WASAPI frames into a shared ring and silences the game locally (no echo); WASAPI process loopback is the automatic fallback | `coop_hook.dll` + `coop_host.exe` | done | | Host ↔ hook IPC | Named shared memory (seqlock for input, status back-channel) | `common/` | done | ## Limitations @@ -38,9 +38,13 @@ XInput game becomes Remote-Play-Together-able. DirectInput-only / RawInput-only games are not handled. - **x64 only:** the host and hook DLL must match the game's bitness, and only x64 is built today. 32-bit games need the x86 hook + injector (see Roadmap). -- **Local audio echo:** process-loopback capture does not mute the game, so the - game's audio plays locally *and* the host re-renders it — the local machine - hears it twice. Guests hear it once. Fixing this is on the Roadmap. +- **Local audio echo (fixed via the hook; falls back otherwise):** when the + render-hook is active it silences the game's local playback while mirroring it, + so there is no echo. If the hook can't attach or the game uses an + unhooked/exotic render path, the host automatically falls back to + process-loopback capture, which does *not* mute the game — so on the fallback + path the local machine still hears the audio twice (guests hear it once). The + Audio panel shows which path is active. - **Debug-oriented UI:** the ImGui overlay is always visible and laid out for diagnosing the pipeline, not for end use. It can't yet be toggled off. @@ -62,6 +66,15 @@ All phases below are implemented and verified. - **Phase 2 — audio mirror. ✅** The host captures the game's audio by PID via WASAPI process loopback and re-renders it on the default endpoint, so RPT carries game audio to guests. +- **Audio render-hook (echo fix). ✅ (pending manual end-to-end)** The injected + hook intercepts the game's WASAPI render path (`IAudioRenderClient`), copies + the frames into a shared audio ring for the host to re-render, and releases the + game's buffer silenced — so the operator no longer hears the audio twice. The + host owns an enable flag and re-renders the game's format via `AUTOCONVERTPCM`; + if the hook doesn't publish a format in time it reverts to process loopback. + The Audio panel shows the active source and a render-stream-count debug table. + Validated in-process by `audio_hook_test`; the real-game pass is the remaining + manual step. ## Roadmap @@ -72,8 +85,6 @@ Future work, roughly in priority order: - **Generalize the UI:** rework the panels from bug-specific debug readouts into general-purpose status, and add broader debug info (latency, frame timing, per-channel stats). -- **Fix the local audio echo:** mute the game's local render (or otherwise avoid - the double playback) while still capturing it for the mirror. - **Present-hook video path:** capture the game's frames by hooking `IDXGISwapChain::Present` in the injected DLL and sharing the backbuffer via a shared D3D11 texture, as a lower-latency / more stable alternative to WGC. @@ -119,10 +130,17 @@ ctest --test-dir build -C Debug --output-on-failure - **`hook_selftest`** — in-process check of the IPC + XInput hook core (no game, no controller needed). +- **`audio_ring_test`** — unit test of the shared audio ring (lock-free SPSC + push/pop, wrap-around, format handshake, overrun/drop). No device needed. +- **`audio_hook_test`** — in-process self-test of the WASAPI render-hook: installs + the hooks, renders a tone through WASAPI in the same process, and asserts the + COM vtables were discovered, the frames reached the ring (non-silent), the + primary stream was silenced, and exactly one render stream was counted. Skips + cleanly if the machine has no audio endpoint. - **`audio_loopback_test`** — spawns `coop_tone.exe` (a standalone WASAPI sine-wave source under [`tools/audio_tone`](tools/audio_tone)) and verifies the - shipping process-loopback capture receives its audio by PID. Skips cleanly if - the machine has no audio endpoint. + shipping process-loopback capture (the fallback path) receives its audio by + PID. Skips cleanly if the machine has no audio endpoint. ## Running the tool (manual, end-to-end) @@ -149,8 +167,11 @@ person/account to receive the stream. the host window now shows a live, letterboxed copy of the game. 4. **Mirror audio:** in the **Audio mirror** panel, tick **Mirror game audio**. - (You'll hear the game twice locally; that's the known echo — guests hear it - once.) + With the hook injected, **Source** shows **Hooked (no echo)** and the game's + local playback goes silent while guests still hear it. If it shows **Loopback + (echo)** the hook's render path wasn't caught and you'll hear the game twice + locally (guests still hear it once). The **Render streams** table shows how + many WASAPI streams the game emits (v1 mirrors the first/primary). 5. **Start Remote Play Together** from Steam and invite a guest. Verify the guest sees the mirrored video, hears the audio, and that their controller drives the @@ -183,5 +204,15 @@ Non-obvious things that cost time and constrain the design: also needs the Windows 10 20H1 headers — build with `NTDDI_VERSION ≥ 0x0A00000B`. - **WGC captures occluded windows but not minimized ones.** The game may sit behind the host window, but must not be minimized. -- **Process-loopback capture doesn't mute the source**, hence the local audio - echo — capturing a process's render does not stop it reaching the speakers. +- **Process-loopback capture doesn't mute the source** — capturing a process's + render does not stop it reaching the speakers. That's why the echo fix instead + injects a WASAPI render-hook that silences the game's own buffer + (`AUDCLNT_BUFFERFLAGS_SILENT`) after copying it for the mirror; loopback stays + as the fallback. +- **COM has no exports, so the render-hook walks vtables — and the indices are + easy to miscount.** All instances of a COM coclass share one vtable, so hooking + one object's method (resolved by frozen-ABI vtable index) catches every + instance. But the indices must be exact: `IAudioClient::GetService` is **14**, + not 13 — `SetEventHandle` (13) sits between `Reset` and `GetService`. Count the + full interface (including every inherited `IUnknown`/base method) when adding a + new COM hook. diff --git a/docs/audio-render-hook-plan.md b/docs/audio-render-hook-plan.md index eb5487a..2fa7711 100644 --- a/docs/audio-render-hook-plan.md +++ b/docs/audio-render-hook-plan.md @@ -1,7 +1,12 @@ # Plan: fix the local audio echo via an injection render-hook (Option B) -Status: **scoped, not started.** This document is the green-lit design; implement -against it. +Status: **implemented (M1–M4), pending manual end-to-end (M5).** Milestones 1–4 +below are built, committed, and covered by `audio_ring_test` + `audio_hook_test` +(in-process). The remaining step is M5: confirm in a real game with a guest that +there's no local echo, the guest still hears audio, and the stream count reads +correctly. This document is the green-lit design; the only correction applied +during implementation was the `IAudioClient::GetService` vtable index (14, not +13 — `SetEventHandle` is 13). ## Problem