Audio render-hook M1: shared audio ring + protocol diag fields
First milestone of the injection render-hook audio path (see docs/audio-render-hook-plan.md) that fixes the local audio echo without a virtual device. - common/include/coop/audio_ring.hpp: new lock-free SPSC shared-memory ring for PCM, separate from the input/status SharedBlock. Free-running 64-bit positions (release/acquire), format handshake, host-owned capture_enabled gate, drop-whole-packet overrun policy. - common/include/coop/protocol.hpp: add AudioStreamInfo + audio_streams_seen / audio_streams[] to the always-present HookStatus for the render-stream-count debug view; bump kProtocolVersion 3->4 (new members appended). - tests/audio_ring_test.cpp: in-process unit test (push/pop integrity, wrap-around, format handshake, overrun/drop). No hook or audio device. - docs/audio-render-hook-plan.md: the green-lit design this implements. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
223
docs/audio-render-hook-plan.md
Normal file
223
docs/audio-render-hook-plan.md
Normal file
@@ -0,0 +1,223 @@
|
||||
# Plan: fix the local audio echo via an injection render-hook (Option B)
|
||||
|
||||
Status: **scoped, not started.** This document is the green-lit design; implement
|
||||
against it.
|
||||
|
||||
## Problem
|
||||
|
||||
`coop_host.exe` mirrors the game's audio so Steam Remote Play Together (which
|
||||
streams only the host process's own audio session) carries game sound to guests.
|
||||
Today the host captures the game via **WASAPI process loopback** and re-renders it
|
||||
on the default endpoint. The game *also* still plays locally, so the operator's
|
||||
default endpoint carries two copies of the audio ("double audio" / echo). The
|
||||
guest hears one copy (the host re-render); only the local operator hears it twice.
|
||||
|
||||
The fix must suppress the game's *direct* local playback **without** killing the
|
||||
signal that feeds the host re-render.
|
||||
|
||||
## Why the cheaper options are out
|
||||
|
||||
- **Option A — per-session mute (`ISimpleAudioVolume` / the Volume Mixer).**
|
||||
Tested manually: muting or mixing down the game's session also mutes/mixes down
|
||||
the mirror. The Windows Volume Mixer *is* the `ISimpleAudioVolume` per-session
|
||||
API (same calls), so there is no API-vs-mixer difference to exploit. This proves
|
||||
the process-loopback tap sits **downstream** of the session volume gate.
|
||||
**Conclusively out.**
|
||||
- **Option C — redirect the game to a separate sink (`IAudioPolicyConfig`).** The
|
||||
per-app endpoint redirect is real and usable (it backs Windows' "App volume and
|
||||
device preferences"), but it only *routes* — it needs a destination endpoint
|
||||
that is silent to the operator yet capturable by us, i.e. a virtual sink.
|
||||
**Windows has no public API to instantiate a virtual audio endpoint at runtime
|
||||
without a driver** (endpoints are driver-backed; every "virtual cable" ships an
|
||||
installed signed kernel driver). The desired *ad-hoc, driverless, auto-removed*
|
||||
virtual device does not exist. **Out.**
|
||||
|
||||
The only place to both grab the audio *and* stop it reaching the shared endpoint
|
||||
is **before it leaves the game process** — i.e. injection, which we already do for
|
||||
input. That is Option B.
|
||||
|
||||
## Approach (Option B)
|
||||
|
||||
In `coop_hook.dll`, hook the game's WASAPI render path:
|
||||
|
||||
```
|
||||
IMMDevice::Activate(IID_IAudioClient) → IAudioClient
|
||||
IAudioClient::Initialize(format) ← capture WAVEFORMATEX here
|
||||
IAudioClient::GetService(IID_IAudioRenderClient)→ IAudioRenderClient (= a "stream")
|
||||
loop: GetBuffer(n,&p) → game writes PCM → ReleaseBuffer(n,flags)
|
||||
```
|
||||
|
||||
On `ReleaseBuffer`, copy the just-written frames into a shared audio ring (the host
|
||||
re-renders them for RPT), then release with `AUDCLNT_BUFFERFLAGS_SILENT` so WASAPI
|
||||
emits silence locally. Result: operator hears one copy (the host re-render), guest
|
||||
hears one copy, no virtual device, no echo.
|
||||
|
||||
### Reaching the vtables
|
||||
|
||||
COM methods aren't exports, so we resolve them via vtable indices (frozen COM ABI)
|
||||
and inline-hook the resolved addresses with SafetyHook (same engine as the XInput
|
||||
hooks):
|
||||
|
||||
- **Anchor (only shared-vtable assumption):** the worker thread `CoCreateInstance`s
|
||||
its *own* `IMMDeviceEnumerator`, gets the default render `IMMDevice`, and hooks
|
||||
`IMMDevice::Activate` (vtable idx 3). All `IMMDevice` instances in the process
|
||||
share that vtable (single coclass), so the game's `Activate` calls are caught.
|
||||
- **Everything else is hooked off the live pointers the game received** (no further
|
||||
assumptions), install-once guarded:
|
||||
- `Activate` hook → if IID is `IAudioClient`/`2`/`3`, hook `Initialize` (idx 3)
|
||||
and `GetService` (idx 13) on that object.
|
||||
- `Initialize` hook → capture the `WAVEFORMATEX` (rate / channels / bits / tag).
|
||||
Fallback for games using `IAudioClient3::InitializeSharedAudioStream`: read the
|
||||
format via the original `GetMixFormat` in the `GetService` hook.
|
||||
- `GetService` hook → if IID is `IAudioRenderClient`, register the stream and hook
|
||||
its `GetBuffer` (idx 3) and `ReleaseBuffer` (idx 4).
|
||||
|
||||
Vtable indices: `IMMDevice::Activate`=3; `IAudioClient::Initialize`=3,
|
||||
`GetService`=13; `IAudioRenderClient::GetBuffer`=3, `ReleaseBuffer`=4.
|
||||
|
||||
The worker thread `CoInitializeEx(MTA)` for the lifetime of the DLL (needed for the
|
||||
enumerator instance); installs are retried on the existing 250 ms worker tick, like
|
||||
the XInput / focus installs.
|
||||
|
||||
### Capture + silence
|
||||
|
||||
- **GetBuffer hook:** call original; stash `pData` + `numFrames` thread-local (the
|
||||
pair is always called on one thread, never nested).
|
||||
- **ReleaseBuffer hook:** act only when `this == primary render client`. If not
|
||||
already silent: `memcpy` `numFrames × nBlockAlign` into the ring, then call the
|
||||
original `ReleaseBuffer(numFrames, flags | AUDCLNT_BUFFERFLAGS_SILENT)`. Copy
|
||||
happens before the original call (buffer is valid until release). memset-to-zero
|
||||
is kept as a fallback if any driver mishandles the SILENT flag.
|
||||
- No allocation and no locks on the game's audio thread — only a lock-free ring
|
||||
write.
|
||||
|
||||
### Hooks always install; only copy+silence is gated
|
||||
|
||||
The Activate/Initialize/GetService/GetBuffer/ReleaseBuffer hooks install whenever
|
||||
the DLL is injected, so **stream counting works even when audio mirroring is off**.
|
||||
A `capture_enabled` flag (host-owned) gates *only* the copy+silence behavior in
|
||||
`ReleaseBuffer`. Flag off → audio passes through untouched (game audible locally,
|
||||
no mirror, but streams are still counted for the debug view).
|
||||
|
||||
## Audio IPC ring (new shared mapping)
|
||||
|
||||
The 20-byte `SharedBlock` can't hold PCM, so a **separate named mapping**
|
||||
`Local\coop_audio_<pid>`, ~1 MB (>1 s at typical formats):
|
||||
|
||||
```
|
||||
AudioRing header: magic, version, capture_enabled,
|
||||
format_valid, format_generation,
|
||||
sample_rate, channels, bits, format_tag, block_align,
|
||||
capacity, atomic<u64> write_pos, read_pos,
|
||||
frames_produced, overruns; // + reserved tail
|
||||
data[capacity]: PCM ring
|
||||
```
|
||||
|
||||
- **Lock-free SPSC** (hook produces, host consumes): free-running 64-bit
|
||||
`write_pos` / `read_pos`, release on publish / acquire on read — same
|
||||
cross-process atomic model as the existing input seqlock. On full → drop the
|
||||
packet and bump `overruns` (host should always keep up).
|
||||
- **Format handshake:** hook sets the format fields + `format_valid` once; the host
|
||||
spins on `format_valid` before creating its render client. `format_generation`
|
||||
is in the layout now so a future mid-session device re-init is forward-compatible
|
||||
(v1 handles format-set-once).
|
||||
- **Ownership / toggle:** the **host creates the mapping and owns `capture_enabled`**;
|
||||
the hook holds it open and copies+silences only while the flag is set. Toggling
|
||||
the Audio panel checkbox just flips the flag — no injection churn, no teardown
|
||||
races. Closing the host's handle while the hook holds it is safe (the section is
|
||||
refcounted by the OS).
|
||||
|
||||
## Stream-count debug visualization (required)
|
||||
|
||||
Goal: easily see when a game emits more than one render stream, since v1 captures
|
||||
only the first. These diagnostics live in the **always-present** `HookStatus`
|
||||
back-channel in `SharedBlock` (not the audio ring), so the count is visible even
|
||||
before/without enabling audio mirror. This bumps `kProtocolVersion` (3 → 4); new
|
||||
members go at the **end** of `HookStatus` so existing offsets never shift.
|
||||
|
||||
Add to `HookStatus`:
|
||||
|
||||
```
|
||||
uint32 audio_streams_seen; // distinct render clients ever created
|
||||
AudioStreamInfo audio_streams[kMaxAudioStreams]; // kMaxAudioStreams = 4
|
||||
|
||||
struct AudioStreamInfo {
|
||||
uint32 is_primary; // 1 = the stream we capture
|
||||
uint32 sample_rate;
|
||||
uint16 channels;
|
||||
uint16 bits;
|
||||
uint32 format_tag;
|
||||
uint64 frames_rendered; // cumulative; host derives "live vs idle" from deltas
|
||||
};
|
||||
```
|
||||
|
||||
- The hook assigns each distinct `IAudioRenderClient` a slot, marks the first as
|
||||
`is_primary`, and bumps `frames_rendered` on each `ReleaseBuffer`.
|
||||
- **Audio panel UI:** show `Render streams: N`, and a small table — one row per
|
||||
stream with format + a frames counter, the primary row tagged, and rows whose
|
||||
`frames_rendered` is advancing highlighted as live. This makes a multi-stream
|
||||
game obvious at a glance.
|
||||
- If `audio_streams_seen > kMaxAudioStreams` (more streams than slots), still show
|
||||
the total count and note the overflow.
|
||||
|
||||
## Host changes
|
||||
|
||||
`AudioMirror` keeps its event-driven render client and prime/underrun logic almost
|
||||
verbatim; only the **source** changes from the `ProcessLoopbackCapture` callback to
|
||||
popping the shared audio ring. The host initializes its render client with the
|
||||
*game's* published format and lets shared-mode WASAPI convert game-format →
|
||||
endpoint. `audio_panel` gains a source indicator (Hooked vs Loopback) plus the
|
||||
stream table above.
|
||||
|
||||
## Fallback — never regress
|
||||
|
||||
If the render hooks don't install, or `format_valid` never appears within ~1 s
|
||||
(unusual COM setup, an uncaught backend, anti-cheat, etc.), the host
|
||||
**automatically falls back to the existing process-loopback path** (works, but with
|
||||
the echo) and surfaces that in the panel. Worst case = today's behavior.
|
||||
`host/src/audio/process_loopback_capture.*` stays in the tree as the fallback.
|
||||
|
||||
## Known limitations (v1)
|
||||
|
||||
- **Primary stream only.** With multiple simultaneous render streams we capture +
|
||||
silence only the first; secondaries stay local and unmirrored (graceful degrade,
|
||||
no corruption). The debug view exists precisely to detect this. Per-stream rings
|
||||
+ host-side mix is a follow-up if it ever matters.
|
||||
- **`ActivateAudioInterfaceAsync` activation path not hooked in v1** (most games use
|
||||
`IMMDevice::Activate`; the rest hit the loopback fallback).
|
||||
- Exclusive-mode and DirectSound/XAudio2 backends still bottom out in an
|
||||
`IAudioClient`, so they're covered; truly exotic backends fall back.
|
||||
|
||||
## Work breakdown
|
||||
|
||||
| File | Change | Size |
|
||||
| --- | --- | --- |
|
||||
| `common/include/coop/audio_ring.hpp` | NEW — ring layout + lock-free push/pop + format/flag | ~120 ln |
|
||||
| `common/include/coop/protocol.hpp` | add `AudioStreamInfo` + audio diag fields to `HookStatus`; bump version 3→4 | small |
|
||||
| `hook/src/audio_hook.{hpp,cpp}` | NEW — vtable discovery, install/remove, the 4 hooks, ring producer, stream counting | ~350 ln (the risk) |
|
||||
| `hook/src/dllmain.cpp` | open `coop_audio_<pid>`, install audio hooks in worker loop | small |
|
||||
| `hook/CMakeLists.txt` | link `ole32 mmdevapi`; COM-init the worker thread | small |
|
||||
| `host/src/audio/audio_loopback.{cpp,hpp}` | "hooked ring" source mode + create mapping + auto-fallback | medium |
|
||||
| `host/src/audio_panel.{cpp,hpp}` | source indicator + stream-count debug table | small |
|
||||
| `host/CMakeLists.txt` | new sources | small |
|
||||
| `tests/audio_hook_test.cpp` | NEW — in-process: render a tone, install hooks, assert ring gets non-silent frames AND local output went silent; assert stream count == 1 | ~150 ln |
|
||||
|
||||
The in-process self-test is the key de-risker: it exercises vtable discovery +
|
||||
GetBuffer/ReleaseBuffer interception with no game and no second Steam account, the
|
||||
same way `hook_selftest` covers the XInput core.
|
||||
|
||||
**Estimate: ~1.5–2 days.** `audio_hook.cpp` is the only real risk; the rest mirrors
|
||||
patterns already in the repo.
|
||||
|
||||
## Milestones (build order)
|
||||
|
||||
1. `audio_ring.hpp` + `HookStatus` audio fields + host ring consumer + ring unit
|
||||
test (no hook yet).
|
||||
2. `audio_hook.cpp` + the in-process tone → ring → silenced self-test, including the
|
||||
stream count. ← *proves the concept*
|
||||
3. Wire into `dllmain` + host "hooked" mode + automatic loopback fallback.
|
||||
4. Audio panel UI: source indicator + stream-count debug table.
|
||||
5. Manual end-to-end in a real game: confirm no local echo, guest still hears audio,
|
||||
and the stream count reads correctly.
|
||||
|
||||
If milestone 2 passes, the rest is plumbing.
|
||||
Reference in New Issue
Block a user