CoopAllTheThings/docs/audio-render-hook-plan.md

# Plan: fix the local audio echo via an injection render-hook (Option B)

Status: **implemented (M1–M4), pending manual end-to-end (M5).** Milestones 1–4
below are built, committed, and covered by `audio_ring_test` + `audio_hook_test`
(in-process). The remaining step is M5: confirm in a real game with a guest that
there's no local echo, the guest still hears audio, and the stream count reads
correctly. This document is the green-lit design; the only correction applied
during implementation was the `IAudioClient::GetService` vtable index (14, not
13 — `SetEventHandle` is 13).

## Problem

`coop_host.exe` mirrors the game's audio so Steam Remote Play Together (which
streams only the host process's own audio session) carries game sound to guests.
Today the host captures the game via **WASAPI process loopback** and re-renders it
on the default endpoint. The game *also* still plays locally, so the operator's
default endpoint carries two copies of the audio ("double audio" / echo). The
guest hears one copy (the host re-render); only the local operator hears it twice.

The fix must suppress the game's *direct* local playback **without** killing the
signal that feeds the host re-render.

## Why the cheaper options are out

- **Option A — per-session mute (`ISimpleAudioVolume` / the Volume Mixer).**
  Tested manually: muting or mixing down the game's session also mutes/mixes down
  the mirror. The Windows Volume Mixer *is* the `ISimpleAudioVolume` per-session
  API (same calls), so there is no API-vs-mixer difference to exploit. This proves
  the process-loopback tap sits **downstream** of the session volume gate.
  **Conclusively out.**
- **Option C — redirect the game to a separate sink (`IAudioPolicyConfig`).** The
  per-app endpoint redirect is real and usable (it backs Windows' "App volume and
  device preferences"), but it only *routes* — it needs a destination endpoint
  that is silent to the operator yet capturable by us, i.e. a virtual sink.
  **Windows has no public API to instantiate a virtual audio endpoint at runtime
  without a driver** (endpoints are driver-backed; every "virtual cable" ships an
  installed signed kernel driver). The desired *ad-hoc, driverless, auto-removed*
  virtual device does not exist. **Out.**

The only place to both grab the audio *and* stop it reaching the shared endpoint
is **before it leaves the game process** — i.e. injection, which we already do for
input. That is Option B.

## Approach (Option B)

In `coop_hook.dll`, hook the game's WASAPI render path:

```
IMMDevice::Activate(IID_IAudioClient)          → IAudioClient
IAudioClient::Initialize(format)                ← capture WAVEFORMATEX here
IAudioClient::GetService(IID_IAudioRenderClient)→ IAudioRenderClient  (= a "stream")
  loop:  GetBuffer(n,&p) → game writes PCM → ReleaseBuffer(n,flags)
```

On `ReleaseBuffer`, copy the just-written frames into a shared audio ring (the host
re-renders them for RPT), then release with `AUDCLNT_BUFFERFLAGS_SILENT` so WASAPI
emits silence locally. Result: operator hears one copy (the host re-render), guest
hears one copy, no virtual device, no echo.

### Reaching the vtables

COM methods aren't exports, so we resolve them via vtable indices (frozen COM ABI)
and inline-hook the resolved addresses with SafetyHook (same engine as the XInput
hooks):

- **Anchor (only shared-vtable assumption):** the worker thread `CoCreateInstance`s
  its *own* `IMMDeviceEnumerator`, gets the default render `IMMDevice`, and hooks
  `IMMDevice::Activate` (vtable idx 3). All `IMMDevice` instances in the process
  share that vtable (single coclass), so the game's `Activate` calls are caught.
- **Everything else is hooked off the live pointers the game received** (no further
  assumptions), install-once guarded:
  - `Activate` hook → if IID is `IAudioClient`/`2`/`3`, hook `Initialize` (idx 3)
    and `GetService` (idx 14) on that object.
  - `Initialize` hook → capture the `WAVEFORMATEX` (rate / channels / bits / tag).
    Fallback for games using `IAudioClient3::InitializeSharedAudioStream`: read the
    format via the original `GetMixFormat` in the `GetService` hook.
  - `GetService` hook → if IID is `IAudioRenderClient`, register the stream and hook
    its `GetBuffer` (idx 3) and `ReleaseBuffer` (idx 4).

Vtable indices: `IMMDevice::Activate`=3; `IAudioClient::Initialize`=3,
`GetService`=14 (after `SetEventHandle`=13); `IAudioRenderClient::GetBuffer`=3,
`ReleaseBuffer`=4.

The worker thread `CoInitializeEx(MTA)` for the lifetime of the DLL (needed for the
enumerator instance); installs are retried on the existing 250 ms worker tick, like
the XInput / focus installs.

### Capture + silence

- **GetBuffer hook:** call original; stash `pData` + `numFrames` thread-local (the
  pair is always called on one thread, never nested).
- **ReleaseBuffer hook:** act only when `this == primary render client`. If not
  already silent: `memcpy` `numFrames × nBlockAlign` into the ring, then call the
  original `ReleaseBuffer(numFrames, flags | AUDCLNT_BUFFERFLAGS_SILENT)`. Copy
  happens before the original call (buffer is valid until release). memset-to-zero
  is kept as a fallback if any driver mishandles the SILENT flag.
- No allocation and no locks on the game's audio thread — only a lock-free ring
  write.

### Hooks always install; only copy+silence is gated

The Activate/Initialize/GetService/GetBuffer/ReleaseBuffer hooks install whenever
the DLL is injected, so **stream counting works even when audio mirroring is off**.
A `capture_enabled` flag (host-owned) gates *only* the copy+silence behavior in
`ReleaseBuffer`. Flag off → audio passes through untouched (game audible locally,
no mirror, but streams are still counted for the debug view).

## Audio IPC ring (new shared mapping)

The 20-byte `SharedBlock` can't hold PCM, so a **separate named mapping**
`Local\coop_audio_<pid>`, ~1 MB (>1 s at typical formats):

```
AudioRing header:  magic, version, capture_enabled,
                   format_valid, format_generation,
                   sample_rate, channels, bits, format_tag, block_align,
                   capacity, atomic<u64> write_pos, read_pos,
                   frames_produced, overruns;     // + reserved tail
data[capacity]:    PCM ring
```

- **Lock-free SPSC** (hook produces, host consumes): free-running 64-bit
  `write_pos` / `read_pos`, release on publish / acquire on read — same
  cross-process atomic model as the existing input seqlock. On full → drop the
  packet and bump `overruns` (host should always keep up).
- **Format handshake:** hook sets the format fields + `format_valid` once; the host
  spins on `format_valid` before creating its render client. `format_generation`
  is in the layout now so a future mid-session device re-init is forward-compatible
  (v1 handles format-set-once).
- **Ownership / toggle:** the **host creates the mapping and owns `capture_enabled`**;
  the hook holds it open and copies+silences only while the flag is set. Toggling
  the Audio panel checkbox just flips the flag — no injection churn, no teardown
  races. Closing the host's handle while the hook holds it is safe (the section is
  refcounted by the OS).

## Stream-count debug visualization (required)

Goal: easily see when a game emits more than one render stream, since v1 captures
only the first. These diagnostics live in the **always-present** `HookStatus`
back-channel in `SharedBlock` (not the audio ring), so the count is visible even
before/without enabling audio mirror. This bumps `kProtocolVersion` (3 → 4); new
members go at the **end** of `HookStatus` so existing offsets never shift.

Add to `HookStatus`:

```
uint32 audio_streams_seen;                       // distinct render clients ever created
AudioStreamInfo audio_streams[kMaxAudioStreams]; // kMaxAudioStreams = 4

struct AudioStreamInfo {
  uint32 is_primary;       // 1 = the stream we capture
  uint32 sample_rate;
  uint16 channels;
  uint16 bits;
  uint32 format_tag;
  uint64 frames_rendered;  // cumulative; host derives "live vs idle" from deltas
};
```

- The hook assigns each distinct `IAudioRenderClient` a slot, marks the first as
  `is_primary`, and bumps `frames_rendered` on each `ReleaseBuffer`.
- **Audio panel UI:** show `Render streams: N`, and a small table — one row per
  stream with format + a frames counter, the primary row tagged, and rows whose
  `frames_rendered` is advancing highlighted as live. This makes a multi-stream
  game obvious at a glance.
- If `audio_streams_seen > kMaxAudioStreams` (more streams than slots), still show
  the total count and note the overflow.

## Host changes

`AudioMirror` keeps its event-driven render client and prime/underrun logic almost
verbatim; only the **source** changes from the `ProcessLoopbackCapture` callback to
popping the shared audio ring. The host initializes its render client with the
*game's* published format and lets shared-mode WASAPI convert game-format →
endpoint. `audio_panel` gains a source indicator (Hooked vs Loopback) plus the
stream table above.

## Fallback — never regress

If the render hooks don't install, or `format_valid` never appears within ~1 s
(unusual COM setup, an uncaught backend, anti-cheat, etc.), the host
**automatically falls back to the existing process-loopback path** (works, but with
the echo) and surfaces that in the panel. Worst case = today's behavior.
`host/src/audio/process_loopback_capture.*` stays in the tree as the fallback.

## Known limitations (v1)

- **Primary stream only.** With multiple simultaneous render streams we capture +
  silence only the first; secondaries stay local and unmirrored (graceful degrade,
  no corruption). The debug view exists precisely to detect this. Per-stream rings
  + host-side mix is a follow-up if it ever matters.
- **`ActivateAudioInterfaceAsync` activation path not hooked in v1** (most games use
  `IMMDevice::Activate`; the rest hit the loopback fallback).
- Exclusive-mode and DirectSound/XAudio2 backends still bottom out in an
  `IAudioClient`, so they're covered; truly exotic backends fall back.

## Work breakdown

| File | Change | Size |
| --- | --- | --- |
| `common/include/coop/audio_ring.hpp` | NEW — ring layout + lock-free push/pop + format/flag | ~120 ln |
| `common/include/coop/protocol.hpp` | add `AudioStreamInfo` + audio diag fields to `HookStatus`; bump version 3→4 | small |
| `hook/src/audio_hook.{hpp,cpp}` | NEW — vtable discovery, install/remove, the 4 hooks, ring producer, stream counting | ~350 ln (the risk) |
| `hook/src/dllmain.cpp` | open `coop_audio_<pid>`, install audio hooks in worker loop | small |
| `hook/CMakeLists.txt` | link `ole32 mmdevapi`; COM-init the worker thread | small |
| `host/src/audio/audio_loopback.{cpp,hpp}` | "hooked ring" source mode + create mapping + auto-fallback | medium |
| `host/src/audio_panel.{cpp,hpp}` | source indicator + stream-count debug table | small |
| `host/CMakeLists.txt` | new sources | small |
| `tests/audio_hook_test.cpp` | NEW — in-process: render a tone, install hooks, assert ring gets non-silent frames AND local output went silent; assert stream count == 1 | ~150 ln |

The in-process self-test is the key de-risker: it exercises vtable discovery +
GetBuffer/ReleaseBuffer interception with no game and no second Steam account, the
same way `hook_selftest` covers the XInput core.

**Estimate: ~1.5–2 days.** `audio_hook.cpp` is the only real risk; the rest mirrors
patterns already in the repo.

## Milestones (build order)

1. `audio_ring.hpp` + `HookStatus` audio fields + host ring consumer + ring unit
   test (no hook yet).
2. `audio_hook.cpp` + the in-process tone → ring → silenced self-test, including the
   stream count. ← *proves the concept*
3. Wire into `dllmain` + host "hooked" mode + automatic loopback fallback.
4. Audio panel UI: source indicator + stream-count debug table.
5. Manual end-to-end in a real game: confirm no local echo, guest still hears audio,
   and the stream count reads correctly.

If milestone 2 passes, the rest is plumbing.