CoopAllTheThings/README.md

# CoopAllTheThings

Steam **Remote Play Together (RPT)** for any XInput game — without breaking DRM,
achievements, or playtime.

Existing "donor game" tools (e.g. RemotePlayWhatever) copy a target game's files
into a donor game's folder and rename the executable so Steam streams the target
under the donor's appid. That breaks DRM-protected games, breaks achievements,
and credits playtime to the donor.

CoopAllTheThings takes a different approach: the **real game runs normally under
its own appid** (so DRM, achievements, and playtime all work), while a lightweight
**mirror app runs under the donor appid**. The mirror presents a borderless
window that is a live copy of the game's video + audio, and forwards the guests'
input back into the real game. Steam's RPT captures the mirror window — so any
XInput game becomes Remote-Play-Together-able.

## Architecture

| Concern | Mechanism | Component | Status |
| --- | --- | --- | --- |
| Receive guest input | XInput (RPT delivers guest pads to the focused window) | `coop_host.exe` | done |
| Forward input to game | DLL injection + XInput hook (SafetyHook) — game sees *only* our pad | `coop_hook.dll` | done |
| Keep game running unfocused | Hook spoofs focus so the game polls while the host holds OS focus | `coop_hook.dll` | done |
| Mirror video | Windows Graphics Capture of the game window, letterboxed into the host window | `coop_host.exe` | done |
| Mirror audio | Injected render-hook copies the game's WASAPI frames into a shared ring and silences the game locally (no echo); WASAPI process loopback is the automatic fallback | `coop_hook.dll` + `coop_host.exe` | done |
| Host ↔ hook IPC | Named shared memory (seqlock for input, status back-channel) | `common/` | done |

## Limitations

- **Anti-cheat:** the input path injects `coop_hook.dll` into the target game.
  Games protected by kernel-level anti-cheat (Easy Anti-Cheat, BattlEye,
  Vanguard, etc.) will detect the injected module and may **kick the player or
  issue a ban**. Such games are explicitly **out of scope and unsupported** — do
  not use CoopAllTheThings with them. The tool targets single-player and
  co-op/local-multiplayer titles without active anti-cheat.
- **XInput only:** the game must read controllers via XInput (the common case).
  DirectInput-only / RawInput-only games are not handled.
- **x64 only:** the host and hook DLL must match the game's bitness, and only x64
  is built today. 32-bit games need the x86 hook + injector (see Roadmap).
- **Local audio echo (fixed via the hook; falls back otherwise):** when the
  render-hook is active it silences the game's local playback while mirroring it,
  so there is no echo. If the hook can't attach or the game uses an
  unhooked/exotic render path, the host automatically falls back to
  process-loopback capture, which does *not* mute the game — so on the fallback
  path the local machine still hears the audio twice (guests hear it once). The
  Audio panel shows which path is active.
- **Debug-oriented UI:** the ImGui overlay is always visible and laid out for
  diagnosing the pipeline, not for end use. It can't yet be toggled off.

## Status

All phases below are implemented and verified.

- **Phase 0 — donor spike. ✅** Confirmed Steam RPT streams an arbitrary
  borderless window under a donor appid and routes guest gamepads into it as
  XInput (correct slot assignment). This is the premise the whole tool rests on.
- **Phase 1a — input forwarding + focus spoofing. ✅** The host injects
  `coop_hook.dll`; the DLL hooks XInput (SafetyHook) so the game reads the
  forwarded pad state and *only* that state, and spoofs focus so the game keeps
  running while the host holds the real OS focus. A hook→host status
  back-channel reports attach state and poll rate.
- **Phase 1b — video mirror (WGC). ✅** The host captures the injected game's
  window with Windows Graphics Capture and draws it letterboxed as its
  background, so RPT streams a live mirror.
- **Phase 2 — audio mirror. ✅** The host captures the game's audio by PID via
  WASAPI process loopback and re-renders it on the default endpoint, so RPT
  carries game audio to guests.
- **Audio render-hook (echo fix). ✅ (capture proven on a real game; full RPT
  pass pending)** The injected hook intercepts the game's WASAPI render path
  (`IAudioRenderClient`), copies the frames into a shared audio ring for the host
  to re-render, and releases the game's buffer silenced — so the operator no
  longer hears the audio twice. It hooks the render vtables *proactively* so a
  game that's already playing when injected is still captured. The host owns an
  enable flag and re-renders the game's format via `AUTOCONVERTPCM`; if the hook
  doesn't publish a format in time it reverts to process loopback. The Audio panel
  shows the active source and a render-stream-count debug table. Validated
  in-process by `audio_hook_test` and against a real already-playing game
  (Phantom Brave) with `coop_audio_probe`: the pre-existing render client is
  detected and real, non-silent audio reaches the ring with zero overruns.

### Remaining verification

The full-app capture path now works: the hook publishes the captured format to the
ring whenever the host attaches it, so toggling **Mirror game audio** switches to
**Source: Hooked (no echo)** instead of falling back to loopback (verified with
`coop_audio_probe` reproducing the app's inject-then-create-ring ordering against
Phantom Brave). These still need a human / full setup to confirm:

- **Hooked audio over RPT, end-to-end.** Run the host, inject, tick **Mirror game
  audio**, and confirm **Source: Hooked (no echo)**, the game goes locally silent,
  and a guest on Remote Play Together still hears it.
- **Format guess for non-mix-format games.** For a stream that already exists at
  injection time the hook can't see the game's `Initialize`, so it assumes the
  device **mix format**. Phantom Brave matched it exactly (48 kHz/2ch/float). A
  game that initialized shared mode with a different format would come out
  wrong-pitched/garbled and needs per-stream format detection — not yet handled.
- **Multi-stream games.** v1 captures only the first ("primary") render stream;
  confirm the Audio panel's render-stream table makes a multi-stream game obvious
  (secondary streams stay local until per-stream mixing is added).

## Roadmap

Done:

- **Usable overlay. ✅** F1 hides the whole overlay so the window is a clean
  mirror for Remote Play Together; a fading hint shows the way back. The
  pipelines keep running while hidden.
- **Generalized UI. ✅** A top menu bar carries a consolidated FPS / frame-time
  (with jitter) readout and a **View** menu that toggles each panel and a global
  **Debug details** switch. Panels default to general-purpose status (attached,
  forwarding, controller polling rate, mirror resolution, audio source + buffered
  ms) and reveal the verbose diagnostics (per-slot poll table, focus-API counts,
  input-path detection, per-render-stream table) only when Debug details is on.

Future work, roughly in priority order:

- **Present-hook video path:** capture the game's frames by hooking
  `IDXGISwapChain::Present` in the injected DLL and sharing the backbuffer via a
  shared D3D11 texture, as a lower-latency / more stable alternative to WGC.
- **Steam Input:** consume guest input through the Steam Input API directly
  rather than XInput.
- **x86 support:** add an x86 build of `coop_hook.dll` plus an x86 injector
  helper the x64 host spawns, so 32-bit games work (a 64-bit process can't
  cleanly inject a 32-bit one). Detect target bitness with `IsWow64Process2`.
  The shared-memory IPC layout is already fixed-width / bitness-stable.

## Building

Requirements: Windows 10/11, Visual Studio 2022 (MSVC + C++ workload), CMake ≥ 3.21.

```sh
git clone --recurse-submodules <repo-url>
# or, if already cloned:
git submodule update --init --recursive

cmake -S . -B build -G "Visual Studio 17 2022" -A x64
cmake --build build --config Debug
# output: bin/Debug/coop_host.exe (+ coop_hook.dll, test exes)
```

Third-party dependencies (Dear ImGui, SafetyHook) are git submodules under
`third_party/`; the Steamworks SDK is vendored manually there when wired. No
vcpkg / package manager is used.

### clangd / IDE setup

The Visual Studio CMake generator does **not** emit `compile_commands.json`, so
clangd has no include paths and reports false errors. Run
[`gen-compile-commands.bat`](gen-compile-commands.bat) once (and after adding
sources or include dirs); it configures a parallel Ninja build in `build-clangd/`
that produces the database, which [`.clangd`](.clangd) points clangd at. clangd's
clang-cl driver resolves the MSVC / Windows SDK system includes on its own.

## Tests

```sh
ctest --test-dir build -C Debug --output-on-failure
```

- **`hook_selftest`** — in-process check of the IPC + XInput hook core (no game,
  no controller needed).
- **`audio_ring_test`** — unit test of the shared audio ring (lock-free SPSC
  push/pop, wrap-around, format handshake, overrun/drop). No device needed.
- **`audio_hook_test`** — in-process self-test of the WASAPI render-hook: installs
  the hooks, renders a tone through WASAPI in the same process, and asserts the
  COM vtables were discovered, the frames reached the ring (non-silent), the
  primary stream was silenced, and exactly one render stream was counted. Skips
  cleanly if the machine has no audio endpoint.
- **`audio_loopback_test`** — spawns `coop_tone.exe` (a standalone WASAPI
  sine-wave source under [`tools/audio_tone`](tools/audio_tone)) and verifies the
  shipping process-loopback capture (the fallback path) receives its audio by
  PID. Skips cleanly if the machine has no audio endpoint.

### Debugging the render-hook against a real game

[`tools/audio_probe`](tools/audio_probe) (`coop_audio_probe.exe <pid> [seconds]`)
brings up the audio render-hook without Steam / RPT / the host UI: it creates the
IPC block + audio ring the hook expects, injects `coop_hook.dll` into the target
game, then drains the ring and prints per-stream format, captured-frame counts,
peak amplitude (proves the audio is real, not silence), and overruns. It enables
the hook's file trace (`%TEMP%\coop_hook.log`) for the run. Run it from
`bin/<config>/`. **Kill the game between runs** — the loaded DLL locks
`coop_hook.dll` against the next rebuild.

## Running the tool (manual, end-to-end)

This needs Steam, a donor game that supports Remote Play Together, and a second
person/account to receive the stream.

1. **Launch the host under a donor appid.** Find the donor's appid (the number in
   its store URL); the donor only needs RPT support and is never actually played:

   ```text
   "C:\Program Files (x86)\Steam\steam.exe" -applaunch <donorAppId> "D:\dev\CoopAllTheThings\bin\Debug\coop_host.exe"
   ```

   The borderless window appears and Steam marks the donor "running". If the donor
   ignores the trailing path, set the host as the donor's **Launch Options**
   (`"D:\...\coop_host.exe" %command%`) or use a launcher like RemotePlayDetached.

2. **Start the real game** windowed or borderless (not exclusive fullscreen — see
   Lessons learned). In the host's **Injection** panel, filter for the game's
   `.exe`, select it, and click **Inject & Connect**. Watch **Hook status** for
   **Attached**, a non-zero **XInput polled: N/s**, and **Focus spoof: active**.

3. **Mirror video:** in the **Video mirror** panel, tick **Mirror game window** —
   the host window now shows a live, letterboxed copy of the game.

4. **Mirror audio:** in the **Audio mirror** panel, tick **Mirror game audio**.
   With the hook injected, **Source** shows **Hooked (no echo)** and the game's
   local playback goes silent while guests still hear it. If it shows **Loopback
   (echo)** the hook's render path wasn't caught and you'll hear the game twice
   locally (guests still hear it once). The **Render streams** table shows how
   many WASAPI streams the game emits (v1 mirrors the first/primary).

5. **Start Remote Play Together** from Steam and invite a guest. Verify the guest
   sees the mirrored video, hears the audio, and that their controller drives the
   real game.

Useful checks while developing without RPT: tick **Forward synthetic test input**
in the Injection panel to make the game move on its own (proving forwarding is the
source), and click away from the game to confirm focus spoofing keeps it running.

> Injection access error → run the host as administrator. "Target is 32-bit" →
> that game needs the x86 hook (see Roadmap).

## Lessons learned

Non-obvious things that cost time and constrain the design:

- **RPT only streams the *focused* window.** The game therefore can't hold focus
  itself; the hook spoofs focus (`GetForegroundWindow` / `GetActiveWindow` /
  `GetFocus`, plus swallowing window-deactivation messages) so the game keeps
  polling and rendering while the host owns the real OS focus.
- **Run target games windowed or borderless, never exclusive fullscreen.**
  Exclusive fullscreen minimizes on focus loss (defeating the focus spoof) and
  can't be window-captured. While unfocused the game gets no OS keyboard/mouse —
  only the forwarded controller.
- **`ActivateAudioInterfaceAsync` requires an *agile* completion handler.** If the
  handler doesn't answer `QueryInterface` for `IAgileObject`, the call is rejected
  **synchronously** with `E_ILLEGAL_METHOD_CALL` (`0x8000000E`) — regardless of
  COM apartment, MFStartup, device path, or activation params. (WRL/wil-based
  samples hide this because they make the handler agile for you.) Process loopback
  also needs the Windows 10 20H1 headers — build with `NTDDI_VERSION ≥ 0x0A00000B`.
- **WGC captures occluded windows but not minimized ones.** The game may sit
  behind the host window, but must not be minimized.
- **Process-loopback capture doesn't mute the source** — capturing a process's
  render does not stop it reaching the speakers. That's why the echo fix instead
  injects a WASAPI render-hook that silences the game's own buffer
  (`AUDCLNT_BUFFERFLAGS_SILENT`) after copying it for the mirror; loopback stays
  as the fallback.
- **COM has no exports, so the render-hook walks vtables — and the indices are
  easy to miscount.** All instances of a COM coclass share one vtable, so hooking
  one object's method (resolved by frozen-ABI vtable index) catches every
  instance. But the indices must be exact: `IAudioClient::GetService` is **14**,
  not 13 — `SetEventHandle` (13) sits between `Reset` and `GetService`. Count the
  full interface (including every inherited `IUnknown`/base method) when adding a
  new COM hook.