CoopAllTheThings/README.md

# CoopAllTheThings

Steam **Remote Play Together (RPT)** for any XInput game — without breaking DRM,
achievements, or playtime.

Existing "donor game" tools (e.g. RemotePlayWhatever) copy a target game's files
into a donor game's folder and rename the executable so Steam streams the target
under the donor's appid. That breaks DRM-protected games, breaks achievements,
and credits playtime to the donor.

CoopAllTheThings takes a different approach: the **real game runs normally under
its own appid** (so DRM, achievements, and playtime all work), while a lightweight
**mirror app runs under the donor appid**. The mirror presents a borderless
window that is a live copy of the game's video + audio, and forwards the guests'
input back into the real game. Steam's RPT captures the mirror window — so any
XInput game becomes Remote-Play-Together-able.

## Architecture

| Concern | Mechanism | Component | Status |
| --- | --- | --- | --- |
| Receive guest input | XInput (RPT delivers guest pads to the focused window); optional, opt-in Steam Input when built with the Steamworks SDK | `coop_host.exe` | done |
| Forward input to game | DLL injection + XInput hook (SafetyHook) — game sees *only* our pad | `coop_hook.dll` | done |
| Keep game running unfocused | Hook spoofs focus so the game polls while the host holds OS focus | `coop_hook.dll` | done |
| Mirror video | Windows Graphics Capture of the game window, letterboxed into the host window | `coop_host.exe` | done |
| Mirror video (alt) | Injected Present-hook copies the DXGI backbuffer into a shared keyed-mutex texture the host samples (lower latency, no capture border) | `coop_hook.dll` + `coop_host.exe` | done |
| Mirror audio | Injected render-hook copies the game's WASAPI frames into a shared ring and silences the game locally (no echo); WASAPI process loopback is the automatic fallback | `coop_hook.dll` + `coop_host.exe` | done |
| Host ↔ hook IPC | Named shared memory (seqlock for input, status back-channel) | `common/` | done |

## Limitations

- **Anti-cheat:** the input path injects `coop_hook.dll` into the target game.
  Games protected by kernel-level anti-cheat (Easy Anti-Cheat, BattlEye,
  Vanguard, etc.) will detect the injected module and may **kick the player or
  issue a ban**. Such games are explicitly **out of scope and unsupported** — do
  not use CoopAllTheThings with them. The tool targets single-player and
  co-op/local-multiplayer titles without active anti-cheat.
- **XInput only:** the game must read controllers via XInput (the common case).
  DirectInput-only / RawInput-only games are not handled.
- **32-bit games supported via a helper:** the host is x64, but the build also
  produces an x86 hook DLL (`coop_hook_x86.dll`) and a 32-bit injector helper
  (`coop_inject_x86.exe`). When the target is a 32-bit (WOW64) process the host
  detects it (`IsWow64Process2`) and shells out to the helper to load the x86 DLL
  (a 64-bit process can't cleanly inject a 32-bit one). The shared-memory IPC is
  fixed-width / bitness-stable, so the x64 host and x86 hook interoperate.
- **Local audio echo (fixed via the hook; falls back otherwise):** when the
  render-hook is active it silences the game's local playback while mirroring it,
  so there is no echo. If the hook can't attach or the game uses an
  unhooked/exotic render path, the host automatically falls back to
  process-loopback capture, which does *not* mute the game — so on the fallback
  path the local machine still hears the audio twice (guests hear it once). The
  Audio panel shows which path is active.
- **Debug-oriented UI:** the ImGui overlay is always visible and laid out for
  diagnosing the pipeline, not for end use. It can't yet be toggled off.

## Status

All phases below are implemented and verified.

- **Phase 0 — donor spike. ✅** Confirmed Steam RPT streams an arbitrary
  borderless window under a donor appid and routes guest gamepads into it as
  XInput (correct slot assignment). This is the premise the whole tool rests on.
- **Phase 1a — input forwarding + focus spoofing. ✅** The host injects
  `coop_hook.dll`; the DLL hooks XInput (SafetyHook) so the game reads the
  forwarded pad state and *only* that state, and spoofs focus so the game keeps
  running while the host holds the real OS focus. A hook→host status
  back-channel reports attach state and poll rate.
- **Phase 1b — video mirror (WGC). ✅** The host captures the injected game's
  window with Windows Graphics Capture and draws it letterboxed as its
  background, so RPT streams a live mirror.
- **Phase 2 — audio mirror. ✅** The host captures the game's audio by PID via
  WASAPI process loopback and re-renders it on the default endpoint, so RPT
  carries game audio to guests.
- **Audio render-hook (echo fix). ✅ (capture proven on a real game; full RPT
  pass pending)** The injected hook intercepts the game's WASAPI render path
  (`IAudioRenderClient`), copies the frames into a shared audio ring for the host
  to re-render, and releases the game's buffer silenced — so the operator no
  longer hears the audio twice. It hooks the render vtables *proactively* so a
  game that's already playing when injected is still captured. The host owns an
  enable flag and re-renders the game's format via `AUTOCONVERTPCM`; if the hook
  doesn't publish a format in time it reverts to process loopback. The Audio panel
  shows the active source and a render-stream-count debug table. Validated
  in-process by `audio_hook_test` and against a real already-playing game
  (Phantom Brave) with `coop_audio_probe`: the pre-existing render client is
  detected and real, non-silent audio reaches the ring with zero overruns.

### Remaining verification

The full-app capture path now works: the hook publishes the captured format to the
ring whenever the host attaches it, so toggling **Mirror game audio** switches to
**Source: Hooked (no echo)** instead of falling back to loopback (verified with
`coop_audio_probe` reproducing the app's inject-then-create-ring ordering against
Phantom Brave). These still need a human / full setup to confirm:

- **Hooked audio over RPT, end-to-end.** Run the host, inject, tick **Mirror game
  audio**, and confirm **Source: Hooked (no echo)**, the game goes locally silent,
  and a guest on Remote Play Together still hears it.
- **Format guess for non-mix-format games.** For a stream that already exists at
  injection time the hook can't see the game's `Initialize`, so it assumes the
  device **mix format**. Phantom Brave matched it exactly (48 kHz/2ch/float). A
  game that initialized shared mode with a different format would come out
  wrong-pitched/garbled and needs per-stream format detection — not yet handled.
- **Multi-stream games.** v1 captures only the first ("primary") render stream;
  confirm the Audio panel's render-stream table makes a multi-stream game obvious
  (secondary streams stay local until per-stream mixing is added).

## Roadmap

Done:

- **Usable overlay. ✅** F1 hides the whole overlay so the window is a clean
  mirror for Remote Play Together; a fading hint shows the way back. The
  pipelines keep running while hidden.
- **Generalized UI. ✅** A top menu bar carries a consolidated FPS / frame-time
  (with jitter) readout and a **View** menu that toggles each panel and a global
  **Debug details** switch. Panels default to general-purpose status (attached,
  forwarding, controller polling rate, mirror resolution, audio source + buffered
  ms) and reveal the verbose diagnostics (per-slot poll table, focus-API counts,
  input-path detection, per-render-stream table) only when Debug details is on.
- **Installed-hooks list. ✅** The DLL keeps a registry of every individual hook
  it installs (XInput, focus, audio), with a running call counter per hook,
  reported over IPC. The Injection panel shows it grouped by subsystem
  (input / focus / audio) so you can see exactly what's hooked and how busy each
  hook is. `coop_audio_probe` prints the same table headless.
- **Per-subsystem hook control. ✅** The three injectable subsystems — input
  forwarding (XInput), focus spoof, and the audio render-hook — are independently
  controllable. The Injection panel has a checkbox per subsystem that
  installs/removes its hooks at runtime over a host→hook control channel; the hook
  reconciles each tick. Dependent features are guarded: the synthetic-input
  control is disabled when input forwarding is off, and the Audio panel notes when
  the render-hook is off (mirroring then uses loopback). Defaults to all-on so
  behavior is unchanged unless you toggle something.
- **In-app Log window. ✅** The injected DLL streams its log lines to the host
  over a shared log ring (`coop_log_<pid>`, a lossy multi-producer ring), and the
  host shows them in a **Log** window with auto-scroll, a text filter, and clear.
  Replaces tailing `%TEMP%\coop_hook.log` (which stays as an opt-in file mirror).
  `coop_audio_probe` drains and prints the same stream headless.

- **Hooked video path (Present + OpenGL). ✅** The injected DLL captures the
  game's frames into a shared keyed-mutex texture (`coop_video_<pid>`) that the
  host opens by name and samples — a lower-latency, border-free alternative to WGC.
  It's an opt-in subsystem (the **Video Present-hook** checkbox in the Injection
  panel, or just pick **Source: Hooked (Present)** in the Video mirror panel, which
  installs it). Two producers cover the common graphics APIs:
  - **Direct3D (DXGI):** hooks `IDXGISwapChain::Present` / `Present1` and copies
    the backbuffer. Catches D3D10/11/12 games whose backbuffer is an
    `ID3D11Texture2D` (the common D3D11 case). Validated by `present_hook_test`
    and against Slaps and Beans (32-bit D3D11) and Life is Strange: Before the
    Storm. The host samples the copy as plain UNORM (see `srgb_to_unorm`) so games
    with an `*_SRGB` backbuffer (Life is Strange is `R8G8B8A8_UNORM_SRGB`) mirror
    with correct brightness instead of being darkened by an sRGB→linear decode.
  - **OpenGL:** hooks `SwapBuffers` / `wglSwapBuffers`, reads the backbuffer with
    `glReadPixels`, and uploads it into the shared texture. Catches OpenGL games
    that never touch DXGI (e.g. **Phantom Brave**, which is OpenGL — that's why
    the Present hook alone showed no image). Validated by `opengl_hook_test` and
    against Phantom Brave. `glReadPixels` forces a per-frame GPU→CPU readback, so
    it's heavier than the D3D copy, but fine for the typically-2D OpenGL titles.

  **Vulkan** games (present via `vkQueuePresentKHR`) aren't hooked yet — that needs
  a Vulkan layer / device-dispatch hook plus a `vkCmdCopyImage` to a readable
  image, a larger effort. **Use WGC** (the default Video source) for Vulkan, D3D9,
  or anything the hooked path doesn't capture — WGC works for any window.

- **x86 (32-bit) game support. ✅** A nested Win32 sub-build (CMake
  `ExternalProject`, driven from the normal x64 build) produces `coop_hook_x86.dll`
  and a 32-bit `coop_inject_x86.exe`, staged next to the x64 binaries. The host
  detects a WOW64 target with `IsWow64Process2` and spawns the helper to inject
  the x86 DLL. Validated end-to-end against Slaps and Beans (a 32-bit D3D11 game):
  all subsystems hooked, and the IPC channels (status, audio ring, video share,
  log) all flow across the x64↔x86 boundary.

- **Steam Input (opt-in). ✅** When the host is built with the Steamworks SDK
  (auto-detected under `third_party/steamworks_sdk/`), a **Use Steam Input**
  checkbox in the Controllers panel switches the input backend to the Steam Input
  API (action-based) at runtime; it loads a bundled action manifest
  (`steam_input_actions.vdf`) via `SetInputActionManifestFilePath`, so it needs no
  partner-backend config. **It's off by default and XInput is the primary path**:
  merely initializing Steam Input activates Steam's in-process XInput interception,
  which *hides* controllers from XInput unless they're bound to our action set for
  the running appid — so defaulting to it silently broke input forwarding. Enabling
  it is only useful once a controller is bound to Steam Input for the donor appid;
  otherwise leave it off and the proven XInput path carries the guest input.
  Verified to initialize and enumerate controllers against the live Steam client
  (`coop_steam_input_probe`), and that toggling it off restores XInput.

All planned phases are now implemented. Possible later work: per-stream audio
mixing for multi-stream games, per-stream format detection for the audio hook, and
hooking the remaining present paths for the video hook (Vulkan `vkQueuePresentKHR`,
D3D9, pure-D3D12) — WGC already covers those today.

## Building

Requirements: Windows 10/11, Visual Studio 2022 (MSVC + C++ workload), CMake ≥ 3.21.

```sh
git clone --recurse-submodules <repo-url>
# or, if already cloned:
git submodule update --init --recursive

cmake -S . -B build -G "Visual Studio 17 2022" -A x64
cmake --build build --config Debug
# output: bin/Debug/coop_host.exe (+ coop_hook.dll, test exes)
```

The x64 build also drives a nested Win32 sub-build (CMake `ExternalProject`,
configured into `build/x86/`) that produces `coop_hook_x86.dll` and
`coop_inject_x86.exe` for 32-bit games, staged next to the x64 binaries. Disable
it with `-DCOOP_BUILD_X86_HELPER=OFF` if you don't need 32-bit support.

Third-party dependencies (Dear ImGui, SafetyHook) are git submodules under
`third_party/`. No vcpkg / package manager is used.

**Steam Input is optional.** It's enabled automatically when the Steamworks SDK is
vendored at `third_party/steamworks_sdk/` (extract the `steamworks_sdk_*.zip`
there). The SDK isn't redistributable, so it's gitignored and never committed; if
it's absent the host builds XInput-only (no other features depend on it). When
present, the build links `steam_api64.lib`, stages `steam_api64.dll` and the
action manifest next to the host, and also builds `coop_steam_input_probe`.

### clangd / IDE setup

The Visual Studio CMake generator does **not** emit `compile_commands.json`, so
clangd has no include paths and reports false errors. Run
[`gen-compile-commands.bat`](gen-compile-commands.bat) once (and after adding
sources or include dirs); it configures a parallel Ninja build in `build-clangd/`
that produces the database, which [`.clangd`](.clangd) points clangd at. clangd's
clang-cl driver resolves the MSVC / Windows SDK system includes on its own.

## Tests

```sh
ctest --test-dir build -C Debug --output-on-failure
```

- **`hook_selftest`** — in-process check of the IPC + XInput hook core (no game,
  no controller needed).
- **`audio_ring_test`** — unit test of the shared audio ring (lock-free SPSC
  push/pop, wrap-around, format handshake, overrun/drop). No device needed.
- **`audio_hook_test`** — in-process self-test of the WASAPI render-hook: installs
  the hooks, renders a tone through WASAPI in the same process, and asserts the
  COM vtables were discovered, the frames reached the ring (non-silent), the
  primary stream was silenced, and exactly one render stream was counted. Skips
  cleanly if the machine has no audio endpoint.
- **`srgb_format_test`** — unit test of the `srgb_to_unorm` mapping the hooked
  video path uses so `*_SRGB`-backbuffer games aren't darkened. No device.
- **`opengl_hook_test`** — in-process self-test of the OpenGL capture path:
  installs the swap hooks, drives a real OpenGL context (clears the backbuffer to a
  known color, calls `SwapBuffers`), and asserts the detour fired, the frame was
  `glReadPixels`'d into the shared texture, and a second device reads the exact
  pixels back by name. Skips cleanly without an OpenGL / D3D11 device.
- **`present_hook_test`** — in-process self-test of the Present-hook video path:
  installs the hook, drives a real D3D11 swapchain in the same process (clears the
  backbuffer to a known color and calls `Present`), and asserts the detour fired,
  the backbuffer reached the shared keyed-mutex texture, and a second device can
  open it by name and read the exact pixels back. Skips cleanly if the machine has
  no D3D11 device.
- **`audio_loopback_test`** — spawns `coop_tone.exe` (a standalone WASAPI
  sine-wave source under [`tools/audio_tone`](tools/audio_tone)) and verifies the
  shipping process-loopback capture (the fallback path) receives its audio by
  PID. Skips cleanly if the machine has no audio endpoint.

### Debugging the render-hook against a real game

[`tools/audio_probe`](tools/audio_probe) (`coop_audio_probe.exe <pid> [seconds]`)
brings up the audio render-hook without Steam / RPT / the host UI: it creates the
IPC block + audio ring the hook expects, injects `coop_hook.dll` into the target
game, then drains the ring and prints per-stream format, captured-frame counts,
peak amplitude (proves the audio is real, not silence), and overruns. It enables
the hook's file trace (`%TEMP%\coop_hook.log`) for the run. Run it from
`bin/<config>/`. **Kill the game between runs** — the loaded DLL locks
`coop_hook.dll` against the next rebuild.

## Running the tool (manual, end-to-end)

This needs Steam, a donor game that supports Remote Play Together, and a second
person/account to receive the stream.

1. **Launch the host under a donor appid.** Find the donor's appid (the number in
   its store URL); the donor only needs RPT support and is never actually played:

   ```text
   "C:\Program Files (x86)\Steam\steam.exe" -applaunch <donorAppId> "D:\dev\CoopAllTheThings\bin\Debug\coop_host.exe"
   ```

   The borderless window appears and Steam marks the donor "running". If the donor
   ignores the trailing path, set the host as the donor's **Launch Options**
   (`"D:\...\coop_host.exe" %command%`) or use a launcher like RemotePlayDetached.

2. **Start the real game** windowed or borderless (not exclusive fullscreen — see
   Lessons learned). In the host's **Injection** panel, filter for the game's
   `.exe`, select it, and click **Inject & Connect**. Watch **Hook status** for
   **Attached**, a non-zero **XInput polled: N/s**, and **Focus spoof: active**.

3. **Mirror video:** in the **Video mirror** panel, tick **Mirror game window** —
   the host window now shows a live, letterboxed copy of the game. **Source**
   picks how the frames are grabbed: **WGC** (default, Windows Graphics Capture —
   works for any window) or **Hooked (Present)** (the injected Present-hook's
   shared texture — lower latency and no capture border, but only for DXGI /
   D3D11 games; selecting it installs the video subsystem in the game).

4. **Mirror audio:** in the **Audio mirror** panel, tick **Mirror game audio**.
   With the hook injected, **Source** shows **Hooked (no echo)** and the game's
   local playback goes silent while guests still hear it. If it shows **Loopback
   (echo)** the hook's render path wasn't caught and you'll hear the game twice
   locally (guests still hear it once). The **Render streams** table shows how
   many WASAPI streams the game emits (v1 mirrors the first/primary).

5. **Start Remote Play Together** from Steam and invite a guest. Verify the guest
   sees the mirrored video, hears the audio, and that their controller drives the
   real game.

Useful checks while developing without RPT: tick **Forward synthetic test input**
in the Injection panel to make the game move on its own (proving forwarding is the
source), and click away from the game to confirm focus spoofing keeps it running.

> Injection access error → run the host as administrator. A 32-bit (WOW64) target
> is injected automatically via `coop_inject_x86.exe` + `coop_hook_x86.dll`; if
> those aren't next to the host, rebuild (the x86 sub-build stages them there).

## Lessons learned

Non-obvious things that cost time and constrain the design:

- **RPT only streams the *focused* window.** The game therefore can't hold focus
  itself; the hook spoofs focus (`GetForegroundWindow` / `GetActiveWindow` /
  `GetFocus`, plus swallowing window-deactivation messages) so the game keeps
  polling and rendering while the host owns the real OS focus.
- **Run target games windowed or borderless, never exclusive fullscreen.**
  Exclusive fullscreen minimizes on focus loss (defeating the focus spoof) and
  can't be window-captured. While unfocused the game gets no OS keyboard/mouse —
  only the forwarded controller.
- **`ActivateAudioInterfaceAsync` requires an *agile* completion handler.** If the
  handler doesn't answer `QueryInterface` for `IAgileObject`, the call is rejected
  **synchronously** with `E_ILLEGAL_METHOD_CALL` (`0x8000000E`) — regardless of
  COM apartment, MFStartup, device path, or activation params. (WRL/wil-based
  samples hide this because they make the handler agile for you.) Process loopback
  also needs the Windows 10 20H1 headers — build with `NTDDI_VERSION ≥ 0x0A00000B`.
- **WGC captures occluded windows but not minimized ones.** The game may sit
  behind the host window, but must not be minimized.
- **Process-loopback capture doesn't mute the source** — capturing a process's
  render does not stop it reaching the speakers. That's why the echo fix instead
  injects a WASAPI render-hook that silences the game's own buffer
  (`AUDCLNT_BUFFERFLAGS_SILENT`) after copying it for the mirror; loopback stays
  as the fallback.
- **COM has no exports, so the render-hook walks vtables — and the indices are
  easy to miscount.** All instances of a COM coclass share one vtable, so hooking
  one object's method (resolved by frozen-ABI vtable index) catches every
  instance. But the indices must be exact: `IAudioClient::GetService` is **14**,
  not 13 — `SetEventHandle` (13) sits between `Reset` and `GetService`. Count the
  full interface (including every inherited `IUnknown`/base method) when adding a
  new COM hook.