CoopAllTheThings/README.md

# CoopAllTheThings

Steam **Remote Play Together (RPT)** for any XInput game — without breaking DRM,
achievements, or playtime.

Existing "donor game" tools (e.g. RemotePlayWhatever) copy a target game's files
into a donor game's folder and rename the executable so Steam streams the target
under the donor's appid. That breaks DRM-protected games, breaks achievements,
and credits playtime to the donor.

CoopAllTheThings takes a different approach: the **real game runs normally under
its own appid** (so DRM, achievements, and playtime all work), while a lightweight
**mirror app runs under the donor appid**. The mirror presents a borderless
window that is a live copy of the game's video + audio, and forwards the guests'
input back into the real game. Steam's RPT captures the mirror window — so any
XInput game becomes Remote-Play-Together-able.

The end-to-end path is working: launched under a donor appid, the host streams a
live video + audio mirror of a separately-running game over Remote Play Together
and forwards guest controllers back into it.

## Architecture

| Concern | Mechanism | Component |
| --- | --- | --- |
| Receive guest input | XInput (RPT delivers guest pads to the focused window); optional, opt-in Steam Input when built with the Steamworks SDK | `coop_host.exe` |
| Forward input to game | DLL injection + XInput hook (SafetyHook) — game sees *only* our pad | `coop_hook.dll` |
| Keep game running unfocused | Hook spoofs focus so the game polls while the host holds OS focus | `coop_hook.dll` |
| Mirror video (default) | Windows Graphics Capture of the game window, letterboxed into the host window | `coop_host.exe` |
| Mirror video (hooked) | Injected Present / OpenGL hook copies the backbuffer into a shared keyed-mutex texture the host samples (lower latency, no capture border) | `coop_hook.dll` + `coop_host.exe` |
| Mirror audio | Injected render-hook copies the game's WASAPI frames into a shared ring and silences the game locally (no echo); WASAPI process loopback is the automatic fallback | `coop_hook.dll` + `coop_host.exe` |
| Host ↔ hook IPC | Named shared memory (seqlock for input, status back-channel, video/audio/log shares) | `common/` |

The hooked video path has two producers: **Direct3D (DXGI)** hooks
`IDXGISwapChain::Present` / `Present1` and copies the backbuffer (D3D10/11 games
whose backbuffer is an `ID3D11Texture2D`); **OpenGL** hooks
`SwapBuffers` / `wglSwapBuffers` and reads the backbuffer with `glReadPixels` (for
games that never touch DXGI, e.g. Phantom Brave). The host samples the copy as
plain UNORM (`srgb_to_unorm`) so `*_SRGB`-backbuffer games mirror at correct
brightness. **WGC remains the default** and covers anything the hooked path
doesn't (Vulkan, D3D9, DX12 — see Roadmap).

## Limitations

- **Anti-cheat:** the input path injects `coop_hook.dll` into the target game.
  Games protected by kernel-level anti-cheat (Easy Anti-Cheat, BattlEye,
  Vanguard, etc.) will detect the injected module and may **kick the player or
  issue a ban**. Such games are explicitly **out of scope and unsupported** — do
  not use CoopAllTheThings with them. The tool targets single-player and
  co-op/local-multiplayer titles without active anti-cheat.
- **XInput only:** the game must read controllers via XInput (the common case).
  DirectInput-only / RawInput-only games are not handled.
- **32-bit games supported via a helper:** the host is x64, but the build also
  produces an x86 hook DLL (`coop_hook_x86.dll`) and a 32-bit injector helper
  (`coop_inject_x86.exe`). When the target is a 32-bit (WOW64) process the host
  detects it (`IsWow64Process2`) and shells out to the helper to load the x86 DLL
  (a 64-bit process can't cleanly inject a 32-bit one). The shared-memory IPC is
  fixed-width / bitness-stable, so the x64 host and x86 hook interoperate.
- **Local audio echo on the fallback path:** when the render-hook is active it
  silences the game's local playback while mirroring it, so there is no echo. If
  the hook can't attach or the game uses an unhooked render path, the host falls
  back to process-loopback capture, which does *not* mute the game — so the local
  machine hears the audio twice (guests hear it once). The Audio panel shows which
  path is active.
- **Debug-oriented UI:** the ImGui overlay is laid out for diagnosing the
  pipeline, not for end use. F1 hides it entirely so the window is a clean mirror
  for RPT.

## Roadmap

### Planned (next up)

The current focus is making specific games work end-to-end. Each item is a
milestone with its own tests and commit.

1. **Release the mouse cursor for cursor-clipping games.** Games that confine the
   cursor while focused (e.g. Trails through Daybreak via `ClipCursor` /
   per-frame `SetCursorPos` re-centering) trap the operator's mouse permanently,
   because the focus spoof makes the game believe it's always focused — so the
   operator can't reach the ImGui overlay. Add a cursor-release capability to the
   Focus subsystem: hook `ClipCursor` (force `ClipCursor(NULL)` and swallow the
   game's clip) and the re-centering `SetCursorPos`, gated by a new host→hook flag
   driven by a host toggle + hotkey. Defaults to released (the guest plays via the
   pad, so the game's own cursor clip is operator-only), with the option to
   re-enable clipping per game.

2. **Real capture metrics + latency stats.** The current FPS readout only measures
   how fast the host renders its own window, which hides capture stutter. Add a
   three-line frametime/FPS graph — **game present rate** (from `VideoShare`
   present deltas), **capture rate** (generation deltas / WGC arrivals), and
   **tool render rate** — plus a **capture→display latency** stat: stamp each
   published frame with a `QueryPerformanceCounter` value in `VideoShare`, and the
   host reports `host-present QPC − game-present QPC` (min/avg/max ms) for the
   matched frame. QPC is system-wide, so the two processes' timestamps compare
   directly.

3. **DX12 hooked capture.** Marvel's Spider-Man is D3D12, so the Present hook fires
   but `GetBuffer(0)` as `ID3D11Texture2D` fails (the backbuffer is an
   `ID3D12Resource`) and the hook idles; WGC works but stutters. Add a D3D12 path
   via a **D3D11On12 bridge**: capture the game's D3D12 command queue (hook
   `ID3D12CommandQueue::ExecuteCommandLists`), create an `ID3D11On12Device`,
   `CreateWrappedResource` around the backbuffer, and `CopyResource` into the
   *existing* D3D11 shared keyed-mutex texture — so the host side is unchanged.

4. **Multi-stream audio capture + mixing.** Games with several concurrent WASAPI
   render streams (e.g. Spider-Man) only get their first ("primary") stream
   mirrored today; the rest keep playing locally and never reach the guest. Capture
   every tracked render stream into its own shared ring, silence each, and add a
   host-side mixer that resamples each ring to the render format and sums them
   (with soft-clip).

### Future work

- **Vulkan video hook.** Vulkan games present via `vkQueuePresentKHR`; hooking
  them needs a Vulkan layer / device-dispatch hook plus a `vkCmdCopyImage` to a
  readable image. Use WGC in the meantime.
- **D3D9 hooked path.** Covered by WGC today; a dedicated `IDirect3DDevice9::Present`
  hook would be the lower-latency upgrade.
- **Per-stream audio format detection.** A render stream that already exists when
  we inject is never seen at `Initialize`, so the hook assumes the device **mix
  format**. A stream initialized in shared mode at a different format would come
  out wrong-pitched. Detecting the real per-stream format would remove that guess.
- **Rumble / haptics forwarding.** `XInputSetState` is currently swallowed; routing
  it back to the guest is a later phase.

## Building

Requirements: Windows 10/11, Visual Studio 2022 (MSVC + C++ workload), CMake ≥ 3.21.

```sh
git clone --recurse-submodules <repo-url>
# or, if already cloned:
git submodule update --init --recursive

cmake -S . -B build -G "Visual Studio 17 2022" -A x64
cmake --build build --config Debug
# output: bin/Debug/coop_host.exe (+ coop_hook.dll, test exes)
```

The x64 build also drives a nested Win32 sub-build (CMake `ExternalProject`,
configured into `build/x86/`) that produces `coop_hook_x86.dll` and
`coop_inject_x86.exe` for 32-bit games, staged next to the x64 binaries. Disable
it with `-DCOOP_BUILD_X86_HELPER=OFF` if you don't need 32-bit support.

Third-party dependencies (Dear ImGui, SafetyHook) are git submodules under
`third_party/`. No vcpkg / package manager is used.

**Steam Input is optional.** It's enabled automatically when the Steamworks SDK is
vendored at `third_party/steamworks_sdk/` (extract the `steamworks_sdk_*.zip`
there). The SDK isn't redistributable, so it's gitignored and never committed; if
it's absent the host builds XInput-only (no other features depend on it). When
present, the build links `steam_api64.lib`, stages `steam_api64.dll` and the
action manifest next to the host, and also builds `coop_steam_input_probe`.

Steam Input is **off by default and XInput is the primary path**: merely
initializing Steam Input activates Steam's in-process XInput interception, which
hides controllers from XInput unless they're bound to our action set for the
running appid. Enable it (Controllers panel → **Use Steam Input**) only once a
controller is bound to Steam Input for the donor appid.

### clangd / IDE setup

The Visual Studio CMake generator does **not** emit `compile_commands.json`, so
clangd has no include paths and reports false errors. Run
[`gen-compile-commands.bat`](gen-compile-commands.bat) once (and after adding
sources or include dirs); it configures a parallel Ninja build in `build-clangd/`
that produces the database, which [`.clangd`](.clangd) points clangd at. clangd's
clang-cl driver resolves the MSVC / Windows SDK system includes on its own.

## Tests

```sh
ctest --test-dir build -C Debug --output-on-failure
```

- **`hook_selftest`** — in-process check of the IPC + XInput hook core (no game,
  no controller needed).
- **`audio_ring_test`** — unit test of the shared audio ring (lock-free SPSC
  push/pop, wrap-around, format handshake, overrun/drop). No device needed.
- **`audio_hook_test`** — in-process self-test of the WASAPI render-hook: installs
  the hooks, renders a tone through WASAPI in the same process, and asserts the
  COM vtables were discovered, the frames reached the ring (non-silent), the
  primary stream was silenced, and the render stream was counted. Skips cleanly if
  the machine has no audio endpoint.
- **`srgb_format_test`** — unit test of the `srgb_to_unorm` mapping the hooked
  video path uses so `*_SRGB`-backbuffer games aren't darkened. No device.
- **`opengl_hook_test`** — in-process self-test of the OpenGL capture path:
  installs the swap hooks, drives a real OpenGL context (clears the backbuffer to a
  known color, calls `SwapBuffers`), and asserts the detour fired, the frame was
  `glReadPixels`'d into the shared texture, and a second device reads the exact
  pixels back by name. Skips cleanly without an OpenGL / D3D11 device.
- **`present_hook_test`** — in-process self-test of the Present-hook video path:
  installs the hook, drives a real D3D11 swapchain in the same process (clears the
  backbuffer to a known color and calls `Present`), and asserts the detour fired,
  the backbuffer reached the shared keyed-mutex texture, and a second device can
  open it by name and read the exact pixels back. Skips cleanly if the machine has
  no D3D11 device.
- **`audio_loopback_test`** — spawns `coop_tone.exe` (a standalone WASAPI
  sine-wave source under [`tools/audio_tone`](tools/audio_tone)) and verifies the
  shipping process-loopback capture (the fallback path) receives its audio by
  PID. Skips cleanly if the machine has no audio endpoint.

### Debugging the hooks against a real game

[`tools/audio_probe`](tools/audio_probe) (`coop_audio_probe.exe <pid> [seconds]`)
brings up the audio render-hook without Steam / RPT / the host UI: it creates the
IPC block + audio ring the hook expects, injects `coop_hook.dll` into the target
game, then drains the ring and prints per-stream format, captured-frame counts,
peak amplitude (proves the audio is real, not silence), and overruns. It enables
the hook's file trace (`%TEMP%\coop_hook.log`) for the run.

[`tools/input_probe`](tools/input_probe)
(`coop_input_probe.exe <pid> [seconds] [disable_mask]`) does the same for input: it
injects, reports one connected pad, and toggles a button each second so the game's
input layer sees a real state change. `disable_mask` (hex bits `0x1`=input
`0x2`=focus `0x4`=audio `0x8`=video) skips installing a subsystem, so you can
**bisect which injected subsystem affects a game** — this is how the 32-bit
Present-hook crash was isolated.

Both auto-detect a 32-bit (WOW64) target and inject via `coop_inject_x86.exe` +
`coop_hook_x86.dll`, exactly like the host. Run them from `bin/<config>/`.
**Kill the game between runs** — the loaded DLL locks `coop_hook.dll` against the
next rebuild.

## Running the tool (manual, end-to-end)

This needs Steam, a donor game that supports Remote Play Together, and a second
person/account to receive the stream.

1. **Launch the host under a donor appid.** Find the donor's appid (the number in
   its store URL); the donor only needs RPT support and is never actually played:

   ```text
   "C:\Program Files (x86)\Steam\steam.exe" -applaunch <donorAppId> "D:\dev\CoopAllTheThings\bin\Debug\coop_host.exe"
   ```

   The borderless window appears and Steam marks the donor "running". If the donor
   ignores the trailing path, set the host as the donor's **Launch Options**
   (`"D:\...\coop_host.exe" %command%`) or use a launcher like RemotePlayDetached.

2. **Start the real game** windowed or borderless (not exclusive fullscreen — see
   Lessons learned). In the host's **Injection** panel, filter for the game's
   `.exe`, select it, and click **Inject & Connect**. Watch **Hook status** for
   **Attached**, a non-zero **XInput polled: N/s**, and **Focus spoof: active**.

3. **Mirror video:** in the **Video mirror** panel, tick **Mirror game window** —
   the host window now shows a live, letterboxed copy of the game. **Source**
   picks how the frames are grabbed: **WGC** (default, Windows Graphics Capture —
   works for any window) or **Hooked (Present)** (the injected hook's shared
   texture — lower latency and no capture border, for DXGI / D3D11 and OpenGL
   games; selecting it installs the video subsystem in the game).

4. **Mirror audio:** in the **Audio mirror** panel, tick **Mirror game audio**.
   With the hook injected, **Source** shows **Hooked (no echo)** and the game's
   local playback goes silent while guests still hear it. If it shows **Loopback
   (echo)** the hook's render path wasn't caught and you'll hear the game twice
   locally (guests still hear it once). The **Render streams** table shows how
   many WASAPI streams the game emits.

5. **Start Remote Play Together** from Steam and invite a guest. Verify the guest
   sees the mirrored video, hears the audio, and that their controller drives the
   real game.

Useful checks while developing without RPT: tick **Forward synthetic test input**
in the Injection panel to make the game move on its own (proving forwarding is the
source), and click away from the game to confirm focus spoofing keeps it running.

> Injection access error → run the host as administrator. A 32-bit (WOW64) target
> is injected automatically via `coop_inject_x86.exe` + `coop_hook_x86.dll`; if
> those aren't next to the host, rebuild (the x86 sub-build stages them there).

## Lessons learned

Non-obvious things that cost time and constrain the design:

- **RPT only streams the *focused* window.** The game therefore can't hold focus
  itself; the hook spoofs focus (`GetForegroundWindow` / `GetActiveWindow` /
  `GetFocus`, plus swallowing window-deactivation messages) so the game keeps
  polling and rendering while the host owns the real OS focus.
- **Run target games windowed or borderless, never exclusive fullscreen.**
  Exclusive fullscreen minimizes on focus loss (defeating the focus spoof) and
  can't be window-captured. While unfocused the game gets no OS keyboard/mouse —
  only the forwarded controller.
- **`ActivateAudioInterfaceAsync` requires an *agile* completion handler.** If the
  handler doesn't answer `QueryInterface` for `IAgileObject`, the call is rejected
  **synchronously** with `E_ILLEGAL_METHOD_CALL` (`0x8000000E`) — regardless of
  COM apartment, MFStartup, device path, or activation params. (WRL/wil-based
  samples hide this because they make the handler agile for you.) Process loopback
  also needs the Windows 10 20H1 headers — build with `NTDDI_VERSION ≥ 0x0A00000B`.
- **WGC captures occluded windows but not minimized ones.** The game may sit
  behind the host window, but must not be minimized.
- **Process-loopback capture doesn't mute the source** — capturing a process's
  render does not stop it reaching the speakers. That's why the echo fix instead
  injects a WASAPI render-hook that silences the game's own buffer
  (`AUDCLNT_BUFFERFLAGS_SILENT`) after copying it for the mirror; loopback stays
  as the fallback.
- **COM has no exports, so the render-hook walks vtables — and the indices are
  easy to miscount.** All instances of a COM coclass share one vtable, so hooking
  one object's method (resolved by frozen-ABI vtable index) catches every
  instance. But the indices must be exact: `IAudioClient::GetService` is **14**,
  not 13 — `SetEventHandle` (13) sits between `Reset` and `GetService`. Count the
  full interface (including every inherited `IUnknown`/base method) when adding a
  new COM hook.
- **SafetyHook's `call()` is `__cdecl` on x86 — use `stdcall()` for `__stdcall`
  targets.** `InlineHook::call()` invokes the trampoline through a pointer with the
  compiler's default convention, which is `__cdecl` on 32-bit. Most things we hook
  are `__stdcall` (COM methods like `IDXGISwapChain::Present` and the WASAPI render
  interfaces, plus `WINAPI` `SwapBuffers`). On x64 every convention collapses to one,
  so `call()` is fine; on x86 it double-cleans the stack → ESP imbalance → an
  instant crash (Debug builds surface it as **Run-Time Check Failure #0**). This
  froze 32-bit games (Slaps and Beans) the moment the Present hook ran. Always call
  trampolines with the matching convention — `stdcall()` for these — which is a
  no-op on x64. The XInput/focus hooks dodged it only because they never call the
  trampoline (they return synthesized data).
- **Steam Input init suppresses XInput.** Initializing the Steam Input API turns on
  Steam's in-process XInput interception, which hides controllers from
  `XInputGetState` unless they're bound to the running appid's action set —
  defaulting to it silently broke input forwarding. XInput is the primary path;
  Steam Input is opt-in.