# CoopAllTheThings Steam **Remote Play Together (RPT)** for any XInput game — without breaking DRM, achievements, or playtime. Existing "donor game" tools (e.g. RemotePlayWhatever) copy a target game's files into a donor game's folder and rename the executable so Steam streams the target under the donor's appid. That breaks DRM-protected games, breaks achievements, and credits playtime to the donor. CoopAllTheThings takes a different approach: the **real game runs normally under its own appid** (so DRM, achievements, and playtime all work), while a lightweight **mirror app runs under the donor appid**. The mirror presents a borderless window that is a live copy of the game's video + audio, and forwards the guests' input back into the real game. Steam's RPT captures the mirror window — so any XInput game becomes Remote-Play-Together-able. The end-to-end path is working: launched under a donor appid, the host streams a live video + audio mirror of a separately-running game over Remote Play Together and forwards guest controllers back into it. ## Architecture | Concern | Mechanism | Component | | --- | --- | --- | | Receive guest input | XInput (RPT delivers guest pads to the focused window); optional, opt-in Steam Input when built with the Steamworks SDK | `coop_host.exe` | | Forward input to game | DLL injection + XInput hook (SafetyHook) — game sees *only* our pad | `coop_hook.dll` | | Keep game running unfocused | Hook spoofs focus so the game polls while the host holds OS focus | `coop_hook.dll` | | Mirror video (default) | Windows Graphics Capture of the game window, letterboxed into the host window | `coop_host.exe` | | Mirror video (hooked) | Injected Present / OpenGL hook copies the backbuffer into a shared keyed-mutex texture the host samples (lower latency, no capture border) | `coop_hook.dll` + `coop_host.exe` | | Mirror audio | Injected render-hook copies the game's WASAPI frames into a shared ring and silences the game locally (no echo); WASAPI process loopback is the automatic fallback | `coop_hook.dll` + `coop_host.exe` | | Host ↔ hook IPC | Named shared memory (seqlock for input, status back-channel, video/audio/log shares) | `common/` | The hooked video path has two producers: **Direct3D (DXGI)** hooks `IDXGISwapChain::Present` / `Present1` and copies the backbuffer (D3D10/11 games whose backbuffer is an `ID3D11Texture2D`); **OpenGL** hooks `SwapBuffers` / `wglSwapBuffers` and reads the backbuffer with `glReadPixels` (for games that never touch DXGI, e.g. Phantom Brave). The host samples the copy as plain UNORM (`srgb_to_unorm`) so `*_SRGB`-backbuffer games mirror at correct brightness. **WGC remains the default** and covers anything the hooked path doesn't (Vulkan, D3D9, DX12 — see Roadmap). ## Limitations - **Anti-cheat:** the input path injects `coop_hook.dll` into the target game. Games protected by kernel-level anti-cheat (Easy Anti-Cheat, BattlEye, Vanguard, etc.) will detect the injected module and may **kick the player or issue a ban**. Such games are explicitly **out of scope and unsupported** — do not use CoopAllTheThings with them. The tool targets single-player and co-op/local-multiplayer titles without active anti-cheat. - **XInput only:** the game must read controllers via XInput (the common case). DirectInput-only / RawInput-only games are not handled. - **32-bit games supported via a helper:** the host is x64, but the build also produces an x86 hook DLL (`coop_hook_x86.dll`) and a 32-bit injector helper (`coop_inject_x86.exe`). When the target is a 32-bit (WOW64) process the host detects it (`IsWow64Process2`) and shells out to the helper to load the x86 DLL (a 64-bit process can't cleanly inject a 32-bit one). The shared-memory IPC is fixed-width / bitness-stable, so the x64 host and x86 hook interoperate. - **Local audio echo on the fallback path:** when the render-hook is active it silences the game's local playback while mirroring it, so there is no echo. If the hook can't attach or the game uses an unhooked render path, the host falls back to process-loopback capture, which does *not* mute the game — so the local machine hears the audio twice (guests hear it once). The Audio panel shows which path is active. - **Debug-oriented UI:** the ImGui overlay is laid out for diagnosing the pipeline, not for end use. F1 hides it entirely so the window is a clean mirror for RPT. ## Roadmap ### Planned (next up) Worked top-to-bottom: each item is a milestone with its own tests and commit, and is removed from this list once done — so the top item is always next. The self-verifiable tooling / UI / input items come first; the game-pipeline items that need a real game (and Remote Play) to fully validate come last. 1. **Stage only deployable artifacts directly under `bin/`.** Today every target — host, both hook DLLs, the injector helper, tests, probes, `coop_tone` — lands in `bin//`, so deploying to a donor folder means hand-picking files. Keep the **deployable set** at the `bin//` root (`coop_host.exe`, `coop_hook.dll`, `coop_hook_x86.dll`, `coop_inject_x86.exe`, and — when Steam is built — `steam_api64.dll` + `steam_input_actions.vdf`) and push everything else into subfolders: tests → `bin//tests/`, debug/probe tools (`coop_audio_probe`, `coop_input_probe`, `coop_tone`, `coop_steam_input_probe`) → `bin//tools/`. *How:* give those targets a per-target `RUNTIME_OUTPUT_DIRECTORY[_]` (the global `CMAKE_RUNTIME_OUTPUT_DIRECTORY` stays the deployable root; a small helper or `set_target_properties` overrides the non-deployable ones). The x86 sub-build must still stage `coop_hook_x86.dll` + `coop_inject_x86.exe` into the deployable root while its x86 *test* exes go to `tests/`. Watch the cross-target paths: `audio_loopback_test` spawns `coop_tone.exe`, and the host's post-build copy of `steam_api64.dll` / the `.vdf` must follow the host. End result: copying `bin//` non-recursively yields a clean deployable bundle. 2. **Detect a terminated target and reflect it in the UI.** The Injection panel keeps showing "Attached" after the game exits. Add a **Terminated** state: the host already knows the target pid and tracks a DLL heartbeat (`InjectionPanel`); on top of that, hold the `OpenProcess` handle from injection (or re-open with `PROCESS_QUERY_LIMITED_INFORMATION`) and poll `GetExitCodeProcess` / `WaitForSingleObject(h, 0)` each tick. When the process is gone, switch to Terminated, gray out / disable the per-subsystem controls and mirror toggles, and show a clear banner; the Video and Audio panels should drop to idle (their hook channels are stale) rather than freezing on the last live frame/state. Note a live process isn't proof it's running — also flag a **stalled heartbeat** (no advance for ~2 s while the process still exists) as a distinct "hung / not responding" state, since games here can freeze without exiting. 3. **Re-attach to a relaunched target.** A killed-and-relaunched game gets a new pid, but the UI still holds the stale one. In the Terminated state (task 2), remember the target's image name (the panel already keeps the selected exe name) and offer a **Re-attach** button that injects only if a live process with that *same name* exists, rebinding the IPC server to the new pid via the existing `inject_dll` path. If several live processes share that name, don't guess — surface the matches in the picker (task 4) for a manual choice. 4. **Select targets by window, not just process.** A flat process list is fine as an advanced/debug view, but the default should be a **window list** — there are far fewer top-level windows than processes, and a window directly yields the HWND the WGC capturer and focus spoof already want. *How:* add a window enumerator (`EnumWindows`, keeping visible, titled, non-tool top-level windows — `IsWindowVisible`, `GetWindowTextLength > 0`, exclude `WS_EX_TOOLWINDOW` and our own HWND, resolve to the root owner) and map each via `GetWindowThreadProcessId` → pid → image name. Show **title + process name + pid** with a filter box like the process list; injecting by window injects into its pid and hands the HWND straight to capture. Keep the process list behind "Debug details" as the advanced path. 5. **Auto-size and lay out the overlay windows so none need manual resizing.** Panels currently `Begin` at default cascade positions, so they overlap and clip. Give each `ImGuiWindowFlags_AlwaysAutoResize` and an initial position computed from `ImGui::GetMainViewport()->WorkPos/WorkSize`, applied with `ImGuiCond_FirstUseEver` (still movable), plus a **View → Reset layout** menu item that re-applies it. Target layout: **Injection** left/top (room to grow downward for hook diagnostics); **Controllers** top-center; **Video mirror** center, below Controllers; **Audio mirror** below Video; **Log** right edge, full height (most room for the log stream). Auto-resize fits these because they're all control/debug panels — the live mirror image is drawn to the whole host window *behind* the overlay, not inside a panel. 6. **Fix the Audio panel "live" column.** It overlays a green dot and grey "idle" because liveness is recomputed each frame from the per-stream `frames_rendered` delta, which is zero on most frames (buffers release in bursts), so it flickers. Replace it with a **debounced activity indicator**: keep a per-stream "last advanced" timestamp (the panel already stores the previous frame counts) and show **live** if frames advanced within the last ~300–500 ms, else **idle** — optionally a small frames/s or activity bar so multi-stream games read clearly. 7. **Move the synthetic-input toggle to the Controllers panel, under Debug details.** The "Forward synthetic test input" checkbox is a controller-debugging aid, so move it out of the Injection panel into `ControllersPanel` and gate it behind `debug_details`. The publish path is unchanged (the Injection panel already substitutes the synthetic pattern in `publish`); the flag just moves with it (or is passed from Controllers into the publish call). 8. **Mouse & keyboard forwarding (messages + polling-state hooks).** Forward guest clicks and keystrokes into the unfocused game via a new **MKB hook subsystem** in `coop_hook.dll` — the toggle *is* the hook (not installed → no forwarding). *Delivery:* `PostMessage` window-message input (`WM_KEYDOWN`/`WM_KEYUP`/`WM_CHAR`, `WM_*BUTTONDOWN`/`UP`, `WM_MOUSEWHEEL`) to the game HWND, **plus** hook `GetAsyncKeyState` / `GetKeyboardState` / `GetCursorPos` in the DLL so polling games see the synthesized keyboard/cursor state (RawInput and DirectInput games are out of scope for this version). *Keyboard* is always forwarded; *mouse* only while video is mirrored (otherwise the operator can't see where they click), and only **clicks + wheel, not movement** (one cursor can't be in two places). *Coordinate mapping* (the part that must be exact): WGC + decorated windowed → translate by the window decoration / client-area offset; hooked capture → relative to the mirrored viewport only (decorations aren't mirrored); borderless → the same under both backends. *Critical gating:* forward only when the host's main window is focused **and** ImGui doesn't want the event (`ImGuiIO::WantCaptureMouse` / `WantCaptureKeyboard`), so interacting with the overlay's own windows never leaks input into the game. The host sends MKB events to the hook over a new (or extended) IPC region. 9. **Rumble / haptics forwarding (both backends).** Currently unsupported — the XInput hook swallows `XInputSetState`. Add a reverse path: the hook captures the game's `XInputSetState` (left/right motor) and publishes it over a hook→host channel (the back-channel already exists), and the host drives the guest's actuators per backend — **XInput:** call `XInputSetState` on the guest's slot (the viability unknown is whether Steam's RPT virtual pad accepts vibration and routes it to the guest); **Steam Input:** `SteamInput()->TriggerVibration` / `Legacy_TriggerHapticPulse`. Map each guest slot to the right actuator. 10. **Per-backend input debug visualization.** To separate "wrong input *into* the tool" from "wrong input *out to* the game", show three distinct views in the Controllers panel (under Debug details): (a) **received via XInput** (raw `XInputSource` state), (b) **received via Steam Input** (raw `SteamInputSource` action values) — so it's obvious which backend delivered what — and (c) **forwarded to the game** (the `PadInfo` we write to shared memory, alongside what the game actually read back via the hook's per-slot channel). The hook already reports per-slot poll counts; extend it to echo the last state the game read so (c) is a true round-trip. 11. **Release the mouse cursor for cursor-clipping games.** Games that confine the cursor while focused (e.g. Trails through Daybreak via `ClipCursor` / per-frame `SetCursorPos` re-centering) trap the operator's mouse permanently, because the focus spoof makes the game believe it's always focused — so the operator can't reach the ImGui overlay. Add a cursor-release capability to the Focus subsystem: hook `ClipCursor` (force `ClipCursor(NULL)` and swallow the game's clip) and the re-centering `SetCursorPos`, gated by a new host→hook flag driven by a host toggle + hotkey. Defaults to released (the guest plays via the pad, so the game's own cursor clip is operator-only), with the option to re-enable clipping per game. 12. **Real capture metrics + latency stats.** The current FPS readout only measures how fast the host renders its own window, which hides capture stutter. Add a three-line frametime/FPS graph — **game present rate** (from `VideoShare` present deltas), **capture rate** (generation deltas / WGC arrivals), and **tool render rate** — plus a **capture→display latency** stat: stamp each published frame with a `QueryPerformanceCounter` value in `VideoShare`, and the host reports `host-present QPC − game-present QPC` (min/avg/max ms) for the matched frame. QPC is system-wide, so the two processes' timestamps compare directly. 13. **DX12 hooked capture (Spider-Man: Miles Morales).** Miles Morales is D3D12, so the Present hook fires but `GetBuffer(0)` as `ID3D11Texture2D` fails (the backbuffer is an `ID3D12Resource`) and the hook idles; WGC works but stutters. Add a D3D12 path via a **D3D11On12 bridge**: capture the game's D3D12 command queue (hook `ID3D12CommandQueue::ExecuteCommandLists`), create an `ID3D11On12Device`, `CreateWrappedResource` around the backbuffer, and `CopyResource` into the *existing* D3D11 shared keyed-mutex texture — so the host side is unchanged. 14. **Multi-stream audio capture + mixing, with per-stream format detection.** Games with several concurrent WASAPI render streams (e.g. Miles Morales) only get their first ("primary") stream mirrored today; the rest keep playing locally and never reach the guest. Capture every tracked render stream into its own shared ring, silence each, and add a host-side mixer that resamples each ring to the render format and sums them (with soft-clip). **Fold in real per-stream format detection** here, since it touches the same hook + ring plumbing: a stream that already existed when we injected is never seen at `Initialize`, so the hook currently assumes the device **mix format** and a shared-mode stream opened at a different format comes out wrong-pitched. Resolve each stream's true format (the stream's own `Initialize` when caught, else the original `GetMixFormat`) so every mixed ring is pitched correctly. ### Future work - **Vulkan video hook.** Vulkan games present via `vkQueuePresentKHR`; hooking them needs a Vulkan layer / device-dispatch hook plus a `vkCmdCopyImage` to a readable image. Use WGC in the meantime. - **D3D9 hooked path.** Covered by WGC today; a dedicated `IDirect3DDevice9::Present` hook would be the lower-latency upgrade. ## Building Requirements: Windows 10/11, Visual Studio 2022 (MSVC + C++ workload), CMake ≥ 3.21. ```sh git clone --recurse-submodules # or, if already cloned: git submodule update --init --recursive cmake -S . -B build -G "Visual Studio 17 2022" -A x64 cmake --build build --config Debug # output: bin/Debug/coop_host.exe (+ coop_hook.dll, test exes) ``` The x64 build also drives a nested Win32 sub-build (CMake `ExternalProject`, configured into `build/x86/`) that produces `coop_hook_x86.dll` and `coop_inject_x86.exe` for 32-bit games, staged next to the x64 binaries. Disable it with `-DCOOP_BUILD_X86_HELPER=OFF` if you don't need 32-bit support. Third-party dependencies (Dear ImGui, SafetyHook) are git submodules under `third_party/`. No vcpkg / package manager is used. **Steam Input is optional.** It's enabled automatically when the Steamworks SDK is vendored at `third_party/steamworks_sdk/` (extract the `steamworks_sdk_*.zip` there). The SDK isn't redistributable, so it's gitignored and never committed; if it's absent the host builds XInput-only (no other features depend on it). When present, the build links `steam_api64.lib`, stages `steam_api64.dll` and the action manifest next to the host, and also builds `coop_steam_input_probe`. Steam Input is **off by default and XInput is the primary path**: merely initializing Steam Input activates Steam's in-process XInput interception, which hides controllers from XInput unless they're bound to our action set for the running appid. Enable it (Controllers panel → **Use Steam Input**) only once a controller is bound to Steam Input for the donor appid. ### clangd / IDE setup The Visual Studio CMake generator does **not** emit `compile_commands.json`, so clangd has no include paths and reports false errors. Run [`gen-compile-commands.bat`](gen-compile-commands.bat) once (and after adding sources or include dirs); it configures a parallel Ninja build in `build-clangd/` that produces the database, which [`.clangd`](.clangd) points clangd at. clangd's clang-cl driver resolves the MSVC / Windows SDK system includes on its own. ## Tests ```sh ctest --test-dir build -C Debug --output-on-failure ``` - **`hook_selftest`** — in-process check of the IPC + XInput hook core (no game, no controller needed). - **`audio_ring_test`** — unit test of the shared audio ring (lock-free SPSC push/pop, wrap-around, format handshake, overrun/drop). No device needed. - **`audio_hook_test`** — in-process self-test of the WASAPI render-hook: installs the hooks, renders a tone through WASAPI in the same process, and asserts the COM vtables were discovered, the frames reached the ring (non-silent), the primary stream was silenced, and the render stream was counted. Skips cleanly if the machine has no audio endpoint. - **`srgb_format_test`** — unit test of the `srgb_to_unorm` mapping the hooked video path uses so `*_SRGB`-backbuffer games aren't darkened. No device. - **`opengl_hook_test`** — in-process self-test of the OpenGL capture path: installs the swap hooks, drives a real OpenGL context (clears the backbuffer to a known color, calls `SwapBuffers`), and asserts the detour fired, the frame was `glReadPixels`'d into the shared texture, and a second device reads the exact pixels back by name. Skips cleanly without an OpenGL / D3D11 device. - **`present_hook_test`** — in-process self-test of the Present-hook video path: installs the hook, drives a real D3D11 swapchain in the same process (clears the backbuffer to a known color and calls `Present`), and asserts the detour fired, the backbuffer reached the shared keyed-mutex texture, and a second device can open it by name and read the exact pixels back. Skips cleanly if the machine has no D3D11 device. - **`audio_loopback_test`** — spawns `coop_tone.exe` (a standalone WASAPI sine-wave source under [`tools/audio_tone`](tools/audio_tone)) and verifies the shipping process-loopback capture (the fallback path) receives its audio by PID. Skips cleanly if the machine has no audio endpoint. ### Debugging the hooks against a real game [`tools/audio_probe`](tools/audio_probe) (`coop_audio_probe.exe [seconds]`) brings up the audio render-hook without Steam / RPT / the host UI: it creates the IPC block + audio ring the hook expects, injects `coop_hook.dll` into the target game, then drains the ring and prints per-stream format, captured-frame counts, peak amplitude (proves the audio is real, not silence), and overruns. It enables the hook's file trace (`%TEMP%\coop_hook.log`) for the run. [`tools/input_probe`](tools/input_probe) (`coop_input_probe.exe [seconds] [disable_mask]`) does the same for input: it injects, reports one connected pad, and toggles a button each second so the game's input layer sees a real state change. `disable_mask` (hex bits `0x1`=input `0x2`=focus `0x4`=audio `0x8`=video) skips installing a subsystem, so you can **bisect which injected subsystem affects a game** — this is how the 32-bit Present-hook crash was isolated. Both auto-detect a 32-bit (WOW64) target and inject via `coop_inject_x86.exe` + `coop_hook_x86.dll`, exactly like the host. Run them from `bin//`. **Kill the game between runs** — the loaded DLL locks `coop_hook.dll` against the next rebuild. ## Running the tool (manual, end-to-end) This needs Steam, a donor game that supports Remote Play Together, and a second person/account to receive the stream. 1. **Launch the host under a donor appid.** Find the donor's appid (the number in its store URL); the donor only needs RPT support and is never actually played: ```text "C:\Program Files (x86)\Steam\steam.exe" -applaunch "D:\dev\CoopAllTheThings\bin\Debug\coop_host.exe" ``` The borderless window appears and Steam marks the donor "running". If the donor ignores the trailing path, set the host as the donor's **Launch Options** (`"D:\...\coop_host.exe" %command%`) or use a launcher like RemotePlayDetached. 2. **Start the real game** windowed or borderless (not exclusive fullscreen — see Lessons learned). In the host's **Injection** panel, filter for the game's `.exe`, select it, and click **Inject & Connect**. Watch **Hook status** for **Attached**, a non-zero **XInput polled: N/s**, and **Focus spoof: active**. 3. **Mirror video:** in the **Video mirror** panel, tick **Mirror game window** — the host window now shows a live, letterboxed copy of the game. **Source** picks how the frames are grabbed: **WGC** (default, Windows Graphics Capture — works for any window) or **Hooked (Present)** (the injected hook's shared texture — lower latency and no capture border, for DXGI / D3D11 and OpenGL games; selecting it installs the video subsystem in the game). 4. **Mirror audio:** in the **Audio mirror** panel, tick **Mirror game audio**. With the hook injected, **Source** shows **Hooked (no echo)** and the game's local playback goes silent while guests still hear it. If it shows **Loopback (echo)** the hook's render path wasn't caught and you'll hear the game twice locally (guests still hear it once). The **Render streams** table shows how many WASAPI streams the game emits. 5. **Start Remote Play Together** from Steam and invite a guest. Verify the guest sees the mirrored video, hears the audio, and that their controller drives the real game. Useful checks while developing without RPT: tick **Forward synthetic test input** in the Injection panel to make the game move on its own (proving forwarding is the source), and click away from the game to confirm focus spoofing keeps it running. > Injection access error → run the host as administrator. A 32-bit (WOW64) target > is injected automatically via `coop_inject_x86.exe` + `coop_hook_x86.dll`; if > those aren't next to the host, rebuild (the x86 sub-build stages them there). ## Lessons learned Non-obvious things that cost time and constrain the design: - **RPT only streams the *focused* window.** The game can't hold focus itself, so the hook spoofs it (`GetForegroundWindow` / `GetActiveWindow` / `GetFocus` + swallowing deactivation messages) to keep the game polling and rendering while the host owns real OS focus. - **Run target games windowed or borderless, never exclusive fullscreen** — exclusive fullscreen minimizes on focus loss (defeating the spoof) and can't be window-captured. While unfocused the game gets no OS keyboard/mouse, only the forwarded pad. - **WGC captures occluded windows but not minimized ones.** - **Process-loopback capture doesn't mute the source.** Capturing a process's render doesn't stop it reaching the speakers, so the no-echo path instead injects a WASAPI render-hook that copies each buffer then releases it with `AUDCLNT_BUFFERFLAGS_SILENT`; loopback stays as the (echoing) fallback. - **`ActivateAudioInterfaceAsync` needs an *agile* completion handler.** If the handler doesn't answer `QueryInterface` for `IAgileObject`, the call is rejected **synchronously** with `E_ILLEGAL_METHOD_CALL` (`0x8000000E`) — regardless of apartment, device path, or activation params. (WRL/wil samples make the handler agile for you.) Process loopback also needs the Win10 20H1 headers (`NTDDI_VERSION ≥ 0x0A00000B`). - **COM methods have no exports, so hooks walk vtables by frozen-ABI index — count exactly.** All instances of a coclass share one vtable, so hooking one object's slot catches every instance; but `IAudioClient::GetService` is **14**, not 13 (`SetEventHandle` sits at 13 between `Reset` and `GetService`). Count every inherited `IUnknown`/base method when adding a hook. - **SafetyHook on x86 has two traps that froze 32-bit Slaps and Beans.** (1) `InlineHook::call()` invokes the trampoline as `__cdecl`, but most targets are `__stdcall` (COM methods like `IDXGISwapChain::Present`, the WASAPI interfaces, `WINAPI` `SwapBuffers`); on 32-bit that double-cleans the stack → ESP imbalance → crash (Debug: **Run-Time Check Failure #0**). Use **`stdcall()`** (a no-op on x64). (2) Don't *inline-hook* COM methods on x86 at all: MMDevApi/AudioSes prologues do `push ebp; mov ebp,esp; and esp,-8` and read args **EBP-relative**, which SafetyHook's trampoline relocation breaks (the original then runs with garbage args and faults). Hook COM methods by **swapping the vtable entry** instead (`VirtualProtect` the slot, overwrite the pointer, call the saved original) — no code patching, pristine stack regardless of prologue. Inline hooking stays fine for `Present`/`SwapBuffers` (clean prologues). Guarded by the x86 hook tests. - **Steam Input init suppresses XInput.** Initializing Steam Input turns on Steam's in-process XInput interception, which hides controllers from `XInputGetState` unless they're bound to the running appid's action set — defaulting to it silently broke forwarding. XInput is primary; Steam Input is opt-in.