BlackMark 9977ad5d5a Add OpenGL capture path for the hooked video mirror
The Present hook never fired in Phantom Brave because it's an OpenGL game
(OPENGL32.dll loaded, IDXGISwapChain::Present calls=0), so the hooked video source
showed no image. Add an OpenGL producer under the video subsystem: inline-hook
gdi32!SwapBuffers + opengl32!wglSwapBuffers (with a re-entrancy guard, since
SwapBuffers calls wglSwapBuffers), glReadPixels the backbuffer, flip it, and upload
it into the same shared keyed-mutex texture the host already samples -- so the host
is unchanged. DXGI games still hit the Present hook; both producers are installed
and whichever the game uses fills the texture.

Validated by opengl_hook_test (real GL context, clears to a known color, reads the
exact pixels back through the shared texture) and against Phantom Brave (SwapBuffers
~75/s, present=0, shared texture 1920x1080, generation advancing). Vulkan
(vkQueuePresentKHR) still needs WGC -- documented. All 7 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 14:33:47 +02:00

CoopAllTheThings

Steam Remote Play Together (RPT) for any XInput game — without breaking DRM, achievements, or playtime.

Existing "donor game" tools (e.g. RemotePlayWhatever) copy a target game's files into a donor game's folder and rename the executable so Steam streams the target under the donor's appid. That breaks DRM-protected games, breaks achievements, and credits playtime to the donor.

CoopAllTheThings takes a different approach: the real game runs normally under its own appid (so DRM, achievements, and playtime all work), while a lightweight mirror app runs under the donor appid. The mirror presents a borderless window that is a live copy of the game's video + audio, and forwards the guests' input back into the real game. Steam's RPT captures the mirror window — so any XInput game becomes Remote-Play-Together-able.

Architecture

Concern Mechanism Component Status
Receive guest input XInput (RPT delivers guest pads to the focused window); optional, opt-in Steam Input when built with the Steamworks SDK coop_host.exe done
Forward input to game DLL injection + XInput hook (SafetyHook) — game sees only our pad coop_hook.dll done
Keep game running unfocused Hook spoofs focus so the game polls while the host holds OS focus coop_hook.dll done
Mirror video Windows Graphics Capture of the game window, letterboxed into the host window coop_host.exe done
Mirror video (alt) Injected Present-hook copies the DXGI backbuffer into a shared keyed-mutex texture the host samples (lower latency, no capture border) coop_hook.dll + coop_host.exe done
Mirror audio Injected render-hook copies the game's WASAPI frames into a shared ring and silences the game locally (no echo); WASAPI process loopback is the automatic fallback coop_hook.dll + coop_host.exe done
Host ↔ hook IPC Named shared memory (seqlock for input, status back-channel) common/ done

Limitations

  • Anti-cheat: the input path injects coop_hook.dll into the target game. Games protected by kernel-level anti-cheat (Easy Anti-Cheat, BattlEye, Vanguard, etc.) will detect the injected module and may kick the player or issue a ban. Such games are explicitly out of scope and unsupported — do not use CoopAllTheThings with them. The tool targets single-player and co-op/local-multiplayer titles without active anti-cheat.
  • XInput only: the game must read controllers via XInput (the common case). DirectInput-only / RawInput-only games are not handled.
  • 32-bit games supported via a helper: the host is x64, but the build also produces an x86 hook DLL (coop_hook_x86.dll) and a 32-bit injector helper (coop_inject_x86.exe). When the target is a 32-bit (WOW64) process the host detects it (IsWow64Process2) and shells out to the helper to load the x86 DLL (a 64-bit process can't cleanly inject a 32-bit one). The shared-memory IPC is fixed-width / bitness-stable, so the x64 host and x86 hook interoperate.
  • Local audio echo (fixed via the hook; falls back otherwise): when the render-hook is active it silences the game's local playback while mirroring it, so there is no echo. If the hook can't attach or the game uses an unhooked/exotic render path, the host automatically falls back to process-loopback capture, which does not mute the game — so on the fallback path the local machine still hears the audio twice (guests hear it once). The Audio panel shows which path is active.
  • Debug-oriented UI: the ImGui overlay is always visible and laid out for diagnosing the pipeline, not for end use. It can't yet be toggled off.

Status

All phases below are implemented and verified.

  • Phase 0 — donor spike. ✅ Confirmed Steam RPT streams an arbitrary borderless window under a donor appid and routes guest gamepads into it as XInput (correct slot assignment). This is the premise the whole tool rests on.
  • Phase 1a — input forwarding + focus spoofing. ✅ The host injects coop_hook.dll; the DLL hooks XInput (SafetyHook) so the game reads the forwarded pad state and only that state, and spoofs focus so the game keeps running while the host holds the real OS focus. A hook→host status back-channel reports attach state and poll rate.
  • Phase 1b — video mirror (WGC). ✅ The host captures the injected game's window with Windows Graphics Capture and draws it letterboxed as its background, so RPT streams a live mirror.
  • Phase 2 — audio mirror. ✅ The host captures the game's audio by PID via WASAPI process loopback and re-renders it on the default endpoint, so RPT carries game audio to guests.
  • Audio render-hook (echo fix). ✅ (capture proven on a real game; full RPT pass pending) The injected hook intercepts the game's WASAPI render path (IAudioRenderClient), copies the frames into a shared audio ring for the host to re-render, and releases the game's buffer silenced — so the operator no longer hears the audio twice. It hooks the render vtables proactively so a game that's already playing when injected is still captured. The host owns an enable flag and re-renders the game's format via AUTOCONVERTPCM; if the hook doesn't publish a format in time it reverts to process loopback. The Audio panel shows the active source and a render-stream-count debug table. Validated in-process by audio_hook_test and against a real already-playing game (Phantom Brave) with coop_audio_probe: the pre-existing render client is detected and real, non-silent audio reaches the ring with zero overruns.

Remaining verification

The full-app capture path now works: the hook publishes the captured format to the ring whenever the host attaches it, so toggling Mirror game audio switches to Source: Hooked (no echo) instead of falling back to loopback (verified with coop_audio_probe reproducing the app's inject-then-create-ring ordering against Phantom Brave). These still need a human / full setup to confirm:

  • Hooked audio over RPT, end-to-end. Run the host, inject, tick Mirror game audio, and confirm Source: Hooked (no echo), the game goes locally silent, and a guest on Remote Play Together still hears it.
  • Format guess for non-mix-format games. For a stream that already exists at injection time the hook can't see the game's Initialize, so it assumes the device mix format. Phantom Brave matched it exactly (48 kHz/2ch/float). A game that initialized shared mode with a different format would come out wrong-pitched/garbled and needs per-stream format detection — not yet handled.
  • Multi-stream games. v1 captures only the first ("primary") render stream; confirm the Audio panel's render-stream table makes a multi-stream game obvious (secondary streams stay local until per-stream mixing is added).

Roadmap

Done:

  • Usable overlay. ✅ F1 hides the whole overlay so the window is a clean mirror for Remote Play Together; a fading hint shows the way back. The pipelines keep running while hidden.

  • Generalized UI. ✅ A top menu bar carries a consolidated FPS / frame-time (with jitter) readout and a View menu that toggles each panel and a global Debug details switch. Panels default to general-purpose status (attached, forwarding, controller polling rate, mirror resolution, audio source + buffered ms) and reveal the verbose diagnostics (per-slot poll table, focus-API counts, input-path detection, per-render-stream table) only when Debug details is on.

  • Installed-hooks list. ✅ The DLL keeps a registry of every individual hook it installs (XInput, focus, audio), with a running call counter per hook, reported over IPC. The Injection panel shows it grouped by subsystem (input / focus / audio) so you can see exactly what's hooked and how busy each hook is. coop_audio_probe prints the same table headless.

  • Per-subsystem hook control. ✅ The three injectable subsystems — input forwarding (XInput), focus spoof, and the audio render-hook — are independently controllable. The Injection panel has a checkbox per subsystem that installs/removes its hooks at runtime over a host→hook control channel; the hook reconciles each tick. Dependent features are guarded: the synthetic-input control is disabled when input forwarding is off, and the Audio panel notes when the render-hook is off (mirroring then uses loopback). Defaults to all-on so behavior is unchanged unless you toggle something.

  • In-app Log window. ✅ The injected DLL streams its log lines to the host over a shared log ring (coop_log_<pid>, a lossy multi-producer ring), and the host shows them in a Log window with auto-scroll, a text filter, and clear. Replaces tailing %TEMP%\coop_hook.log (which stays as an opt-in file mirror). coop_audio_probe drains and prints the same stream headless.

  • Hooked video path (Present + OpenGL). ✅ The injected DLL captures the game's frames into a shared keyed-mutex texture (coop_video_<pid>) that the host opens by name and samples — a lower-latency, border-free alternative to WGC. It's an opt-in subsystem (the Video Present-hook checkbox in the Injection panel, or just pick Source: Hooked (Present) in the Video mirror panel, which installs it). Two producers cover the common graphics APIs:

    • Direct3D (DXGI): hooks IDXGISwapChain::Present / Present1 and copies the backbuffer. Catches D3D10/11/12 games whose backbuffer is an ID3D11Texture2D (the common D3D11 case). Validated by present_hook_test and against Slaps and Beans (32-bit D3D11) and Life is Strange: Before the Storm. The host samples the copy as plain UNORM (see srgb_to_unorm) so games with an *_SRGB backbuffer (Life is Strange is R8G8B8A8_UNORM_SRGB) mirror with correct brightness instead of being darkened by an sRGB→linear decode.
    • OpenGL: hooks SwapBuffers / wglSwapBuffers, reads the backbuffer with glReadPixels, and uploads it into the shared texture. Catches OpenGL games that never touch DXGI (e.g. Phantom Brave, which is OpenGL — that's why the Present hook alone showed no image). Validated by opengl_hook_test and against Phantom Brave. glReadPixels forces a per-frame GPU→CPU readback, so it's heavier than the D3D copy, but fine for the typically-2D OpenGL titles.

    Vulkan games (present via vkQueuePresentKHR) aren't hooked yet — that needs a Vulkan layer / device-dispatch hook plus a vkCmdCopyImage to a readable image, a larger effort. Use WGC (the default Video source) for Vulkan, D3D9, or anything the hooked path doesn't capture — WGC works for any window.

  • x86 (32-bit) game support. ✅ A nested Win32 sub-build (CMake ExternalProject, driven from the normal x64 build) produces coop_hook_x86.dll and a 32-bit coop_inject_x86.exe, staged next to the x64 binaries. The host detects a WOW64 target with IsWow64Process2 and spawns the helper to inject the x86 DLL. Validated end-to-end against Slaps and Beans (a 32-bit D3D11 game): all subsystems hooked, and the IPC channels (status, audio ring, video share, log) all flow across the x64↔x86 boundary.

  • Steam Input (opt-in). ✅ When the host is built with the Steamworks SDK (auto-detected under third_party/steamworks_sdk/), a Use Steam Input checkbox in the Controllers panel switches the input backend to the Steam Input API (action-based) at runtime; it loads a bundled action manifest (steam_input_actions.vdf) via SetInputActionManifestFilePath, so it needs no partner-backend config. It's off by default and XInput is the primary path: merely initializing Steam Input activates Steam's in-process XInput interception, which hides controllers from XInput unless they're bound to our action set for the running appid — so defaulting to it silently broke input forwarding. Enabling it is only useful once a controller is bound to Steam Input for the donor appid; otherwise leave it off and the proven XInput path carries the guest input. Verified to initialize and enumerate controllers against the live Steam client (coop_steam_input_probe), and that toggling it off restores XInput.

All planned phases are now implemented. Possible later work: per-stream audio mixing for multi-stream games, per-stream format detection for the audio hook, and hooking the remaining present paths for the video hook (Vulkan vkQueuePresentKHR, D3D9, pure-D3D12) — WGC already covers those today.

Building

Requirements: Windows 10/11, Visual Studio 2022 (MSVC + C++ workload), CMake ≥ 3.21.

git clone --recurse-submodules <repo-url>
# or, if already cloned:
git submodule update --init --recursive

cmake -S . -B build -G "Visual Studio 17 2022" -A x64
cmake --build build --config Debug
# output: bin/Debug/coop_host.exe (+ coop_hook.dll, test exes)

The x64 build also drives a nested Win32 sub-build (CMake ExternalProject, configured into build/x86/) that produces coop_hook_x86.dll and coop_inject_x86.exe for 32-bit games, staged next to the x64 binaries. Disable it with -DCOOP_BUILD_X86_HELPER=OFF if you don't need 32-bit support.

Third-party dependencies (Dear ImGui, SafetyHook) are git submodules under third_party/. No vcpkg / package manager is used.

Steam Input is optional. It's enabled automatically when the Steamworks SDK is vendored at third_party/steamworks_sdk/ (extract the steamworks_sdk_*.zip there). The SDK isn't redistributable, so it's gitignored and never committed; if it's absent the host builds XInput-only (no other features depend on it). When present, the build links steam_api64.lib, stages steam_api64.dll and the action manifest next to the host, and also builds coop_steam_input_probe.

clangd / IDE setup

The Visual Studio CMake generator does not emit compile_commands.json, so clangd has no include paths and reports false errors. Run gen-compile-commands.bat once (and after adding sources or include dirs); it configures a parallel Ninja build in build-clangd/ that produces the database, which .clangd points clangd at. clangd's clang-cl driver resolves the MSVC / Windows SDK system includes on its own.

Tests

ctest --test-dir build -C Debug --output-on-failure
  • hook_selftest — in-process check of the IPC + XInput hook core (no game, no controller needed).
  • audio_ring_test — unit test of the shared audio ring (lock-free SPSC push/pop, wrap-around, format handshake, overrun/drop). No device needed.
  • audio_hook_test — in-process self-test of the WASAPI render-hook: installs the hooks, renders a tone through WASAPI in the same process, and asserts the COM vtables were discovered, the frames reached the ring (non-silent), the primary stream was silenced, and exactly one render stream was counted. Skips cleanly if the machine has no audio endpoint.
  • srgb_format_test — unit test of the srgb_to_unorm mapping the hooked video path uses so *_SRGB-backbuffer games aren't darkened. No device.
  • opengl_hook_test — in-process self-test of the OpenGL capture path: installs the swap hooks, drives a real OpenGL context (clears the backbuffer to a known color, calls SwapBuffers), and asserts the detour fired, the frame was glReadPixels'd into the shared texture, and a second device reads the exact pixels back by name. Skips cleanly without an OpenGL / D3D11 device.
  • present_hook_test — in-process self-test of the Present-hook video path: installs the hook, drives a real D3D11 swapchain in the same process (clears the backbuffer to a known color and calls Present), and asserts the detour fired, the backbuffer reached the shared keyed-mutex texture, and a second device can open it by name and read the exact pixels back. Skips cleanly if the machine has no D3D11 device.
  • audio_loopback_test — spawns coop_tone.exe (a standalone WASAPI sine-wave source under tools/audio_tone) and verifies the shipping process-loopback capture (the fallback path) receives its audio by PID. Skips cleanly if the machine has no audio endpoint.

Debugging the render-hook against a real game

tools/audio_probe (coop_audio_probe.exe <pid> [seconds]) brings up the audio render-hook without Steam / RPT / the host UI: it creates the IPC block + audio ring the hook expects, injects coop_hook.dll into the target game, then drains the ring and prints per-stream format, captured-frame counts, peak amplitude (proves the audio is real, not silence), and overruns. It enables the hook's file trace (%TEMP%\coop_hook.log) for the run. Run it from bin/<config>/. Kill the game between runs — the loaded DLL locks coop_hook.dll against the next rebuild.

Running the tool (manual, end-to-end)

This needs Steam, a donor game that supports Remote Play Together, and a second person/account to receive the stream.

  1. Launch the host under a donor appid. Find the donor's appid (the number in its store URL); the donor only needs RPT support and is never actually played:

    "C:\Program Files (x86)\Steam\steam.exe" -applaunch <donorAppId> "D:\dev\CoopAllTheThings\bin\Debug\coop_host.exe"
    

    The borderless window appears and Steam marks the donor "running". If the donor ignores the trailing path, set the host as the donor's Launch Options ("D:\...\coop_host.exe" %command%) or use a launcher like RemotePlayDetached.

  2. Start the real game windowed or borderless (not exclusive fullscreen — see Lessons learned). In the host's Injection panel, filter for the game's .exe, select it, and click Inject & Connect. Watch Hook status for Attached, a non-zero XInput polled: N/s, and Focus spoof: active.

  3. Mirror video: in the Video mirror panel, tick Mirror game window — the host window now shows a live, letterboxed copy of the game. Source picks how the frames are grabbed: WGC (default, Windows Graphics Capture — works for any window) or Hooked (Present) (the injected Present-hook's shared texture — lower latency and no capture border, but only for DXGI / D3D11 games; selecting it installs the video subsystem in the game).

  4. Mirror audio: in the Audio mirror panel, tick Mirror game audio. With the hook injected, Source shows Hooked (no echo) and the game's local playback goes silent while guests still hear it. If it shows Loopback (echo) the hook's render path wasn't caught and you'll hear the game twice locally (guests still hear it once). The Render streams table shows how many WASAPI streams the game emits (v1 mirrors the first/primary).

  5. Start Remote Play Together from Steam and invite a guest. Verify the guest sees the mirrored video, hears the audio, and that their controller drives the real game.

Useful checks while developing without RPT: tick Forward synthetic test input in the Injection panel to make the game move on its own (proving forwarding is the source), and click away from the game to confirm focus spoofing keeps it running.

Injection access error → run the host as administrator. A 32-bit (WOW64) target is injected automatically via coop_inject_x86.exe + coop_hook_x86.dll; if those aren't next to the host, rebuild (the x86 sub-build stages them there).

Lessons learned

Non-obvious things that cost time and constrain the design:

  • RPT only streams the focused window. The game therefore can't hold focus itself; the hook spoofs focus (GetForegroundWindow / GetActiveWindow / GetFocus, plus swallowing window-deactivation messages) so the game keeps polling and rendering while the host owns the real OS focus.
  • Run target games windowed or borderless, never exclusive fullscreen. Exclusive fullscreen minimizes on focus loss (defeating the focus spoof) and can't be window-captured. While unfocused the game gets no OS keyboard/mouse — only the forwarded controller.
  • ActivateAudioInterfaceAsync requires an agile completion handler. If the handler doesn't answer QueryInterface for IAgileObject, the call is rejected synchronously with E_ILLEGAL_METHOD_CALL (0x8000000E) — regardless of COM apartment, MFStartup, device path, or activation params. (WRL/wil-based samples hide this because they make the handler agile for you.) Process loopback also needs the Windows 10 20H1 headers — build with NTDDI_VERSION ≥ 0x0A00000B.
  • WGC captures occluded windows but not minimized ones. The game may sit behind the host window, but must not be minimized.
  • Process-loopback capture doesn't mute the source — capturing a process's render does not stop it reaching the speakers. That's why the echo fix instead injects a WASAPI render-hook that silences the game's own buffer (AUDCLNT_BUFFERFLAGS_SILENT) after copying it for the mirror; loopback stays as the fallback.
  • COM has no exports, so the render-hook walks vtables — and the indices are easy to miscount. All instances of a COM coclass share one vtable, so hooking one object's method (resolved by frozen-ABI vtable index) catches every instance. But the indices must be exact: IAudioClient::GetService is 14, not 13 — SetEventHandle (13) sits between Reset and GetService. Count the full interface (including every inherited IUnknown/base method) when adding a new COM hook.
Description
No description provided
Readme 845 KiB
Languages
C++ 95.7%
CMake 4.2%