BlackMark 784c31a9b5 Capture every audio stream into its own ring and mix them on the host
Games with several concurrent WASAPI render streams (e.g. Spider-Man: Miles
Morales) only had their first ("primary") stream mirrored; the rest kept playing
locally and never reached the guest. Now the render-hook captures + silences EVERY
tracked stream into its own ring (coop_audio_<pid>[_<index>]), each published with
that stream's own detected format (Initialize when caught, else GetMixFormat -- the
per-stream format detection, now actually used per ring rather than only for the
primary). The host creates a ring per stream and mixes the same-format streams with
a soft clip (host/src/audio/audio_mix.hpp); streams whose format differs from the
primary are still silenced (no echo) but skipped from the mix (would need
resampling).

The single-stream case is byte-for-byte unchanged: when only one stream is active
the host passes it through without the mixer, so the common path has no overhead or
fidelity change.

Verified: new audio_mix_test covers the decode/sum/soft-clip/encode math (float32 +
int16); audio_hook_test (x64 + x86) still passes, guarding the primary
capture+silence path against regression; full build x64 + x86 clean; ctest x64
11/11, x86 3/3. Multi-stream mixing against a real multi-stream game needs a live
session to fully confirm.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 05:45:47 +02:00

CoopAllTheThings

Steam Remote Play Together (RPT) for any XInput game — without breaking DRM, achievements, or playtime.

Existing "donor game" tools (e.g. RemotePlayWhatever) copy a target game's files into a donor game's folder and rename the executable so Steam streams the target under the donor's appid. That breaks DRM-protected games, breaks achievements, and credits playtime to the donor.

CoopAllTheThings takes a different approach: the real game runs normally under its own appid (so DRM, achievements, and playtime all work), while a lightweight mirror app runs under the donor appid. The mirror presents a borderless window that is a live copy of the game's video + audio, and forwards the guests' input back into the real game. Steam's RPT captures the mirror window — so any XInput game becomes Remote-Play-Together-able.

The end-to-end path is working: launched under a donor appid, the host streams a live video + audio mirror of a separately-running game over Remote Play Together and forwards guest controllers back into it.

Architecture

Concern Mechanism Component
Receive guest input XInput (RPT delivers guest pads to the focused window); optional, opt-in Steam Input when built with the Steamworks SDK coop_host.exe
Forward input to game DLL injection + XInput hook (SafetyHook) — game sees only our pad coop_hook.dll
Forward mouse + keyboard Opt-in MKB subsystem: host streams its window's clicks/keys, the hook posts the matching window messages and synthesizes GetAsyncKeyState/GetKeyboardState/GetCursorPos for polling games coop_hook.dll + coop_host.exe
Keep game running unfocused Hook spoofs focus so the game polls while the host holds OS focus coop_hook.dll
Mirror video (default) Windows Graphics Capture of the game window, letterboxed into the host window coop_host.exe
Mirror video (hooked) Injected Present / OpenGL hook copies the backbuffer into a shared keyed-mutex texture the host samples (lower latency, no capture border) coop_hook.dll + coop_host.exe
Mirror audio Injected render-hook copies each of the game's WASAPI render streams into its own shared ring and silences the game locally (no echo); the host mixes the streams (soft-clipped); WASAPI process loopback is the automatic fallback coop_hook.dll + coop_host.exe
Host ↔ hook IPC Named shared memory (seqlock for input, status back-channel, video/audio/log shares) common/

The hooked video path has two producers: Direct3D (DXGI) hooks IDXGISwapChain::Present / Present1 and copies the backbuffer — directly for D3D10/11 games (the backbuffer is an ID3D11Texture2D), and via a D3D11On12 bridge for D3D12 games (wrap the ID3D12Resource backbuffer, CopyResource into the shared texture); OpenGL hooks SwapBuffers / wglSwapBuffers and reads the backbuffer with glReadPixels (for games that never touch DXGI, e.g. Phantom Brave). The host samples the copy as plain UNORM (srgb_to_unorm) so *_SRGB-backbuffer games mirror at correct brightness. WGC remains the default and covers anything the hooked path doesn't (Vulkan, D3D9 — see Roadmap).

Limitations

  • Anti-cheat: the input path injects coop_hook.dll into the target game. Games protected by kernel-level anti-cheat (Easy Anti-Cheat, BattlEye, Vanguard, etc.) will detect the injected module and may kick the player or issue a ban. Such games are explicitly out of scope and unsupported — do not use CoopAllTheThings with them. The tool targets single-player and co-op/local-multiplayer titles without active anti-cheat.
  • XInput only: the game must read controllers via XInput (the common case). DirectInput-only / RawInput-only games are not handled.
  • 32-bit games supported via a helper: the host is x64, but the build also produces an x86 hook DLL (coop_hook_x86.dll) and a 32-bit injector helper (coop_inject_x86.exe). When the target is a 32-bit (WOW64) process the host detects it (IsWow64Process2) and shells out to the helper to load the x86 DLL (a 64-bit process can't cleanly inject a 32-bit one). The shared-memory IPC is fixed-width / bitness-stable, so the x64 host and x86 hook interoperate.
  • Local audio echo on the fallback path: when the render-hook is active it silences the game's local playback while mirroring it, so there is no echo. If the hook can't attach or the game uses an unhooked render path, the host falls back to process-loopback capture, which does not mute the game — so the local machine hears the audio twice (guests hear it once). The Audio panel shows which path is active.
  • Debug-oriented UI: the ImGui overlay is laid out for diagnosing the pipeline, not for end use. F1 hides it entirely so the window is a clean mirror for RPT.

Roadmap

Planned (next up)

Nothing queued — the previous backlog (bin restructure, terminated/hung detection, re-attach, window-based target picker, overlay auto-layout, Audio "live" column, moving the synthetic-input toggle, mouse + keyboard forwarding, rumble forwarding, per-backend input debug view, cursor release, capture metrics + latency, DX12 hooked capture, multi-stream audio + per-stream formats) is all shipped. See Future work for what's left.

Future work

  • Vulkan video hook. Vulkan games present via vkQueuePresentKHR; hooking them needs a Vulkan layer / device-dispatch hook plus a vkCmdCopyImage to a readable image. Use WGC in the meantime.
  • D3D9 hooked path. Covered by WGC today; a dedicated IDirect3DDevice9::Present hook would be the lower-latency upgrade.

Building

Requirements: Windows 10/11, Visual Studio 2022 (MSVC + C++ workload), CMake ≥ 3.21.

git clone --recurse-submodules <repo-url>
# or, if already cloned:
git submodule update --init --recursive

cmake -S . -B build -G "Visual Studio 17 2022" -A x64
cmake --build build --config Debug
# output: bin/Debug/coop_host.exe (+ coop_hook.dll, test exes)

The x64 build also drives a nested Win32 sub-build (CMake ExternalProject, configured into build/x86/) that produces coop_hook_x86.dll and coop_inject_x86.exe for 32-bit games, staged next to the x64 binaries. Disable it with -DCOOP_BUILD_X86_HELPER=OFF if you don't need 32-bit support.

Third-party dependencies (Dear ImGui, SafetyHook) are git submodules under third_party/. No vcpkg / package manager is used.

Steam Input is optional. It's enabled automatically when the Steamworks SDK is vendored at third_party/steamworks_sdk/ (extract the steamworks_sdk_*.zip there). The SDK isn't redistributable, so it's gitignored and never committed; if it's absent the host builds XInput-only (no other features depend on it). When present, the build links steam_api64.lib, stages steam_api64.dll and the action manifest next to the host, and also builds coop_steam_input_probe.

Steam Input is off by default and XInput is the primary path: merely initializing Steam Input activates Steam's in-process XInput interception, which hides controllers from XInput unless they're bound to our action set for the running appid. Enable it (Controllers panel → Use Steam Input) only once a controller is bound to Steam Input for the donor appid.

clangd / IDE setup

The Visual Studio CMake generator does not emit compile_commands.json, so clangd has no include paths and reports false errors. Run gen-compile-commands.bat once (and after adding sources or include dirs); it configures a parallel Ninja build in build-clangd/ that produces the database, which .clangd points clangd at. clangd's clang-cl driver resolves the MSVC / Windows SDK system includes on its own.

Tests

ctest --test-dir build -C Debug --output-on-failure
  • hook_selftest — in-process check of the IPC + XInput hook core (no game, no controller needed).
  • audio_ring_test — unit test of the shared audio ring (lock-free SPSC push/pop, wrap-around, format handshake, overrun/drop). No device needed.
  • audio_mix_test — unit test of the multi-stream mixer math (decode / sum / soft-clip / encode for float32 + int16). No device needed.
  • audio_hook_test — in-process self-test of the WASAPI render-hook: installs the hooks, renders a tone through WASAPI in the same process, and asserts the COM vtables were discovered, the frames reached the ring (non-silent), the primary stream was silenced, and the render stream was counted. Skips cleanly if the machine has no audio endpoint.
  • srgb_format_test — unit test of the srgb_to_unorm mapping the hooked video path uses so *_SRGB-backbuffer games aren't darkened. No device.
  • opengl_hook_test — in-process self-test of the OpenGL capture path: installs the swap hooks, drives a real OpenGL context (clears the backbuffer to a known color, calls SwapBuffers), and asserts the detour fired, the frame was glReadPixels'd into the shared texture, and a second device reads the exact pixels back by name. Skips cleanly without an OpenGL / D3D11 device.
  • dx12_present_hook_test — in-process self-test of the Present hook's D3D12 path: drives a real D3D12 swapchain through the (shared) IDXGISwapChain::Present vtable and asserts the D3D11On12 bridge wraps the ID3D12Resource backbuffer and copies it into the shared texture, then reads the exact rendered color back by name. Skips cleanly without a D3D12 device.
  • present_hook_test — in-process self-test of the Present-hook video path: installs the hook, drives a real D3D11 swapchain in the same process (clears the backbuffer to a known color and calls Present), and asserts the detour fired, the backbuffer reached the shared keyed-mutex texture, and a second device can open it by name and read the exact pixels back. Skips cleanly if the machine has no D3D11 device.
  • audio_loopback_test — spawns coop_tone.exe (a standalone WASAPI sine-wave source under tools/audio_tone) and verifies the shipping process-loopback capture (the fallback path) receives its audio by PID. Skips cleanly if the machine has no audio endpoint.

Debugging the hooks against a real game

tools/audio_probe (coop_audio_probe.exe <pid> [seconds]) brings up the audio render-hook without Steam / RPT / the host UI: it creates the IPC block + audio ring the hook expects, injects coop_hook.dll into the target game, then drains the ring and prints per-stream format, captured-frame counts, peak amplitude (proves the audio is real, not silence), and overruns. It enables the hook's file trace (%TEMP%\coop_hook.log) for the run.

tools/input_probe (coop_input_probe.exe <pid> [seconds] [disable_mask]) does the same for input: it injects, reports one connected pad, and toggles a button each second so the game's input layer sees a real state change. disable_mask (hex bits 0x1=input 0x2=focus 0x4=audio 0x8=video) skips installing a subsystem, so you can bisect which injected subsystem affects a game — this is how the 32-bit Present-hook crash was isolated.

Both auto-detect a 32-bit (WOW64) target and inject via coop_inject_x86.exe + coop_hook_x86.dll, exactly like the host. The probes build into bin/<config>/tools/ (the deployable bin/<config>/ root holds only shipping artifacts; tests build into bin/<config>/tests/) and resolve coop_hook.dll from the root one level up, so run them from there. Kill the game between runs — the loaded DLL locks coop_hook.dll against the next rebuild.

Running the tool (manual, end-to-end)

This needs Steam, a donor game that supports Remote Play Together, and a second person/account to receive the stream.

  1. Launch the host under a donor appid. Find the donor's appid (the number in its store URL); the donor only needs RPT support and is never actually played:

    "C:\Program Files (x86)\Steam\steam.exe" -applaunch <donorAppId> "D:\dev\CoopAllTheThings\bin\Debug\coop_host.exe"
    

    The borderless window appears and Steam marks the donor "running". If the donor ignores the trailing path, set the host as the donor's Launch Options ("D:\...\coop_host.exe" %command%) or use a launcher like RemotePlayDetached.

  2. Start the real game windowed or borderless (not exclusive fullscreen — see Lessons learned). In the host's Injection panel, filter for the game's .exe, select it, and click Inject & Connect. Watch Hook status for Attached, a non-zero XInput polled: N/s, and Focus spoof: active.

  3. Mirror video: in the Video mirror panel, tick Mirror game window — the host window now shows a live, letterboxed copy of the game. Source picks how the frames are grabbed: WGC (default, Windows Graphics Capture — works for any window) or Hooked (Present) (the injected hook's shared texture — lower latency and no capture border, for DXGI / D3D11 and OpenGL games; selecting it installs the video subsystem in the game).

  4. Mirror audio: in the Audio mirror panel, tick Mirror game audio. With the hook injected, Source shows Hooked (no echo) and the game's local playback goes silent while guests still hear it. If it shows Loopback (echo) the hook's render path wasn't caught and you'll hear the game twice locally (guests still hear it once). The Render streams table shows how many WASAPI streams the game emits.

  5. Start Remote Play Together from Steam and invite a guest. Verify the guest sees the mirrored video, hears the audio, and that their controller drives the real game.

Useful checks while developing without RPT: tick Forward synthetic test input in the Injection panel to make the game move on its own (proving forwarding is the source), and click away from the game to confirm focus spoofing keeps it running.

Injection access error → run the host as administrator. A 32-bit (WOW64) target is injected automatically via coop_inject_x86.exe + coop_hook_x86.dll; if those aren't next to the host, rebuild (the x86 sub-build stages them there).

Lessons learned

Non-obvious things that cost time and constrain the design:

  • RPT only streams the focused window. The game can't hold focus itself, so the hook spoofs it (GetForegroundWindow / GetActiveWindow / GetFocus + swallowing deactivation messages) to keep the game polling and rendering while the host owns real OS focus.
  • Run target games windowed or borderless, never exclusive fullscreen — exclusive fullscreen minimizes on focus loss (defeating the spoof) and can't be window-captured. While unfocused the game gets no OS keyboard/mouse, only the forwarded pad.
  • WGC captures occluded windows but not minimized ones.
  • Process-loopback capture doesn't mute the source. Capturing a process's render doesn't stop it reaching the speakers, so the no-echo path instead injects a WASAPI render-hook that copies each buffer then releases it with AUDCLNT_BUFFERFLAGS_SILENT; loopback stays as the (echoing) fallback.
  • ActivateAudioInterfaceAsync needs an agile completion handler. If the handler doesn't answer QueryInterface for IAgileObject, the call is rejected synchronously with E_ILLEGAL_METHOD_CALL (0x8000000E) — regardless of apartment, device path, or activation params. (WRL/wil samples make the handler agile for you.) Process loopback also needs the Win10 20H1 headers (NTDDI_VERSION ≥ 0x0A00000B).
  • COM methods have no exports, so hooks walk vtables by frozen-ABI index — count exactly. All instances of a coclass share one vtable, so hooking one object's slot catches every instance; but IAudioClient::GetService is 14, not 13 (SetEventHandle sits at 13 between Reset and GetService). Count every inherited IUnknown/base method when adding a hook.
  • SafetyHook on x86 has two traps that froze 32-bit Slaps and Beans. (1) InlineHook::call() invokes the trampoline as __cdecl, but most targets are __stdcall (COM methods like IDXGISwapChain::Present, the WASAPI interfaces, WINAPI SwapBuffers); on 32-bit that double-cleans the stack → ESP imbalance → crash (Debug: Run-Time Check Failure #0). Use stdcall() (a no-op on x64). (2) Don't inline-hook COM methods on x86 at all: MMDevApi/AudioSes prologues do push ebp; mov ebp,esp; and esp,-8 and read args EBP-relative, which SafetyHook's trampoline relocation breaks (the original then runs with garbage args and faults). Hook COM methods by swapping the vtable entry instead (VirtualProtect the slot, overwrite the pointer, call the saved original) — no code patching, pristine stack regardless of prologue. Inline hooking stays fine for Present/SwapBuffers (clean prologues). Guarded by the x86 hook tests.
  • Steam Input init suppresses XInput. Initializing Steam Input turns on Steam's in-process XInput interception, which hides controllers from XInputGetState unless they're bound to the running appid's action set — defaulting to it silently broke forwarding. XInput is primary; Steam Input is opt-in.
Description
No description provided
Readme 845 KiB
Languages
C++ 95.7%
CMake 4.2%