The mirror renders into the host window, so the host's render frame timing is
the mirror's performance. FrameStats now retains a ~2 s ring of frame-time
samples; the Video mirror panel plots them as a frametime graph (0-33 ms
scale) and an FPS graph (0-144 scale), each with an avg overlay, and prints
avg/min/max for both frametime and FPS underneath.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rework the spike-era debug readouts into general-purpose status, with the
verbose diagnostics gated behind the menu bar's "Debug details" switch:
- Controllers (was "Phase 0 spike"): always shows slot/source + live
buttons; the raw stick/trigger numbers are debug-only.
- Injection hook status: general view is attached + focus spoof + a single
"Game reading controller: N polls/s" summary; the per-slot poll table,
focus-API counts, and input-path detection are debug-only.
- Audio: general view adds a "Buffered: N ms" health/latency proxy
(AudioMirror now tracks buffered audio in both render paths) and keeps the
render-stream count; the per-stream table is debug-only.
Default overlay is now clean general status; flip Debug details for the full
diagnostics. Completes the "generalize the UI" roadmap task. All tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a top menu bar (host/src/ui/app_chrome) as the overlay shell:
- View menu toggles each panel (Controllers / Injection / Video / Audio)
and a global "Debug details" switch (consumed by panels in a follow-up).
- A right-aligned performance readout: stable 1 s FPS plus the frame-time
spread (avg, min-max) so stutter is visible; it turns amber on a >25 ms
spike. FrameStats keeps the rolling window.
Consolidates the FPS that was duplicated in the controllers and video
panels into this one readout.
First structural step of the "generalize the UI" roadmap task; content
rework (debug-vs-general split, renaming the spike-era readouts) follows.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The ImGui overlay was always visible, so the mirror window Steam Remote Play
Together captures always had debug panels drawn over the game. F1 now toggles
the whole overlay off; the input/video/audio pipelines keep running
underneath. When hidden, a small "F1: show overlay" hint fades out over ~4s so
the operator can find the way back, then the window is fully clean.
First item of the "make the tool usable" roadmap task.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
List the human/full-setup checks that producer-side capture can't prove on
its own: hooked audio over RPT end-to-end, the mix-format assumption for
streams that predate injection, and multi-stream games.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Update the plan's status (validated against a real already-playing game),
add the "reactive-only hooking fails on late injection" lesson and the
self-deadlock ordering sub-lesson, and document tools/audio_probe in the
README. Mirrors commit c7be4ee.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The render-hook only installed IAudioClient/IAudioRenderClient hooks
reactively, when it saw the game call IMMDevice::Activate -> GetService.
But we attach to a game that is already running and playing audio, so its
render client was created before injection: those calls never fire again,
no primary stream is ever registered, nothing is captured, and the host
always falls back to process loopback (the echo). Every game tested did so.
Fix: at anchor time, build our own probe IAudioClient + IAudioRenderClient
with raw calls and hook GetBuffer/ReleaseBuffer (plus Initialize/GetService)
on their vtables. Every instance of a COM coclass shares one vtable, so this
patches the shared vtables and intercepts the game's pre-existing render
client too. The first render client seen actively releasing buffers is
adopted as primary on the audio thread (try-lock, one-time) using the device
mix format as its assumed format (we never saw its Initialize). Streams
created after injection still register via the reactive path with their real
format.
Also fixes a self-deadlock: installing the Activate hook before the probe's
own device->Activate call re-entered hk_Activate -> install_audioclient_hooks,
which blocked on the setup mutex the installer already held, freezing the
worker (and any game thread that later called Activate -> crash). The probe
objects are now created raw, before any hook is installed.
Validated against Phantom Brave (injected while already playing): the
pre-existing 48 kHz/2ch/float render client is detected and registered as
primary, real non-silent audio reaches the ring (peak tracks the game's
levels), and a draining consumer sees zero overruns.
Tooling for iterating on real games without Steam/RPT/the host UI:
- tools/audio_probe: creates the IPC block + audio ring, injects the hook,
drains the ring and prints stream/format/peak/overrun diagnostics by pid.
- hook/src/debug_log: opt-in file trace (%TEMP%\coop_hook.log), enabled by
the COOP_HOOK_LOG env var or the %TEMP%\coop_hook.log.on sentinel the probe
drops; off in normal use.
All four tests still pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- README: audio mirror is now the injected render-hook with process-loopback as
automatic fallback; update Architecture/Limitations/Status/Roadmap and the
manual run step (Source indicator + render-stream table). Add tests
(audio_ring_test, audio_hook_test) and a lessons-learned note on COM vtable
indices (IAudioClient::GetService is 14, not 13).
- plan doc: mark M1-M4 implemented/committed, M5 (real-game end-to-end) pending,
and record the GetService index correction.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Surfaces the render-hook diagnostics in the Audio panel.
- HookStatusView / IpcServer::hook_status(): carry audio_streams_seen +
audio_streams[] from the hook's back-channel.
- InjectionPanel: expose hook_status() so the audio panel can read the counts.
- AudioPanel::draw_ui(HookStatusView): show the active Source (green Hooked
vs amber Loopback); only warn about the local echo on the loopback path. Add
the required render-stream table: one row per stream with format, cumulative
frames, the primary tagged, and a live/idle dot derived from frame-count
deltas. Notes overflow when the game exceeds kMaxAudioStreams slots.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
End-to-end plumbing of the render-hook audio path.
Hook (coop_hook.dll):
- CMake: build audio_hook.cpp, link ole32/mmdevapi, NTDDI_WIN10_CO.
- dllmain: CoInitializeEx(MTA) on the worker thread; install the audio hooks
even before the ring exists (so streams are counted); open the host's
coop_audio_<pid> ring when it appears and attach it (enabling capture+silence);
remove_audio_hooks on clean detach.
Host (coop_host.exe):
- AudioMirror now creates the shared audio ring (owns capture_enabled) and tries
the Hooked path first: waits ~1s for the hook to publish a format, then
re-renders the game's frames from the ring with AUTOCONVERTPCM (no echo, since
the hook silences the game locally).
- Automatic fallback: if the ring can't be created, no format arrives in time,
or the render client won't initialize, it disables capture (so the game stays
audible) and reverts to the existing process-loopback path (echo, no regress).
- Exposes Source (Hooked/Loopback/None) for the upcoming panel indicator.
Loopback render loop kept intact as run_loopback. Manual end-to-end (M5) and the
panel UI (M4) are next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the WASAPI render-hook (hook/src/audio_hook.{hpp,cpp}) and an
in-process self-test that proves COM vtable discovery and GetBuffer/ReleaseBuffer
interception with no game and no second Steam account.
- audio_hook.cpp: anchors on IMMDevice::Activate (idx 3) off our own default
endpoint (shared vtable), then hooks IAudioClient::Initialize (3) /
GetService (14) and IAudioRenderClient::GetBuffer (3) / ReleaseBuffer (4) off
live game pointers. Copies primary-stream frames into the audio ring and
releases with AUDCLNT_BUFFERFLAGS_SILENT (+ memset belt-and-suspenders), only
while the host-owned capture_enabled flag is set. Stream counting runs always;
on a ring overrun it keeps playing locally rather than going silent.
- ipc_client.hpp: publish_audio_stream / note_audio_frames /
set_audio_streams_seen write the render-stream debug fields into HookStatus.
- tests/audio_hook_test.cpp: installs the hooks, renders a tone through WASAPI
in-process, and asserts exactly one stream, frames pushed to the ring, the
ring carries the non-silent tone, and the primary was silenced. PASS:
streams_seen=1, frames_captured=32640.
- plan doc: correct GetService vtable index 13 -> 14 (SetEventHandle is 13).
coop_hook DLL wiring + host consumer/fallback come next (M3).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First milestone of the injection render-hook audio path (see
docs/audio-render-hook-plan.md) that fixes the local audio echo without a
virtual device.
- common/include/coop/audio_ring.hpp: new lock-free SPSC shared-memory ring
for PCM, separate from the input/status SharedBlock. Free-running 64-bit
positions (release/acquire), format handshake, host-owned capture_enabled
gate, drop-whole-packet overrun policy.
- common/include/coop/protocol.hpp: add AudioStreamInfo + audio_streams_seen /
audio_streams[] to the always-present HookStatus for the render-stream-count
debug view; bump kProtocolVersion 3->4 (new members appended).
- tests/audio_ring_test.cpp: in-process unit test (push/pop integrity,
wrap-around, format handshake, overrun/drop). No hook or audio device.
- docs/audio-render-hook-plan.md: the green-lit design this implements.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Reflect audio mirror (Phase 2) and video mirror (Phase 1b) as implemented;
drop stale "(current)"/"still ahead" framing and the broken docs/ link.
- Add a Roadmap section for the planned work: toggle/generalize the debug UI,
fix the local audio echo, Present-hook video path, Steam Input, x86 support.
- Add Lessons learned (RPT focus requirement, no exclusive fullscreen, the
ActivateAudioInterfaceAsync agile-handler gotcha, WGC occlusion, audio echo).
- Document clangd/compile_commands setup and the CTest suite.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mirror the real game's audio so Steam Remote Play Together (which streams the
host's own audio session) carries it to the guest. The host captures the game
by PID via WASAPI process loopback and re-renders it on the default endpoint;
the game still plays locally too (accepted "double audio" for now).
- ProcessLoopbackCapture: process-loopback capture client, frame-sink + stats.
The completion handler must be agile (IAgileObject) or
ActivateAudioInterfaceAsync rejects every call with E_ILLEGAL_METHOD_CALL.
- AudioMirror: wraps capture with an event-driven render client and a primed
ring buffer; AudioPanel drives it from the injected game's window/PID.
- coop_tone: standalone WASAPI sine-wave process used as a known audio source.
- audio_loopback_test (CTest): captures coop_tone by PID and asserts non-silent
audio arrives, so the path is verifiable without a second Steam account.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Visual Studio CMake generator does not emit compile_commands.json, so
clangd had no include paths or flags and reported hundreds of false errors.
Add a parallel Ninja config (build-clangd/) that produces the database:
- CMAKE_EXPORT_COMPILE_COMMANDS ON (honored by Ninja, ignored by VS)
- gen-compile-commands.bat configures build-clangd/ under vcvars64
- .clangd points clangd's CompilationDatabase at build-clangd/
- ignore /build-clangd
clangd's clang-cl driver auto-resolves the MSVC/Windows SDK system includes,
so the cl.exe-based database is sufficient.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mirror the injected game's window into the host's borderless window so Steam
RPT streams a live copy of the game. No injection needed for video.
- capture/window_capture: WGC capture of a target HWND. Wraps our D3D11 device
as an IDirect3DDevice, creates a free-threaded frame pool + capture session,
hides the cursor and (best-effort) the capture border. Frames arrive on a WGC
thread and are handed to the render thread, which copies the newest into a
shader-resource texture and draws it -- keeping all D3D11 context use on one
thread. Handles window resize via frame-pool Recreate.
- capture/frame_renderer: fullscreen-triangle shader that blits the captured
texture letterboxed (aspect-preserved) into the window.
- capture_panel: "Mirror game window" toggle + lifecycle; target HWND comes
from the hook's reported game window.
- main: winrt apartment init; mirrored frame drawn as background, ImGui on top.
- CMake: link windowsapp + d3dcompiler.
Verified: builds clean; host starts (apartment init + shader compile succeed).
Visual capture quality/latency to be judged on a real game.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Surfaced by a game (Life is Strange: Before the Storm) that ignores
controller input when it lacks true OS focus even though it still polls
XInput. To find the focus-gated detection path, instrument the hook.
Protocol v3 status back-channel now reports:
- per-slot XInputGetState and XInputGetCapabilities counters (replacing the
single aggregate), so the overlay shows exactly which slots the game polls
and how fast;
- focus-API call counts (GetForegroundWindow/GetActiveWindow/GetFocus) to see
whether the game consults the APIs we spoof;
- input-path diagnostics: whether the process registered Raw Input for a
gamepad usage and whether it set RIDEV_INPUTSINK (background delivery), and
whether a DirectInput dll is loaded.
Host overlay gains a per-slot poll table and an "Input path" section. The DLL
refreshes input diagnostics each worker tick via GetRegisteredRawInputDevices.
hook_selftest updated for per-slot counters; passes.
This is diagnostic-only: once a real run shows which path LiS uses, the
targeted focus fix (e.g. forcing RIDEV_INPUTSINK or DI background coop) follows.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two problems surfaced in testing: (1) no way to tell whether the injected
hook was actually the input source, and (2) the final design needs the tool
window focused for Steam RPT capture, which would pause/silence games that
react to focus loss. Both are addressed here.
- Focus spoofing (hook/focus_spoof): find the game's main window, subclass it
to rewrite/swallow WM_ACTIVATE/ACTIVATEAPP/NCACTIVATE/KILLFOCUS, and inline-
hook GetForegroundWindow/GetActiveWindow/GetFocus to always report the game
as active. The game keeps running and polling while unfocused.
- Status back-channel (protocol v2): the DLL reports attached/focus-spoof
flags, game pid/hwnd, a heartbeat, and a cumulative XInputGetState counter.
The host overlay turns the counter into a live poll rate, so "is the hook
working" is directly observable.
- Synthetic test-input toggle in the host: forwards a known automated pattern
(stick circle + periodic A) to prove forwarding independent of the physical
pad.
- hook_selftest extended to assert the status channel; passes.
Documented the windowed/borderless requirement and the new observable test
flow in the README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The host can now inject coop_hook.dll into a running game and forward
controller state to it over shared memory, so the game reads the host's
(eventually the guest's) input and nothing else.
- hook/: coop_hook.dll. DllMain spawns a worker that opens the shared-memory
channel (named by the game's pid) and installs SafetyHook inline hooks on
XInputGetState/GetStateEx/GetCapabilities/SetState. Detours synthesize state
from shared memory; unmanaged slots report disconnected, hiding physical pads.
- host/: process picker (Toolhelp32), CreateRemoteThread(LoadLibraryW) injector
with an IsWow64Process2 bitness guard, IPC server publishing pads each frame,
and an ImGui Injection panel wiring it together.
- tests/: hook_selftest exercises the IPC seqlock + hook detours in-process
(no game/controller needed); passes.
Build: SafetyHook wired in (COOP_BUILD_HOOK=ON), Zydis via FetchContent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Scaffold CoopAllTheThings: Remote Play Together for any XInput game via a
mirror app under a donor appid (real game keeps its own appid, so DRM,
achievements, and playtime stay intact).
- Build: CMake skeleton, ImGui + SafetyHook submodules (no vcpkg)
- common/: host<->hook IPC contract (seqlock pad state, shared-memory RAII)
- host/: borderless D3D11 window + ImGui overlay listing visible XInput pads,
behind an InputSource interface (Steam Input slots in later)
- README documents the Phase 0 donor-launch validation procedure, anti-cheat
limitation, and XInput/bitness constraints
Phase 0 validates the riskiest assumption (Steam RPT streams an arbitrary
window under a donor appid and routes guest input to it) before capture and
injection are built.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
