The hook-removal fix evolved from disable->drain->destroy to the persistent-hook
model (never free the trampoline during the session); update the roadmap summary
to match the Lessons-learned detail.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- poll_input() each frame (XInputGetState / GetAsyncKeyState / GetKeyboardState /
GetForegroundWindow), like a real game, so mock_game_test's hook/unhook storm
actually exercises removing the input/focus/MKB hooks while their detours are in
flight -- the coverage gap that let those removal races go untested.
- Window title shows the backend + a once-per-second-smoothed fps.
- A vectored-exception crash logger prints the faulting module+offset (named the
storm's intermittent crashes during this work; inert otherwise).
- detour_gate_test: fast, deterministic guard for DetourGate -- drain() must block
while a Guard is in flight and return promptly otherwise, plus a concurrency
stress that asserts no body runs against freed state. (A synthetic install-race
unit test was tried but flaked on SafetyHook's own enable/disable atomicity under
~30M calls/s, unrelated to our code, so the storm is the install/remove guard.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The uncapped, input-polling mock_game_test storm (thousands of presents/s, now
also driving the input/focus/MKB hooks) drove out a family of install/remove races
the slow vsync'd mock had masked. Fixes (hook/src/hook_install.hpp + hook_guard.hpp):
- Persistent hooks. The old model created a hook on install and DESTROYED it on
remove (= {}), freeing the trampoline; a detour about to call it (.stdcall) then
hit freed memory -> 0xC0000005. drain() can't fully close that window (a thread
can be inside the detour but not past its Guard ctor). So hooks are now created
ONCE and only enable()/disable()d across install/remove cycles -- never destroyed
during the session -- so a stale detour always calls a live trampoline (disabled,
it just runs the original). Reused, so no churn and no leak. remove_* therefore
disable()s + drain()s but does not destroy; install guards check .enabled().
- Install race. create_inline() enables the hook before the result is move-assigned
into the global the detour reads; a call landing in the detour mid-assign reads a
torn hook -> AV. install_inline() creates StartDisabled, assigns, then enable()s.
- drain() Sleep(1)s BEFORE each zero-check, so a thread that entered the detour but
hasn't reached its Guard registers before we conclude zero.
- Focus: publish g_orig_proc before SetWindowLongPtr activates the subclass (and
subclass_proc falls back to DefWindowProc if null); and disable the focus-query
hooks in reverse install order, because GetForegroundWindow shares user32 code
with GetActiveWindow (keep GFW hooked until GAW is unhooked).
- disable()/enable() [[nodiscard]] results are handled (logged), not (void)-discarded.
Storm now survives on every backend across repeated runs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The mock backends presented with vsync ("a game-like cadence") -- wrong for a
perf/stress fixture: it does trivial work on an RTX 4090, so it must run as fast
as it can. Vsync capped them to tens of fps (dx9 30, dx10 23, dx11 63, dx12 126),
which hid both capture-induced slowdowns and the hook-removal race. Uncapped now:
dx9/dx10 INTERVAL_IMMEDIATE / Present(0,0) (BLT), dx11/dx12 ALLOW_TEARING +
Present(0, ALLOW_TEARING) (flip), gl wglSwapIntervalEXT(0), vk IMMEDIATE/MAILBOX.
Measured no-hook: dx9 ~21000, dx10 ~2800, dx11 ~17000, dx12 ~12000, gl ~26000, vk
~24000 fps.
mock_game_test now adds a present-rate floor per backend (>= 300/s while
capturing): with the hook live every backend stays in the hundreds-thousands
(vk 13500, dx11 9000+, gl 1800, dx9/10 ~1000-1600, dx12 2500). This is the
dimension the frame-advance checks missed -- the Vulkan 144->3 FPS stall still
advanced frames -- so it catches a present-thread stall OR an accidental vsync.
The faster storm exposed the hook-removal UAF fixed in the previous commit.
README roadmap + lessons-learned updated (incl. correcting the old "reset makes
in-flight trampoline calls safe" claim). Full suite 21/21.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Uncapping the mock game (next commit) turned mock_game_test's hook/unhook storm
into a real stress test (thousands of presents/s instead of tens), which reliably
crashed the game on remove (0xC0000005) for dx9/dx11/dx12. Two races the slow
vsync'd mock had masked:
1. Trampoline use-after-free. remove_*_hooks did `hook = {}` (destroy) BEFORE the
DetourGate drain. Destroying a SafetyHook InlineHook frees its trampoline
immediately, but an in-flight detour about to call the original via .stdcall()
(the trampoline) then used freed memory. Fix: disable() first (restores the
original bytes under thread suspension, but KEEPS the trampoline alive) -> drain
-> only then destroy. Applied to present/d3d9/opengl/vk/xinput/mkb/focus.
2. Entry-window race in DetourGate::drain(). It returned the instant the active
count read zero, but a thread can be inside the detour yet not have reached its
Guard constructor (the prologue is unguarded), so the count reads zero while a
detour is about to run -- and the freed state is then used. Fix: Sleep(1) BEFORE
each zero-check; with the hook disabled no new detour starts, so any
already-entered thread registers within that window. This alone fixed dx11 (the
highest present rate, ~11000/s, which hit the window every storm).
Audio is unaffected (it uses vtable swaps, which keep a real original pointer, not
a trampoline). Full suite 21/21, and the storm now survives on every backend.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Investigated the DX12 present-thread overhead (~0.38 ms vs DX11 ~0.05 / GL ~0.09).
Per-stage timing of the D3D11On12 path showed the cost is NOT where you'd assume:
fence 0.005 + wrap(CreateWrappedResource+Acquire) 0.012 + copy 0.133 + flush 0.057 ms
CreateWrappedResource is cheap. The cost is the CopyResource issued on the 11On12
immediate context plus the mandatory Flush to make the shared copy visible to the
host -- both inherent to the bridge and not paid by the native-D3D11 path. The
per-frame GetDevice can't be skipped either (it's how device recreation is
detected). Documented this in the capture path.
Improving it means a native-D3D12 copy-queue path into a D3D12-shared texture, but
the host consumes the shared surface via IDXGIKeyedMutex (a D3D11 concept), so that
also requires switching the DX12 producer<->host sync to a shared ID3D12Fence -- a
cross-API rewrite. Deferred: the overhead is ~5% of a 144 Hz frame and correct.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The capture must never change vsync, and must not frame-limit itself.
- Removed the ~150 Hz capture throttle from VkCapture. It was wrong: vsync already
paces capture (a 144 Hz FIFO game presents 144x/s, so we mirror 144x/s). The only
limiter left is ring backpressure (skip a frame if the reaper is behind), which is
correctness, not a cap, and never touches the game's present thread or sync mode.
- The layer/hook already pass VkSwapchainCreateInfoKHR straight through, so the
present mode (= the sync mode) is untouched. Added a presentMode log to prove it.
Measured on Sphere Spectacle (direct launch, layer attached): presentMode=2 (FIFO),
steady 144.0 fps, and with the throttle gone the mirror now tracks it at 144/s
(was capped ~130). The earlier 400-600 fps reading was a direct-launch artifact --
a non-foreground windowed FIFO app isn't throttled by DWM -- not the layer, and not
the case through Steam (144). vk_validate now states the mirror follows the present
rate (no throttle) and still asserts a present-rate floor.
Full suite 21/21.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-ups from real-game testing:
1. The Vulkan capture must not change the swapchain sync mode, and must not
throttle/frame-limit capture (that's vsync's job). Remove the ~150 Hz throttle
and prove the present mode/rate is unchanged with vs without the layer.
2. DX12 capture overhead (~0.34 ms) is higher than DX11 (~0.05) / GL (~0.09) due
to the D3D11On12 bridge -- investigate and improve.
3. The mock game only drives D3D11/D3D12; add Vulkan/OpenGL/D3D9 renderers so every
capture path has game-driven coverage (this gap is why the Vulkan stall slipped
through). Make the mock game a real-world test across all backends.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The earlier validation concluded suspended-inject was "not applicable -- the
title requires launching through Steam." That was wrong; it was two bugs:
1. coop_vk_validate's inject mode launched the exe with CreateProcessW and a
null working directory, so the game couldn't load steam_api64.dll / resources/
(loaded relative to cwd) and never rendered -> no presents. Launch with the
game's own folder as cwd and it runs fine directly, no Steam needed.
2. The game resolves vkQueuePresentKHR / vkCreateSwapchainKHR via
vkGetInstanceProcAddr (volk's volkLoadInstance does this), but vk_hook only
substituted our detours when they were resolved via vkGetDeviceProcAddr -- so
the present bypassed the hook. Intercept those names in hk_vkGetInstanceProcAddr
too (our detours already gate on g_capture_enabled/g_device, so handing them out
before the device exists is safe).
With both fixed, inject mode captures Sphere Spectacle correctly: 1920x1080,
correct colors/orientation (screenshot), ~480 fps present while mirroring at the
~150 Hz throttle -- no present-thread impact (the VkCapture fix is shared).
Also makes the validator ASSERT a present-rate floor while capturing (it used to
report the rate and rationalize it, which is exactly what hid the 144->3 FPS
stall), and reports the true mirror rate from video.generation. Division of labor
is about who launches the game: layer for Steam-launched (can't suspend), inject
when we control the launch. README lessons-learned corrected accordingly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Against Sphere Spectacle (144 FPS, runs without Steam) the implicit-layer
capture dropped the game to ~3 FPS. Measured cause (per-stage trace in the
layer): the read-back ran on the game's PRESENT THREAD and spent ~370 ms per
1080p frame -- not the GPU copy (~2 ms) but the CPU swizzle, because the staging
buffer was a plain HOST_VISIBLE|HOST_COHERENT type (write-combined / uncached on
a discrete GPU), where a scattered CPU read runs at PCIe latency. 3 captures/s =
the 3 FPS the user saw.
Test-first: tests/vk_capture_perf_test reproduces the stall as a deterministic
unit test (372 ms/present, ratio 1.0 -> FAIL via `--sync`), then proves the fix
(0.02 ms/present, byte-correct BGRA->RGBA, ratio ~0 -> PASS).
Fix: extract the near-identical read-back from vk_hook.cpp and coop_vk_layer.cpp
into one shared coop::hook::VkCapture that:
* has the present thread only record + submit the copy (sub-ms) and return;
* runs a dedicated reaper thread for the fence wait + swizzle + D3D upload, off
the critical path, with a ring of in-flight slots (game never waits);
* allocates HOST_CACHED staging (fast CPU read), invalidating when non-coherent;
* throttles capture to ~150 Hz (a guest stream is <= the host refresh; no point
mirroring an uncapped 400+ FPS game and burning reaper CPU).
Real-game A/B: present rate now matches the no-capture baseline (605->470 vs
593->405 over the same ramp) with the mirror at ~130 fps -- no measurable impact.
Also adds present-thread overhead guards to the other GPU backends' hook tests
(present_overhead.hpp): DX11 0.05 ms, DX12 0.34 ms, OpenGL 0.09 ms overhead, all
asserted < one 60 Hz frame, so any future synchronous-stall regression fails.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Real-game testing of Sphere Spectacle (144 FPS, no focus throttle, runs without
Steam) surfaced two genuine defects the earlier "validation" missed:
1. The implicit-layer capture drops the game to ~3 FPS -- it stalls the present
thread on a full GPU readback every frame. Vulkan support is meaningless until
capture is (near-)free.
2. The inline-hook (suspended-inject) path must actually work on this title, or
be proven a true limitation -- Steam is not the reason it currently doesn't.
Final verification of both paths is to be done with the game launched through
Steam. Tracked as Current tasks; will move to Done as each lands.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds coop_vk_validate, a harness that drives the Vulkan capture path end-to-end
against a shipping title (default Sphere Spectacle, a pure-Vulkan game) and asserts
frames reach the shared texture and advance, the captured resolution/colors are
sane, saves a BMP screenshot for visual confirmation, and reports the present rate.
Findings:
- Layer method WORKS: coop_vk_layer mirrors the game correctly at 1920x1080 --
right colors/brightness, no BGRA/RGBA swizzle, no sRGB darkening (screenshot
confirmed). The layer captures every present (no drops).
- Suspended-inject "Auto-attach" is NOT applicable to a Steam title that must launch
through Steam: its exe renders nothing when launched directly, so there's no Vulkan
present to catch. The layer is the method for Steam Vulkan games (the early-inject
mechanism itself is covered by mock_game_test's suspended-launch path).
- The layer does video, but the game needs coop_hook.dll co-injected for focus-spoof
or an unfocused, event-driven game throttles itself to a few fps (looks like a
capture slowdown but isn't). A present-rate number alone can't prove "no FPS impact"
-- it's the game's own cadence; capture stays off the critical path (every present
copied, read-back on its own queue + present-semaphore re-chain).
Also adds SharedTextureSource::read_frame (bulk RGBA readback) for the screenshot.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The MKB subsystem covered message-loop (PostMessage) and polling
(GetAsyncKeyState/GetKeyboardState/GetCursorPos) games. Add the two remaining
read paths, both fed from the same synthesized state:
- DirectInput: vtable-swap IDirectInputDevice8::GetDeviceState (a COM method ->
vtable swap, not inline, per the x86 COM-prologue trap), reading the shared
vtable from a kept-alive probe device (the game made its devices before we
injected). Dispatch on cbData: 256 = keyboard BYTE[256] indexed by DIK
scan-code (map VK->DIK via MapVirtualKey VK_TO_VSC); DIMOUSESTATE = mouse
buttons + relative deltas from the forwarded cursor.
- Raw Input: games get no WM_INPUT while unfocused, so mkb_pump synthesizes it
(PostMessage WM_INPUT with lParam = one of our RAWINPUT slots) and the hooked
GetRawInputData serves that slot back (RID_HEADER + RID_INPUT). Covers keys +
buttons; relative raw-mouse movement isn't in the position-based MKB stream.
Both detours use the DetourGate safe-unhook guard, and the mock_game_test storm
exercises their install/remove. dinput_hook_test drives the real path end-to-end:
forward a key + mouse button through the ring, then a real DI keyboard/mouse
device's GetDeviceState returns them. The Raw Input round-trip is operator-
validated against a real game (too brittle to assert in-process). vtable_hook.hpp
factors the COM vtable-swap helper out for reuse.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extends the two-path correlation from rate-only to the full layout, removing the
"channels/bit-depth assumed = device" limitation. correlate_format tries each
candidate de-interleaving (float32 / int16; mono..7.1) of the hook capture, runs
the rate correlation per layout, and keeps whichever aligns with the loopback;
a wrong de-interleaving is noise and won't.
The catch: the hook can't know a guessed stream's real frame size, so its verify
tap pads each render buffer to the device block -- which over-reads stale staging
bytes for a stream with fewer channels/bits, scrambling the audio. So the tap is
now self-describing: it prefixes each buffer with its frame count
([count][count*device_block bytes]), and the host strips the padding per candidate
layout (take the real count*real_block of each chunk) before de-interleaving.
- audio_correlate.hpp: ChunkedCapture + chunk-aware correlate_format + candidate
layouts; absolute-margin confidence gate (the true layout scores ~1.0, a truly
ambiguous alternative within ~0.001 -- 2ch@R == 1ch@2R for identical channels --
is correctly left unconfident).
- audio_hook.cpp: chunked verify tap (free-space-checked so framing can't tear).
- audio_format_verifier: parse chunks; recover_layout path. AudioMirror now corrects
the full format.
- audio_correlation_test: layout recovery from padded chunks (stereo float, 16-bit
PCM, 5.1, mono). audio_verify_test gains scenario (b): 2ch on a multichannel
endpoint with distinct per-channel content (new env-gated ToneSource mode) ->
recovers ch=2/32-bit float end-to-end.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When the host attaches to an already-running game it never saw the stream's
Initialize, so the render-hook assumes the device mix format and measures only
the sample rate from the render cadence -- which a jittery game can make wrong
(intermittent pitch shift). But during the measurement window the game is still
audible, so we have the same audio twice: the hook (pre-mix, unknown format) and
a process-loopback (post-mix, the known device format). Cross-correlating them
pins the true rate from ground truth.
- common/include/coop/audio_correlate.hpp: the pure correlator. Resample the hook
by each candidate standard rate up to the device rate and score how well it
aligns with the loopback across the window (drift-detecting). audio_correlation_test
recovers every rate (score ~1.0 vs ~0.01 for wrong ones), incl. 44100-vs-48000,
and rejects unrelated signals.
- Hook measurement tap: a host-set verify_capture ring flag makes the hook push a
still-being-measured (guessed) stream's raw pre-mix bytes WITHOUT silencing, so
the host can co-capture both signals (a silenced game's loopback is silent).
Inert by default -- the shipping no-echo path is untouched.
- host/src/audio/audio_format_verifier: co-captures hook + loopback and correlates,
feeding a correction into the existing override channel. Wired into AudioMirror's
measurement window (hidden in the gap loopback already covers, so exact streams
pay nothing). audio_verify_test drives it end-to-end against coop_mock_game.
Rate vs layout are coupled (correlating the waveform needs the right channel
de-interleaving), so this step assumes the hook layout matches the device (the
common stereo-on-stereo case); recovering a different channel count / bit depth
is step b.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Spamming a subsystem toggle (the "Mirror video" button) could crash the game:
remove_*_hooks freed a hook's shared D3D / Vulkan / IPC state immediately, while a
capture detour was still mid-flight on the game's render thread -> use-after-free.
Only the audio hooks had the safe-unhook drain; the video (Present/D3D9/D3D10/GL/
Vulkan) and XInput/focus/MKB hooks did not.
Test-first: mock_game_test now runs an aggressive hook/unhook storm -- a separate
thread thrashes every subsystem on/off while the game presents, across all backends.
It crashed gl + vk (0xC0000005) and failed dx9 capture-resume before the fix.
Fix (hook/src/hook_guard.hpp, DetourGate): each detour wraps its body in an RAII
active-count Guard; remove_* restores the hook first (so no new detour starts),
drains the in-flight detours to zero, and only then frees the shared state. Vulkan
is special-cased -- the game caches hk_vkQueuePresentKHR, so removal closes an
atomic capture gate (detours then pass through to the real present), drains, then
frees the read-back resources. Storm now passes on every backend.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a fourth Current Task: exercise the Vulkan capture path end-to-end on
Sphere Spectacle (Steam appid 1123040) via both early-presence methods
(Auto-attach and the implicit coop_vk_layer), verifying capture, image
correctness, and no performance regression.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the Future-work section with a single "Current Tasks" list:
- Injection hardening: toggling "Mirror video" has crashed a real game
(Brotato) via a hook install/remove race. Test-first task -- make
mock_game_test's hook/unhook stress aggressive enough to reproduce the
crash (tight video-subsystem toggles from a separate thread while the game
presents, across all backends), then apply the audio hooks' safe-unhook
guard (epoch bump + restore-then-drain in-flight detours) to the Present /
D3D9 / D3D10 / OpenGL / Vulkan hooks.
- Audio format by correlating the loopback (known device format) and
render-hook (unknown format) captures instead of guessing: (a) rate
verification/correction, (b) channel + bit-depth recovery.
- Mouse + keyboard forwarding (Raw Input / DirectInput) folded in.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The hooked path must capture the game's frames AND mute its local playback
("no echo"). The mute was implemented as zero-the-buffer (memset) + release
with AUDCLNT_BUFFERFLAGS_SILENT. Zeroing num_frames*block is only safe when
block is the real frame size; for a guessed format (late attach -- the
Brotato case, where we never saw Initialize) the guessed block can exceed
the real buffer, so the conservative code skipped the whole mute for guessed
streams. That left the game audible: it played locally AND the mirror
re-rendered the same audio a few ms later = a metallic, out-of-sync double.
Fix: AUDCLNT_BUFFERFLAGS_SILENT already makes WASAPI ignore the buffer
contents and play silence -- it mutes with no write at all, so it's safe for
any format. Decouple the two: always mute via the flag; keep the memset only
for an exact/override format (belt-and-suspenders). One-line behavior change;
the byte-incompatible cases confirm the flag-mute never over-writes.
Test-first (now a documented rule, README "Tests"): added an
audio_frames_silenced() counter and a mute assertion to audio_hook_test for
both the exact and guessed paths. The guessed assertion FAILS on the unfixed
code (3 rates) and passes after the fix -- the regression guard for this bug.
README also updates the now-correct no-echo limitation and adds a
lessons-learned writeup.
ctest 17/17. Brotato confirmed fixed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Build coop_audio_validate, a tool that turns "the mirror audio sounds off"
into numbers. It plays a known sine (coop_tone, 44.1 kHz on a 48 kHz
endpoint -- the Godot/Brotato case), injects the hook as the host does, and
runs a fidelity analyzer (coop/tone_analysis.hpp: pitch error in cents,
SNR/THD, click + dropout counts), dumping a .wav to listen to. Modes:
--render drives the real AudioMirror and measures its rendered output;
--baseline/--selfcheck give the measurement floor; --listen <pid> records a
live coop_host's output; --wav analyzes a recording. Analyzer + WAV I/O are
unit-tested (tone_analysis_test) against synthesized defects.
Using it, the capture ring measures pristine (~68 dB, 0 gaps) while the
render path dropped to ~18 dB with gaps -- localizing a real defect in
AudioMirror::run_hooked: it re-primed (withheld the feed until ~30 ms had
rebuffered) on any partial fill (to_write < avail). A partial fill is normal
producer jitter, and withholding the feed drains the device, so a one-frame
ring dip became a full ~30 ms drop-out; on a jittery game it fired
constantly. Fix: feed whatever is available each tick and re-prime only on a
genuine starvation (device empty AND ring empty). The policy is factored
into a pure RenderPacer reused by run_hooked + run_loopback and proven by
render_pacer_test (the old policy withholds available data ~168x and drains
to one period from silence on a jittery schedule; the new one never
withholds).
ctest 17/17.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All of M1 (UI-fit + live inspection) and M2 (every backend mock + capture, incl.
the Vulkan implicit layer + checkbox) are done, so the "Current work" roadmap
section is removed -- only Future work (Raw Input / DirectInput MKB) remains.
Architecture lists the Vulkan layer as the implemented early-presence path;
lessons-learned add the chain-dispatch + loader sType 47/48 gotcha; build/test
docs note the layer artifact + coverage; submodule list updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Host helper register_vk_layer/unregister_vk_layer registers coop_vk_layer's
manifest as a per-user (HKCU, no admin) implicit Vulkan layer and writes the
scoping file (%TEMP%\coop_vk_target.txt) naming the target game's exe, so the
layer captures only that game and stays inert for every other Vulkan app. The
Injection panel gains an opt-in "Set up Vulkan layer" checkbox (next to Auto
re-attach, which the too-late banner points at); it registers on tick,
unregisters on untick, and the panel destructor unregisters on host exit so no
stale registration is left behind.
Verified live: a registry-registered layer is loaded by the loader for a fresh
vk launch (no env vars) and the scoping matches (active=1), then cleans up.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A real chain-aware Vulkan implicit layer the loader inserts at vkCreateInstance
-- the reliable early-presence path for games that init Vulkan immediately,
which the inline-hook vk_hook can't catch. It intercepts vkCreateInstance /
Device / CreateSwapchainKHR / QueuePresentKHR via proper layer-chain dispatch
and does the same read-back capture (vkCmdCopyImageToBuffer -> swizzle ->
hook-owned D3D11 shared texture, with present-semaphore re-chaining) as vk_hook.
The loader/layer link structs (VkLayer*CreateInfo, VkNegotiateLayerInterface)
aren't in Vulkan-Headers, so they're hand-declared to interface version 2. Key
gotcha found via tracing: the loader tags those link structs with small internal
sType values (LOADER_INSTANCE_CREATE_INFO=47, _DEVICE=48), not the 1000000000
range -- matching the wrong value made the device-chain walk fail.
Scoping: an implicit layer loads into every Vulkan app, so it only *captures*
when COOP_VK_LAYER_FORCE is set (tests) or this process's image matches
%TEMP%\coop_vk_target.txt (the host writes it); otherwise pure pass-through.
mock_game_test registers it via VK_LAYER_PATH/VK_INSTANCE_LAYERS and decodes
frames through it. Ships at the bin root with its JSON manifest.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The opt-in implicit-layer piece requires the loader/layer interface header
vk_layer.h, which Vulkan-Headers doesn't ship (it's in Vulkan-Loader / the SDK)
-- so it needs a new submodule or hand-declared link structs, confirming it's a
separate, higher-risk component rather than a quick add.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Architecture now lists Vulkan as a hooked producer (GPA interception + read-back,
early-presence required). Adds a lessons-learned bullet (can't late-hook Vulkan;
the present-semaphore re-chaining trap; the too-late heuristic; early-load
testing). mock_game_test docs note the suspended-launch capture + too-late check.
The Vulkan milestone is trimmed to its one remaining piece -- the opt-in implicit
layer for immediate-init games -- with the capture/banner/best-effort marked done.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The hook reports vk_too_late when vulkan-1.dll is loaded and the GPA hook has
been in for >4s but it never caught the app creating its device -- i.e. the app
resolved its Vulkan functions before we hooked (injected too late). Surfaced
through a new HookStatus.vk_too_late field (protocol 15->16) and shown by the
host as a red banner on the mirror window: "Detected a Vulkan game, but the
mirror hook attached too late... enable Auto re-attach (and Set up Vulkan layer
if it persists) and relaunch." WGC keeps mirroring meanwhile.
mock_game_test gains a too-late detection check (late-inject a Vulkan game ->
vk_too_late trips). Verified live: the banner shows over the running tool when a
Vulkan game is injected late with the video subsystem on (added a debug-harness
`video on/off` command to drive it). 15/15 ctest.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New vk_hook.cpp inline-hooks the vulkan-1.dll vkGetInstanceProcAddr export and
hands back our wrappers for vkCreateInstance / vkCreateDevice / vkGetDeviceProcAddr
/ vkCreateSwapchainKHR / vkQueuePresentKHR, so a volk-using (loader-bypass) app
resolves our hooks. On present it reads the swap-chain image back with
vkCmdCopyImageToBuffer into a host-visible buffer (same read-back model as
D3D10/D3D9/OpenGL), swizzles BGRA->RGBA, and uploads it into the shared
keyed-mutex texture on a hook-owned D3D11 device. The read-back submit re-chains
the present's wait semaphores (consume the originals, signal our own that the
real present waits on) so capture orders after rendering without double-waiting.
Wired into the video subsystem with a lazy retry (vulkan-1.dll loads late). Hook
links the official Vulkan-Headers (headers only, VK_NO_PROTOTYPES) via the
include dir so the x86 sub-build builds too.
Because Vulkan caches its present pointer at init, late injection can't hook it:
mock_game_test launches the mock **suspended**, injects, resumes, and under
COOP_MOCK_VK_EARLY the mock loads Vulkan and waits so the hook arms first -- then
decodes frames through the hook like the other backends. 15/15 ctest (x64 +
x86); skips cleanly without a Vulkan driver.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reflect that the Vulkan mock backend + submodules are done, and scope the
remaining Vulkan capture honestly: hook vkGetInstance/DeviceProcAddr +
intercept device/swapchain creation, and read the presented image back with
vkCmdCopyImageToBuffer (the same read-back pattern as D3D10/D3D9/OpenGL --
simpler/robuster than a VK_KHR_external_memory keyed-mutex blit). Note testing
needs the early-load path (late-inject can't catch a Vulkan present), and Vulkan
games mirror via WGC meanwhile.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The headless ui_fit_test proves panels *fit* their windows but not that the
overlay *looks* right. Document that closing M1 requires running the actual
coop_host.exe (-DCOOP_TEST_HARNESS=ON), driving it to maximum info, taking an
F10 screenshot, and eyeballing the real overlay -- a green unit test is not a
substitute for looking at the product. Verified live: every panel fits and reads
correctly at the monitor resolution, and the live `uifit` reports "fit".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add render_vk.cpp (selectable as `vk`): brings up a real Vulkan
instance/device/swap chain via volk (which dlopens vulkan-1.dll -- the
loader-bypass case the capture hook must handle) and clears the swap-chain image
to the frame-counter colour each frame with vkCmdClearColorImage (no pipeline,
no shaders, no SPIR-V) and presents. The whole image encodes the frame number,
so it animates and stale frames are detectable.
Adds the official Khronos Vulkan-Headers + zeux/volk submodules and a
coop_require_submodule() CMake helper that fails with a clear "git submodule
update --init --recursive" message rather than auto-cloning. volk is pinned to
the project's dynamic CRT (no LNK4098).
mock_game_test gets a vk liveness check (its present pointer is cached at init,
so late injection can't hook it -- the capture path needs the early-load path,
to come). 16/16 ctest, skips cleanly without a Vulkan driver.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add render_gl.cpp (selectable as `gl`): renders the animated pattern with
scissored clears (glClear + glScissor -- GL 1.1, exported straight from
opengl32) and presents with SwapBuffers. No glad/submodule needed: a legacy
wglCreateContext + <GL/gl.h> suffices, so the planned loader dependency was
dropped. GL is bottom-left origin and the capture flips top-down, so the
frame-counter block is drawn at the GL top to land top-left in the captured
image. Window class gains CS_OWNDC for a stable GL DC; best-effort vsync via a
runtime wglSwapIntervalEXT lookup.
The GL SwapBuffers/wglSwapBuffers hook now bumps the shared present counter
(it's the GL present), so present_calls works for GL games too. mock_game_test
decodes GL frames through the existing glReadPixels capture path; 15/15 ctest.
Roadmap: OpenGL milestone done and removed (Vulkan renumbered to M2);
architecture/lessons/test docs updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New d3d9_hook.cpp (own file like opengl_hook): inline-hooks
IDirect3DDevice9::Present (vtable index 17, discovered from a throwaway device;
clean prologue so inline is safe, stdcall() on x86) and read-backs the backbuffer
with GetRenderTargetData into a D3DPOOL_SYSTEMMEM surface, swizzles BGRA->RGBA,
and uploads it into the shared keyed-mutex texture on a hook-owned D3D11 device
(a D3D9 surface isn't D3D11-shareable). Both plain D3D9 and D3D9Ex call this same
Present, so one read-back path serves both -- the GPU shared-surface fast path
the plan sketched for D3D9Ex wasn't worth it. Wired into the video subsystem
(install/remove in dllmain); hook links d3d9.
mock_game_test decodes DX9 + DX9Ex frames through the hook; 15/15 ctest (incl.
the x86 sub-build). Roadmap: DX9 milestone done and removed (remaining
renumbered); architecture + lessons updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add render_dx09.cpp (selectable as `dx9ex` / `dx9`): renders the animated
pattern with Clear + ColorFill (D3D9's built-in rect fill -- no shaders) and
presents through a real D3D9 / D3D9Ex device. Two modes so the two D3D9 capture
paths each have a matching game. Smoke-verified: both present frames and exit
cleanly. The capture hook + frame-decode test land next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A pure-D3D10 game's backbuffer QIs to ID3D11Texture2D but that view reads back
empty (content lives on the game's D3D10 device), and a D3D11 backbuffer also
QIs to ID3D10Texture2D -- so GetBuffer can't discriminate. The reliable signal
is that a feature-level-10 device rejects CreateTexture2D with the NT-handle
keyed-mutex share flags (E_INVALIDARG). The present hook now tries the D3D11
fast path and, on that failure, switches (sticky) to reading the backbuffer
through the game's own D3D10 device into a staging texture and uploading it into
the shared texture on a hook-owned D3D11 device (Map blocks until the GPU copy
completes -> no cross-device race). The host reader is unchanged.
mock_game_test now decodes DX10 frames through the hook (monotonic/advancing)
alongside DX11/DX12; 15/15 ctest. Roadmap: DX10 milestone done and removed
(remaining renumbered); architecture + lessons updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add render_dx10.cpp (selectable as `dx10`): composes the animated pattern
(background + moving bar + top-left frame-counter block) on the CPU each frame
and CopyResource's it into a real D3D10 DXGI swap-chain back buffer (DX10 has no
rect-clear; no shaders). Smoke-verified: `coop_mock_game dx10 2` presents frames
and exits cleanly. The frame-accurate decode-through-the-hook assertion lands
with the DX10 capture commit next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add panel overflow instrumentation (record_panel_fit reading ImGui ScrollMax),
a forced layout-reference + debug-aware center split, and host harness commands
(uisize/uifit). New headless ui_fit_test drives the real Controllers + Audio
panels at reference resolutions with Debug details on and asserts no panel
overflows its assigned size; the Audio panel gains a demo mode so its richest
content renders without a live mirror.
Fixes from the measured overflow: widen the center column (was too narrow ->
horizontal overflow), merge the Controllers poll/round-trip tables and fold the
trigger line into the slot line, and make the center height split
Debug-details-aware (Video's height is mirroring-driven, not debug-driven, so a
static split can't serve both modes). Controllers + Audio now fit at 1920x1080
with max info. Video/Injection/Log fit is finalized at the end via the live
uifit harness (M1 stays open until then).
15/15 ctest.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Restructure the roadmap into one "Current work" section where each rendering
API is a single milestone that builds its mock-game backend and the injected
capture for that same API side by side, so each API reaches verified
end-to-end before the next. Adds an M1 end-user UI fit pass (drive the overlay
to maximum info via the harness, screenshot, assert every panel fits its
window; resize/rearrange otherwise) kept green by every later milestone.
Folds in DX10, DX9 (D3D9Ex + plain non-Ex), OpenGL, and Vulkan (best-effort
inject + opt-in implicit layer, too-late red prompt). Drops the old Vulkan/
D3D9 future-work bullets.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mock_game_test now launches coop_mock_game at 44100/48000/96000 (PCM + float)
and asserts the hook measures each variant's rate through the full inject path
(+ non-silent capture) -- the "all audio variants work" part of the suite.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Since guessed streams are now captured-but-not-silenced (the over-write fix),
the hooked path is only no-echo for an exact/override format. The panel
inferred "Hooked (no echo)" unconditionally, which was misleading. Show
"Hooked (echo -- guessed format)" (amber) for a guessed primary stream and
"Hooked (no echo)" (green) only for exact/override; drop the unconditional
"no echo" from source_name() and the mirror status string.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mock_game_test launches coop_mock_game (DX11/DX12, frame-numbered A/V), injects
coop_hook.dll, opens the hook's shared video texture, and decodes the frame
number from the captured pixels to assert a monotonic, advancing mirror for
both backends (catches dropped/stale/out-of-order frames). Then it runs an
A/V + hook/unhook stress pass and confirms no crash + capture resumes. Adds a
read_pixel() readback to the shipping SharedTextureSource for the decode.
Robust to repeated/CI runs (kills stray games, retries inject). Trim the
roadmap (the mock-game item is done) and document the test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Found by the mock-game capture/audio/hook stress test (toggling the audio
subsystem while a game renders):
1. Guessed-stream silence over-WRITE: hk_ReleaseBuffer zeroed num_frames *
guessed_block bytes, but a guess can be larger than the real per-frame size
(e.g. an 8ch device guess for a 2ch game), so the memset wrote past the real
buffer into adjacent audio memory -> intermittent access violation in the
game. Fix: capture but do NOT silence a guessed stream (it stays audible --
echo); only an exact/override format, whose frame size is known, gets the
no-echo silence.
2. VtableHook::remove nulled m_original, racing an in-flight detour into a null
call -> keep it valid (the original function stays mapped).
3. g_ipc was a non-atomic pointer read on the hot path while unhook nulled it
(TOCTOU) -> make it atomic, load once.
4. Stale GetBuffer/ReleaseBuffer pairing across a toggle -> epoch-stamp the
GetBuffer and only capture in the same hooked epoch.
5. COM-object churn: re-creating the probe client every enable raced AudioSes ->
build the probe once, keep it across toggles (only swap vtable slots);
release on detach (shutdown_audio_hooks). Plus drain in-flight detours before
tearing down state.
Stress test: 0 crashes in many repeated runs (was ~50%). Guessed streams now
echo (the no-echo path is reached via an exact/auto-attach format or override).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A tiny test "game" for exercising the capture/audio/hook paths. Opens a normal
window and renders an animated pattern -- moving bar + per-frame background so
motion (and dropped frames) are obvious -- with a top-left block whose RGB
encodes the exact frame number, so a capture test can decode it and detect
dropped / duplicated / stale frames. Selectable backend (dx11 / dx12 today),
behind a RenderBackend interface so OpenGL/Vulkan can be added. With audio args
it also plays a configurable WASAPI tone (shared ToneSource), so it's a full A/V
source with a window (unlike coop_tone).
coop_mock_game.exe [dx11|dx12] [seconds] [rate] [channels] [bits] [pcm|float]
Milestone 1 of the mock-game roadmap item (the stress-test suite follows).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an "Auto re-attach this game on relaunch" checkbox (default off, never
persisted, shown while attached or terminated). While on and the target has
terminated, the host polls the process list for the same image name and
re-injects the moment it reappears -- built on the existing
re-attach-by-image-name path, so there's no target to pick. The kill+relaunch
recovery for a wrong audio format: re-attaching early catches
IAudioClient::Initialize and reads the exact format.
Validated live (harness): inject coop_tone -> kill -> relaunch -> auto
re-injected into the new pid, and early enough that the format came back as
Exact. Trim README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Persist audio overrides keyed by game image name (coop_audio_overrides.ini
next to the exe) so a known-bad game is auto-corrected on its next launch:
on attach to a guessed stream the host applies any saved override, and a
manual Override now saves too. A format the hook catches exactly at
IAudioClient::Initialize is auto-saved as that game's override (ground truth);
if it overwrites a differing stored value, a warning is logged. Host-originated
log lines now reach the Log window via IpcServer::host_log (color-coded).
Validated live (harness): a pre-seeded override for coop_tone is auto-applied
over the guess (state -> manual override, 48000/2/16). audio_overrides_test
covers persist/reload/case-insensitive lookup/differing-overwrite. Trim README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a per-stream op channel in AudioRingHeader (op_seq + op_* fields, version
2): the host posts re-measure / override commands, the hook applies them and
re-publishes (bumping format_generation), and the host rebuilds its render
client live on the change. The Audio panel (under Debug details) gains a
"Re-measure rate" button and a rate/channels/bit-depth/format override -- for
when detection is wrong or the channels/bit-depth were unrecoverable.
Also add a debug-only IPC test harness (-DCOOP_TEST_HARNESS, off by default,
absent from the shipped host): a file-based command channel that drives the
host's real UI code paths (inject / audio / re-measure / override / screenshot
/ status) for scripted validation, instead of unreliable synthetic mouse input.
Used to validate live: late-attach to coop_tone@44100 -> measured 44100,
promoted to hooked; override -> 2ch state, re-measure -> reconverge.
Trim the README roadmap to what's left; document the harness + rate_estimator_test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The host used to wait a fixed 1 s for the hook to publish a format and, on
timeout, fall to loopback permanently with no explanation -- which the now
slower (consensus) measurement made common. Restructure the audio thread to
keep the rings live the whole session and alternate: prefer hooked, and while
it isn't ready run loopback (echo) so guests still hear audio, watching the
ring to promote to the no-echo hooked path the instant the hook publishes a
format. Surface the concrete reason loopback is active (no format yet / not
renderable / no ring) in the Audio panel, in amber.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Harden the guessed-stream sample-rate measurement that produced wrong rates
(e.g. 44100 read as ~46205). New rate_estimator.hpp measures over longer
~0.5 s windows, rejects any window that doesn't snap to a standard rate
(standard rates are >8% apart, so a quantization/burst error big enough to
miss one lands in no-man's-land, never on a wrong neighbour), and requires
consensus across windows before committing. If consensus isn't reached it
commits a low-confidence estimate (new AudioFormat_LowConfidence, shown red)
rather than spinning or publishing garbage. Pure logic, unit-tested with
adversarial cadences (rate_estimator_test) incl. the real 46205 bug value.
Add log severity levels: hook logw/loge set LogRecord.level; the host Log
window colors warnings amber and errors red. The low-confidence rate logs a
warning. Protocol -> v15 (new format states); also reserves AudioFormat_Override.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Capture the rendered back buffer to a timestamped PNG next to the exe via
WIC, triggered by F10 (delivered even when unfocused). The capture runs in
render_frame just before Present so it includes the ImGui overlay, and reads
off the GPU so it works regardless of window focus, z-order, or occlusion. A
brief toast confirms the save (drawn the next frame, so it's never in the shot).
F10 chosen to avoid Steam's F12; its WM_SYSKEYDOWN is swallowed so Windows
doesn't enter menu mode. Documented in the Help menu + README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document the diagnosis and plan for the unreliable hooked-audio format
detection: robust rate measurement (longer window, atomic endpoints,
consensus, reject non-standard rates), visible + red-flagged loopback
fallback with auto-promote, a re-measure button and per-game persisted
format overrides via an AudioRingHeader op channel, session-only
auto-re-attach on relaunch, and color-coded log levels. Drop the stale
"previous backlog is all shipped" note.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
