- The remove_* comments still described the superseded "disable -> drain -> destroy"
flow; the code keeps hooks alive (persistent) and re-enables on re-install. Updated
the comments to match, and corrected the XInput note (its detours return synthesized
state and never call the trampoline, so destroying its vector is safe -- unlike the
trampoline-calling present/MKB/focus-cursor hooks).
- hook_install_test: a fast, single-threaded contract test for hook_install.hpp --
install_inline creates the hook once and reuses the SAME trampoline across 50
install/remove cycles (never freed -> no stale-detour UAF), toggling enable/disable
cleanly. Fills the guard the removed (flaky, concurrency-bound) reproducer left, with
no threads so it can't flake on SafetyHook's enable/disable atomicity.
x64 23/23, x86 3/3.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The uncapped, input-polling mock_game_test storm (thousands of presents/s, now
also driving the input/focus/MKB hooks) drove out a family of install/remove races
the slow vsync'd mock had masked. Fixes (hook/src/hook_install.hpp + hook_guard.hpp):
- Persistent hooks. The old model created a hook on install and DESTROYED it on
remove (= {}), freeing the trampoline; a detour about to call it (.stdcall) then
hit freed memory -> 0xC0000005. drain() can't fully close that window (a thread
can be inside the detour but not past its Guard ctor). So hooks are now created
ONCE and only enable()/disable()d across install/remove cycles -- never destroyed
during the session -- so a stale detour always calls a live trampoline (disabled,
it just runs the original). Reused, so no churn and no leak. remove_* therefore
disable()s + drain()s but does not destroy; install guards check .enabled().
- Install race. create_inline() enables the hook before the result is move-assigned
into the global the detour reads; a call landing in the detour mid-assign reads a
torn hook -> AV. install_inline() creates StartDisabled, assigns, then enable()s.
- drain() Sleep(1)s BEFORE each zero-check, so a thread that entered the detour but
hasn't reached its Guard registers before we conclude zero.
- Focus: publish g_orig_proc before SetWindowLongPtr activates the subclass (and
subclass_proc falls back to DefWindowProc if null); and disable the focus-query
hooks in reverse install order, because GetForegroundWindow shares user32 code
with GetActiveWindow (keep GFW hooked until GAW is unhooked).
- disable()/enable() [[nodiscard]] results are handled (logged), not (void)-discarded.
Storm now survives on every backend across repeated runs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Uncapping the mock game (next commit) turned mock_game_test's hook/unhook storm
into a real stress test (thousands of presents/s instead of tens), which reliably
crashed the game on remove (0xC0000005) for dx9/dx11/dx12. Two races the slow
vsync'd mock had masked:
1. Trampoline use-after-free. remove_*_hooks did `hook = {}` (destroy) BEFORE the
DetourGate drain. Destroying a SafetyHook InlineHook frees its trampoline
immediately, but an in-flight detour about to call the original via .stdcall()
(the trampoline) then used freed memory. Fix: disable() first (restores the
original bytes under thread suspension, but KEEPS the trampoline alive) -> drain
-> only then destroy. Applied to present/d3d9/opengl/vk/xinput/mkb/focus.
2. Entry-window race in DetourGate::drain(). It returned the instant the active
count read zero, but a thread can be inside the detour yet not have reached its
Guard constructor (the prologue is unguarded), so the count reads zero while a
detour is about to run -- and the freed state is then used. Fix: Sleep(1) BEFORE
each zero-check; with the hook disabled no new detour starts, so any
already-entered thread registers within that window. This alone fixed dx11 (the
highest present rate, ~11000/s, which hit the window every storm).
Audio is unaffected (it uses vtable swaps, which keep a real original pointer, not
a trampoline). Full suite 21/21, and the storm now survives on every backend.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Spamming a subsystem toggle (the "Mirror video" button) could crash the game:
remove_*_hooks freed a hook's shared D3D / Vulkan / IPC state immediately, while a
capture detour was still mid-flight on the game's render thread -> use-after-free.
Only the audio hooks had the safe-unhook drain; the video (Present/D3D9/D3D10/GL/
Vulkan) and XInput/focus/MKB hooks did not.
Test-first: mock_game_test now runs an aggressive hook/unhook storm -- a separate
thread thrashes every subsystem on/off while the game presents, across all backends.
It crashed gl + vk (0xC0000005) and failed dx9 capture-resume before the fix.
Fix (hook/src/hook_guard.hpp, DetourGate): each detour wraps its body in an RAII
active-count Guard; remove_* restores the hook first (so no new detour starts),
drains the in-flight detours to zero, and only then frees the shared state. Vulkan
is special-cased -- the game caches hk_vkQueuePresentKHR, so removal closes an
atomic capture gate (detours then pass through to the real present), drains, then
frees the read-back resources. Storm now passes on every backend.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New d3d9_hook.cpp (own file like opengl_hook): inline-hooks
IDirect3DDevice9::Present (vtable index 17, discovered from a throwaway device;
clean prologue so inline is safe, stdcall() on x86) and read-backs the backbuffer
with GetRenderTargetData into a D3DPOOL_SYSTEMMEM surface, swizzles BGRA->RGBA,
and uploads it into the shared keyed-mutex texture on a hook-owned D3D11 device
(a D3D9 surface isn't D3D11-shareable). Both plain D3D9 and D3D9Ex call this same
Present, so one read-back path serves both -- the GPU shared-surface fast path
the plan sketched for D3D9Ex wasn't worth it. Wired into the video subsystem
(install/remove in dllmain); hook links d3d9.
mock_game_test decodes DX9 + DX9Ex frames through the hook; 15/15 ctest (incl.
the x86 sub-build). Roadmap: DX9 milestone done and removed (remaining
renumbered); architecture + lessons updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
