Commit Graph

7 Commits

Author SHA1 Message Date
f843c56f5b Fix stale removal comments (persistent, not destroy) + add deterministic install test
- The remove_* comments still described the superseded "disable -> drain -> destroy"
  flow; the code keeps hooks alive (persistent) and re-enables on re-install. Updated
  the comments to match, and corrected the XInput note (its detours return synthesized
  state and never call the trampoline, so destroying its vector is safe -- unlike the
  trampoline-calling present/MKB/focus-cursor hooks).
- hook_install_test: a fast, single-threaded contract test for hook_install.hpp --
  install_inline creates the hook once and reuses the SAME trampoline across 50
  install/remove cycles (never freed -> no stale-detour UAF), toggling enable/disable
  cleanly. Fills the guard the removed (flaky, concurrency-bound) reproducer left, with
  no threads so it can't flake on SafetyHook's enable/disable atomicity.

x64 23/23, x86 3/3.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 11:51:48 +02:00
79582f9fa6 Make inline-hook install AND remove safe to spam
The uncapped, input-polling mock_game_test storm (thousands of presents/s, now
also driving the input/focus/MKB hooks) drove out a family of install/remove races
the slow vsync'd mock had masked. Fixes (hook/src/hook_install.hpp + hook_guard.hpp):

- Persistent hooks. The old model created a hook on install and DESTROYED it on
  remove (= {}), freeing the trampoline; a detour about to call it (.stdcall) then
  hit freed memory -> 0xC0000005. drain() can't fully close that window (a thread
  can be inside the detour but not past its Guard ctor). So hooks are now created
  ONCE and only enable()/disable()d across install/remove cycles -- never destroyed
  during the session -- so a stale detour always calls a live trampoline (disabled,
  it just runs the original). Reused, so no churn and no leak. remove_* therefore
  disable()s + drain()s but does not destroy; install guards check .enabled().

- Install race. create_inline() enables the hook before the result is move-assigned
  into the global the detour reads; a call landing in the detour mid-assign reads a
  torn hook -> AV. install_inline() creates StartDisabled, assigns, then enable()s.

- drain() Sleep(1)s BEFORE each zero-check, so a thread that entered the detour but
  hasn't reached its Guard registers before we conclude zero.

- Focus: publish g_orig_proc before SetWindowLongPtr activates the subclass (and
  subclass_proc falls back to DefWindowProc if null); and disable the focus-query
  hooks in reverse install order, because GetForegroundWindow shares user32 code
  with GetActiveWindow (keep GFW hooked until GAW is unhooked).

- disable()/enable() [[nodiscard]] results are handled (logged), not (void)-discarded.

Storm now survives on every backend across repeated runs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 11:17:13 +02:00
958c355126 Harden hook removal: disable -> drain -> destroy, and drain settles first
Uncapping the mock game (next commit) turned mock_game_test's hook/unhook storm
into a real stress test (thousands of presents/s instead of tens), which reliably
crashed the game on remove (0xC0000005) for dx9/dx11/dx12. Two races the slow
vsync'd mock had masked:

1. Trampoline use-after-free. remove_*_hooks did `hook = {}` (destroy) BEFORE the
   DetourGate drain. Destroying a SafetyHook InlineHook frees its trampoline
   immediately, but an in-flight detour about to call the original via .stdcall()
   (the trampoline) then used freed memory. Fix: disable() first (restores the
   original bytes under thread suspension, but KEEPS the trampoline alive) -> drain
   -> only then destroy. Applied to present/d3d9/opengl/vk/xinput/mkb/focus.

2. Entry-window race in DetourGate::drain(). It returned the instant the active
   count read zero, but a thread can be inside the detour yet not have reached its
   Guard constructor (the prologue is unguarded), so the count reads zero while a
   detour is about to run -- and the freed state is then used. Fix: Sleep(1) BEFORE
   each zero-check; with the hook disabled no new detour starts, so any
   already-entered thread registers within that window. This alone fixed dx11 (the
   highest present rate, ~11000/s, which hit the window every storm).

Audio is unaffected (it uses vtable swaps, which keep a real original pointer, not
a trampoline). Full suite 21/21, and the storm now survives on every backend.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 09:44:42 +02:00
21c15b162b Harden every hook against the install/remove use-after-free
Spamming a subsystem toggle (the "Mirror video" button) could crash the game:
remove_*_hooks freed a hook's shared D3D / Vulkan / IPC state immediately, while a
capture detour was still mid-flight on the game's render thread -> use-after-free.
Only the audio hooks had the safe-unhook drain; the video (Present/D3D9/D3D10/GL/
Vulkan) and XInput/focus/MKB hooks did not.

Test-first: mock_game_test now runs an aggressive hook/unhook storm -- a separate
thread thrashes every subsystem on/off while the game presents, across all backends.
It crashed gl + vk (0xC0000005) and failed dx9 capture-resume before the fix.

Fix (hook/src/hook_guard.hpp, DetourGate): each detour wraps its body in an RAII
active-count Guard; remove_* restores the hook first (so no new detour starts),
drains the in-flight detours to zero, and only then frees the shared state. Vulkan
is special-cased -- the game caches hk_vkQueuePresentKHR, so removal closes an
atomic capture gate (detours then pass through to the real present), drains, then
frees the read-back resources. Storm now passes on every backend.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 01:56:36 +02:00
e996188aec M2(OpenGL): GL mock backend (no loader) + mock-backed capture coverage
Add render_gl.cpp (selectable as `gl`): renders the animated pattern with
scissored clears (glClear + glScissor -- GL 1.1, exported straight from
opengl32) and presents with SwapBuffers. No glad/submodule needed: a legacy
wglCreateContext + <GL/gl.h> suffices, so the planned loader dependency was
dropped. GL is bottom-left origin and the capture flips top-down, so the
frame-counter block is drawn at the GL top to land top-left in the captured
image. Window class gains CS_OWNDC for a stable GL DC; best-effort vsync via a
runtime wglSwapIntervalEXT lookup.

The GL SwapBuffers/wglSwapBuffers hook now bumps the shared present counter
(it's the GL present), so present_calls works for GL games too. mock_game_test
decodes GL frames through the existing glReadPixels capture path; 15/15 ctest.
Roadmap: OpenGL milestone done and removed (Vulkan renumbered to M2);
architecture/lessons/test docs updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 12:03:43 +02:00
435ab9d30f Fix 32-bit game crash: call hooked __stdcall functions with stdcall()
SafetyHook's InlineHook::call() invokes the trampoline through a __cdecl
pointer (the compiler default on x86). The functions we hook are __stdcall
(IDXGISwapChain::Present/Present1, the WASAPI render interfaces, and the
WINAPI SwapBuffers/wglSwapBuffers), so on 32-bit both sides cleaned the
stack -> ESP imbalance -> Run-Time Check Failure #0 and an instant crash.
On x64 every convention collapses to one, so it only bit 32-bit games:
Slaps and Beans (Unity/Rewired, 32-bit D3D11) froze the moment the Present
hook ran. The user's "crashes as soon as a button is pressed" was the
Present, not the button.

Switch every __stdcall trampoline call to SafetyHook's stdcall() (a no-op
on x64). The XInput/focus hooks were unaffected because they never call
the trampoline -- they return synthesized data.

Reproduction + regression coverage:
- tools/input_probe (coop_input_probe): injects, reports a connected pad,
  toggles a button, and takes a disable_mask to bisect which subsystem
  affects a game. Isolated the freeze to the video subsystem live.
- hook_selftest_x86 + present_hook_test_x86: the x86 sub-build now builds
  and runs these (the x64 present_hook_test can't see a one-convention
  bug). present_hook_test_x86 drives a real swapchain through the
  trampoline -- it would hit RTC #0 before this fix.
- hook_selftest strengthened to exercise every loaded xinput DLL's full
  export set (GetState, ordinal-100 GetStateEx, GetCapabilities, rumble
  SetState) and to dump the SharedBlock layout.
- protocol.hpp: static_asserts lock the cross-bitness front-of-block
  offsets (verified byte-identical on x86 and x64).

README roadmap trimmed (this milestone done) and a lessons-learned note
added on the call()/stdcall() convention trap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 20:07:17 +02:00
9977ad5d5a Add OpenGL capture path for the hooked video mirror
The Present hook never fired in Phantom Brave because it's an OpenGL game
(OPENGL32.dll loaded, IDXGISwapChain::Present calls=0), so the hooked video source
showed no image. Add an OpenGL producer under the video subsystem: inline-hook
gdi32!SwapBuffers + opengl32!wglSwapBuffers (with a re-entrancy guard, since
SwapBuffers calls wglSwapBuffers), glReadPixels the backbuffer, flip it, and upload
it into the same shared keyed-mutex texture the host already samples -- so the host
is unchanged. DXGI games still hit the Present hook; both producers are installed
and whichever the game uses fills the texture.

Validated by opengl_hook_test (real GL context, clears to a known color, reads the
exact pixels back through the shared texture) and against Phantom Brave (SwapBuffers
~75/s, present=0, shared texture 1920x1080, generation advancing). Vulkan
(vkQueuePresentKHR) still needs WGC -- documented. All 7 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 14:33:47 +02:00