Deduplicate the hook DLL and scrub history from its comments

Consolidate four copies of the keyed-mutex shared-texture setup
(present/opengl/d3d9/vk_capture) into one RAII SharedVideoTexture,
two copies of find_main_window into find_window.hpp, audio_hook's
hand-rolled detour guard into the shared DetourGate, the duplicated
vtable_method into vtable_hook.hpp, and the near-identical
Present/Present1 and SwapBuffers/wglSwapBuffers detour pairs into one
shared body each. The vk_layer and vk_capture_perf_test targets now
compile debug_log.cpp since the shared texture code logs.

Comments no longer narrate the past: drop stress-test/game anecdotes,
"used to"/"the old model" phrasing, plan-step labels, and pointers to
docs that do not exist; fix present_hook.hpp/opengl_hook.hpp claims
that predate the D3D12/D3D9/Vulkan backends. Net -266 lines, no
behavior change (full x64 + x86 suites pass, including the mock-game
hook/unhook storm).
This commit is contained in:
2026-07-12 08:53:58 +02:00
parent 05039ab104
commit 635ef51283
22 changed files with 358 additions and 624 deletions

View File

@@ -1,17 +1,17 @@
// Safe INSTALL of a SafetyHook inline hook -- the symmetric partner of hook_guard.hpp's safe removal.
//
// Two problems this avoids, both surfaced by mock_game_test's hook/unhook storm (a game polling a
// hooked API at thousands/s while the subsystem is toggled on/off every 60 ms):
// Two problems this avoids, both hit when a game polls a hooked API at thousands of calls/s while
// the subsystem is toggled on/off (mock_game_test's hook/unhook storm exercises exactly that):
//
// 1. Enable-before-assign. create_inline() builds the hook AND enables it (patches the target's bytes
// to jump to the detour), THEN the result is move-assigned into the global the detour reads to
// reach the trampoline. A call landing in the detour during that assign reads a torn global -> AV.
// So we create StartDisabled, let dst be populated, and only THEN enable().
//
// 2. Trampoline use-after-free on re-install. The old model recreated the hook on every install and
// DESTROYED it on every remove (= {}), which frees the trampoline. A detour that has entered but
// not yet reached its DetourGate::Guard (the unguarded prologue) can then call a freed trampoline
// -> AV at a garbage address. drain() narrows that window but can't fully close it under heavy
// 2. Trampoline use-after-free on re-install. Recreating the hook on every install and destroying
// it on every remove (= {}) frees the trampoline. A detour that has entered but not yet reached
// its DetourGate::Guard (the unguarded prologue) can then call a freed trampoline -> AV at a
// garbage address. drain() narrows that window but can't fully close it under heavy
// preemption. So instead we treat hooks as PERSISTENT: create each once and thereafter only
// enable()/disable() it across install/remove cycles. The trampoline is allocated once and never
// freed during the session, so a stale detour always calls a live trampoline (which, when