Harden hook removal: disable -> drain -> destroy, and drain settles first

Uncapping the mock game (next commit) turned mock_game_test's hook/unhook storm
into a real stress test (thousands of presents/s instead of tens), which reliably
crashed the game on remove (0xC0000005) for dx9/dx11/dx12. Two races the slow
vsync'd mock had masked:

1. Trampoline use-after-free. remove_*_hooks did `hook = {}` (destroy) BEFORE the
   DetourGate drain. Destroying a SafetyHook InlineHook frees its trampoline
   immediately, but an in-flight detour about to call the original via .stdcall()
   (the trampoline) then used freed memory. Fix: disable() first (restores the
   original bytes under thread suspension, but KEEPS the trampoline alive) -> drain
   -> only then destroy. Applied to present/d3d9/opengl/vk/xinput/mkb/focus.

2. Entry-window race in DetourGate::drain(). It returned the instant the active
   count read zero, but a thread can be inside the detour yet not have reached its
   Guard constructor (the prologue is unguarded), so the count reads zero while a
   detour is about to run -- and the freed state is then used. Fix: Sleep(1) BEFORE
   each zero-check; with the hook disabled no new detour starts, so any
   already-entered thread registers within that window. This alone fixed dx11 (the
   highest present rate, ~11000/s, which hit the window every storm).

Audio is unaffected (it uses vtable swaps, which keep a real original pointer, not
a trampoline). Full suite 21/21, and the storm now survives on every backend.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-23 09:44:42 +02:00
parent c480dfe152
commit 958c355126
8 changed files with 79 additions and 25 deletions

View File

@@ -366,11 +366,15 @@ bool install_d3d9_hooks(IpcClient& ipc)
void remove_d3d9_hooks() void remove_d3d9_hooks()
{ {
// Reset the hook first (restores Present's bytes under thread suspension -> no new detour), // DISABLE (not destroy) first: restores Present's bytes under thread suspension (no new detour)
// then drain any Present detour still in-flight before freeing the D3D state it reads (UAF). // but keeps the trampoline alive, so an in-flight detour about to call g_hk_present9.stdcall()
g_hk_present9 = {}; // (the trampoline) doesn't have it freed under it. Destroying (= {}) before the drain frees the
// trampoline immediately -- a UAF the uncapped mock-game storm (thousands of presents/s) hits
// reliably (0xC0000005). Disable -> drain -> only then destroy.
(void)g_hk_present9.disable();
hook_set_installed(g_id_present9, false); hook_set_installed(g_id_present9, false);
g_gate.drain(); g_gate.drain();
g_hk_present9 = {}; // no detour in-flight or able to start now -> safe to free the trampoline
release_shared(); release_shared();
release_sysmem(); release_sysmem();
if (g_ctx != nullptr) if (g_ctx != nullptr)

View File

@@ -289,9 +289,17 @@ void remove_focus_spoof()
SetWindowLongPtrA(g_game_hwnd, GWLP_WNDPROC, reinterpret_cast<LONG_PTR>(g_orig_proc)); SetWindowLongPtrA(g_game_hwnd, GWLP_WNDPROC, reinterpret_cast<LONG_PTR>(g_orig_proc));
} }
} }
g_focus_hooks.clear(); // Disable (restore original bytes) the inline focus hooks first so no new detour starts, but KEEP
g_hk_clipcursor = {}; // restore ClipCursor / SetCursorPos before clearing state // the trampolines alive for any in-flight detour calling its trampoline; destroy only after the
g_hk_setcursorpos = {}; // drain. Destroying before the drain frees a trampoline under a detour about to call it (the same
// UAF class as the present-storm crash). The WNDPROC subclass uses a saved g_orig_proc pointer
// (restored above), not a trampoline, so it has nothing to free early.
for (auto& h : g_focus_hooks)
{
(void)h.disable();
}
(void)g_hk_clipcursor.disable();
(void)g_hk_setcursorpos.disable();
ClipCursor(nullptr); // leave the cursor free when the spoof is removed ClipCursor(nullptr); // leave the cursor free when the spoof is removed
hook_set_installed(g_id_foreground, false); hook_set_installed(g_id_foreground, false);
hook_set_installed(g_id_active, false); hook_set_installed(g_id_active, false);
@@ -299,11 +307,14 @@ void remove_focus_spoof()
hook_set_installed(g_id_wndproc, false); hook_set_installed(g_id_wndproc, false);
hook_set_installed(g_id_clipcursor, false); hook_set_installed(g_id_clipcursor, false);
hook_set_installed(g_id_setcursorpos, false); hook_set_installed(g_id_setcursorpos, false);
// The WNDPROC is restored and the inline hooks reset above, so no NEW detour can start. Drain // The WNDPROC is restored and the inline hooks disabled above, so no NEW detour can start. Drain
// any focus / WNDPROC detour still in-flight on the game's window thread before nulling the // any focus / WNDPROC detour still in-flight on the game's window thread before nulling the
// state they read (g_orig_proc / g_game_hwnd / g_focus_ipc) -- otherwise a dispatch mid-flight // state they read (g_orig_proc / g_game_hwnd / g_focus_ipc) -- otherwise a dispatch mid-flight
// could call a null original WNDPROC or a dangling IPC pointer. // could call a null original WNDPROC or a dangling IPC pointer.
g_gate.drain(); g_gate.drain();
g_focus_hooks.clear(); // no detour in-flight or able to start now -> safe to free the trampolines
g_hk_clipcursor = {};
g_hk_setcursorpos = {};
if (g_focus_ipc != nullptr) if (g_focus_ipc != nullptr)
{ {
g_focus_ipc->mark_focus_spoof(false, 0); g_focus_ipc->mark_focus_spoof(false, 0);

View File

@@ -53,15 +53,23 @@ public:
// Wait (bounded ~400 ms) for all in-flight detours to finish. Call AFTER the hook is // Wait (bounded ~400 ms) for all in-flight detours to finish. Call AFTER the hook is
// restored / the gate is closed, so no new detour can start -- otherwise this may never // restored / the gate is closed, so no new detour can start -- otherwise this may never
// reach zero on a busy render thread. // reach zero on a busy render thread.
//
// We Sleep(1) *before* each zero-check (not after) to close an entry-window race: a thread can
// have already jumped into the detour but not yet reached its Guard constructor (the few-
// instruction prologue is unguarded), so m_active reads 0 even though a detour is about to run.
// Returning then would free the state out from under it. With the hook disabled no NEW detour can
// start, so any such thread reaches its Guard within nanoseconds; a 1 ms settle before concluding
// "zero" lets it register. Without this, a backend presenting at thousands/s (the uncapped
// mock-game storm) reliably crashed on remove (0xC0000005); with it, the count is accurate.
void drain() void drain()
{ {
for (int spins = 0; spins < 400; ++spins) for (int spins = 0; spins < 400; ++spins)
{ {
Sleep(1);
if (m_active.load(std::memory_order_acquire) == 0) if (m_active.load(std::memory_order_acquire) == 0)
{ {
return; return;
} }
Sleep(1);
} }
} }

View File

@@ -514,12 +514,21 @@ void remove_mkb_hooks()
return; return;
} }
g_active.store(false, std::memory_order_release); g_active.store(false, std::memory_order_release);
g_hk_async = {}; // InlineHook destructor restores the original bytes -> no new detour starts // Disable (restore original bytes) the inline hooks first so no new detour starts, but KEEP the
g_hk_kbstate = {}; // trampolines alive for any in-flight detour calling its trampoline; destroy only after the
g_hk_cursor = {}; // drain. Destroying before the drain frees the trampoline under a detour about to call it -- the
g_hk_getrawinputdata = {}; // restore GetRawInputData // same UAF class as the present-storm crash. (The DI hook is a vtable swap: remove() restores the
// slot and keeps m_original valid, so it has no trampoline to free early.)
(void)g_hk_async.disable();
(void)g_hk_kbstate.disable();
(void)g_hk_cursor.disable();
(void)g_hk_getrawinputdata.disable();
g_vh_di_getstate.remove(); // restore the DI GetDeviceState slot (probe kept alive for re-enable) g_vh_di_getstate.remove(); // restore the DI GetDeviceState slot (probe kept alive for re-enable)
g_gate.drain(); // wait for any in-flight polling / DI / raw detour before clearing state g_gate.drain(); // wait for any in-flight polling / DI / raw detour before clearing state
g_hk_async = {}; // no detour in-flight or able to start now -> safe to free the trampolines
g_hk_kbstate = {};
g_hk_cursor = {};
g_hk_getrawinputdata = {};
hook_set_installed(g_id_di_getstate, false); hook_set_installed(g_id_di_getstate, false);
hook_set_installed(g_id_rawinput, false); hook_set_installed(g_id_rawinput, false);
for (int vk = 0; vk < 256; ++vk) for (int vk = 0; vk < 256; ++vk)

View File

@@ -322,14 +322,18 @@ bool install_opengl_hooks(IpcClient& ipc)
void remove_opengl_hooks() void remove_opengl_hooks()
{ {
// Reset the hooks first (restores the original SwapBuffers bytes under thread suspension, so // DISABLE (not destroy) first: restores the SwapBuffers bytes under thread suspension (no new
// no NEW detour starts), then drain any swap detour still in-flight on the render thread // detour) but keeps the trampolines alive, so an in-flight detour about to call .stdcall() (the
// BEFORE freeing the D3D state it reads -- otherwise the detour uses freed memory (UAF). // trampoline) doesn't have it freed under it. Destroying (= {}) before the drain frees the
g_hk_swapbuffers = {}; // trampoline immediately -- a UAF the uncapped mock-game storm (thousands of swaps/s) can hit.
g_hk_wglswap = {}; // Disable -> drain -> only then destroy.
(void)g_hk_swapbuffers.disable();
(void)g_hk_wglswap.disable();
hook_set_installed(g_id_swapbuffers, false); hook_set_installed(g_id_swapbuffers, false);
hook_set_installed(g_id_wglswap, false); hook_set_installed(g_id_wglswap, false);
g_gate.drain(); g_gate.drain();
g_hk_swapbuffers = {}; // no detour in-flight or able to start now -> safe to free the trampolines
g_hk_wglswap = {};
release_shared(); release_shared();
if (g_ctx != nullptr) if (g_ctx != nullptr)
{ {

View File

@@ -994,16 +994,22 @@ bool install_present_hooks(IpcClient& ipc)
void remove_present_hooks() void remove_present_hooks()
{ {
// Reset the hooks first (restores Present/Present1/ECL bytes under thread suspension, so no // DISABLE (not destroy) the hooks first: this restores the Present/Present1/ECL bytes under
// NEW detour starts), then drain any detour still in-flight on the render thread BEFORE // thread suspension so no NEW detour starts, but KEEPS the trampolines alive -- an in-flight
// freeing the shared texture / On12 bridge / present queue it reads -- otherwise UAF. // detour about to call g_hk_present.stdcall() (the trampoline) must not have it freed under it.
g_hk_present = {}; // Destroying here (= {}) frees the trampoline immediately; at a few hundred presents/s that race
g_hk_present1 = {}; // was rarely hit, but the uncapped mock-game storm (thousands/s) hits it reliably (0xC0000005).
g_hk_ecl = {}; // So: disable -> drain (in-flight detours finish on the live trampoline) -> only THEN destroy.
(void)g_hk_present.disable();
(void)g_hk_present1.disable();
(void)g_hk_ecl.disable();
hook_set_installed(g_id_present, false); hook_set_installed(g_id_present, false);
hook_set_installed(g_id_present1, false); hook_set_installed(g_id_present1, false);
hook_set_installed(g_id_ecl, false); hook_set_installed(g_id_ecl, false);
g_gate.drain(); g_gate.drain();
g_hk_present = {}; // no detour is in-flight or can start now -> freeing the trampoline is safe
g_hk_present1 = {};
g_hk_ecl = {};
g_present_queue.store(nullptr, std::memory_order_relaxed); g_present_queue.store(nullptr, std::memory_order_relaxed);
g_logged_presents_n = 0; // let a fresh injection re-log the present pattern g_logged_presents_n = 0; // let a fresh injection re-log the present pattern
g_logged_swapchains_n = 0; g_logged_swapchains_n = 0;

View File

@@ -334,8 +334,12 @@ void remove_vk_hooks()
// game's thread BEFORE shutting the capture down. The game keeps calling our cached present // game's thread BEFORE shutting the capture down. The game keeps calling our cached present
// detour, but with the gate closed it now passes straight through to the real present. // detour, but with the gate closed it now passes straight through to the real present.
g_capture_enabled.store(false, std::memory_order_release); g_capture_enabled.store(false, std::memory_order_release);
g_hk_gipa = {}; // Disable (not destroy) the GIPA hook first so the trampoline stays alive for any in-flight
// resolution detour calling real_gipa() (the trampoline); destroy only after the drain. (The
// present detour already passes through to the saved g_real_present, so it's unaffected.)
(void)g_hk_gipa.disable();
g_gate.drain(); g_gate.drain();
g_hk_gipa = {};
g_cap.shutdown(); // joins the reaper, drains the device, frees the read-back resources g_cap.shutdown(); // joins the reaper, drains the device, frees the read-back resources

View File

@@ -234,12 +234,20 @@ bool install_xinput_hooks(IpcClient& ipc)
void remove_xinput_hooks() void remove_xinput_hooks()
{ {
g_hooks.clear(); // InlineHook destructor restores the original bytes -> no new detour starts // Disable (restore original bytes) first so no new detour starts, but KEEP the trampolines alive
// for any in-flight detour calling its trampoline; destroy only after the drain. Clearing the
// vector here would free the trampolines immediately -- a UAF a game polling XInput at a high
// rate could hit (the same class as the mock-game present-storm crash).
for (auto& h : g_hooks)
{
(void)h.disable();
}
hook_set_installed(g_id_getstate, false); hook_set_installed(g_id_getstate, false);
hook_set_installed(g_id_getstateex, false); hook_set_installed(g_id_getstateex, false);
hook_set_installed(g_id_getcaps, false); hook_set_installed(g_id_getcaps, false);
hook_set_installed(g_id_setstate, false); hook_set_installed(g_id_setstate, false);
g_gate.drain(); // wait for any in-flight detour before nulling the IPC pointer it reads g_gate.drain(); // wait for any in-flight detour before nulling the IPC pointer it reads
g_hooks.clear(); // no detour in-flight or able to start now -> safe to free the trampolines
if (g_ipc != nullptr) if (g_ipc != nullptr)
{ {
g_ipc->mark_detached(); g_ipc->mark_detached();