BlackMark/CoopAllTheThings
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Harden every hook against the install/remove use-after-free

Browse Source 
Spamming a subsystem toggle (the "Mirror video" button) could crash the game:
remove_*_hooks freed a hook's shared D3D / Vulkan / IPC state immediately, while a
capture detour was still mid-flight on the game's render thread -> use-after-free.
Only the audio hooks had the safe-unhook drain; the video (Present/D3D9/D3D10/GL/
Vulkan) and XInput/focus/MKB hooks did not.

Test-first: mock_game_test now runs an aggressive hook/unhook storm -- a separate
thread thrashes every subsystem on/off while the game presents, across all backends.
It crashed gl + vk (0xC0000005) and failed dx9 capture-resume before the fix.

Fix (hook/src/hook_guard.hpp, DetourGate): each detour wraps its body in an RAII
active-count Guard; remove_* restores the hook first (so no new detour starts),
drains the in-flight detours to zero, and only then frees the shared state. Vulkan
is special-cased -- the game caches hk_vkQueuePresentKHR, so removal closes an
atomic capture gate (detours then pass through to the real present), drains, then
frees the read-back resources. Storm now passes on every backend.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
BlackMark
2026-06-23 01:56:36 +02:00
parent 79399e88f1
commit 21c15b162b
10 changed files with 365 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
hook/src/xinput_hook.cpp
View File

@@ -9,6 +9,7 @@
#include <safetyhook.hpp>
#include "hook_guard.hpp"
#include "hook_registry.hpp"
namespace coop::hook
@@ -17,6 +18,8 @@ namespace coop::hook
namespace
{
DetourGate g_gate; // drains in-flight XInput detours before remove nulls the IPC pointer
// XInput guide-button bit, reported only by the undocumented ordinal-100
// XInputGetStateEx that many games use. Mirrors how Steam/x360ce expose it.
constexpr std::uint16_t kGuideButton = 0x0400;
@@ -99,18 +102,21 @@ DWORD query_state(DWORD user_index, XINPUT_STATE* state, bool keep_guide)
DWORD WINAPI hk_XInputGetState(DWORD user_index, XINPUT_STATE* state)
{
DetourGate::Guard guard(g_gate); // keep g_ipc valid for this whole detour
hook_note_call(g_id_getstate);
return query_state(user_index, state, /*keep_guide=*/false);
}
DWORD WINAPI hk_XInputGetStateEx(DWORD user_index, XINPUT_STATE* state)
{
DetourGate::Guard guard(g_gate); // keep g_ipc valid for this whole detour
hook_note_call(g_id_getstateex);
return query_state(user_index, state, /*keep_guide=*/true);
}
DWORD WINAPI hk_XInputGetCapabilities(DWORD user_index, DWORD /*flags*/, XINPUT_CAPABILITIES* caps)
{
DetourGate::Guard guard(g_gate); // keep g_ipc valid for this whole detour
hook_note_call(g_id_getcaps);
if (caps == nullptr || user_index >= kMaxPads)
{
@@ -147,6 +153,7 @@ DWORD WINAPI hk_XInputGetCapabilities(DWORD user_index, DWORD /*flags*/, XINPUT_
// Still report success so the game's logic is happy.
DWORD WINAPI hk_XInputSetState(DWORD user_index, XINPUT_VIBRATION* vibration)
{
DetourGate::Guard guard(g_gate); // keep g_ipc valid for this whole detour
hook_note_call(g_id_setstate);
if (user_index >= kMaxPads || !g_cache[user_index].connected)
{
@@ -227,11 +234,12 @@ bool install_xinput_hooks(IpcClient& ipc)
void remove_xinput_hooks()
{
g_hooks.clear(); // InlineHook destructor restores the original bytes
g_hooks.clear(); // InlineHook destructor restores the original bytes -> no new detour starts
hook_set_installed(g_id_getstate, false);
hook_set_installed(g_id_getstateex, false);
hook_set_installed(g_id_getcaps, false);
hook_set_installed(g_id_setstate, false);
g_gate.drain(); // wait for any in-flight detour before nulling the IPC pointer it reads
if (g_ipc != nullptr)
{
g_ipc->mark_detached();