BlackMark 4985239222 Fix 32-bit FMOD audio crash: hook WASAPI COM methods via vtable swap ...

The stdcall() fix stopped the Present-hook crash but 32-bit games (Slaps
and Beans, FMOD) still crashed the instant audio init ran through the
hook. Root cause: SafetyHook's inline hook relocates the target's
overwritten prologue into a trampoline, but MMDevApi/AudioSes COM methods
on x86 open with `push ebp; mov ebp,esp; and esp,-8` (dynamic stack
alignment) and read arguments EBP-relative. The relocated copy leaves EBP
wrong, so the original runs with garbage arguments and faults (AV writing
*ppInterface inside CEndpointDevice::Activate+0x3d).

Switch all five WASAPI COM hooks (IMMDevice::Activate, IAudioClient::
Initialize/GetService, IAudioRenderClient::GetBuffer/ReleaseBuffer) from
safetyhook::create_inline to a small VtableHook helper: VirtualProtect the
shared vtable slot, overwrite the function pointer, call the saved original
directly. No code patching, no trampoline, pristine stack regardless of
prologue. One swap covers every instance (a coclass shares one vtable), so
the existing shared-vtable strategy is preserved. Inline hooking stays for
Present/SwapBuffers, whose prologues relocate cleanly.

Reproduced in-process with a new x86 build of the audio render-hook test
(audio_hook_test_x86): it installs the hooks, then drives a fresh
IAudioClient through them and renders -- segfaulted before, passes now.
The x64 audio_hook_test passes regardless of the bug, so the 32-bit build
is the regression guard.

ctest: x64 7/7, x86 3/3.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-06-20 20:39:29 +02:00

6.1 KiB

Raw Blame History

View Raw