Fix 32-bit game crash: call hooked __stdcall functions with stdcall()

SafetyHook's InlineHook::call() invokes the trampoline through a __cdecl
pointer (the compiler default on x86). The functions we hook are __stdcall
(IDXGISwapChain::Present/Present1, the WASAPI render interfaces, and the
WINAPI SwapBuffers/wglSwapBuffers), so on 32-bit both sides cleaned the
stack -> ESP imbalance -> Run-Time Check Failure #0 and an instant crash.
On x64 every convention collapses to one, so it only bit 32-bit games:
Slaps and Beans (Unity/Rewired, 32-bit D3D11) froze the moment the Present
hook ran. The user's "crashes as soon as a button is pressed" was the
Present, not the button.

Switch every __stdcall trampoline call to SafetyHook's stdcall() (a no-op
on x64). The XInput/focus hooks were unaffected because they never call
the trampoline -- they return synthesized data.

Reproduction + regression coverage:
- tools/input_probe (coop_input_probe): injects, reports a connected pad,
  toggles a button, and takes a disable_mask to bisect which subsystem
  affects a game. Isolated the freeze to the video subsystem live.
- hook_selftest_x86 + present_hook_test_x86: the x86 sub-build now builds
  and runs these (the x64 present_hook_test can't see a one-convention
  bug). present_hook_test_x86 drives a real swapchain through the
  trampoline -- it would hit RTC #0 before this fix.
- hook_selftest strengthened to exercise every loaded xinput DLL's full
  export set (GetState, ordinal-100 GetStateEx, GetCapabilities, rumble
  SetState) and to dump the SharedBlock layout.
- protocol.hpp: static_asserts lock the cross-bitness front-of-block
  offsets (verified byte-identical on x86 and x64).

README roadmap trimmed (this milestone done) and a lessons-learned note
added on the call()/stdcall() convention trap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-20 20:07:17 +02:00
parent 9977ad5d5a
commit 435ab9d30f
9 changed files with 528 additions and 164 deletions

View File

@@ -4,6 +4,7 @@
#pragma once
#include <atomic>
#include <cstddef>
#include <cstdint>
namespace coop
@@ -184,6 +185,15 @@ static_assert(std::atomic<std::uint32_t>::is_always_lock_free,
static_assert(std::atomic<std::uint64_t>::is_always_lock_free,
"status counters need a lock-free 64-bit atomic for cross-process use");
// The x64 host and the x86 hook map this same block, so its layout must be
// byte-identical across bitness. These offsets (verified equal on both arches)
// lock the front of the block -- the seqlock + pad state the input hot path reads;
// a future field reorder that diverges between x86 and x64 fails to compile on the
// arch that disagrees. (Fixed-width POD + no pointers is what keeps it stable.)
static_assert(offsetof(SharedBlock, sequence) == 12, "cross-bitness: sequence offset moved");
static_assert(offsetof(SharedBlock, pads) == 16, "cross-bitness: pad-state offset moved");
static_assert(offsetof(SharedBlock, status) == 96, "cross-bitness: status offset moved");
// --- Seqlock helpers -------------------------------------------------------
// Writer side: publish a fresh set of pad states. Called from the host.