Fix Vulkan capture perf collapse: read back off the present thread

Against Sphere Spectacle (144 FPS, runs without Steam) the implicit-layer
capture dropped the game to ~3 FPS. Measured cause (per-stage trace in the
layer): the read-back ran on the game's PRESENT THREAD and spent ~370 ms per
1080p frame -- not the GPU copy (~2 ms) but the CPU swizzle, because the staging
buffer was a plain HOST_VISIBLE|HOST_COHERENT type (write-combined / uncached on
a discrete GPU), where a scattered CPU read runs at PCIe latency. 3 captures/s =
the 3 FPS the user saw.

Test-first: tests/vk_capture_perf_test reproduces the stall as a deterministic
unit test (372 ms/present, ratio 1.0 -> FAIL via `--sync`), then proves the fix
(0.02 ms/present, byte-correct BGRA->RGBA, ratio ~0 -> PASS).

Fix: extract the near-identical read-back from vk_hook.cpp and coop_vk_layer.cpp
into one shared coop::hook::VkCapture that:
  * has the present thread only record + submit the copy (sub-ms) and return;
  * runs a dedicated reaper thread for the fence wait + swizzle + D3D upload, off
    the critical path, with a ring of in-flight slots (game never waits);
  * allocates HOST_CACHED staging (fast CPU read), invalidating when non-coherent;
  * throttles capture to ~150 Hz (a guest stream is <= the host refresh; no point
    mirroring an uncapped 400+ FPS game and burning reaper CPU).

Real-game A/B: present rate now matches the no-capture baseline (605->470 vs
593->405 over the same ramp) with the mirror at ~130 fps -- no measurable impact.

Also adds present-thread overhead guards to the other GPU backends' hook tests
(present_overhead.hpp): DX11 0.05 ms, DX12 0.34 ms, OpenGL 0.09 ms overhead, all
asserted < one 60 Hz frame, so any future synchronous-stall regression fails.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-23 08:19:47 +02:00
parent 60957775b7
commit 304857dcf0
13 changed files with 1564 additions and 799 deletions

View File

@@ -110,11 +110,6 @@ default** and covers anything the hooked path doesn't.
### Current tasks
- **Vulkan capture has a catastrophic performance cost.** Against a real game (Sphere Spectacle,
144 FPS uncapped, no focus throttling) the implicit-layer capture path drops the game to ~3 FPS —
unplayable, which makes Vulkan support meaningless. The capture stalls the game's present thread
every frame; it must be made (near-)free so the game holds its frame rate while mirroring. Applies
to the shared read-back design in both `vk_layer/coop_vk_layer.cpp` and `hook/src/vk_hook.cpp`.
- **The inline-hook (suspended-inject) Vulkan path must work, or be proven a true limitation.**
Sphere Spectacle does *not* require Steam — the exe can be launched directly (and suspended, with
`coop_hook.dll` injected before it runs any code). Either make the early inline `vk_hook` capture
@@ -126,10 +121,14 @@ default** and covers anything the hooked path doesn't.
The earlier tracked tasks are complete: injection hardening (the cross-backend safe-unhook drain),
two-path audio-format correlation (rate + channels/bit-depth recovery), mouse + keyboard forwarding
for DirectInput and Raw Input games, and real-game Vulkan validation (`coop_vk_validate` against
Sphere Spectacle). See **Lessons learned** and the test suite for each. Open directions: per-game
profiles, multi-guest virtual-pad mapping, and continuous raw-mouse *movement* forwarding (the MKB
event stream is position-based today).
for DirectInput and Raw Input games, real-game Vulkan validation (`coop_vk_validate` against Sphere
Spectacle), and the **Vulkan capture performance fix** — the read-back that dropped a 144 FPS game
to ~3 FPS now runs off the present thread (shared `coop::hook::VkCapture`), so the game keeps its
frame rate while mirroring (guarded by `vk_capture_perf_test`; every GPU backend's hook test also
asserts a present-thread overhead bound). See **Lessons learned** and the test suite for each. Open
directions: per-game profiles, multi-guest
virtual-pad mapping, and continuous raw-mouse *movement* forwarding (the MKB event stream is
position-based today).
## Building