Docs: Vulkan hooked capture in architecture + lessons; scope the opt-in layer

Architecture now lists Vulkan as a hooked producer (GPA interception + read-back,
early-presence required). Adds a lessons-learned bullet (can't late-hook Vulkan;
the present-semaphore re-chaining trap; the too-late heuristic; early-load
testing). mock_game_test docs note the suspended-launch capture + too-late check.
The Vulkan milestone is trimmed to its one remaining piece -- the opt-in implicit
layer for immediate-init games -- with the capture/banner/best-effort marked done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-22 12:56:26 +02:00
parent 894285459c
commit 36ccdcc3ad

101
README.md
View File

@@ -44,9 +44,15 @@ read-backs the backbuffer with `GetRenderTargetData` (a D3D9 surface isn't D3D11
swizzling BGRA→RGBA and uploading via a hook-owned D3D11 device — one path covers both plain swizzling BGRA→RGBA and uploading via a hook-owned D3D11 device — one path covers both plain
D3D9 and D3D9Ex; **OpenGL** hooks `SwapBuffers` / `wglSwapBuffers` and reads the D3D9 and D3D9Ex; **OpenGL** hooks `SwapBuffers` / `wglSwapBuffers` and reads the
backbuffer with `glReadPixels` (for games that never touch DXGI, e.g. Phantom backbuffer with `glReadPixels` (for games that never touch DXGI, e.g. Phantom
Brave). The host samples the copy as plain UNORM (`srgb_to_unorm`) so Brave); and **Vulkan** inline-hooks the `vulkan-1.dll` `vkGetInstanceProcAddr` export to
`*_SRGB`-backbuffer games mirror at correct brightness. **WGC remains the default** intercept the resolution chain (`vkCreateInstance` / `vkCreateDevice` /
and covers anything the hooked path doesn't (Vulkan — see Roadmap). `vkCreateSwapchainKHR` / `vkQueuePresentKHR`) and reads the presented image back with
`vkCmdCopyImageToBuffer` — but only when the hook is present **before** the game initializes
Vulkan (it caches its present pointer at init), so the Vulkan path needs early presence via
Auto-attach (and an opt-in implicit layer — see Roadmap); a too-late attach shows a red
relaunch banner and falls back to WGC. The host samples the copy as plain UNORM
(`srgb_to_unorm`) so `*_SRGB`-backbuffer games mirror at correct brightness. **WGC remains the
default** and covers anything the hooked path doesn't.
## Limitations ## Limitations
@@ -147,38 +153,24 @@ Conventions for every milestone below:
nothing clipped/overlapping, colours/labels correct. A green unit test is not a substitute nothing clipped/overlapping, colours/labels correct. A green unit test is not a substitute
for looking at the product; this real-screenshot check is required, not optional. for looking at the product; this real-screenshot check is required, not optional.
- **M2 — Vulkan capture.** The Vulkan **mock backend is done** (`render_vk.cpp`, selectable `vk`): - **M2 — Vulkan: opt-in reliability layer (the one piece left).** The Vulkan **mock backend and
volk + Vulkan-Headers submodules, a real instance / device / swap chain, and a per-frame hooked capture are done and tested**: the hook inline-hooks the `vulkan-1.dll`
`vkCmdClearColorImage` of the frame-counter colour (no pipeline/shaders → **no SPIR-V `vkGetInstanceProcAddr` export and intercepts the resolution chain (`vkCreateInstance` /
toolchain**); it loads Vulkan via **volk** on purpose (the loader-bypass case the capture hook `vkCreateDevice` / `vkCreateSwapchainKHR` / `vkQueuePresentKHR`), reads the presented image back
must handle). `mock_game_test` runs a **liveness** check on it — its present pointer is cached with `vkCmdCopyImageToBuffer`, and re-chains the present's wait semaphores so the copy orders
at init, so late injection can't hook it, which is exactly why the capture below is the hard after rendering. `mock_game_test` decodes Vulkan frames through it (via a suspended-launch +
part. The remaining **capture hook** (Vulkan games mirror via **WGC** meanwhile — the default): early-load path) and verifies the **too-late detection**; the host shows the **red relaunch
1. **Present interception (needs early presence).** Hook `vkGetInstanceProcAddr` / banner** (verified live). **Best-effort auto-attach** already works (the existing poll-and-inject
`vkGetDeviceProcAddr` (the `vulkan-1.dll` exports) so a volk-using app resolves *our* re-injects on relaunch, the vk hook retries until `vulkan-1.dll` loads), enough for games that
wrappers, and intercept `vkCreateDevice` / `vkCreateSwapchainKHR` to capture the don't initialize Vulkan instantly. Remaining:
device/queue/images/format. Read the presented image back with `vkCmdCopyImageToBuffer` into - **Opt-in implicit Vulkan layer** for games that init Vulkan *immediately* (where even a
a host-visible buffer (the same read-back pattern as D3D10/D3D9/OpenGL — simpler and more relaunch+auto-attach injects too late). A real chain-aware Vulkan layer (`coop_vk_layer.dll`
robust than a `VK_KHR_external_memory_win32` + keyed-mutex blit), swizzle if BGRA, and upload + JSON manifest) the loader loads at `vkCreateInstance` — guaranteed before init — registered
via the hook-owned D3D11 device. **Testing needs the early-load path** (the mock loads the per-user (HKCU, no admin) and scoped to the target image, with an **opt-in** "Set up Vulkan
hook before `vkCreateInstance`, e.g. via an env-var or the layer); the normal late-inject layer" Injection-panel checkbox that registers/unregisters it (self-deactivating for non-target
`mock_game_test` flow can't catch a Vulkan present. apps so a stale entry after a host crash is harmless). This is a separate component from the
2. **Attach via auto-attach — best-effort by default, opt-in layer for reliability.** By inline-hook capture (layer-chain dispatch + manifest/registry + the checkbox + IPC handshake);
default, arming Auto-attach for a detected Vulkan game injects as early as possible on until it lands, the banner directs immediate-init Vulkan games to WGC.
relaunch (the existing poll-and-inject) — enough for games that initialize Vulkan a little
into startup. For games that init Vulkan immediately, an **opt-in** Injection-panel checkbox
("Set up Vulkan layer") registers a per-user (HKCU, no admin) implicit Vulkan layer (thin
`coop_vk_layer.dll` + JSON manifest) scoped to that game's image name; on relaunch the loader
loads it at `vkCreateInstance` — guaranteed before init — and it wires up present capture +
IPC. The layer is removed when the option is unticked or the host exits, and self-deactivates
for any non-target app, so a stale registration (e.g. after a host crash) is harmless.
3. **Too-late detection + red banner.** A late-injected hook that finds `vulkan-1.dll` loaded
but no working hooked-present reports a *"Vulkan, injected too late"* status; the host
overlays a **red** banner on the mirror window: "Detected a Vulkan game — the hook attached
too late to mirror it with low latency. Enable Auto-attach (and, if it persists, 'Set up
Vulkan layer') and relaunch the game." **WGC keeps mirroring meanwhile** so the session stays
usable; the banner clears once the hooked path goes live. This is new end-user UI, so it must
keep M1's fit test (and the visual inspection) green.
Each of 1–3 is its own commit; test the early/layer path against the mock (decode frames) and Each of 1–3 is its own commit; test the early/layer path against the mock (decode frames) and
the too-late path (assert the status + banner fire). the too-late path (assert the status + banner fire).
@@ -292,17 +284,20 @@ ctest --test-dir build -C Debug --output-on-failure
endpoint format). Skips cleanly if the machine has no audio endpoint. endpoint format). Skips cleanly if the machine has no audio endpoint.
- **`mock_game_test`** — comprehensive capture/audio/hook stress test against - **`mock_game_test`** — comprehensive capture/audio/hook stress test against
**`coop_mock_game`** (an animated, frame-numbered A/V test game under **`coop_mock_game`** (an animated, frame-numbered A/V test game under
[`tools/mock_game`](tools/mock_game) with selectable **DX9 / DX9Ex / DX10 / DX11 / DX12 / OpenGL** backends and a [`tools/mock_game`](tools/mock_game) with selectable **DX9 / DX9Ex / DX10 / DX11 / DX12 /
configurable WASAPI tone). It launches the game, injects `coop_hook.dll`, opens the OpenGL / Vulkan** backends and a configurable WASAPI tone). It launches the game, injects
hook's shared video texture, and **decodes the frame number out of the captured pixels** `coop_hook.dll`, opens the hook's shared video texture, and **decodes the frame number out of
to assert the mirror sees a *monotonic, advancing* sequence for each backend (the bar the captured pixels** to assert the mirror sees a *monotonic, advancing* sequence for each
for no dropped / stale / out-of-order frames — what the DX12 rotating-backbuffer bug backend (the bar for no dropped / stale / out-of-order frames — what the DX12 rotating-backbuffer
broke). It launches the game at several **audio formats** (44100/48000/96000, PCM + bug broke). **Vulkan** is special: since its present pointer is cached at init, the test launches
float) and asserts the hook measures each one's rate through the full inject path, then the mock **suspended**, injects, then resumes (the mock loads Vulkan and waits so the hook arms
injects with audio + video, checks both stream, cycles the audio subsystem off/on first) to decode frames; it also late-injects a Vulkan game and asserts the **too-late** flag
(hook/unhook stress), and confirms the game never crashes and capture resumes. This trips (which drives the host's relaunch banner). It launches the game at several **audio formats**
suite drove out five real audio races (see Lessons learned). Skips cleanly without a (44100/48000/96000, PCM + float) and asserts the hook measures each one's rate through the full
D3D11 device. inject path, then injects with audio + video, checks both stream, cycles the audio subsystem
off/on (hook/unhook stress), and confirms the game never crashes and capture resumes. This suite
drove out five real audio races (see Lessons learned). Skips cleanly without a D3D11 / Vulkan
device.
### Debugging the hooks against a real game ### Debugging the hooks against a real game
@@ -460,6 +455,20 @@ Non-obvious things that cost time and constrain the design:
the frame-counter block at the GL *top* (`y = h - block`) to land at the captured image's the frame-counter block at the GL *top* (`y = h - block`) to land at the captured image's
top-left. The GL `SwapBuffers` hook now also bumps the shared present counter (it's the GL top-left. The GL `SwapBuffers` hook now also bumps the shared present counter (it's the GL
present), so the Video panel's present rate works for GL games too. present), so the Video panel's present rate works for GL games too.
- **Vulkan can't be late-hooked, and capturing its present needs semaphore surgery.** Vulkan
games cache `vkQueuePresentKHR` at init (often via volk, which bypasses the loader trampoline),
so late injection misses it — the hook must be in **before `vkCreateInstance`**. Catch the
resolution chain instead: inline-hook the `vulkan-1.dll` `vkGetInstanceProcAddr` export and hand
back wrappers for `vkCreateInstance` / `vkCreateDevice` / `vkCreateSwapchainKHR` /
`vkQueuePresentKHR` (so a volk app resolves *ours*), tracking the device / queue / images /
format; read the presented image back with `vkCmdCopyImageToBuffer` and upload it to the shared
texture on a hook-owned D3D11 device. The trap: the read-back submit must **wait on the present's
wait semaphores** (consume them) and **signal a fresh semaphore the real present then waits on**
— both your copy and the present can't wait the same binary semaphore. "Injected too late" is
detected as *vulkan-1.dll loaded + hook in for >4 s + we never saw `vkCreateDevice`* (the app set
everything up before us) → red relaunch banner, WGC meanwhile. Testing needs an **early-load**
path (suspended launch + inject + resume; the mock loads Vulkan and waits), since the normal
late-inject flow can't catch a Vulkan present.
- **A render client that predates our injection has no knowable format — measure it.** - **A render client that predates our injection has no knowable format — measure it.**
We inject into already-running games, so we usually never see the game's We inject into already-running games, so we usually never see the game's
`IAudioClient::Initialize`; the render-hook then assumes the device mix format for that `IAudioClient::Initialize`; the render-hook then assumes the device mix format for that