Docs: Vulkan hooked capture in architecture + lessons; scope the opt-in layer
Architecture now lists Vulkan as a hooked producer (GPA interception + read-back, early-presence required). Adds a lessons-learned bullet (can't late-hook Vulkan; the present-semaphore re-chaining trap; the too-late heuristic; early-load testing). mock_game_test docs note the suspended-launch capture + too-late check. The Vulkan milestone is trimmed to its one remaining piece -- the opt-in implicit layer for immediate-init games -- with the capture/banner/best-effort marked done. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
101
README.md
101
README.md
@@ -44,9 +44,15 @@ read-backs the backbuffer with `GetRenderTargetData` (a D3D9 surface isn't D3D11
|
||||
swizzling BGRA→RGBA and uploading via a hook-owned D3D11 device — one path covers both plain
|
||||
D3D9 and D3D9Ex; **OpenGL** hooks `SwapBuffers` / `wglSwapBuffers` and reads the
|
||||
backbuffer with `glReadPixels` (for games that never touch DXGI, e.g. Phantom
|
||||
Brave). The host samples the copy as plain UNORM (`srgb_to_unorm`) so
|
||||
`*_SRGB`-backbuffer games mirror at correct brightness. **WGC remains the default**
|
||||
and covers anything the hooked path doesn't (Vulkan — see Roadmap).
|
||||
Brave); and **Vulkan** inline-hooks the `vulkan-1.dll` `vkGetInstanceProcAddr` export to
|
||||
intercept the resolution chain (`vkCreateInstance` / `vkCreateDevice` /
|
||||
`vkCreateSwapchainKHR` / `vkQueuePresentKHR`) and reads the presented image back with
|
||||
`vkCmdCopyImageToBuffer` — but only when the hook is present **before** the game initializes
|
||||
Vulkan (it caches its present pointer at init), so the Vulkan path needs early presence via
|
||||
Auto-attach (and an opt-in implicit layer — see Roadmap); a too-late attach shows a red
|
||||
relaunch banner and falls back to WGC. The host samples the copy as plain UNORM
|
||||
(`srgb_to_unorm`) so `*_SRGB`-backbuffer games mirror at correct brightness. **WGC remains the
|
||||
default** and covers anything the hooked path doesn't.
|
||||
|
||||
## Limitations
|
||||
|
||||
@@ -147,38 +153,24 @@ Conventions for every milestone below:
|
||||
nothing clipped/overlapping, colours/labels correct. A green unit test is not a substitute
|
||||
for looking at the product; this real-screenshot check is required, not optional.
|
||||
|
||||
- **M2 — Vulkan capture.** The Vulkan **mock backend is done** (`render_vk.cpp`, selectable `vk`):
|
||||
volk + Vulkan-Headers submodules, a real instance / device / swap chain, and a per-frame
|
||||
`vkCmdClearColorImage` of the frame-counter colour (no pipeline/shaders → **no SPIR-V
|
||||
toolchain**); it loads Vulkan via **volk** on purpose (the loader-bypass case the capture hook
|
||||
must handle). `mock_game_test` runs a **liveness** check on it — its present pointer is cached
|
||||
at init, so late injection can't hook it, which is exactly why the capture below is the hard
|
||||
part. The remaining **capture hook** (Vulkan games mirror via **WGC** meanwhile — the default):
|
||||
1. **Present interception (needs early presence).** Hook `vkGetInstanceProcAddr` /
|
||||
`vkGetDeviceProcAddr` (the `vulkan-1.dll` exports) so a volk-using app resolves *our*
|
||||
wrappers, and intercept `vkCreateDevice` / `vkCreateSwapchainKHR` to capture the
|
||||
device/queue/images/format. Read the presented image back with `vkCmdCopyImageToBuffer` into
|
||||
a host-visible buffer (the same read-back pattern as D3D10/D3D9/OpenGL — simpler and more
|
||||
robust than a `VK_KHR_external_memory_win32` + keyed-mutex blit), swizzle if BGRA, and upload
|
||||
via the hook-owned D3D11 device. **Testing needs the early-load path** (the mock loads the
|
||||
hook before `vkCreateInstance`, e.g. via an env-var or the layer); the normal late-inject
|
||||
`mock_game_test` flow can't catch a Vulkan present.
|
||||
2. **Attach via auto-attach — best-effort by default, opt-in layer for reliability.** By
|
||||
default, arming Auto-attach for a detected Vulkan game injects as early as possible on
|
||||
relaunch (the existing poll-and-inject) — enough for games that initialize Vulkan a little
|
||||
into startup. For games that init Vulkan immediately, an **opt-in** Injection-panel checkbox
|
||||
("Set up Vulkan layer") registers a per-user (HKCU, no admin) implicit Vulkan layer (thin
|
||||
`coop_vk_layer.dll` + JSON manifest) scoped to that game's image name; on relaunch the loader
|
||||
loads it at `vkCreateInstance` — guaranteed before init — and it wires up present capture +
|
||||
IPC. The layer is removed when the option is unticked or the host exits, and self-deactivates
|
||||
for any non-target app, so a stale registration (e.g. after a host crash) is harmless.
|
||||
3. **Too-late detection + red banner.** A late-injected hook that finds `vulkan-1.dll` loaded
|
||||
but no working hooked-present reports a *"Vulkan, injected too late"* status; the host
|
||||
overlays a **red** banner on the mirror window: "Detected a Vulkan game — the hook attached
|
||||
too late to mirror it with low latency. Enable Auto-attach (and, if it persists, 'Set up
|
||||
Vulkan layer') and relaunch the game." **WGC keeps mirroring meanwhile** so the session stays
|
||||
usable; the banner clears once the hooked path goes live. This is new end-user UI, so it must
|
||||
keep M1's fit test (and the visual inspection) green.
|
||||
- **M2 — Vulkan: opt-in reliability layer (the one piece left).** The Vulkan **mock backend and
|
||||
hooked capture are done and tested**: the hook inline-hooks the `vulkan-1.dll`
|
||||
`vkGetInstanceProcAddr` export and intercepts the resolution chain (`vkCreateInstance` /
|
||||
`vkCreateDevice` / `vkCreateSwapchainKHR` / `vkQueuePresentKHR`), reads the presented image back
|
||||
with `vkCmdCopyImageToBuffer`, and re-chains the present's wait semaphores so the copy orders
|
||||
after rendering. `mock_game_test` decodes Vulkan frames through it (via a suspended-launch +
|
||||
early-load path) and verifies the **too-late detection**; the host shows the **red relaunch
|
||||
banner** (verified live). **Best-effort auto-attach** already works (the existing poll-and-inject
|
||||
re-injects on relaunch, the vk hook retries until `vulkan-1.dll` loads), enough for games that
|
||||
don't initialize Vulkan instantly. Remaining:
|
||||
- **Opt-in implicit Vulkan layer** for games that init Vulkan *immediately* (where even a
|
||||
relaunch+auto-attach injects too late). A real chain-aware Vulkan layer (`coop_vk_layer.dll`
|
||||
+ JSON manifest) the loader loads at `vkCreateInstance` — guaranteed before init — registered
|
||||
per-user (HKCU, no admin) and scoped to the target image, with an **opt-in** "Set up Vulkan
|
||||
layer" Injection-panel checkbox that registers/unregisters it (self-deactivating for non-target
|
||||
apps so a stale entry after a host crash is harmless). This is a separate component from the
|
||||
inline-hook capture (layer-chain dispatch + manifest/registry + the checkbox + IPC handshake);
|
||||
until it lands, the banner directs immediate-init Vulkan games to WGC.
|
||||
|
||||
Each of 1–3 is its own commit; test the early/layer path against the mock (decode frames) and
|
||||
the too-late path (assert the status + banner fire).
|
||||
@@ -292,17 +284,20 @@ ctest --test-dir build -C Debug --output-on-failure
|
||||
endpoint format). Skips cleanly if the machine has no audio endpoint.
|
||||
- **`mock_game_test`** — comprehensive capture/audio/hook stress test against
|
||||
**`coop_mock_game`** (an animated, frame-numbered A/V test game under
|
||||
[`tools/mock_game`](tools/mock_game) with selectable **DX9 / DX9Ex / DX10 / DX11 / DX12 / OpenGL** backends and a
|
||||
configurable WASAPI tone). It launches the game, injects `coop_hook.dll`, opens the
|
||||
hook's shared video texture, and **decodes the frame number out of the captured pixels**
|
||||
to assert the mirror sees a *monotonic, advancing* sequence for each backend (the bar
|
||||
for no dropped / stale / out-of-order frames — what the DX12 rotating-backbuffer bug
|
||||
broke). It launches the game at several **audio formats** (44100/48000/96000, PCM +
|
||||
float) and asserts the hook measures each one's rate through the full inject path, then
|
||||
injects with audio + video, checks both stream, cycles the audio subsystem off/on
|
||||
(hook/unhook stress), and confirms the game never crashes and capture resumes. This
|
||||
suite drove out five real audio races (see Lessons learned). Skips cleanly without a
|
||||
D3D11 device.
|
||||
[`tools/mock_game`](tools/mock_game) with selectable **DX9 / DX9Ex / DX10 / DX11 / DX12 /
|
||||
OpenGL / Vulkan** backends and a configurable WASAPI tone). It launches the game, injects
|
||||
`coop_hook.dll`, opens the hook's shared video texture, and **decodes the frame number out of
|
||||
the captured pixels** to assert the mirror sees a *monotonic, advancing* sequence for each
|
||||
backend (the bar for no dropped / stale / out-of-order frames — what the DX12 rotating-backbuffer
|
||||
bug broke). **Vulkan** is special: since its present pointer is cached at init, the test launches
|
||||
the mock **suspended**, injects, then resumes (the mock loads Vulkan and waits so the hook arms
|
||||
first) to decode frames; it also late-injects a Vulkan game and asserts the **too-late** flag
|
||||
trips (which drives the host's relaunch banner). It launches the game at several **audio formats**
|
||||
(44100/48000/96000, PCM + float) and asserts the hook measures each one's rate through the full
|
||||
inject path, then injects with audio + video, checks both stream, cycles the audio subsystem
|
||||
off/on (hook/unhook stress), and confirms the game never crashes and capture resumes. This suite
|
||||
drove out five real audio races (see Lessons learned). Skips cleanly without a D3D11 / Vulkan
|
||||
device.
|
||||
|
||||
### Debugging the hooks against a real game
|
||||
|
||||
@@ -460,6 +455,20 @@ Non-obvious things that cost time and constrain the design:
|
||||
the frame-counter block at the GL *top* (`y = h - block`) to land at the captured image's
|
||||
top-left. The GL `SwapBuffers` hook now also bumps the shared present counter (it's the GL
|
||||
present), so the Video panel's present rate works for GL games too.
|
||||
- **Vulkan can't be late-hooked, and capturing its present needs semaphore surgery.** Vulkan
|
||||
games cache `vkQueuePresentKHR` at init (often via volk, which bypasses the loader trampoline),
|
||||
so late injection misses it — the hook must be in **before `vkCreateInstance`**. Catch the
|
||||
resolution chain instead: inline-hook the `vulkan-1.dll` `vkGetInstanceProcAddr` export and hand
|
||||
back wrappers for `vkCreateInstance` / `vkCreateDevice` / `vkCreateSwapchainKHR` /
|
||||
`vkQueuePresentKHR` (so a volk app resolves *ours*), tracking the device / queue / images /
|
||||
format; read the presented image back with `vkCmdCopyImageToBuffer` and upload it to the shared
|
||||
texture on a hook-owned D3D11 device. The trap: the read-back submit must **wait on the present's
|
||||
wait semaphores** (consume them) and **signal a fresh semaphore the real present then waits on**
|
||||
— both your copy and the present can't wait the same binary semaphore. "Injected too late" is
|
||||
detected as *vulkan-1.dll loaded + hook in for >4 s + we never saw `vkCreateDevice`* (the app set
|
||||
everything up before us) → red relaunch banner, WGC meanwhile. Testing needs an **early-load**
|
||||
path (suspended launch + inject + resume; the mock loads Vulkan and waits), since the normal
|
||||
late-inject flow can't catch a Vulkan present.
|
||||
- **A render client that predates our injection has no knowable format — measure it.**
|
||||
We inject into already-running games, so we usually never see the game's
|
||||
`IAudioClient::Initialize`; the render-hook then assumes the device mix format for that
|
||||
|
||||
Reference in New Issue
Block a user