BlackMark/CoopAllTheThings
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

DX12 capture: fence the copy off the game's queue + add drop detection

Browse Source 
Resolves the DX12 mirror stutter and makes dropped frames observable.

Decouple the D3D11On12 copy from the game's present queue. Submitting the
copy on the game's own present queue (the prior approach) ordered it
correctly but stalled the game's presents: GPU back-pressure, plus the
shared keyed-mutex AcquireSync is a CPU-blocking call on the render
thread. Running it on an independent queue avoids the stall but races the
game's render -> stale frames. Do both: run the copy on our own queue and
order it after the frame with an ID3D12Fence the game's present queue
signals (near-free) and our queue waits on. The present queue is still
recovered for late injection via the ExecuteCommandLists hook (now used to
signal the fence, not host the copy). Producer AcquireSync stays
non-blocking (timeout 0) so a busy mutex drops a mirror frame instead of
stalling the game.

Add drop detection (protocol v12 -> v13). The hook counts captures skipped
because the keyed mutex was busy (VideoShare.frames_dropped); the host
counts published frames it never displayed (generation gaps). The Video
panel shows "Frames lost: N/s capture  N/s display", red when nonzero.
This confirmed the game-window-vs-mirror behavior is a display-path
artifact (unfocused windows lose VRR/independent flip), not a capture loss.

Add a one-shot present-pattern log: per distinct swapchain (size/format/
buffer index) and per distinct present-flags value, with DXGI_PRESENT_TEST
spelled out as an occlusion probe that draws nothing -- which is why
Miles Morales shows ~2 presents per captured frame (the test present is
counted but produces no frame).

Docs: add the DX12 capture lessons to the README (rotating back buffer,
fence/own-queue, capture-at-Present decoupling from DWM) and drop the now
-moot DX12 overhead future-work item.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
BlackMark
2026-06-21 19:55:27 +02:00
parent efc16b5eea
commit ffaad6c4ae
10 changed files with 241 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
hook/src/ipc_client.hpp
View File

@@ -207,6 +207,16 @@ public:
}
}
// Record that a produced frame couldn't be copied into the shared texture because the
// keyed mutex was held by the host (we skip rather than block the game's render thread).
void note_video_dropped()
{
if (block_ != nullptr)
{
block_->video.frames_dropped += 1;
}
}
// Publish that a fresh backbuffer copy is in the shared texture (named
// coop_video_<pid>) with these dimensions/format; bumps the generation the host
// polls. The texture itself is shared out-of-band by name, not through here.

209
hook/src/present_hook.cpp
View File

@@ -60,21 +60,23 @@ bool g_unsupported_logged = false;
ID3D11On12Device* g_on12 = nullptr;
ID3D11Device* g_on12_d3d11 = nullptr;
ID3D11DeviceContext* g_on12_ctx = nullptr;
ID3D12CommandQueue* g_on12_queue = nullptr;
ID3D12CommandQueue* g_on12_queue = nullptr; // our OWN copy queue (never the game's)
ID3D12Device* g_on12_d3d12 = nullptr;
// The game's own present queue we built the bridge against (null = we fell back to
// creating our own queue because it hadn't been captured yet). When this changes we
// rebuild the bridge so the copy keeps running on the game's queue.
ID3D12CommandQueue* g_on12_src_queue = nullptr;
// Cross-queue ordering fence, created on the game's device. The game's present queue
// signals it after the frame's render (a near-free operation); our own copy queue waits
// on it before copying. This orders our copy correctly *after* the frame WITHOUT putting
// any copy or keyed-mutex work on the game's present queue -- doing that stalled the
// game's own presents (GPU back-pressure, plus the keyed mutex is a CPU-blocking acquire
// on the render thread). Lifetime tied to the bridge; guarded by g_tex_mutex.
ID3D12Fence* g_copy_fence = nullptr;
UINT64 g_copy_fence_val = 0;
// ExecuteCommandLists hook: recovers the game's D3D12 *present* command queue so the
// On12 copy can be submitted on the game's own queue (ordered after the frame's
// rendering, like the D3D11 immediate-context path) instead of an independent queue
// that races it -- the cause of the occasional stale-frame stutter under GPU load.
// We hook the per-frame method rather than swapchain/queue creation, so it works for
// late injection (the queue already exists). ID3D12CommandQueue method index 10:
// IUnknown 0-2, ID3D12Object 3-6, ID3D12DeviceChild 7 (ID3D12Pageable adds none),
// then ID3D12CommandQueue UpdateTileMappings 8, CopyTileMappings 9, ExecuteCommandLists 10.
// ExecuteCommandLists hook: recovers the game's D3D12 *present* command queue so we can
// signal the ordering fence (above) on it. We hook the per-frame method rather than
// swapchain/queue creation, so it works for late injection (the queue already exists).
// ID3D12CommandQueue method index 10: IUnknown 0-2, ID3D12Object 3-6, ID3D12DeviceChild
// 7 (ID3D12Pageable adds none), then UpdateTileMappings 8, CopyTileMappings 9,
// ExecuteCommandLists 10.
constexpr unsigned kIdx_ID3D12CommandQueue_ExecuteCommandLists = 10;
safetyhook::InlineHook g_hk_ecl;
int g_id_ecl = -1;
@@ -85,6 +87,60 @@ int g_id_ecl = -1;
thread_local ID3D12CommandQueue* t_present_queue = nullptr;
std::atomic<ID3D12CommandQueue*> g_present_queue{nullptr};
// Present-pattern inventory. The first time we see each swapchain -- and each distinct
// present-flags value per swapchain -- we log it once, mapping how a game drives DXGI:
// how many swapchains it presents, their size/format, and how often it issues no-op
// DXGI_PRESENT_TEST presents (occlusion probes that draw nothing). That last point
// explains games whose raw Present count outpaces the frames actually captured, since a
// test present is counted but produces no frame. Render-thread only; small fixed tables.
constexpr int kMaxLoggedPresents = 16;
constexpr int kMaxLoggedSwapchains = 8;
struct LoggedPresent
{
void* swapchain;
UINT flags;
};
LoggedPresent g_logged_presents[kMaxLoggedPresents] = {};
int g_logged_presents_n = 0;
void* g_logged_swapchains[kMaxLoggedSwapchains] = {};
int g_logged_swapchains_n = 0;
// True the first time this (swapchain, flags) pair is presented, so the caller logs once.
bool first_present_with_flags(void* swapchain, UINT flags)
{
for (int i = 0; i < g_logged_presents_n; ++i)
{
if (g_logged_presents[i].swapchain == swapchain && g_logged_presents[i].flags == flags)
{
return false;
}
}
if (g_logged_presents_n >= kMaxLoggedPresents)
{
return false;
}
g_logged_presents[g_logged_presents_n++] = {swapchain, flags};
return true;
}
// True the first time this swapchain feeds the capture, so the caller logs it once.
bool first_capture_from(void* swapchain)
{
for (int i = 0; i < g_logged_swapchains_n; ++i)
{
if (g_logged_swapchains[i] == swapchain)
{
return false;
}
}
if (g_logged_swapchains_n >= kMaxLoggedSwapchains)
{
return false;
}
g_logged_swapchains[g_logged_swapchains_n++] = swapchain;
return true;
}
void* vtable_method(void* obj, unsigned index)
{
return (*reinterpret_cast<void***>(obj))[index];
@@ -210,45 +266,41 @@ void release_on12_locked()
g_on12_d3d12->Release();
g_on12_d3d12 = nullptr;
}
g_on12_src_queue = nullptr;
if (g_copy_fence != nullptr)
{
g_copy_fence->Release();
g_copy_fence = nullptr;
}
g_copy_fence_val = 0;
}
// Build the D3D11On12 device for the game's D3D12 `dev`, submitting our copy work on
// `game_queue` (the game's present queue, captured via the ExecuteCommandLists hook)
// so it's ordered after the frame's rendering. If `game_queue` is null we fall back to
// a queue of our own (correct frame, but can race the game's render under load -- the
// old behavior) until the real queue is captured. Caller holds g_tex_mutex. Returns
// true when the bridge is ready.
bool ensure_on12_locked(ID3D12Device* dev, ID3D12CommandQueue* game_queue)
// Build the D3D11On12 device for the game's D3D12 `dev` on a copy queue of our own, plus
// a fence on `dev` for ordering our copy after the game's frame (signaled by the game's
// present queue in capture_backbuffer_d3d12, waited by our queue). Keeping the copy on
// our own queue is what stops our work from stalling the game's present queue. Caller
// holds g_tex_mutex. Returns true when the bridge is ready.
bool ensure_on12_locked(ID3D12Device* dev)
{
if (g_on12 != nullptr && g_on12_d3d12 == dev && g_on12_src_queue == game_queue)
if (g_on12 != nullptr && g_on12_d3d12 == dev)
{
return true;
}
release_on12_locked();
// Hold a ref on whichever queue the bridge uses, so Release on teardown is uniform.
ID3D12CommandQueue* queue = game_queue;
if (queue != nullptr)
D3D12_COMMAND_QUEUE_DESC qd{};
qd.Type = D3D12_COMMAND_LIST_TYPE_DIRECT;
ID3D12CommandQueue* queue = nullptr;
HRESULT hr = dev->CreateCommandQueue(&qd, __uuidof(ID3D12CommandQueue), reinterpret_cast<void**>(&queue));
if (FAILED(hr) || queue == nullptr)
{
queue->AddRef();
}
else
{
D3D12_COMMAND_QUEUE_DESC qd{};
qd.Type = D3D12_COMMAND_LIST_TYPE_DIRECT;
HRESULT hr = dev->CreateCommandQueue(&qd, __uuidof(ID3D12CommandQueue), reinterpret_cast<void**>(&queue));
if (FAILED(hr) || queue == nullptr)
{
logf("present(d3d12): CreateCommandQueue failed hr=0x%08lX", static_cast<unsigned long>(hr));
return false;
}
logf("present(d3d12): CreateCommandQueue failed hr=0x%08lX", static_cast<unsigned long>(hr));
return false;
}
IUnknown* queues[] = {queue};
ID3D11Device* d11 = nullptr;
ID3D11DeviceContext* ctx = nullptr;
HRESULT hr = D3D11On12CreateDevice(dev, 0, nullptr, 0, queues, 1, 0, &d11, &ctx, nullptr);
hr = D3D11On12CreateDevice(dev, 0, nullptr, 0, queues, 1, 0, &d11, &ctx, nullptr);
if (FAILED(hr) || d11 == nullptr)
{
logf("present(d3d12): D3D11On12CreateDevice failed hr=0x%08lX", static_cast<unsigned long>(hr));
@@ -269,14 +321,26 @@ bool ensure_on12_locked(ID3D12Device* dev, ID3D12CommandQueue* game_queue)
return false;
}
// Ordering fence on the game's device. Non-fatal if it fails -- we simply lose the
// cross-queue ordering (the copy can then race the frame, the pre-fence behavior).
ID3D12Fence* fence = nullptr;
hr = dev->CreateFence(0, D3D12_FENCE_FLAG_NONE, __uuidof(ID3D12Fence), reinterpret_cast<void**>(&fence));
if (FAILED(hr) || fence == nullptr)
{
logf("present(d3d12): CreateFence failed hr=0x%08lX (copy will be unordered)",
static_cast<unsigned long>(hr));
fence = nullptr;
}
g_on12 = on12;
g_on12_d3d11 = d11;
g_on12_ctx = ctx;
g_on12_queue = queue; // we hold a ref (the AddRef'd game queue, or our created one)
g_on12_src_queue = game_queue; // what we built against (null = our own queue)
g_on12_queue = queue;
g_on12_d3d12 = dev;
g_copy_fence = fence;
g_copy_fence_val = 0;
dev->AddRef(); // we hold a reference for the lifetime of the bridge
logf("present(d3d12): D3D11On12 bridge ready (queue=%s)", game_queue != nullptr ? "game" : "own");
logf("present(d3d12): D3D11On12 bridge ready (own copy queue, fence=%d)", fence != nullptr ? 1 : 0);
return true;
}
@@ -316,6 +380,7 @@ void capture_backbuffer_d3d12(IDXGISwapChain* sc)
bb->GetDevice(__uuidof(ID3D12Device), reinterpret_cast<void**>(&dev));
bool shared = false;
bool dropped = false;
UINT w = 0, h = 0;
DXGI_FORMAT fmt = DXGI_FORMAT_UNKNOWN;
// The game's present queue, preferring the one seen on this (the render) thread.
@@ -325,11 +390,30 @@ void capture_backbuffer_d3d12(IDXGISwapChain* sc)
game_queue = g_present_queue.load(std::memory_order_relaxed);
}
// Log each distinct swapchain feeding the capture once (size/format/buffer index).
if (first_capture_from(sc))
{
const D3D12_RESOURCE_DESC rd = bb->GetDesc();
logf("present: swapchain=%p capturing D3D12 backbuffer %llux%u fmt=%d samples=%u bufferindex=%u queue=%s",
sc, static_cast<unsigned long long>(rd.Width), rd.Height, static_cast<int>(rd.Format),
rd.SampleDesc.Count, bb_index, game_queue != nullptr ? "known" : "unknown");
}
if (dev != nullptr)
{
std::scoped_lock lock(g_tex_mutex);
if (ensure_on12_locked(dev, game_queue))
if (ensure_on12_locked(dev))
{
// Order our copy after the game's frame without burdening the game's queue: the
// game queue signals the fence (cheap), our copy queue waits on it. Skipped if the
// queue isn't captured yet or the fence is missing (one possibly-early frame).
if (game_queue != nullptr && g_copy_fence != nullptr)
{
const UINT64 fence_val = ++g_copy_fence_val;
game_queue->Signal(g_copy_fence, fence_val);
g_on12_queue->Wait(g_copy_fence, fence_val);
}
D3D11_RESOURCE_FLAGS rf{};
rf.BindFlags = D3D11_BIND_RENDER_TARGET;
ID3D11Resource* wrapped = nullptr;
@@ -352,12 +436,16 @@ void capture_backbuffer_d3d12(IDXGISwapChain* sc)
if (d.SampleDesc.Count == 1 && ensure_shared_texture_locked(g_on12_d3d11, w, h, fmt) &&
g_shared_mutex != nullptr)
{
if (g_shared_mutex->AcquireSync(kVideoMutexKey, 8) == S_OK)
if (g_shared_mutex->AcquireSync(kVideoMutexKey, 0) == S_OK)
{
g_on12_ctx->CopyResource(g_shared_tex, wtex);
g_shared_mutex->ReleaseSync(kVideoMutexKey);
shared = true;
}
else
{
dropped = true; // host held the mutex -> this frame never reaches the mirror
}
}
wtex->Release();
}
@@ -381,6 +469,10 @@ void capture_backbuffer_d3d12(IDXGISwapChain* sc)
g_ipc->publish_video_frame(w, h, static_cast<std::uint32_t>(fmt));
}
}
else if (dropped && g_ipc != nullptr)
{
g_ipc->note_video_dropped();
}
if (dev != nullptr)
{
dev->Release();
@@ -402,6 +494,13 @@ void capture_backbuffer(IDXGISwapChain* sc)
D3D11_TEXTURE2D_DESC bd{};
backbuf->GetDesc(&bd);
// Log each distinct swapchain feeding the capture once (see the D3D12 path).
if (first_capture_from(sc))
{
logf("present: swapchain=%p capturing D3D11 backbuffer %ux%u fmt=%d samples=%u", sc, bd.Width, bd.Height,
static_cast<int>(bd.Format), bd.SampleDesc.Count);
}
// Multisampled backbuffers would need ResolveSubresource; flip-model swapchains
// are single-sampled. Skip the rare MSAA case rather than mis-copy.
if (bd.SampleDesc.Count != 1)
@@ -419,6 +518,7 @@ void capture_backbuffer(IDXGISwapChain* sc)
}
bool shared = false;
bool dropped = false;
if (device != nullptr && ctx != nullptr)
{
std::scoped_lock lock(g_tex_mutex);
@@ -433,6 +533,10 @@ void capture_backbuffer(IDXGISwapChain* sc)
g_shared_mutex->ReleaseSync(kVideoMutexKey);
shared = true;
}
else
{
dropped = true; // host held the mutex past the wait -> frame lost (rare on D3D11)
}
}
}
@@ -444,6 +548,10 @@ void capture_backbuffer(IDXGISwapChain* sc)
g_ipc->publish_video_frame(bd.Width, bd.Height, static_cast<std::uint32_t>(bd.Format));
}
}
else if (dropped && g_ipc != nullptr)
{
g_ipc->note_video_dropped();
}
if (ctx != nullptr)
{
@@ -516,6 +624,14 @@ HRESULT STDMETHODCALLTYPE hk_Present(IDXGISwapChain* sc, UINT sync_interval, UIN
{
g_ipc->note_present();
}
// Log each distinct (swapchain, flags) once. A DXGI_PRESENT_TEST present draws nothing
// (it only probes occlusion), so those inflate the Present count without producing a
// frame -- which is why some games show more presents than captured frames.
if (first_present_with_flags(sc, flags))
{
logf("present: swapchain=%p Present flags=0x%08X%s", sc, flags,
(flags & DXGI_PRESENT_TEST) ? " (DXGI_PRESENT_TEST: occlusion probe, no frame drawn)" : "");
}
// DXGI_PRESENT_TEST presents nothing; don't bother copying for it.
if ((flags & DXGI_PRESENT_TEST) == 0)
{
@@ -538,6 +654,11 @@ HRESULT STDMETHODCALLTYPE hk_Present1(IDXGISwapChain1* sc, UINT sync_interval, U
{
g_ipc->note_present();
}
if (first_present_with_flags(sc, flags)) // see hk_Present
{
logf("present: swapchain=%p Present1 flags=0x%08X%s", sc, flags,
(flags & DXGI_PRESENT_TEST) ? " (DXGI_PRESENT_TEST: occlusion probe, no frame drawn)" : "");
}
if ((flags & DXGI_PRESENT_TEST) == 0)
{
capture_backbuffer(sc); // IDXGISwapChain1 derives from IDXGISwapChain
@@ -676,6 +797,8 @@ void remove_present_hooks()
hook_set_installed(g_id_present1, false);
hook_set_installed(g_id_ecl, false);
g_present_queue.store(nullptr, std::memory_order_relaxed);
g_logged_presents_n = 0; // let a fresh injection re-log the present pattern
g_logged_swapchains_n = 0;
{
std::scoped_lock lock(g_tex_mutex);
release_shared_locked();