present_calls, frames_dropped (VideoShare) and frames_rendered (AudioStreamInfo)
were plain `+= 1` / stores in the DLL, read by the host cross-process. On an x86
DLL a 64-bit store is two halves, so the x64 host could read a torn value during
a carry. Benign (display-only), but a real data race.
Use std::atomic_ref at the access sites rather than changing the field types:
the structs stay plain POD so the layout/offset asserts are unchanged and
AudioStreamInfo stays trivially copyable (it's published/read as a whole struct).
The DLL writers (note_present / note_video_dropped / note_audio_frames) and the
host readers (IpcServer::video_share / hook_status) now use relaxed atomic_ref;
hook_status reloads frames_rendered atomically after the wholesale struct copy.
The dev-tool readers (vk_validate, audio_probe) keep plain reads -- diagnostics of
diagnostics, and same-bitness in practice.
Validated by present_hook_test (present_calls via atomic_ref) and audio_hook_test
(frames_rendered) -- also confirms no atomic_ref alignment fault.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SharedTextureSource::update() acquired the shared-texture keyed mutex with
`AcquireSync(...) == S_OK`. But WAIT_ABANDONED -- a prior owner (e.g. a host that
crashed mid-acquire, then reconnected) died holding it -- actually GRANTS us
ownership. Treating it as failure skipped the copy AND never released, so the
next AcquireSync blocked forever and the mirror froze permanently after a crash
+ reconnect (directly relevant to the new reconnect path).
Factor the decision into keyed_mutex_acquired(HRESULT) (capture/keyed_mutex.hpp):
S_OK or WAIT_ABANDONED -> copy + release; timeout/hard errors -> skip the frame.
update() now uses it.
Test-first: keyed_mutex_test asserts WAIT_ABANDONED is treated as acquired while
the genuine "didn't get it" cases (timeout, E_FAIL, device-removed) are not. The
full cross-process abandonment is keyed-mutex OS semantics, not re-tested with a
child process here -- the predicate is the regression surface.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
D3D11Window ignored the HRESULTs from Present, ResizeBuffers, and
CreateRenderTargetView, so a host-side TDR / driver reset / GPU hang left the
render loop presenting to a dead device forever with no error.
Now note_device_loss() inspects those HRESULTs; on DXGI_ERROR_DEVICE_REMOVED/RESET
it captures GetDeviceRemovedReason() and sets device_lost(). The main loop checks
it after render_frame, shows a MessageBox with the reason, and stops cleanly.
Per the agreed scope this is detect-surface-halt, not full device re-creation
(which would have to re-init ImGui + the capture pipeline) -- that's future work.
Not unit-testable (TDR isn't deterministically reproducible); fixed by inspection.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
register_ui_settings installs an ImGui settings handler whose UserData points at
the run()-local UiState. ~ImGuiLayer calls DestroyContext, which flushes the .ini
through that handler (ui_settings_write_all dereferences UserData). But UiState
was declared after ImGuiLayer in run(), so it (and the panels between them) were
destroyed first -- the shutdown save read freed/clobbered stack every clean exit
(UB; could corrupt coop_layout.ini / the persisted debug-details flag).
Declare UiState before ImGuiLayer so it outlives the context and is destroyed
last. Not unit-testable (shutdown lifetime ordering); fixed by inspection.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Disconnect -> reconnect now reuses the DLL already in the game instead of
injecting again, including across a tool restart or crash: a connected DLL keeps
its per-pid shared section (and worker) alive after the host goes away, so a
fresh host can find it and re-attach to the same section.
- hook_dll_alive(pid) (host/src/inject/dll_probe.cpp): detect a live DLL by
opening the per-pid section and polling its heartbeat (returns as soon as a
beat lands; a missing section or stalled worker reads as not-alive). It does
not check magic -- a graceful disconnect zeroes magic but the DLL keeps
beating and the worker never re-checks magic post-connect.
- InjectionPanel: the Inject and Connect button branches to reconnect_selected()
when a live DLL is detected -- IpcServer::start() re-attaches to the SAME
section the DLL still holds and re-publishes the subsystem state; no
re-injection. Factored the shared post-connect setup (publish_subsystem_state
/ begin_liveness_tracking). The DLL needed no change -- it just resumes reading
the re-attached section.
- A false not-alive is benign: the inject path still re-attaches an
already-injected DLL (LoadLibrary no-ops), so the timeout only needs to clear
the worker's ~250ms beat period with margin.
Test (mock_game_test test_reconnect): inject -> hooked -> graceful disconnect ->
drop the host handle (simulating a restart while the DLL keeps the section alive)
-> detect via heartbeat -> re-attach to the same section -> hooks re-install
without re-injecting -> and hook_dll_alive goes false once the game is gone.
Roadmap: both current tasks (graceful disconnect, reconnect) done -> removed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On an explicit Disconnect and on graceful tool exit, the host now asks the
injected DLL to remove every subsystem so the game runs exactly as if it was
never hooked (each hook restores its original bytes). The DLL stays injected but
dormant, ready for a later reconnect -- we never eject it.
Before, both paths just dropped the IPC channel (IpcServer::stop) without telling
the DLL, leaving the hooks active with frozen forwarded state until the game
exited.
- IpcServer::request_unhook_all() sets every subsystem_disabled flag (the DLL
reconciles to fully unhooked on its next tick); all_hooks_removed() reads the
hook registry back so the host can confirm the game is vanilla.
- InjectionPanel::disconnect_graceful() requests the unhook, waits (bounded) for
the registry to clear, then stops. Wired into the Disconnect button (700ms) and
the destructor (300ms). The flags persist in the section the DLL keeps alive, so
the unhook completes even if the host exits before confirming.
Tests (failing first):
- ipc_server_test: request_unhook_all() disables all subsystems; all_hooks_removed()
tracks the registry. Deterministic, no game.
- mock_game_test test_graceful_disconnect: inject -> hooks installed -> request
unhook-all -> every hook removed (game vanilla) while the DLL stays alive
(heartbeat advancing). Full suite still passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds coop_vk_validate, a harness that drives the Vulkan capture path end-to-end
against a shipping title (default Sphere Spectacle, a pure-Vulkan game) and asserts
frames reach the shared texture and advance, the captured resolution/colors are
sane, saves a BMP screenshot for visual confirmation, and reports the present rate.
Findings:
- Layer method WORKS: coop_vk_layer mirrors the game correctly at 1920x1080 --
right colors/brightness, no BGRA/RGBA swizzle, no sRGB darkening (screenshot
confirmed). The layer captures every present (no drops).
- Suspended-inject "Auto-attach" is NOT applicable to a Steam title that must launch
through Steam: its exe renders nothing when launched directly, so there's no Vulkan
present to catch. The layer is the method for Steam Vulkan games (the early-inject
mechanism itself is covered by mock_game_test's suspended-launch path).
- The layer does video, but the game needs coop_hook.dll co-injected for focus-spoof
or an unfocused, event-driven game throttles itself to a few fps (looks like a
capture slowdown but isn't). A present-rate number alone can't prove "no FPS impact"
-- it's the game's own cadence; capture stays off the critical path (every present
copied, read-back on its own queue + present-semaphore re-chain).
Also adds SharedTextureSource::read_frame (bulk RGBA readback) for the screenshot.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extends the two-path correlation from rate-only to the full layout, removing the
"channels/bit-depth assumed = device" limitation. correlate_format tries each
candidate de-interleaving (float32 / int16; mono..7.1) of the hook capture, runs
the rate correlation per layout, and keeps whichever aligns with the loopback;
a wrong de-interleaving is noise and won't.
The catch: the hook can't know a guessed stream's real frame size, so its verify
tap pads each render buffer to the device block -- which over-reads stale staging
bytes for a stream with fewer channels/bits, scrambling the audio. So the tap is
now self-describing: it prefixes each buffer with its frame count
([count][count*device_block bytes]), and the host strips the padding per candidate
layout (take the real count*real_block of each chunk) before de-interleaving.
- audio_correlate.hpp: ChunkedCapture + chunk-aware correlate_format + candidate
layouts; absolute-margin confidence gate (the true layout scores ~1.0, a truly
ambiguous alternative within ~0.001 -- 2ch@R == 1ch@2R for identical channels --
is correctly left unconfident).
- audio_hook.cpp: chunked verify tap (free-space-checked so framing can't tear).
- audio_format_verifier: parse chunks; recover_layout path. AudioMirror now corrects
the full format.
- audio_correlation_test: layout recovery from padded chunks (stereo float, 16-bit
PCM, 5.1, mono). audio_verify_test gains scenario (b): 2ch on a multichannel
endpoint with distinct per-channel content (new env-gated ToneSource mode) ->
recovers ch=2/32-bit float end-to-end.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When the host attaches to an already-running game it never saw the stream's
Initialize, so the render-hook assumes the device mix format and measures only
the sample rate from the render cadence -- which a jittery game can make wrong
(intermittent pitch shift). But during the measurement window the game is still
audible, so we have the same audio twice: the hook (pre-mix, unknown format) and
a process-loopback (post-mix, the known device format). Cross-correlating them
pins the true rate from ground truth.
- common/include/coop/audio_correlate.hpp: the pure correlator. Resample the hook
by each candidate standard rate up to the device rate and score how well it
aligns with the loopback across the window (drift-detecting). audio_correlation_test
recovers every rate (score ~1.0 vs ~0.01 for wrong ones), incl. 44100-vs-48000,
and rejects unrelated signals.
- Hook measurement tap: a host-set verify_capture ring flag makes the hook push a
still-being-measured (guessed) stream's raw pre-mix bytes WITHOUT silencing, so
the host can co-capture both signals (a silenced game's loopback is silent).
Inert by default -- the shipping no-echo path is untouched.
- host/src/audio/audio_format_verifier: co-captures hook + loopback and correlates,
feeding a correction into the existing override channel. Wired into AudioMirror's
measurement window (hidden in the gap loopback already covers, so exact streams
pay nothing). audio_verify_test drives it end-to-end against coop_mock_game.
Rate vs layout are coupled (correlating the waveform needs the right channel
de-interleaving), so this step assumes the hook layout matches the device (the
common stereo-on-stereo case); recovering a different channel count / bit depth
is step b.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Build coop_audio_validate, a tool that turns "the mirror audio sounds off"
into numbers. It plays a known sine (coop_tone, 44.1 kHz on a 48 kHz
endpoint -- the Godot/Brotato case), injects the hook as the host does, and
runs a fidelity analyzer (coop/tone_analysis.hpp: pitch error in cents,
SNR/THD, click + dropout counts), dumping a .wav to listen to. Modes:
--render drives the real AudioMirror and measures its rendered output;
--baseline/--selfcheck give the measurement floor; --listen <pid> records a
live coop_host's output; --wav analyzes a recording. Analyzer + WAV I/O are
unit-tested (tone_analysis_test) against synthesized defects.
Using it, the capture ring measures pristine (~68 dB, 0 gaps) while the
render path dropped to ~18 dB with gaps -- localizing a real defect in
AudioMirror::run_hooked: it re-primed (withheld the feed until ~30 ms had
rebuffered) on any partial fill (to_write < avail). A partial fill is normal
producer jitter, and withholding the feed drains the device, so a one-frame
ring dip became a full ~30 ms drop-out; on a jittery game it fired
constantly. Fix: feed whatever is available each tick and re-prime only on a
genuine starvation (device empty AND ring empty). The policy is factored
into a pure RenderPacer reused by run_hooked + run_loopback and proven by
render_pacer_test (the old policy withholds available data ~168x and drains
to one period from silence on a jittery schedule; the new one never
withholds).
ctest 17/17.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Host helper register_vk_layer/unregister_vk_layer registers coop_vk_layer's
manifest as a per-user (HKCU, no admin) implicit Vulkan layer and writes the
scoping file (%TEMP%\coop_vk_target.txt) naming the target game's exe, so the
layer captures only that game and stays inert for every other Vulkan app. The
Injection panel gains an opt-in "Set up Vulkan layer" checkbox (next to Auto
re-attach, which the too-late banner points at); it registers on tick,
unregisters on untick, and the panel destructor unregisters on host exit so no
stale registration is left behind.
Verified live: a registry-registered layer is loaded by the loader for a fresh
vk launch (no env vars) and the scoping matches (active=1), then cleans up.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The hook reports vk_too_late when vulkan-1.dll is loaded and the GPA hook has
been in for >4s but it never caught the app creating its device -- i.e. the app
resolved its Vulkan functions before we hooked (injected too late). Surfaced
through a new HookStatus.vk_too_late field (protocol 15->16) and shown by the
host as a red banner on the mirror window: "Detected a Vulkan game, but the
mirror hook attached too late... enable Auto re-attach (and Set up Vulkan layer
if it persists) and relaunch." WGC keeps mirroring meanwhile.
mock_game_test gains a too-late detection check (late-inject a Vulkan game ->
vk_too_late trips). Verified live: the banner shows over the running tool when a
Vulkan game is injected late with the video subsystem on (added a debug-harness
`video on/off` command to drive it). 15/15 ctest.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add panel overflow instrumentation (record_panel_fit reading ImGui ScrollMax),
a forced layout-reference + debug-aware center split, and host harness commands
(uisize/uifit). New headless ui_fit_test drives the real Controllers + Audio
panels at reference resolutions with Debug details on and asserts no panel
overflows its assigned size; the Audio panel gains a demo mode so its richest
content renders without a live mirror.
Fixes from the measured overflow: widen the center column (was too narrow ->
horizontal overflow), merge the Controllers poll/round-trip tables and fold the
trigger line into the slot line, and make the center height split
Debug-details-aware (Video's height is mirroring-driven, not debug-driven, so a
static split can't serve both modes). Controllers + Audio now fit at 1920x1080
with max info. Video/Injection/Log fit is finalized at the end via the live
uifit harness (M1 stays open until then).
15/15 ctest.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Since guessed streams are now captured-but-not-silenced (the over-write fix),
the hooked path is only no-echo for an exact/override format. The panel
inferred "Hooked (no echo)" unconditionally, which was misleading. Show
"Hooked (echo -- guessed format)" (amber) for a guessed primary stream and
"Hooked (no echo)" (green) only for exact/override; drop the unconditional
"no echo" from source_name() and the mirror status string.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mock_game_test launches coop_mock_game (DX11/DX12, frame-numbered A/V), injects
coop_hook.dll, opens the hook's shared video texture, and decodes the frame
number from the captured pixels to assert a monotonic, advancing mirror for
both backends (catches dropped/stale/out-of-order frames). Then it runs an
A/V + hook/unhook stress pass and confirms no crash + capture resumes. Adds a
read_pixel() readback to the shipping SharedTextureSource for the decode.
Robust to repeated/CI runs (kills stray games, retries inject). Trim the
roadmap (the mock-game item is done) and document the test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an "Auto re-attach this game on relaunch" checkbox (default off, never
persisted, shown while attached or terminated). While on and the target has
terminated, the host polls the process list for the same image name and
re-injects the moment it reappears -- built on the existing
re-attach-by-image-name path, so there's no target to pick. The kill+relaunch
recovery for a wrong audio format: re-attaching early catches
IAudioClient::Initialize and reads the exact format.
Validated live (harness): inject coop_tone -> kill -> relaunch -> auto
re-injected into the new pid, and early enough that the format came back as
Exact. Trim README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Persist audio overrides keyed by game image name (coop_audio_overrides.ini
next to the exe) so a known-bad game is auto-corrected on its next launch:
on attach to a guessed stream the host applies any saved override, and a
manual Override now saves too. A format the hook catches exactly at
IAudioClient::Initialize is auto-saved as that game's override (ground truth);
if it overwrites a differing stored value, a warning is logged. Host-originated
log lines now reach the Log window via IpcServer::host_log (color-coded).
Validated live (harness): a pre-seeded override for coop_tone is auto-applied
over the guess (state -> manual override, 48000/2/16). audio_overrides_test
covers persist/reload/case-insensitive lookup/differing-overwrite. Trim README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a per-stream op channel in AudioRingHeader (op_seq + op_* fields, version
2): the host posts re-measure / override commands, the hook applies them and
re-publishes (bumping format_generation), and the host rebuilds its render
client live on the change. The Audio panel (under Debug details) gains a
"Re-measure rate" button and a rate/channels/bit-depth/format override -- for
when detection is wrong or the channels/bit-depth were unrecoverable.
Also add a debug-only IPC test harness (-DCOOP_TEST_HARNESS, off by default,
absent from the shipped host): a file-based command channel that drives the
host's real UI code paths (inject / audio / re-measure / override / screenshot
/ status) for scripted validation, instead of unreliable synthetic mouse input.
Used to validate live: late-attach to coop_tone@44100 -> measured 44100,
promoted to hooked; override -> 2ch state, re-measure -> reconverge.
Trim the README roadmap to what's left; document the harness + rate_estimator_test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The host used to wait a fixed 1 s for the hook to publish a format and, on
timeout, fall to loopback permanently with no explanation -- which the now
slower (consensus) measurement made common. Restructure the audio thread to
keep the rings live the whole session and alternate: prefer hooked, and while
it isn't ready run loopback (echo) so guests still hear audio, watching the
ring to promote to the no-echo hooked path the instant the hook publishes a
format. Surface the concrete reason loopback is active (no format yet / not
renderable / no ring) in the Audio panel, in amber.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Harden the guessed-stream sample-rate measurement that produced wrong rates
(e.g. 44100 read as ~46205). New rate_estimator.hpp measures over longer
~0.5 s windows, rejects any window that doesn't snap to a standard rate
(standard rates are >8% apart, so a quantization/burst error big enough to
miss one lands in no-man's-land, never on a wrong neighbour), and requires
consensus across windows before committing. If consensus isn't reached it
commits a low-confidence estimate (new AudioFormat_LowConfidence, shown red)
rather than spinning or publishing garbage. Pure logic, unit-tested with
adversarial cadences (rate_estimator_test) incl. the real 46205 bug value.
Add log severity levels: hook logw/loge set LogRecord.level; the host Log
window colors warnings amber and errors red. The low-confidence rate logs a
warning. Protocol -> v15 (new format states); also reserves AudioFormat_Override.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Capture the rendered back buffer to a timestamped PNG next to the exe via
WIC, triggered by F10 (delivered even when unfocused). The capture runs in
render_frame just before Present so it includes the ImGui overlay, and reads
off the GPU so it works regardless of window focus, z-order, or occlusion. A
brief toast confirms the save (drawn the next frame, so it's never in the shot).
F10 chosen to avoid Steam's F12; its WM_SYSKEYDOWN is swallowed so Windows
doesn't enter menu mode. Documented in the Help menu + README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The inline hint still claimed Esc quits (no longer true) and duplicated the
F1 shortcut + RPT-capture note that the Help menu already lists. Remove the
redundant block; the Help menu is the single source for those.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Escape was a leftover spike convenience that quit the tool. Esc is a common
in-game key, so dropping it from a borderless mirror was too easy to do by
accident. Remove the Escape-quit handler and add a File -> Exit menu item
(shown with its Alt+F4 shortcut). Alt+F4 already worked via DefWindowProc ->
WM_CLOSE -> WM_DESTROY; the menu item sets a one-shot flag the main loop reads.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The View-menu verbosity switch reset to off on every launch. Register a
custom ImGui settings handler (a [CoopUI][State] section in coop_layout.ini,
alongside the window layout) so the operator's choice survives restarts.
Toggling it marks settings dirty so ImGui's auto-save writes it back.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hooked audio mirroring played back pitch-shifted on games we inject into
that render at a non-device sample rate (e.g. Godot/Brotato render 44100 Hz
on a 48000 Hz endpoint via WASAPI AUTOCONVERTPCM). We attach to an
already-running game, so the render-hook never saw its IAudioClient::
Initialize and assumed the device mix format -- right channels/bits, wrong
rate -- so 44100 audio was rendered as 48000 (+~1.5 semitones).
Fix: treat a pre-existing client's format as a guess and measure its true
sample rate from the render cadence (frames/sec over a steady-state window,
snapped to the nearest standard rate) before publishing it, deferring
capture until verified. Discard the first measurement window so the
buffer-fill burst at attach time doesn't over-count. Streams created after
we inject still carry their exact Initialize format.
Channels/bit-depth genuinely can't be recovered for a pre-existing client:
AUTOCONVERTPCM hands GetBuffer a fixed staging buffer (no buffer stride to
measure -- confirmed empirically) and WASAPI exposes no API for the format.
They stay the device-mix guess, which is correct for the common case
(engines render stereo float, matching the endpoint). To keep a wrong guess
safe, a VirtualQuery clamp stops the capture copy from ever over-reading the
source buffer when the guessed bytes/frame is too large.
Surface all of this: a per-stream AudioFormatState (known / measuring /
measured rate (ch/bits assumed)) in HookStatus, shown in the Audio panel for
the hooked path and as "device endpoint (known)" for loopback; clear hook
logs; and enriched mirror status strings. Documented in README (Limitations
+ Lessons learned). The loopback fallback was always correct (post-mix at
the device format).
Tests: extract a shared, configurable ToneSource (used by coop_tone and the
hook self-test); coop_tone takes rate/channels/bits/format args. Rewrite
audio_hook_test to a format matrix x both code paths -- see-init (exact) and
guess (rate measured) -- plus a byte-incompatible guess that asserts the
clamp keeps capture safe. The matrix caught the attach-burst over-count.
audio_loopback_test now spawns coop_tone at several source formats to
confirm loopback is format-agnostic. 11/11 x64 + 3/3 x86 pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A game that used the Hooked source with "Sync flip to game frames" left
the overlay sluggish (~5 FPS) after it terminated or was detached: with no
live game the generation never bumps, so wait_for_hooked_frame waited out
its full timeout every iteration.
Gate frame_sync_active() on InjectionPanel::target_state() == Alive, so a
terminated/hung/detached target falls back to normal vsync. Alive means the
process is running and the hook heartbeat is advancing, so this also covers
a frozen game (which would stall the generation the same way). The checkbox
preference is preserved -- frame-sync auto-resumes when a new game is
injected rather than silently unchecking.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolves the DX12 mirror stutter and makes dropped frames observable.
Decouple the D3D11On12 copy from the game's present queue. Submitting the
copy on the game's own present queue (the prior approach) ordered it
correctly but stalled the game's presents: GPU back-pressure, plus the
shared keyed-mutex AcquireSync is a CPU-blocking call on the render
thread. Running it on an independent queue avoids the stall but races the
game's render -> stale frames. Do both: run the copy on our own queue and
order it after the frame with an ID3D12Fence the game's present queue
signals (near-free) and our queue waits on. The present queue is still
recovered for late injection via the ExecuteCommandLists hook (now used to
signal the fence, not host the copy). Producer AcquireSync stays
non-blocking (timeout 0) so a busy mutex drops a mirror frame instead of
stalling the game.
Add drop detection (protocol v12 -> v13). The hook counts captures skipped
because the keyed mutex was busy (VideoShare.frames_dropped); the host
counts published frames it never displayed (generation gaps). The Video
panel shows "Frames lost: N/s capture N/s display", red when nonzero.
This confirmed the game-window-vs-mirror behavior is a display-path
artifact (unfocused windows lose VRR/independent flip), not a capture loss.
Add a one-shot present-pattern log: per distinct swapchain (size/format/
buffer index) and per distinct present-flags value, with DXGI_PRESENT_TEST
spelled out as an occlusion probe that draws nothing -- which is why
Miles Morales shows ~2 presents per captured frame (the test present is
counted but produces no frame).
Docs: add the DX12 capture lessons to the README (rotating back buffer,
fence/own-queue, capture-at-Present decoupling from DWM) and drop the now
-moot DX12 overhead future-work item.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Input thread: controller polling, pad publishing, and rumble forwarding were
driven by the render loop, so a low/synced frame rate throttled how often guest
input reached the game. New InputWorker owns the InputSource and runs poll +
IPC publish + rumble on a dedicated ~1 kHz thread, independent of rendering. The
UI thread reads a copy-safe InputSnapshot for the Controllers panel and relays the
Steam-Input request/active/failed state to/from the worker (Steam init/shutdown now
happen on the worker thread). IpcServer gained a mutex so the worker's publish() /
hook_status() can't race the UI thread starting/stopping the shared-memory channel
(use-after-unmap); InjectionPanel::test_input_ is now atomic. ControllersPanel::draw
takes an InputSnapshot instead of the live InputSource.
Fixed-width numbers: fast-changing readouts (menu-bar FPS/ms, Video pipeline rates +
latency + graph legend, controller poll rates + round-trip sticks, audio buffered ms
+ frames/s) printed with %.0f etc., so they shifted/blurred as values crossed digit
thresholds (99 -> 100) each frame. Padded them to fixed field widths so they stay put.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Frame sync: new "Sync flip to game frames" toggle in the Video mirror panel
(Hooked source only -- WGC frames are delivered by the compositor at monitor
refresh and don't carry the game's true present cadence, so it's disabled there).
When on, the main loop waits for the hook's next published frame (its generation
bump) before rendering and presents with sync interval 0, so the tool flips in
lockstep with the game instead of vsync. The wait pumps messages to stay
responsive and times out after 200 ms so a paused/stalled game can't hang the
overlay. timeBeginPeriod(1) keeps the wait's Sleep(1) granular; links winmm.
render_frame() gained a sync_interval parameter (default 1 = vsync).
Perf graphs: the old graphs drew the tool's frametime and FPS as single same-color
lines. Replaced with a custom multi-series plotter (ImDrawList polylines) that
overlays Tool (blue), Game present (green), and Hook publish (orange) -- or Tool +
WGC capture in WGC mode -- in distinct colors with a colored legend, for both an
FPS (0-144) and a frametime (0-33 ms) view. Game/hook rates come from an EdgeRate
tracker that measures the instantaneous rate the moment each counter advances, so
the lines have real per-frame resolution rather than 0.5 s stair-steps.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Layout persistence: re-enable io.IniFilename (was nullptr "for the spike"),
anchored to a coop_layout.ini next to the exe so window positions/sizes survive
restarts even when Steam launches us under the donor appid (CWD is unreliable).
Path is UTF-8 for ImGui's file IO. When a saved layout is restored at startup,
suppress the computed-default force so it does not clobber the user's positions;
Reset layout (and a fresh install with no .ini) still applies the default.
Log spam: the worker thread re-attaches every audio ring every tick (idempotent),
and set_audio_ring logged unconditionally, flooding the log. Only log when the
ring pointer actually changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Center stack heights set to Controllers 45% / Video 32% / Audio 23% (measured from
the operator's preferred layout): Controllers has the most content, Audio the least.
- Future work: MKB forwarding for Raw Input / DirectInput games (the message +
polling-state path doesn't reach them -- e.g. Trails through Daybreak uses Raw
Input), and reducing the D3D11On12 (DX12) per-frame capture overhead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Feedback from real use: the center column was far too wide and Injection/Log too
narrow; Audio was too tall while Controllers/Video were too short; and the layout
overlapped on launch until "Reset layout" was pressed.
- Columns are now proportional: Injection (left) and Log (right) take 36% each of the
usable width, the center control column 28% -- so the wide panels are wide and the
control panels narrow.
- The center stack heights are evened out (Controllers 30% / Video 38% / Audio 32% of
the column), so Audio no longer hogs it.
- The default layout is now forced (ImGuiCond_Always) for the first few frames after
launch, because the viewport WorkSize isn't trustworthy on frame 0 -- FirstUseEver
was locking in those wrong (overlapping) positions until a manual Reset layout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Games with several concurrent WASAPI render streams (e.g. Spider-Man: Miles
Morales) only had their first ("primary") stream mirrored; the rest kept playing
locally and never reached the guest. Now the render-hook captures + silences EVERY
tracked stream into its own ring (coop_audio_<pid>[_<index>]), each published with
that stream's own detected format (Initialize when caught, else GetMixFormat -- the
per-stream format detection, now actually used per ring rather than only for the
primary). The host creates a ring per stream and mixes the same-format streams with
a soft clip (host/src/audio/audio_mix.hpp); streams whose format differs from the
primary are still silenced (no echo) but skipped from the mix (would need
resampling).
The single-stream case is byte-for-byte unchanged: when only one stream is active
the host passes it through without the mixer, so the common path has no overhead or
fidelity change.
Verified: new audio_mix_test covers the decode/sum/soft-clip/encode math (float32 +
int16); audio_hook_test (x64 + x86) still passes, guarding the primary
capture+silence path against regression; full build x64 + x86 clean; ctest x64
11/11, x86 3/3. Multi-stream mixing against a real multi-stream game needs a live
session to fully confirm.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The FPS readout only measured the host's own render rate, hiding capture stutter.
The Video panel now shows a "Pipeline rates" section:
- Tool render (host FPS / frametime, as before);
- hooked source: Game present (/s, from VideoShare.present_calls deltas), Hook
publish (/s, generation deltas), and capture->display latency avg/min/max ms;
- WGC source: WGC capture (/s) from a new WindowCapture frame-arrival counter
(game present + latency are n/a, since WGC frames aren't game-timestamped).
Latency uses a system-wide clock: protocol v11->v12 adds VideoShare.present_qpc,
stamped by the hook at publish (publish_video_frame); the host measures
now_qpc - present_qpc per newly published frame, windowed to min/avg/max each second.
Verified: present_hook_test (x64 + x86) now asserts present_qpc is stamped; full
build x64 + x86 clean; ctest x64 9/9, x86 3/3. The live rate/latency numbers need a
real game mirroring to read meaningfully; wiring validated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Games that ClipCursor / re-center via SetCursorPos while focused trap the operator's
mouse (the focus spoof makes them think they're always focused), so the operator
can't reach the overlay. The Focus subsystem now inline-hooks ClipCursor and
SetCursorPos (stdcall trampolines): while "release" is requested it forces
ClipCursor(NULL) and swallows the re-centering SetCursorPos; otherwise it passes them
through. It frees any existing clip at install and re-frees each worker tick (covers
one-time clippers and a runtime clip->release toggle).
Host: protocol v10->v11 adds HookControl::allow_cursor_clip (0 = release, the
default). The Injection panel gets a "Release operator cursor" checkbox and an F2
hotkey (InjectionPanel::toggle_cursor_release); default released, since the guest
plays via the pad so the game's clip is operator-only.
Verified: full build x64 + x86 clean; ctest x64 9/9, x86 3/3. The cursor behavior
against a real clipping game (Trails through Daybreak) needs a live injected session
to confirm; logic reviewed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Makes it possible to isolate an input->tool problem from a tool->game one in the
Controllers panel (under Debug details):
- which backend fed each slot is already shown via the per-pad source tag
("XInput #0" / "Steam Input") in the Incoming section;
- a new "Round-trip" table shows, per slot, the state we forwarded (the active
backend's pad) next to what the game actually read back, highlighting matches.
For the round-trip, the XInput hook now echoes the state it returns to the game into
the status (protocol v9->v10: per-slot read_state in HookStatus).
Verified: hook_selftest (x64 + x86) asserts the hook records the game-read state;
full build x64 + x86 clean; ctest x64 9/9, x86 3/3.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The XInput hook used to swallow XInputSetState; now it records the requested
left/right motor speeds into the status back-channel (protocol v8->v9: per-slot
rumble_left/right in HookStatus). Each frame the host reads them and, only on
change, drives the guest's actuator via the active backend:
- XInput: XInputSetState on the guest's slot (the open question is whether Steam's
RPT virtual pad accepts vibration and routes it to the guest -- needs live RPT);
- Steam Input: SteamInput TriggerVibration on the slot's controller handle, with the
XInput fallback for slots Steam isn't driving.
InputSource gains a set_rumble(slot,left,right) hook (default no-op) implemented by
both backends; SteamInputSource now tracks per-slot controller handles + which slots
it drives.
Verified: hook_selftest (x64 + x86) now asserts the hook records the rumble from
XInputSetState into the status; full build x64 + x86 clean; ctest x64 9/9, x86 3/3.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Forward the host window's clicks and keystrokes into the injected game so guests
can drive menus / "Press Start" / text entry that a pad can't.
Protocol (v7->v8): new HookSubsys_Mkb and an SPSC MkbRing of MkbEvents in
SharedBlock (host produces, hook consumes); push/pop helpers.
Hook (hook/src/mkb_hook.cpp, new subsystem): a worker-loop pump drains the ring at
~5 ms and PostMessageW's the matching window messages (WM_KEY*/WM_CHAR, mouse
buttons, WM_MOUSEWHEEL) to the game's main window; it also inline-hooks user32
GetAsyncKeyState / GetKeyboardState / GetCursorPos (stdcall trampolines per the x86
rule) to report a synthesized state so polling games react too. Removing the
subsystem clears all synthesized keys (no stuck input).
Host: the Injection panel gets a "Mouse + keyboard forwarding" subsystem toggle
(opt-in, default off -- the toggle is the hook). host/src/inject/mkb_forward.cpp
reads ImGui IO each frame and forwards only when the host window is focused and
ImGui isn't capturing the event; keyboard always, mouse only while mirroring (clicks
+ wheel, not movement). Mouse coords are mapped through the letterbox to game-client
space (host/src/inject/mkb_map.hpp), accounting for WGC-of-decorated-window vs
hooked/borderless. RawInput/DirectInput games are out of scope for this version.
Verified: new mkb_ring_test + mkb_map_test pass; full build x64 + x86 clean; ctest
x64 9/9 and x86 3/3 green (no regression from the protocol bump). The subsystem is
opt-in, so it can't affect existing behavior unless enabled; the end-to-end
click-into-game path needs live Remote Play + a real game to confirm.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
"Forward synthetic test input" is a controller-debug aid, so it now lives in the
Controllers panel (gated behind Debug details, disabled until the XInput hook is
attached) instead of the Injection panel. ControllersPanel owns the flag and exposes
test_input(); main feeds it into InjectionPanel::set_test_input each frame, so the
existing synthetic-pad substitution in publish() is unchanged.
Verified: x64 build green; review (default view no longer shows it in Injection;
appears in Controllers under Debug details).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The live indicator compared frames_rendered to the previous UI frame's value, but
audio buffers release in bursts so most frames saw no change -- the cell flickered
between a green dot and grey "idle". Now each stream remembers when it last advanced
and reads "live" for a short window (0.4 s) afterwards, with a ~2 Hz frames/s
estimate next to it; otherwise "idle". Steady and readable for multi-stream games.
Verified: x64 build green. Full visual confirmation needs an injected, audio-playing
game with the per-stream table open (Debug details); logic reviewed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Panels used to Begin at cascade positions, so they overlapped and clipped. A new
apply_panel_layout() in app_chrome positions/sizes each panel from the main
viewport work area (ImGuiCond_FirstUseEver, still movable): Injection left column
full height (room for hook diagnostics), Controllers/Video/Audio stacked in the
center column, Log right edge full height (max room for the log stream). Added a
"View -> Reset layout" menu item (request_layout_reset / apply_layout_end_frame
re-apply the defaults once via ImGuiCond_Always). Each panel now calls
apply_panel_layout(Panel::X) instead of its own ad-hoc SetNextWindowPos/Size.
Verified live: captured the host overlay -- Injection (left, full height),
Controllers/Video/Audio (center stack), Log (right, full height), no overlap among
the panels. x64 build + ctest green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New host/src/inject/window_list.{hpp,cpp} enumerates visible, titled, non-tool
top-level (alt-tab-style) windows via EnumWindows -- root-owner only, our own
process excluded -- and maps each to its owning pid + image name. The Injection
panel now defaults to this window list (each row "title [process.exe pid]", with a
filter over title or process), since there are far fewer windows than processes and
a window maps straight to the HWND the capturer wants. The full process list stays
as the advanced picker under Debug details.
Verified live: launched the host and captured its window -- the picker lists real
windows (Discord/Firefox/Explorer/...) in "title [exe pid]" form, filter present,
and the host's own window correctly excluded. x64 build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In the Terminated state the Injection panel now offers a Re-attach button: it
refreshes the process list and finds live processes whose image name matches the
original target's (case-insensitive). Exactly one match -> tear down the stale IPC
channel and inject into the new pid via the normal path; several matches -> don't
guess, filter the picker to the name and prompt the operator to pick one; none ->
report it. Saves hunting for a relaunched game's new pid in the list.
Verified: x64 build + ctest 7/7 green. Re-attach is a Terminated-state button flow,
so its end-to-end behavior is best confirmed live (ImGui clicks can't be scripted);
logic reviewed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Injection panel kept showing "Attached" after the game exited. Now it tracks
target liveness each frame (InjectionPanel::tick from the main loop, independent of
panel visibility):
- a SYNCHRONIZE|QUERY process handle taken at inject time -> WaitForSingleObject
detects the process exiting (Terminated);
- the hook heartbeat stalling for ~2 s while the process still exists flags a
distinct Hung state (games here can freeze without exiting).
The panel shows a clear colored banner per state and disables the subsystem
hook/unhook controls and the synthetic-input toggle when the target isn't alive.
game_hwnd() returns null once Terminated, so the Video and Audio panels drop to
idle instead of chasing a dead window.
Verified: x64 build + ctest 7/7 green; host launches and renders the panels without
regression (screenshot smoke test). The interactive terminated/hung visual against a
real game is best confirmed in a live session (ImGui injection can't be GUI-scripted).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Only the shipping artifacts (coop_host.exe, coop_hook.dll, coop_hook_x86.dll,
coop_inject_x86.exe, steam_api64.dll, steam_input_actions.vdf) now land in the
bin/<config>/ root, so it can be copied wholesale into a donor game folder. Test
exes (plus the coop_tone fixture) build into bin/<config>/tests/ and the dev probes
into bin/<config>/tools/, via a new coop_output_subdir() CMake helper.
The probes resolve coop_hook.dll / the x86 injector from the deployable root one
level up (new common/coop/tool_paths.hpp: deployed_artifact_path checks next-to-exe
then parent). coop_tone is co-located with the tests so audio_loopback_test's
"spawn coop_tone.exe next to me" lookup is unchanged.
Verified from a clean bin/: root holds only deployables; ctest x64 7/7 and x86 3/3
green (incl. audio_loopback_test driving coop_tone from tests/).
Also convert the roadmap Planned list to bullets and drop this (now-done) item.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Making Steam Input the default backend silently broke input forwarding. Merely
initializing Steam Input activates Steam's in-process XInput interception, which
hides controllers from XInputGetState unless they're bound to our action set for
the running appid. With no such binding (the normal case for a donor appid) Steam
Input reports zero controllers AND XInput now sees nothing -> no input at all.
Reproduced with coop_steam_input_probe: without Steam, XInput slot 0 is seen;
with Steam Input initialized, 0 Steam controllers and the XInput fallback goes
empty.
Default to XInput (RPT delivers guest pads there and it works) and make Steam
Input an opt-in Controllers-panel toggle that switches the backend at runtime;
turning it off restores XInput. All 6 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The host sampled the shared backbuffer copy through a view typed exactly like the
game's backbuffer. For games whose backbuffer is an *_SRGB format (e.g. Life is
Strange: Before the Storm -- confirmed R8G8B8A8_UNORM_SRGB / fmt 29 via the hook
log), the GPU decoded sRGB->linear on the sample, and the host then wrote those
linear values straight to its plain-UNORM swapchain with no re-encode, so the
mirror came out noticeably darker than the game.
Sample the copy as the plain-UNORM sibling of the format (srgb_to_unorm) so the
bytes pass through unchanged -- matching what WGC already does. The UNORM and
*_SRGB formats share a typeless group, so CopyResource from the producer's sRGB
texture into the host's UNORM copy is allowed. Non-sRGB formats are unaffected.
Adds srgb_format_test locking the mapping. All 6 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add SteamInputSource: initializes SteamAPI + Steam Input, loads a bundled action
manifest via SetInputActionManifestFilePath (no partner-backend config needed),
and reads the GameControls action set into CoopPadState -- falling back to XInput
per slot, and to pure XInput if Steam isn't available, so the host always runs.
Enabled automatically when the Steamworks SDK is vendored at
third_party/steamworks_sdk/ (auto-detected by CMake; gitignored and never
committed -- the build is XInput-only without it). Stages steam_api64.dll + the
manifest next to the host and builds coop_steam_input_probe (a console smoke test).
Verified: the probe initializes against the live Steam client and enumerates
controllers; the host degrades gracefully when launched standalone. All 5 tests
pass. Reading actual controller state needs a pad bound through Steam Input for
the running (donor) appid, which XInput otherwise covers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Drive a nested Win32 sub-build (CMake ExternalProject, re-entrant via
COOP_X86_HELPER_BUILD) from the normal x64 build to produce coop_hook_x86.dll and
a 32-bit coop_inject_x86.exe, staged next to the x64 binaries. The host detects a
WOW64 target with IsWow64Process2 and spawns the helper to load the x86 DLL, since
a 64-bit process can't cleanly inject a 32-bit one. The shared-memory IPC is
fixed-width / bitness-stable, so the x64 host and x86 hook interoperate.
Validated end-to-end against Slaps and Beans (32-bit D3D11): all 15 hooks
installed, heartbeat advancing, the Present hook engaged (shared a 1920x1080
backbuffer -- the real-game video-hook proof Phantom Brave's D3D9 couldn't give),
and status/audio/video/log IPC all crossed the x64<->x86 boundary. coop_audio_probe
now also delegates to the helper for WOW64 targets. All 5 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an injected IDXGISwapChain::Present / Present1 hook as a lower-latency,
border-free alternative to WGC. The hook copies the swapchain backbuffer into a
shared keyed-mutex texture (coop_video_<pid>); the host opens it by name and
samples it. New opt-in HookSubsys_Video (protocol v6 -> v7); the Video mirror
panel gains a WGC vs Hooked source toggle that installs/removes the subsystem.
Verified by present_hook_test (drives a real D3D11 swapchain end-to-end and reads
the rendered pixels back through the shared texture) and against Phantom Brave
(D3D9: hook installs cleanly and stays idle, WGC fallback). All 5 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
