@@ -75,159 +75,129 @@ is removed from this list once done — so the top item is always next. The
self-verifiable tooling / UI / input items come first; the game-pipeline items that
need a real game (and Remote Play) to fully validate come last.
1. **Stage only deployable artifacts directly under `bin/`. ** Today every target —
host, both hook DLLs, the injecto r helper, tests, probes, `coop_tone` — lands in
`bin/<config>/` , so deploying to a donor folder means hand-picking files. Kee p
the **deployable set ** at the `bin/<config>/` root (`coop_host.exe` ,
`coop_hook.dll` , `coop_hook_x86.dll` , `coop_inject_x86.exe` , and — when Steam is
built — `steam_api64.dll` + `steam_input_actions.vdf` ) and push everything else
into subfolders: tests → `bin/<config>/tests/` , debug/probe tools
(`coop_audio_probe` , `coop_input_probe` , `co op_ tone` , `coop_steam_input_probe` ) →
`bin/<config>/tools/` . * How: * give those targets a per-target
`RUNTIME_OUTPUT_DIRECTORY[_<CONFIG>]` (the global `CMAKE_RUNTIME_OUTPUT_DIRECTORY`
stays the deployable root; a small helper or `set_target_properties` overrides the
non-deployable ones). The x86 sub-build must still stage `coop_hook_x86.dll` +
`coop_inject_x86.exe` into the deployable root while its x86 * test * exes go to
`tests/` . Watch the cross-target paths: `audio_loopback_test` spawns
`coop_tone.exe` , and the host's post-build copy of `steam_api64.dll` / the `.vdf`
must follow the host. End result: copying `bin/<config>/` non-recursively yields a
clean deployable bundle .
2. **Detect a terminated target and reflect it in the UI. ** The Injection panel keeps
showing "Attached" after the game exits. Add a **Terminated ** state: the host
already knows the target pid and tracks a DLL heartbeat (`InjectionPanel` ); on top
of that, hold the `OpenProcess` handle from injection (or re-open wi th
`PROCESS_QUERY_LIMITED_INFORMATION` ) and poll `GetExitCodeProcess` /
`WaitForSingleObject(h, 0)` each tick. When the process is gone, switch to
Terminated, gray out / d isa ble the per-subsystem controls and mirror toggles, and
show a clear banner; the Video and Audio panels should drop to idle (their hook
channels are stale) rather than freezing on the last liv e frame/state. Note a live
process isn't proof it's running — also flag a **stalled heartbeat ** (no advance
for ~2 s while the process still exists) as a distinct "hung / not responding"
state, since games here can freeze without exiting.
3. **Re-attach to a relaunched target. ** A killed-and-relaunched game gets a new pid,
but the UI still holds the stale one. In the Terminated state (task 2), remember
the target's image name (the panel already keeps the selected exe name) and offer a
**Re-attach ** button that injects only if a live process with that * same name *
exists, rebinding the IPC server to the new pid via the existing `inject_dll` path.
If several live processes share that name, don't guess — surface the matches in the
picker (task 4) for a manual choice.
4. **Select targets by window, not just process. ** A flat process list is fine as an
advanced/debug view, but the default should be a **window list ** — there are far
fewer top-level windows than processes, and a window directly yields the HWND the
WGC capturer and focus spoof already want. * How: * add a window enumerator
(`EnumWindows` , keeping visible, titled, non-tool top-level windows —
`IsWindowVisible` , `GetWindowTextLength > 0` , exclude `WS_EX_TOOLWINDOW` and our own
HWND, resolve to the root owner) and map each via `GetWindowThreadProcessId` → pid →
image name. Show **title + process name + pid ** with a filter box like the process
list; injecting by window injects into its pid and hands the HWND straight to
capture. Keep the process list behind "Debug details" as the advanced path.
5. **Auto-size and lay out the overlay windows so none need manual resizing. ** Panels
currently `Begin` at default cascade positions, so they overlap and clip. Give each
`ImGuiWindowFlags_AlwaysAutoResize` and an initial position computed from
`ImGui::GetMainViewport()->WorkPos/WorkSize` , applied with `ImGuiCond_FirstUseEver`
(still movable), plus a **View → Reset layout ** menu item that re-applies it. Target
layout: **Injection ** left/top (room to grow downward for hook diagnostics);
**Controllers ** top-center; **Video mirror ** center, below Controllers; **Audio
mirror** below Video; **Log ** right edge, full height (most room for the log
stream). Auto-resize fits these because they're all control/debug panels — the live
mirror image is drawn to the whole host window * behind * the overlay, not inside a
panel.
6. **Fix the Audio panel "live" column. ** It overlays a green dot and grey "idle"
because liveness is recomputed each frame from the per-stream `frames_rendered`
delta, which is zero on most frames (buffers release in bursts), so it flickers.
Replace it with a **debounced activity indicator ** : keep a per-stream "last
advanced" timestamp (the panel already stores the previous frame counts) and show
**live ** if frames advanced within the last ~300– 500 ms, else **idle ** — optionally
a small frames/s or activity bar so multi-stream games read clearly.
7. **Move the synthetic-input toggle to the Controllers panel, under Debug details. **
The "Forward synthetic test input" checkbox is a controller-debugging aid, so move
it out of the Injection panel into `ControllersP anel` and gate it behind
`debug_details` . The publish path is unchanged (the Injection panel already
substitutes the synthetic pattern in `publish` ); the flag just moves with it (or is
passed from Controller s in to the publish call).
8. **Mouse & keyboard forwarding (messages + polling-state hooks). ** Forward guest
clicks and keystrokes into the unfocused game via a new **MKB hook subsystem * * in
`coop_hook.dll` — the toggle * is * the hook (not installed → no forwarding).
* Delivery: * `PostMessage` window-message input (`WM_KEYDOWN` /`WM_KEYUP` /`WM_CHAR` ,
`WM_*BUTTONDOWN` /`UP` , `WM_MOUSEWHEEL` ) to the game HWND, **plus ** hook
`GetAsyncKeyState` / `GetKeyboardState` / `GetCursorPos` in the DLL so polling games
see the synthesized keyboard/cursor state (RawInput and DirectInput games are out of
scope for this version). * Keyboard * is always forwarded; * mouse * only while video is
mirrored (otherwise the operator can't see where they click), and only **clicks +
wheel, not movement** (one cursor can't be in two places). * Coordinate mapping * (the
part that must be exact): WGC + decorated windowed → translate by the window
decoration / client-area offset; hooked capture → relative to the mirrored viewport
only (decorations aren't mirrored); borderless → the same under both backends.
* Critical gating: * forward only when the host's main window is focused **and ** ImGui
doesn't want the event (`ImGuiIO::WantCaptureMouse` / `WantCaptureKeyboard` ), so
interacting with the overlay's own windows never leaks input into the game. The host
sends MKB events to the hook over a new (or extended) IPC region.
9. **Rumble / haptics forwarding (both backends). ** Currently unsupported — the XInput
hook swallows `XInputSetState` . Add a reverse path: the hook captures the game' s
`XInputSetState` (left/right motor) and publishe s it over a hook→host channel (the
back-channel already exists), and the host drives the guest's actuators per backend —
**XInput: ** call `XInputSetState` on the guest's slot (the viability unknown is
whether Steam's RPT virtual pad accepts vibration and routes it to the guest);
**Steam Input: ** `SteamInput()->TriggerVibration` / `Legacy_TriggerHapticPulse` .
Map each guest slot to the right actuator.
10. **Per-bac ken d input debug visualization. ** To separate "wrong input * into * the
tool" from "wrong input * out to * the game", show three distinct views in the
Controller s p anel (under Debug details): (a) **received via XInput ** (raw
`XInputSource` state), (b) **received via Steam Input ** (raw `SteamInputSource`
action values) — so it's obvious which backend delivered what — and (c)
**forwarded to the game ** (the `PadInfo` we write to shared memory, alongside what
the game actually read back via the hook's per-slot channel). The hook already
reports per-slot poll counts; extend it to echo the last state the game read so (c)
is a true round-trip.
11. **Release the mouse cursor for cursor-clipping games. ** Games that confine the
cursor while focused (e.g. Trails through Daybreak via `ClipCursor` / per-frame
`SetCursorPos` re-centering) trap the operator's mouse permanently, because the
focus spoof mak es the g ame believe it's always focused — so the operator can' t
reach the ImGui overlay. Add a cursor-release capability to the Focus subsystem:
hook `ClipCursor` (force `ClipCursor(NULL)` and swallow the game's clip) and the
re-centering `SetCursorPos` , gated by a new host→hook flag driven by a host
toggle + hotkey. Defaults to released (the guest plays via the pad, so the game's
own cursor clip is operator-only), with the option to re-enable clipping per game .
12. **Real capture metrics + latency stats. ** The current FPS readout only measures
how fast the host renders its own window, which hides capture stutter. Add a
three-line frametime/FPS graph — **game present rate ** (from `VideoShare` present
deltas), **capture rate ** (generation deltas / WGC arrivals), and **tool render
rate** — plus a **capture→display latency ** stat: stamp each published frame with
a `QueryPerformanceCounter` value in `VideoShare` , and the host reports
`host-present QPC − game-present QPC` (min/avg/max ms) for the matched frame. QPC
is system-wide, so the two processes' timestamps compare directly.
13. **DX12 hooked capture (Spider-Man: Miles Morales). ** Miles Morales is D3D12, so
the Present hook fires but `GetBuffer(0)` as `ID3D11Texture2D` fails (the
backbuffer is an `ID3D12Resource` ) and the hook idles; WGC works but stutters. Add
a D3D12 path via a **D3D11On12 bridge ** : capture the game's D3D12 command queue
(hook `ID3D12CommandQueue::ExecuteCommandLists` ), create an `ID3D11On12Device` ,
`CreateWrappedResource` around the backbuffer, and `CopyResource` into the
* existing * D3D11 shared keyed-mutex texture — so the host side is unchanged.
14. **Multi-stream audio capture + mixing, with per-stream format detection. ** Games
with several concurrent WASAPI render streams (e.g. Miles Morales) only get their
first ("primary") stream mirrored today; the rest keep playing locally and never
reach the guest. Capture every tracked render stream into its own shared ring,
silence each, and add a host-side mixer that resamples each ring to the render
format and sums them (with soft-clip). **Fold in real per-stream format
detection** here, since it touches the same hook + ring plumbing: a stream that
already existed when we injected is never seen at `Initialize` , so the hook
currently assumes the device **mix format ** and a shared-mode stream opened at a
different format comes out wrong-pitched. Resolve each stream's true format (the
stream's own `Initialize` when caught, else the original `GetMixFormat` ) so every
mixed ring is pitched correctly.
- **Detect a terminated target and reflect it in the UI. ** The Injection panel keeps
showing "Attached" afte r t he game exits. Add a **Terminated ** state: the host
already knows the target pid and tracks a DLL heartbeat (`InjectionPanel` ); on to p
of that, hold the `OpenProcess` handle from injection (or re-open with
`PROCESS_QUERY_LIMITED_INFORMATION` ) and poll `GetExitCodeProcess` /
`WaitForSingleObject(h, 0)` each tick. When the process is gone, switch to
Terminated, gray out / disable the per-subsystem controls and mirror toggles, and
show a clear banner; the Video and Audio panels should dr op to idle (their hook
channels are stale) rather than freezing on the last live frame/state. Note a live
process isn't proof it's running — also flag a **stalled heartbeat ** (no advance
for ~2 s while the process still exists) as a distinct "hung / not responding"
state, since games here can freeze without exiting.
- **Re-attach to a relaunched target.** A killed-and-relaunched game gets a new pid,
but the UI still holds the stale one. In the Terminated state, remember the
target's image name (the panel already keeps the selected exe name) and offer a
**Re-attach ** button that injects only if a live process with that * same name *
exists, rebinding the IPC server to the new pid via the existing `inject_dll` path .
If several live processes share that name, don't guess — surface the matches in the
window/process picker for a manual choice.
- **Select targets by window, not just process.** A flat process list is fine as an
advanced/debug view, but the default should be a **window list ** — there are far
fewer top-level windows than processes, and a window directly yields the HWND the
WGC capturer and focus spoof already want. * How: * add a window enumerator
( `EnumWindows` , keeping visible, titled, non-tool top-level windows —
`IsWindowV isi ble` , `GetWindowTextLength > 0` , exclude `WS_EX_TOOLWINDOW` and our own
HWND, resolve to the root owner) and map each via `GetWindowThreadProcessId` → pid →
image name. Show **title + process name + pid ** with a filter box lik e the process
list; injecting by window injects into its pid and hands the HWND straight to
capture. Keep the process list behind "Debug details" as the advanced path.
- **Auto-size and lay out the overlay windows so none need manual resizing.** Panels
currently `Begin` at default cascade positions, so they overlap and clip. Give each
`ImGuiWindowFlags_AlwaysAutoResize` and an initial position computed from
`ImGui::GetMainViewport()->WorkPos/WorkSize` , applied with `ImGuiCond_FirstUseEver`
(still movable), plus a **View → Reset layout ** menu item that re-applies it. Target
layout: **Injection ** left/top (room to grow downward for hook diagnostics);
**Controllers ** top-center; **Video mirror ** center, below Controllers; **Audio
mirror** below Video; **Log ** right edge, full height (most room for the log
stream). Auto-resize fits these because they're all control/debug panels — the live
mirror image is drawn to the whole host window * behind * the overlay, not inside a
panel.
- **Fix the Audio panel "live" column.** It overlays a green dot and grey "idle"
because liveness is recomputed each frame from the per-stream `frames_rendered`
delta, which is zero on most frames (buffers release in bursts), so it flickers.
Replace it with a **debounced activity indicator ** : keep a per-stream "last
advanced" timestamp (the panel already stores the previous frame counts) and sh ow
**live ** if frames advanced within the last ~300– 500 ms, else **idle ** — optionally
a small frames/s or activity bar so multi-stream games read clearly.
- **Move the synthetic-input toggle to the Controllers panel, under Debug details.**
The "Forward synthetic test input" checkbox is a controller-debugging aid, so move
it out of the Injection panel into `ControllersPanel` and gate it behind
`debug_details` . The publish path is unchanged (the Injection panel already
substitutes the synthetic pattern in `publish` ); the flag just moves with it (or is
passed from Controllers into the publish call).
- **Mouse & keyboard forwarding (messages + polling-state hooks).** Forward guest
clicks and keystrokes into the unfocused game via a new **MKB hook subsystem ** in
`coop_hook.dll` — the toggle * is * the hook (not installed → no forwarding).
* Delivery: * `PostMessage` window-message input (`WM_KEYDOWN` /`WM_KEYUP` /`WM_CHAR` ,
`WM_*BUTTONDOWN` /`UP` , `WM_MOUSEWHEEL` ) to the game HWND, **plus ** hook
`GetAsyncKeyState` / `GetKeyboardState` / `GetCursorPos` in the DLL so polling games
see the synthesized keyboard/cursor state (RawInput and DirectInput games are out of
scope for this version). * Keyboard * is always forwarded; * mouse * only while video is
mirrored (otherwise the operator can't see where they click), and only **clicks +
wheel, not movement** (one cursor can't be in two places). * Coordinate mapping * (the
part that must be exact): WGC + decorated windowed → translate by the window
decoration / client-area offset; hooked capture → relative to the mirrored viewport
only (decorations aren't mirrored); borderless → the same under both backends.
* Critical gating: * forward only when the host's main window is focused **and ** ImGui
doesn't want the event (`ImGuiIO::WantCaptureMouse` / `WantCaptureKeyboard` ), so
interacting with the overlay's own windows never leaks input into the game. The host
sends MKB events to the hook over a new (or extended) IPC region.
- **Rumble / haptics forwarding (both backends).** Currently unsupported — the XInput
hook swallows `XInputSetState` . Add a reverse path: the hook captures the game's
`XInputSetState` (left/right motor) and publishes it over a hook→host ch ann el (the
back-channel already exists), and the host drives the guest's actuators per backend —
**XInput: ** call `XInputSetState` on the guest's slot (the viability unknown is
whether Steam's RPT virtual pad accepts vibration and route s it to the guest);
**Steam Input: ** `SteamInput()->TriggerVibration` / `Legacy_TriggerHapticPulse` .
Map each guest slot to the right actuator.
- **Per-backend input debug visualization.** To separate "wrong input * into * the
tool" from "wrong input * out to * the game", show three distinct views in the
Controllers panel (under Debug details): (a) **received via XInput ** (raw
`XInputSource` state), (b) **received via Steam Input ** (raw `SteamInputSource`
action values) — so it's obvious which backend delivered what — and (c)
**forwarded to the game ** (the `PadInfo` we write to shared memory, alongside what
the game actually read back via the hook's per-slot channel). The hook already
reports per-slot poll counts; extend it to echo the last state the game read so (c)
is a true round-trip.
- **Release the mo use cursor for cursor-clipping games.** Games that confine the
cursor while focused (e.g. Trails through Daybreak via `ClipCursor` / per-frame
`SetCursorPos` re-centering) trap the operator's mouse permanently, because the
focus spoof makes the game believe it's always focused — so the operator can't
reach the ImGui overlay. Add a cursor-release capability to the Focus subsystem:
hook `ClipCursor` (force `ClipCursor(NULL)` and swallow the game's clip) and the
re-centering `SetCursorPos` , gated by a new host→hook flag driven by a host
toggle + hotkey. Defaults to released (the guest plays via the pad, so the game's
own cursor clip is operator-only), with the option to re-enable clipping per game.
- **Real capture metrics + latency stats.** The current FPS readout only measure s
how fast the host render s its own window, which hides capture stutter. Add a
three-line frametime/FPS graph — **game present rate ** (from `VideoShare` present
deltas), **capture rate ** (generation deltas / WGC arrivals), and **tool render
rate** — plus a **capture→display latency ** stat: stamp each published frame with
a `QueryPerformanceCounter` value in `VideoShare` , and the host reports
`host-present QPC − game-present QPC` (min/avg/max ms) for the matched frame. QPC
is system-wide, so the two processes' timestamps compare directly.
- **DX12 hoo ked capture (Spider-Man: Miles Morales).** Miles Morales is D3D12, so
the Present hook fires but `GetBuffer(0)` as `ID3D11Texture2D` fails ( the
backbuffer i s an `ID3D12Resource` ) and the hook idles; WGC works but stutters. Add
a D3D12 path via a **D3D11On12 bridge ** : capture the game's D3D12 command queue
(hook `ID3D12CommandQueue::ExecuteCommandLists` ), create an `ID3D11On12Device` ,
`CreateWrappedResource` around the backbuffer, and `CopyResource` into the
* existing * D3D11 shared keyed-mutex texture — so the host side is unchanged.
- **Multi-stream audio capture + mixing, with per-stream format detection.** Games
with several concurrent WASAPI render streams (e.g. Miles Morales) only get their
first ("primary") stream mirrored today; the rest keep playing locally and never
reach the guest. Capture every tracked render stream into its own shared ring,
silence each, and add a host-side mixer that resamples each ring to the render
format and sums them (with soft-clip). **Fold in real per-stream format
detection** here, since it touch es the s ame hook + ring plumbing: a stream tha t
al ready existed when we injected is never seen at `Initialize` , so the hook
currently assumes the device **mix format ** and a shared-mode stream opened at a
different format comes out wrong-pitched. Resolve each stream's true format (the
stream's own `Initialize` when caught, else the original `GetMixFormat` ) so every
mixed ring is pitched correctly .
### Future work
@@ -332,9 +302,11 @@ input layer sees a real state change. `disable_mask` (hex bits `0x1`=input
Present-hook crash was isolated.
Both auto-detect a 32-bit (WOW64) target and inject via `coop_inject_x86.exe` +
`coop_hook_x86.dll` , exactly like the host. Run them from `bin/<config>/` .
**Kill the game between runs ** — the loaded DLL locks `coop_hook.dll` against the
next rebuild.
`coop_hook_x86.dll` , exactly like the host. The probes build into
`bin/<config>/tools/` (the deployable `bin/<config>/` root holds only shipping
artifacts; tests build into `bin/<config>/tests/` ) and resolve `coop_hook.dll` from
the root one level up, so run them from there. **Kill the game between runs ** — the
loaded DLL locks `coop_hook.dll` against the next rebuild.
## Running the tool (manual, end-to-end)