@@ -75,159 +75,129 @@ is removed from this list once done — so the top item is always next. The
self-verifiable tooling / UI / input items come first; the game-pipeline items that
self-verifiable tooling / UI / input items come first; the game-pipeline items that
need a real game (and Remote Play) to fully validate come last.
need a real game (and Remote Play) to fully validate come last.
1. Stage only deployable artifacts directly under `bin/`. oday every target —
- Detect a terminated target and reflect it in the UI. ** The Injection panel keeps
host, both hook DLLs, the injecto r helper, tests, probes, `coop_tone` — lands in
showing "Attached" afte r t he game exits. Add a **Terminated ** state: the host
`bin/<config>/` , so deploying to a donor folder means hand-picking files. Kee p
already knows the target pid and tracks a DLL heartbeat (`InjectionPanel` ); on to p
the **deployable set ** at the `bin/<config>/` root (`coop_host.exe` ,
of that, hold the `OpenProcess` handle from injection (or re-open with
`coop_hook.dll` , `coop_hook_x86.dll` , `coop_inject_x86.exe` , and — when Steam is
`PROCESS_QUERY_LIMITED_INFORMATION` ) and poll `GetExitCodeProcess` /
built — `steam_api64.dll` + `steam_input_actions.vdf` ) and push everything else
`WaitForSingleObject(h, 0)` each tick. When the process is gone, switch to
into subfolders: tests → `bin/<config>/tests/` , debug/probe tools
Terminated, gray out / disable the per-subsystem controls and mirror toggles, and
(`coop_audio_probe` , `coop_input_probe` , `co op_ tone` , `coop_steam_input_probe` ) →
show a clear banner; the Video and Audio panels should dr op to idle (their hook
`bin/<config>/tools/` . * How: * give those targets a per-target
channels are stale) rather than freezing on the last live frame/state. Note a live
`RUNTIME_OUTPUT_DIRECTORY[_<CONFIG>]` (the global `CMAKE_RUNTIME_OUTPUT_DIRECTORY`
process isn't proof it's running — also flag a **stalled heartbeat ** (no advance
stays the deployable root; a small helper or `set_target_properties` overrides the
for ~2 s while the process still exists) as a distinct "hung / not responding"
non-deployable ones). The x86 sub-build must still stage `coop_hook_x86.dll` +
state, since games here can freeze without exiting.
`coop_inject_x86.exe` into the deployable root while its x86 * test * exes go to
- **Re-attach to a relaunched target.** A killed-and-relaunched game gets a new pid,
`tests/` . Watch the cross-target paths: `audio_loopback_test` spawns
but the UI still holds the stale one. In the Terminated state, remember the
`coop_tone.exe` , and the host's post-build copy of `steam_api64.dll` / the `.vdf`
target's image name (the panel already keeps the selected exe name) and offer a
must follow the host. End result: copying `bin/<config>/` non-recursively yields a
**Re-attach ** button that injects only if a live process with that * same name *
clean deployable bundle .
exists, rebinding the IPC server to the new pid via the existing `inject_dll` path .
If several live processes share that name, don't guess — surface the matches in the
2. **Detect a terminated target and reflect it in the UI. ** The Injection panel keeps window/process picker for a manual choice.
showing "Attached" after the game exits. Add a **Terminated ** state: the host
- **Select targets by window, not just process.** A flat process list is fine as an
already knows the target pid and tracks a DLL heartbeat (`InjectionPanel` ); on top
advanced/debug view, but the default should be a **window list ** — there are far
of that, hold the `OpenProcess` handle from injection (or re-open wi th
fewer top-level windows than processes, and a window directly yields the HWND the
`PROCESS_QUERY_LIMITED_INFORMATION` ) and poll `GetExitCodeProcess` /
WGC capturer and focus spoof already want. * How: * add a window enumerator
`WaitForSingleObject(h, 0)` each tick. When the process is gone, switch to
( `EnumWindows` , keeping visible, titled, non-tool top-level windows —
Terminated, gray out / d isa ble the per-subsystem controls and mirror toggles, and
`IsWindowV isi ble` , `GetWindowTextLength > 0` , exclude `WS_EX_TOOLWINDOW` and our own
show a clear banner; the Video and Audio panels should drop to idle (their hook
HWND, resolve to the root owner) and map each via `GetWindowThreadProcessId` → pid →
channels are stale) rather than freezing on the last liv e frame/state. Note a live
image name. Show **title + process name + pid ** with a filter box lik e the process
process isn't proof it's running — also flag a **stalled heartbeat ** (no advance
list; injecting by window injects into its pid and hands the HWND straight to
for ~2 s while the process still exists) as a distinct "hung / not responding"
capture. Keep the process list behind "Debug details" as the advanced path.
state, since games here can freeze without exiting.
- **Auto-size and lay out the overlay windows so none need manual resizing.** Panels
currently `Begin` at default cascade positions, so they overlap and clip. Give each
3. **Re-attach to a relaunched target. ** A killed-and-relaunched game gets a new pid, `ImGuiWindowFlags_AlwaysAutoResize` and an initial position computed from
but the UI still holds the stale one. In the Terminated state (task 2), remember
`ImGui::GetMainViewport()->WorkPos/WorkSize` , applied with `ImGuiCond_FirstUseEver`
the target's image name (the panel already keeps the selected exe name) and offer a
(still movable), plus a **View → Reset layout ** menu item that re-applies it. Target
**Re-attach ** button that injects only if a live process with that * same name *
layout: **Injection ** left/top (room to grow downward for hook diagnostics);
exists, rebinding the IPC server to the new pid via the existing `inject_dll` path.
**Controllers ** top-center; **Video mirror ** center, below Controllers; **Audio
If several live processes share that name, don't guess — surface the matches in the
mirror** below Video; **Log ** right edge, full height (most room for the log
picker (task 4) for a manual choice.
stream). Auto-resize fits these because they're all control/debug panels — the live
mirror image is drawn to the whole host window * behind * the overlay, not inside a
4. **Select targets by window, not just process. ** A flat process list is fine as an panel.
advanced/debug view, but the default should be a **window list ** — there are far
- **Fix the Audio panel "live" column.** It overlays a green dot and grey "idle"
fewer top-level windows than processes, and a window directly yields the HWND the
because liveness is recomputed each frame from the per-stream `frames_rendered`
WGC capturer and focus spoof already want. * How: * add a window enumerator
delta, which is zero on most frames (buffers release in bursts), so it flickers.
(`EnumWindows` , keeping visible, titled, non-tool top-level windows —
Replace it with a **debounced activity indicator ** : keep a per-stream "last
`IsWindowVisible` , `GetWindowTextLength > 0` , exclude `WS_EX_TOOLWINDOW` and our own
advanced" timestamp (the panel already stores the previous frame counts) and sh ow
HWND, resolve to the root owner) and map each via `GetWindowThreadProcessId` → pid →
**live ** if frames advanced within the last ~300– **idle ** — optionally
image name. Show **title + process name + pid ** with a filter box like the process
a small frames/s or activity bar so multi-stream games read clearly.
list; injecting by window injects into its pid and hands the HWND straight to
- **Move the synthetic-input toggle to the Controllers panel, under Debug details.**
capture. Keep the process list behind "Debug details" as the advanced path.
The "Forward synthetic test input" checkbox is a controller-debugging aid, so move
it out of the Injection panel into `ControllersPanel` and gate it behind
5. **Auto-size and lay out the overlay windows so none need manual resizing. ** Panels `debug_details` . The publish path is unchanged (the Injection panel already
currently `Begin` at default cascade positions, so they overlap and clip. Give each
substitutes the synthetic pattern in `publish` ); the flag just moves with it (or is
`ImGuiWindowFlags_AlwaysAutoResize` and an initial position computed from
passed from Controllers into the publish call).
`ImGui::GetMainViewport()->WorkPos/WorkSize` , applied with `ImGuiCond_FirstUseEver`
- **Mouse & keyboard forwarding (messages + polling-state hooks).** Forward guest
(still movable), plus a **View → Reset layout ** menu item that re-applies it. Target
clicks and keystrokes into the unfocused game via a new **MKB hook subsystem ** in
layout: **Injection ** left/top (room to grow downward for hook diagnostics);
`coop_hook.dll` — the toggle * is * the hook (not installed → no forwarding).
**Controllers ** top-center; **Video mirror ** center, below Controllers; **Audio
* Delivery: * `PostMessage` window-message input (`WM_KEYDOWN` /`WM_KEYUP` /`WM_CHAR` ,
mirror** below Video; **Log ** right edge, full height (most room for the log
`WM_*BUTTONDOWN` /`UP` , `WM_MOUSEWHEEL` ) to the game HWND, **plus ** hook
stream). Auto-resize fits these because they're all control/debug panels — the live
`GetAsyncKeyState` / `GetKeyboardState` / `GetCursorPos` in the DLL so polling games
mirror image is drawn to the whole host window * behind * the overlay, not inside a
see the synthesized keyboard/cursor state (RawInput and DirectInput games are out of
panel.
scope for this version). * Keyboard * is always forwarded; * mouse * only while video is
mirrored (otherwise the operator can't see where they click), and only **clicks +
6. **Fix the Audio panel "live" column. ** It overlays a green dot and grey "idle" wheel, not movement** (one cursor can't be in two places). * Coordinate mapping * (the
because liveness is recomputed each frame from the per-stream `frames_rendered`
part that must be exact): WGC + decorated windowed → translate by the window
delta, which is zero on most frames (buffers release in bursts), so it flickers.
decoration / client-area offset; hooked capture → relative to the mirrored viewport
Replace it with a **debounced activity indicator ** : keep a per-stream "last
only (decorations aren't mirrored); borderless → the same under both backends.
advanced" timestamp (the panel already stores the previous frame counts) and show
* Critical gating: * forward only when the host's main window is focused **and ** ImGui
**live ** if frames advanced within the last ~300– **idle ** — optionally
doesn't want the event (`ImGuiIO::WantCaptureMouse` / `WantCaptureKeyboard` ), so
a small frames/s or activity bar so multi-stream games read clearly.
interacting with the overlay's own windows never leaks input into the game. The host
sends MKB events to the hook over a new (or extended) IPC region.
7. Move the synthetic-input toggle to the Controllers panel, under Debug details. ** - Rumble / haptics forwarding (both backends).** Currently unsupported — the XInput
The "Forward synthetic test input" checkbox is a controller-debugging aid, so move
hook swallows `XInputSetState` . Add a reverse path: the hook captures the game's
it out of the Injection panel into `ControllersP anel` and gate it behind
`XInputSetState` (left/right motor) and publishes it over a hook→host chn el (the
`debug_details` . The publish path is unchanged (the Injection panel already
back-channel already exists), and the host drives the guest's actuators per backend —
substitutes the synthetic pattern in `publish` ); the flag just moves with it (or is
**XInput: ** call `XInputSetState` on the guest's slot (the viability unknown
passed from Controller s in to the publish call).
whether Steam's RPT virtual pad accepts vibration and route s it to the guest);
**Steam Input: ** `SteamInput()->TriggerVibration` / `Legacy_TriggerHapticPulse` .
8. **Mouse & keyboard forwarding (messages + polling-state hooks). ** Forward guest Map each guest slot to the right actuator.
clicks and keystrokes into the unfocused game via a new **MKB hook subsystem * * in
- **Per-backend input debug visualization.** To separate "wrong input to * the
`coop_hook.dll` — the toggle * is * the hook (not installed → no forwarding).
tool" from "wrong input * out to * the game", show three distinct views in the
* Delivery: * `PostMessage` window-message input (`WM_KEYDOWN` /`WM_KEYUP` /`WM_CHAR` ,
Controllers panel (under Debug details): (a) **received via XInput ** (raw
`WM_*BUTTONDOWN` /`UP` , `WM_MOUSEWHEEL` ) to the game HWND, **plus ** hook
`XInputSource` state), (b) **received via Steam Input ** (raw `SteamInputSource`
`GetAsyncKeyState` / `GetKeyboardState` / `GetCursorPos` in the DLL so polling games
action values) — so it's obvious which backend delivered what — and (c)
see the synthesized keyboard/cursor state (RawInput and DirectInput games are out of
**forwarded to the game ** (the `PadInfo` we write to shared memory, alongside what
scope for this version). * Keyboard * is always forwarded; * mouse * only while video is
the game actually read back via the hook's per-slot channel). The hook already
mirrored (otherwise the operator can't see where they click), and only **clicks +
reports per-slot poll counts; extend it to echo the last state the game read so (c)
wheel, not movement** (one cursor can't be in two places). * Coordinate mapping * (the
is a true round-trip.
part that must be exact): WGC + decorated windowed → translate by the window
- **Releasee mo use cursor for cursor-clipping games.** Games that confine the
decoration / client-area offset; hooked capture → relative to the mirrored viewport
cursor while focused (e.g. Trails through Daybreak via `ClipCursor` / per-frame
only (decorations aren't mirrored); borderless → the same under both backends.
`SetCursorPos` re-centering) trap the operator's mouse permanently, because the
* Critical gating: * forward only when the host's main window is focused **and ** ImGui
focus spoof makes the game believe it's always focused — so the operator can't
doesn't want the event (`ImGuiIO::WantCaptureMouse` / `WantCaptureKeyboard` ), so
reach the ImGui overlay. Add a cursor-release capability to the Focus subsystem:
interacting with the overlay's own windows never leaks input into the game. The host
hook `ClipCursor` (force `ClipCursor(NULL)` and swallow the game's clip) and the
sends MKB events to the hook over a new (or extended) IPC region.
re-centering `SetCursorPos` , gated by a new host→hook flag driven by a host
toggle + hotkey. Defaults to released (the guest plays via the pad, so the game's
9. **Rumble / haptics forwarding (both backends). ** Currently unsupported — the XInput own cursor clip is operator-only), with the option to re-enable clipping per game.
hook swallows `XInputSetState` . Add a reverse path: the hook captures the game' s
- **Real capture metrics + latency stats.** The current FPS readout only measure
`XInputSetState` (left/right motor) and publishe s it over a hook→host channel (the
how fast the host render s its own window, which hides capture stutter. Add a
back-channel already exists), and the host drives the guest's actuators per backend —
three-line frametime/FPS graph — **game present rate ** (from `VideoShare` present
**XInput: ** call `XInputSetState` on the guest's slot (the viability unknown is
deltas), **capture rate ** (generation deltas / WGC arrivals), and **tool render
whether Steam's RPT virtual pad accepts vibration and routes it to the guest);
rate** — plus a **capture→display latency ** stat: stamp each published frame with
**Steam Input: ** `SteamInput()->TriggerVibration` / `Legacy_TriggerHapticPulse` .
a `QueryPerformanceCounter` value in `VideoShare` , and the host reports
Map each guest slot to the right actuator.
`host-present QPC − (min/avg/max ms) for the matched frame. QPC
is system-wide, so the two processes' timestamps compare directly.
10. Per-bac ken d input debug visualization. ** To separate "wrong input * into * the- DX12 hoo ked capture (Spider-Man: Miles Morales).** Miles Morales is D3D12, so
tool" from "wrong input * out to * the game", show three distinct views in the
the Present hook fires but `GetBuffer(0)` as `ID3D11Texture2D` fails ( the
Controller s p anel (under Debug details): (a) **received via XInput ** (raw
backbuffer i s an `ID3D12Resource` ) and the hook idles; WGC works but stutters. Add
`XInputSource` state), (b) **received via Steam Input ** (raw `SteamInputSource`
a D3D12 path via a **D3D11On12 bridge ** : capture the game's D3D12 command queue
action values) — so it's obvious which backend delivered what — and (c)
(hook `ID3D12CommandQueue::ExecuteCommandLists` ), create an `ID3D11On12Device` ,
**forwarded to the game ** (the `PadInfo` we write to shared memory, alongside what
`CreateWrappedResource` around the backbuffer, and `CopyResource` into the
the game actually read back via the hook's per-slot channel). The hook already
* existing * D3D11 shared keyed-mutex texture — so the host side is unchanged.
reports per-slot poll counts; extend it to echo the last state the game read so (c)
- **Multi-stream audio capture + mixing, with per-stream format detection.** Games
is a true round-trip.
with several concurrent WASAPI render streams (e.g. Miles Morales) only get their
first ("primary") stream mirrored today; the rest keep playing locally and never
11. **Release the mouse cursor for cursor-clipping games. ** Games that confine the reach the guest. Capture every tracked render stream into its own shared ring,
cursor while focused (e.g. Trails through Daybreak via `ClipCursor` / per-frame
silence each, and add a host-side mixer that resamples each ring to the render
`SetCursorPos` re-centering) trap the operator's mouse permanently, because the
format and sums them (with soft-clip). **Fold in real per-stream format
focus spoof mak es the g ame believe it's always focused — so the operator can' t
detection** here, since it touch es the s ame hook + ring plumbing: a stream tha t
reach the ImGui overlay. Add a cursor-release capability to the Focus subsystem:
al ready existed when we injected is never seen at `Initialize` , so the hook
hook `ClipCursor` (force `ClipCursor(NULL)` and swallow the game's clip) and the
currently assumes the device **mix format ** and a shared-mode stream opened at a
re-centering `SetCursorPos` , gated by a new host→hook flag driven by a host
different format comes out wrong-pitched. Resolve each stream's true format (the
toggle + hotkey. Defaults to released (the guest plays via the pad, so the game's
stream's own `Initialize` when caught, else the original `GetMixFormat` ) so every
own cursor clip is operator-only), with the option to re-enable clipping per game .
mixed ring is pitched correctly .
12. **Real capture metrics + latency stats. ** The current FPS readout only measures
how fast the host renders its own window, which hides capture stutter. Add a
three-line frametime/FPS graph — **game present rate ** (from `VideoShare` present
deltas), **capture rate ** (generation deltas / WGC arrivals), and **tool render
rate** — plus a **capture→display latency ** stat: stamp each published frame with
a `QueryPerformanceCounter` value in `VideoShare` , and the host reports
`host-present QPC − (min/avg/max ms) for the matched frame. QPC
is system-wide, so the two processes' timestamps compare directly.
13. **DX12 hooked capture (Spider-Man: Miles Morales). ** Miles Morales is D3D12, so
the Present hook fires but `GetBuffer(0)` as `ID3D11Texture2D` fails (the
backbuffer is an `ID3D12Resource` ) and the hook idles; WGC works but stutters. Add
a D3D12 path via a **D3D11On12 bridge ** : capture the game's D3D12 command queue
(hook `ID3D12CommandQueue::ExecuteCommandLists` ), create an `ID3D11On12Device` ,
`CreateWrappedResource` around the backbuffer, and `CopyResource` into the
* existing * D3D11 shared keyed-mutex texture — so the host side is unchanged.
14. **Multi-stream audio capture + mixing, with per-stream format detection. ** Games
with several concurrent WASAPI render streams (e.g. Miles Morales) only get their
first ("primary") stream mirrored today; the rest keep playing locally and never
reach the guest. Capture every tracked render stream into its own shared ring,
silence each, and add a host-side mixer that resamples each ring to the render
format and sums them (with soft-clip). **Fold in real per-stream format
detection** here, since it touches the same hook + ring plumbing: a stream that
already existed when we injected is never seen at `Initialize` , so the hook
currently assumes the device **mix format ** and a shared-mode stream opened at a
different format comes out wrong-pitched. Resolve each stream's true format (the
stream's own `Initialize` when caught, else the original `GetMixFormat` ) so every
mixed ring is pitched correctly.
### Future work
### Future work
@@ -332,9 +302,11 @@ input layer sees a real state change. `disable_mask` (hex bits `0x1`=input
Present-hook crash was isolated.
Present-hook crash was isolated.
Both auto-detect a 32-bit (WOW64) target and inject via `coop_inject_x86.exe` +
Both auto-detect a 32-bit (WOW64) target and inject via `coop_inject_x86.exe` +
`coop_hook_x86.dll` , exactly like the host. Run them from `bin/<config>/` .
`coop_hook_x86.dll` , exactly like the host. The probes build into
**Kill the game between runs ** — the loaded DLL locks `coop_hook.dll` against the`bin/<config>/tools/` (the deployable `bin/<config>/` root holds only shipping
next rebuild.
artifacts; tests build into `bin/<config>/tests/` ) and resolve `coop_hook.dll` from
the root one level up, so run them from there. **Kill the game between runs ** — the
loaded DLL locks `coop_hook.dll` against the next rebuild.
## Running the tool (manual, end-to-end)
## Running the tool (manual, end-to-end)
