Restructure bin/: deployables at root, tests and tools in subfolders
Only the shipping artifacts (coop_host.exe, coop_hook.dll, coop_hook_x86.dll, coop_inject_x86.exe, steam_api64.dll, steam_input_actions.vdf) now land in the bin/<config>/ root, so it can be copied wholesale into a donor game folder. Test exes (plus the coop_tone fixture) build into bin/<config>/tests/ and the dev probes into bin/<config>/tools/, via a new coop_output_subdir() CMake helper. The probes resolve coop_hook.dll / the x86 injector from the deployable root one level up (new common/coop/tool_paths.hpp: deployed_artifact_path checks next-to-exe then parent). coop_tone is co-located with the tests so audio_loopback_test's "spawn coop_tone.exe next to me" lookup is unchanged. Verified from a clean bin/: root holds only deployables; ctest x64 7/7 and x86 3/3 green (incl. audio_loopback_test driving coop_tone from tests/). Also convert the roadmap Planned list to bullets and drop this (now-done) item. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
284
README.md
284
README.md
@@ -75,159 +75,129 @@ is removed from this list once done — so the top item is always next. The
|
||||
self-verifiable tooling / UI / input items come first; the game-pipeline items that
|
||||
need a real game (and Remote Play) to fully validate come last.
|
||||
|
||||
1. **Stage only deployable artifacts directly under `bin/`.** Today every target —
|
||||
host, both hook DLLs, the injector helper, tests, probes, `coop_tone` — lands in
|
||||
`bin/<config>/`, so deploying to a donor folder means hand-picking files. Keep
|
||||
the **deployable set** at the `bin/<config>/` root (`coop_host.exe`,
|
||||
`coop_hook.dll`, `coop_hook_x86.dll`, `coop_inject_x86.exe`, and — when Steam is
|
||||
built — `steam_api64.dll` + `steam_input_actions.vdf`) and push everything else
|
||||
into subfolders: tests → `bin/<config>/tests/`, debug/probe tools
|
||||
(`coop_audio_probe`, `coop_input_probe`, `coop_tone`, `coop_steam_input_probe`) →
|
||||
`bin/<config>/tools/`. *How:* give those targets a per-target
|
||||
`RUNTIME_OUTPUT_DIRECTORY[_<CONFIG>]` (the global `CMAKE_RUNTIME_OUTPUT_DIRECTORY`
|
||||
stays the deployable root; a small helper or `set_target_properties` overrides the
|
||||
non-deployable ones). The x86 sub-build must still stage `coop_hook_x86.dll` +
|
||||
`coop_inject_x86.exe` into the deployable root while its x86 *test* exes go to
|
||||
`tests/`. Watch the cross-target paths: `audio_loopback_test` spawns
|
||||
`coop_tone.exe`, and the host's post-build copy of `steam_api64.dll` / the `.vdf`
|
||||
must follow the host. End result: copying `bin/<config>/` non-recursively yields a
|
||||
clean deployable bundle.
|
||||
|
||||
2. **Detect a terminated target and reflect it in the UI.** The Injection panel keeps
|
||||
showing "Attached" after the game exits. Add a **Terminated** state: the host
|
||||
already knows the target pid and tracks a DLL heartbeat (`InjectionPanel`); on top
|
||||
of that, hold the `OpenProcess` handle from injection (or re-open with
|
||||
`PROCESS_QUERY_LIMITED_INFORMATION`) and poll `GetExitCodeProcess` /
|
||||
`WaitForSingleObject(h, 0)` each tick. When the process is gone, switch to
|
||||
Terminated, gray out / disable the per-subsystem controls and mirror toggles, and
|
||||
show a clear banner; the Video and Audio panels should drop to idle (their hook
|
||||
channels are stale) rather than freezing on the last live frame/state. Note a live
|
||||
process isn't proof it's running — also flag a **stalled heartbeat** (no advance
|
||||
for ~2 s while the process still exists) as a distinct "hung / not responding"
|
||||
state, since games here can freeze without exiting.
|
||||
|
||||
3. **Re-attach to a relaunched target.** A killed-and-relaunched game gets a new pid,
|
||||
but the UI still holds the stale one. In the Terminated state (task 2), remember
|
||||
the target's image name (the panel already keeps the selected exe name) and offer a
|
||||
**Re-attach** button that injects only if a live process with that *same name*
|
||||
exists, rebinding the IPC server to the new pid via the existing `inject_dll` path.
|
||||
If several live processes share that name, don't guess — surface the matches in the
|
||||
picker (task 4) for a manual choice.
|
||||
|
||||
4. **Select targets by window, not just process.** A flat process list is fine as an
|
||||
advanced/debug view, but the default should be a **window list** — there are far
|
||||
fewer top-level windows than processes, and a window directly yields the HWND the
|
||||
WGC capturer and focus spoof already want. *How:* add a window enumerator
|
||||
(`EnumWindows`, keeping visible, titled, non-tool top-level windows —
|
||||
`IsWindowVisible`, `GetWindowTextLength > 0`, exclude `WS_EX_TOOLWINDOW` and our own
|
||||
HWND, resolve to the root owner) and map each via `GetWindowThreadProcessId` → pid →
|
||||
image name. Show **title + process name + pid** with a filter box like the process
|
||||
list; injecting by window injects into its pid and hands the HWND straight to
|
||||
capture. Keep the process list behind "Debug details" as the advanced path.
|
||||
|
||||
5. **Auto-size and lay out the overlay windows so none need manual resizing.** Panels
|
||||
currently `Begin` at default cascade positions, so they overlap and clip. Give each
|
||||
`ImGuiWindowFlags_AlwaysAutoResize` and an initial position computed from
|
||||
`ImGui::GetMainViewport()->WorkPos/WorkSize`, applied with `ImGuiCond_FirstUseEver`
|
||||
(still movable), plus a **View → Reset layout** menu item that re-applies it. Target
|
||||
layout: **Injection** left/top (room to grow downward for hook diagnostics);
|
||||
**Controllers** top-center; **Video mirror** center, below Controllers; **Audio
|
||||
mirror** below Video; **Log** right edge, full height (most room for the log
|
||||
stream). Auto-resize fits these because they're all control/debug panels — the live
|
||||
mirror image is drawn to the whole host window *behind* the overlay, not inside a
|
||||
panel.
|
||||
|
||||
6. **Fix the Audio panel "live" column.** It overlays a green dot and grey "idle"
|
||||
because liveness is recomputed each frame from the per-stream `frames_rendered`
|
||||
delta, which is zero on most frames (buffers release in bursts), so it flickers.
|
||||
Replace it with a **debounced activity indicator**: keep a per-stream "last
|
||||
advanced" timestamp (the panel already stores the previous frame counts) and show
|
||||
**live** if frames advanced within the last ~300–500 ms, else **idle** — optionally
|
||||
a small frames/s or activity bar so multi-stream games read clearly.
|
||||
|
||||
7. **Move the synthetic-input toggle to the Controllers panel, under Debug details.**
|
||||
The "Forward synthetic test input" checkbox is a controller-debugging aid, so move
|
||||
it out of the Injection panel into `ControllersPanel` and gate it behind
|
||||
`debug_details`. The publish path is unchanged (the Injection panel already
|
||||
substitutes the synthetic pattern in `publish`); the flag just moves with it (or is
|
||||
passed from Controllers into the publish call).
|
||||
|
||||
8. **Mouse & keyboard forwarding (messages + polling-state hooks).** Forward guest
|
||||
clicks and keystrokes into the unfocused game via a new **MKB hook subsystem** in
|
||||
`coop_hook.dll` — the toggle *is* the hook (not installed → no forwarding).
|
||||
*Delivery:* `PostMessage` window-message input (`WM_KEYDOWN`/`WM_KEYUP`/`WM_CHAR`,
|
||||
`WM_*BUTTONDOWN`/`UP`, `WM_MOUSEWHEEL`) to the game HWND, **plus** hook
|
||||
`GetAsyncKeyState` / `GetKeyboardState` / `GetCursorPos` in the DLL so polling games
|
||||
see the synthesized keyboard/cursor state (RawInput and DirectInput games are out of
|
||||
scope for this version). *Keyboard* is always forwarded; *mouse* only while video is
|
||||
mirrored (otherwise the operator can't see where they click), and only **clicks +
|
||||
wheel, not movement** (one cursor can't be in two places). *Coordinate mapping* (the
|
||||
part that must be exact): WGC + decorated windowed → translate by the window
|
||||
decoration / client-area offset; hooked capture → relative to the mirrored viewport
|
||||
only (decorations aren't mirrored); borderless → the same under both backends.
|
||||
*Critical gating:* forward only when the host's main window is focused **and** ImGui
|
||||
doesn't want the event (`ImGuiIO::WantCaptureMouse` / `WantCaptureKeyboard`), so
|
||||
interacting with the overlay's own windows never leaks input into the game. The host
|
||||
sends MKB events to the hook over a new (or extended) IPC region.
|
||||
|
||||
9. **Rumble / haptics forwarding (both backends).** Currently unsupported — the XInput
|
||||
hook swallows `XInputSetState`. Add a reverse path: the hook captures the game's
|
||||
`XInputSetState` (left/right motor) and publishes it over a hook→host channel (the
|
||||
back-channel already exists), and the host drives the guest's actuators per backend —
|
||||
**XInput:** call `XInputSetState` on the guest's slot (the viability unknown is
|
||||
whether Steam's RPT virtual pad accepts vibration and routes it to the guest);
|
||||
**Steam Input:** `SteamInput()->TriggerVibration` / `Legacy_TriggerHapticPulse`.
|
||||
Map each guest slot to the right actuator.
|
||||
|
||||
10. **Per-backend input debug visualization.** To separate "wrong input *into* the
|
||||
tool" from "wrong input *out to* the game", show three distinct views in the
|
||||
Controllers panel (under Debug details): (a) **received via XInput** (raw
|
||||
`XInputSource` state), (b) **received via Steam Input** (raw `SteamInputSource`
|
||||
action values) — so it's obvious which backend delivered what — and (c)
|
||||
**forwarded to the game** (the `PadInfo` we write to shared memory, alongside what
|
||||
the game actually read back via the hook's per-slot channel). The hook already
|
||||
reports per-slot poll counts; extend it to echo the last state the game read so (c)
|
||||
is a true round-trip.
|
||||
|
||||
11. **Release the mouse cursor for cursor-clipping games.** Games that confine the
|
||||
cursor while focused (e.g. Trails through Daybreak via `ClipCursor` / per-frame
|
||||
`SetCursorPos` re-centering) trap the operator's mouse permanently, because the
|
||||
focus spoof makes the game believe it's always focused — so the operator can't
|
||||
reach the ImGui overlay. Add a cursor-release capability to the Focus subsystem:
|
||||
hook `ClipCursor` (force `ClipCursor(NULL)` and swallow the game's clip) and the
|
||||
re-centering `SetCursorPos`, gated by a new host→hook flag driven by a host
|
||||
toggle + hotkey. Defaults to released (the guest plays via the pad, so the game's
|
||||
own cursor clip is operator-only), with the option to re-enable clipping per game.
|
||||
|
||||
12. **Real capture metrics + latency stats.** The current FPS readout only measures
|
||||
how fast the host renders its own window, which hides capture stutter. Add a
|
||||
three-line frametime/FPS graph — **game present rate** (from `VideoShare` present
|
||||
deltas), **capture rate** (generation deltas / WGC arrivals), and **tool render
|
||||
rate** — plus a **capture→display latency** stat: stamp each published frame with
|
||||
a `QueryPerformanceCounter` value in `VideoShare`, and the host reports
|
||||
`host-present QPC − game-present QPC` (min/avg/max ms) for the matched frame. QPC
|
||||
is system-wide, so the two processes' timestamps compare directly.
|
||||
|
||||
13. **DX12 hooked capture (Spider-Man: Miles Morales).** Miles Morales is D3D12, so
|
||||
the Present hook fires but `GetBuffer(0)` as `ID3D11Texture2D` fails (the
|
||||
backbuffer is an `ID3D12Resource`) and the hook idles; WGC works but stutters. Add
|
||||
a D3D12 path via a **D3D11On12 bridge**: capture the game's D3D12 command queue
|
||||
(hook `ID3D12CommandQueue::ExecuteCommandLists`), create an `ID3D11On12Device`,
|
||||
`CreateWrappedResource` around the backbuffer, and `CopyResource` into the
|
||||
*existing* D3D11 shared keyed-mutex texture — so the host side is unchanged.
|
||||
|
||||
14. **Multi-stream audio capture + mixing, with per-stream format detection.** Games
|
||||
with several concurrent WASAPI render streams (e.g. Miles Morales) only get their
|
||||
first ("primary") stream mirrored today; the rest keep playing locally and never
|
||||
reach the guest. Capture every tracked render stream into its own shared ring,
|
||||
silence each, and add a host-side mixer that resamples each ring to the render
|
||||
format and sums them (with soft-clip). **Fold in real per-stream format
|
||||
detection** here, since it touches the same hook + ring plumbing: a stream that
|
||||
already existed when we injected is never seen at `Initialize`, so the hook
|
||||
currently assumes the device **mix format** and a shared-mode stream opened at a
|
||||
different format comes out wrong-pitched. Resolve each stream's true format (the
|
||||
stream's own `Initialize` when caught, else the original `GetMixFormat`) so every
|
||||
mixed ring is pitched correctly.
|
||||
- **Detect a terminated target and reflect it in the UI.** The Injection panel keeps
|
||||
showing "Attached" after the game exits. Add a **Terminated** state: the host
|
||||
already knows the target pid and tracks a DLL heartbeat (`InjectionPanel`); on top
|
||||
of that, hold the `OpenProcess` handle from injection (or re-open with
|
||||
`PROCESS_QUERY_LIMITED_INFORMATION`) and poll `GetExitCodeProcess` /
|
||||
`WaitForSingleObject(h, 0)` each tick. When the process is gone, switch to
|
||||
Terminated, gray out / disable the per-subsystem controls and mirror toggles, and
|
||||
show a clear banner; the Video and Audio panels should drop to idle (their hook
|
||||
channels are stale) rather than freezing on the last live frame/state. Note a live
|
||||
process isn't proof it's running — also flag a **stalled heartbeat** (no advance
|
||||
for ~2 s while the process still exists) as a distinct "hung / not responding"
|
||||
state, since games here can freeze without exiting.
|
||||
- **Re-attach to a relaunched target.** A killed-and-relaunched game gets a new pid,
|
||||
but the UI still holds the stale one. In the Terminated state, remember the
|
||||
target's image name (the panel already keeps the selected exe name) and offer a
|
||||
**Re-attach** button that injects only if a live process with that *same name*
|
||||
exists, rebinding the IPC server to the new pid via the existing `inject_dll` path.
|
||||
If several live processes share that name, don't guess — surface the matches in the
|
||||
window/process picker for a manual choice.
|
||||
- **Select targets by window, not just process.** A flat process list is fine as an
|
||||
advanced/debug view, but the default should be a **window list** — there are far
|
||||
fewer top-level windows than processes, and a window directly yields the HWND the
|
||||
WGC capturer and focus spoof already want. *How:* add a window enumerator
|
||||
(`EnumWindows`, keeping visible, titled, non-tool top-level windows —
|
||||
`IsWindowVisible`, `GetWindowTextLength > 0`, exclude `WS_EX_TOOLWINDOW` and our own
|
||||
HWND, resolve to the root owner) and map each via `GetWindowThreadProcessId` → pid →
|
||||
image name. Show **title + process name + pid** with a filter box like the process
|
||||
list; injecting by window injects into its pid and hands the HWND straight to
|
||||
capture. Keep the process list behind "Debug details" as the advanced path.
|
||||
- **Auto-size and lay out the overlay windows so none need manual resizing.** Panels
|
||||
currently `Begin` at default cascade positions, so they overlap and clip. Give each
|
||||
`ImGuiWindowFlags_AlwaysAutoResize` and an initial position computed from
|
||||
`ImGui::GetMainViewport()->WorkPos/WorkSize`, applied with `ImGuiCond_FirstUseEver`
|
||||
(still movable), plus a **View → Reset layout** menu item that re-applies it. Target
|
||||
layout: **Injection** left/top (room to grow downward for hook diagnostics);
|
||||
**Controllers** top-center; **Video mirror** center, below Controllers; **Audio
|
||||
mirror** below Video; **Log** right edge, full height (most room for the log
|
||||
stream). Auto-resize fits these because they're all control/debug panels — the live
|
||||
mirror image is drawn to the whole host window *behind* the overlay, not inside a
|
||||
panel.
|
||||
- **Fix the Audio panel "live" column.** It overlays a green dot and grey "idle"
|
||||
because liveness is recomputed each frame from the per-stream `frames_rendered`
|
||||
delta, which is zero on most frames (buffers release in bursts), so it flickers.
|
||||
Replace it with a **debounced activity indicator**: keep a per-stream "last
|
||||
advanced" timestamp (the panel already stores the previous frame counts) and show
|
||||
**live** if frames advanced within the last ~300–500 ms, else **idle** — optionally
|
||||
a small frames/s or activity bar so multi-stream games read clearly.
|
||||
- **Move the synthetic-input toggle to the Controllers panel, under Debug details.**
|
||||
The "Forward synthetic test input" checkbox is a controller-debugging aid, so move
|
||||
it out of the Injection panel into `ControllersPanel` and gate it behind
|
||||
`debug_details`. The publish path is unchanged (the Injection panel already
|
||||
substitutes the synthetic pattern in `publish`); the flag just moves with it (or is
|
||||
passed from Controllers into the publish call).
|
||||
- **Mouse & keyboard forwarding (messages + polling-state hooks).** Forward guest
|
||||
clicks and keystrokes into the unfocused game via a new **MKB hook subsystem** in
|
||||
`coop_hook.dll` — the toggle *is* the hook (not installed → no forwarding).
|
||||
*Delivery:* `PostMessage` window-message input (`WM_KEYDOWN`/`WM_KEYUP`/`WM_CHAR`,
|
||||
`WM_*BUTTONDOWN`/`UP`, `WM_MOUSEWHEEL`) to the game HWND, **plus** hook
|
||||
`GetAsyncKeyState` / `GetKeyboardState` / `GetCursorPos` in the DLL so polling games
|
||||
see the synthesized keyboard/cursor state (RawInput and DirectInput games are out of
|
||||
scope for this version). *Keyboard* is always forwarded; *mouse* only while video is
|
||||
mirrored (otherwise the operator can't see where they click), and only **clicks +
|
||||
wheel, not movement** (one cursor can't be in two places). *Coordinate mapping* (the
|
||||
part that must be exact): WGC + decorated windowed → translate by the window
|
||||
decoration / client-area offset; hooked capture → relative to the mirrored viewport
|
||||
only (decorations aren't mirrored); borderless → the same under both backends.
|
||||
*Critical gating:* forward only when the host's main window is focused **and** ImGui
|
||||
doesn't want the event (`ImGuiIO::WantCaptureMouse` / `WantCaptureKeyboard`), so
|
||||
interacting with the overlay's own windows never leaks input into the game. The host
|
||||
sends MKB events to the hook over a new (or extended) IPC region.
|
||||
- **Rumble / haptics forwarding (both backends).** Currently unsupported — the XInput
|
||||
hook swallows `XInputSetState`. Add a reverse path: the hook captures the game's
|
||||
`XInputSetState` (left/right motor) and publishes it over a hook→host channel (the
|
||||
back-channel already exists), and the host drives the guest's actuators per backend —
|
||||
**XInput:** call `XInputSetState` on the guest's slot (the viability unknown is
|
||||
whether Steam's RPT virtual pad accepts vibration and routes it to the guest);
|
||||
**Steam Input:** `SteamInput()->TriggerVibration` / `Legacy_TriggerHapticPulse`.
|
||||
Map each guest slot to the right actuator.
|
||||
- **Per-backend input debug visualization.** To separate "wrong input *into* the
|
||||
tool" from "wrong input *out to* the game", show three distinct views in the
|
||||
Controllers panel (under Debug details): (a) **received via XInput** (raw
|
||||
`XInputSource` state), (b) **received via Steam Input** (raw `SteamInputSource`
|
||||
action values) — so it's obvious which backend delivered what — and (c)
|
||||
**forwarded to the game** (the `PadInfo` we write to shared memory, alongside what
|
||||
the game actually read back via the hook's per-slot channel). The hook already
|
||||
reports per-slot poll counts; extend it to echo the last state the game read so (c)
|
||||
is a true round-trip.
|
||||
- **Release the mouse cursor for cursor-clipping games.** Games that confine the
|
||||
cursor while focused (e.g. Trails through Daybreak via `ClipCursor` / per-frame
|
||||
`SetCursorPos` re-centering) trap the operator's mouse permanently, because the
|
||||
focus spoof makes the game believe it's always focused — so the operator can't
|
||||
reach the ImGui overlay. Add a cursor-release capability to the Focus subsystem:
|
||||
hook `ClipCursor` (force `ClipCursor(NULL)` and swallow the game's clip) and the
|
||||
re-centering `SetCursorPos`, gated by a new host→hook flag driven by a host
|
||||
toggle + hotkey. Defaults to released (the guest plays via the pad, so the game's
|
||||
own cursor clip is operator-only), with the option to re-enable clipping per game.
|
||||
- **Real capture metrics + latency stats.** The current FPS readout only measures
|
||||
how fast the host renders its own window, which hides capture stutter. Add a
|
||||
three-line frametime/FPS graph — **game present rate** (from `VideoShare` present
|
||||
deltas), **capture rate** (generation deltas / WGC arrivals), and **tool render
|
||||
rate** — plus a **capture→display latency** stat: stamp each published frame with
|
||||
a `QueryPerformanceCounter` value in `VideoShare`, and the host reports
|
||||
`host-present QPC − game-present QPC` (min/avg/max ms) for the matched frame. QPC
|
||||
is system-wide, so the two processes' timestamps compare directly.
|
||||
- **DX12 hooked capture (Spider-Man: Miles Morales).** Miles Morales is D3D12, so
|
||||
the Present hook fires but `GetBuffer(0)` as `ID3D11Texture2D` fails (the
|
||||
backbuffer is an `ID3D12Resource`) and the hook idles; WGC works but stutters. Add
|
||||
a D3D12 path via a **D3D11On12 bridge**: capture the game's D3D12 command queue
|
||||
(hook `ID3D12CommandQueue::ExecuteCommandLists`), create an `ID3D11On12Device`,
|
||||
`CreateWrappedResource` around the backbuffer, and `CopyResource` into the
|
||||
*existing* D3D11 shared keyed-mutex texture — so the host side is unchanged.
|
||||
- **Multi-stream audio capture + mixing, with per-stream format detection.** Games
|
||||
with several concurrent WASAPI render streams (e.g. Miles Morales) only get their
|
||||
first ("primary") stream mirrored today; the rest keep playing locally and never
|
||||
reach the guest. Capture every tracked render stream into its own shared ring,
|
||||
silence each, and add a host-side mixer that resamples each ring to the render
|
||||
format and sums them (with soft-clip). **Fold in real per-stream format
|
||||
detection** here, since it touches the same hook + ring plumbing: a stream that
|
||||
already existed when we injected is never seen at `Initialize`, so the hook
|
||||
currently assumes the device **mix format** and a shared-mode stream opened at a
|
||||
different format comes out wrong-pitched. Resolve each stream's true format (the
|
||||
stream's own `Initialize` when caught, else the original `GetMixFormat`) so every
|
||||
mixed ring is pitched correctly.
|
||||
|
||||
### Future work
|
||||
|
||||
@@ -332,9 +302,11 @@ input layer sees a real state change. `disable_mask` (hex bits `0x1`=input
|
||||
Present-hook crash was isolated.
|
||||
|
||||
Both auto-detect a 32-bit (WOW64) target and inject via `coop_inject_x86.exe` +
|
||||
`coop_hook_x86.dll`, exactly like the host. Run them from `bin/<config>/`.
|
||||
**Kill the game between runs** — the loaded DLL locks `coop_hook.dll` against the
|
||||
next rebuild.
|
||||
`coop_hook_x86.dll`, exactly like the host. The probes build into
|
||||
`bin/<config>/tools/` (the deployable `bin/<config>/` root holds only shipping
|
||||
artifacts; tests build into `bin/<config>/tests/`) and resolve `coop_hook.dll` from
|
||||
the root one level up, so run them from there. **Kill the game between runs** — the
|
||||
loaded DLL locks `coop_hook.dll` against the next rebuild.
|
||||
|
||||
## Running the tool (manual, end-to-end)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user