diff --git a/README.md b/README.md index 208917d..e835b85 100644 --- a/README.md +++ b/README.md @@ -109,6 +109,122 @@ milestone with its own tests and commit. host-side mixer that resamples each ring to the render format and sums them (with soft-clip). +### Tooling, UI & input + +Operability, UI, and input-debugging improvements, mostly independent of the +game-specific milestones above. Ordered roughly as requested, not by priority. + +1. **Stage only deployable artifacts directly under `bin/`.** Today every target — + host, both hook DLLs, the injector helper, tests, probes, `coop_tone` — lands in + `bin//`, so deploying to a donor folder means hand-picking files. Keep + the **deployable set** at the `bin//` root (`coop_host.exe`, + `coop_hook.dll`, `coop_hook_x86.dll`, `coop_inject_x86.exe`, and — when Steam is + built — `steam_api64.dll` + `steam_input_actions.vdf`) and push everything else + into subfolders: tests → `bin//tests/`, debug/probe tools + (`coop_audio_probe`, `coop_input_probe`, `coop_tone`, `coop_steam_input_probe`) → + `bin//tools/`. *How:* give those targets a per-target + `RUNTIME_OUTPUT_DIRECTORY[_]` (the global `CMAKE_RUNTIME_OUTPUT_DIRECTORY` + stays the deployable root; a small helper or `set_target_properties` overrides the + non-deployable ones). The x86 sub-build must still stage `coop_hook_x86.dll` + + `coop_inject_x86.exe` into the deployable root while its x86 *test* exes go to + `tests/`. Watch the cross-target paths: `audio_loopback_test` spawns + `coop_tone.exe`, and the host's post-build copy of `steam_api64.dll` / the `.vdf` + must follow the host. End result: copying `bin//` non-recursively yields a + clean deployable bundle. + +2. **Detect a terminated target and reflect it in the UI.** The Injection panel keeps + showing "Attached" after the game exits. Add a **Terminated** state: the host + already knows the target pid and tracks a DLL heartbeat (`InjectionPanel`); on top + of that, hold the `OpenProcess` handle from injection (or re-open with + `PROCESS_QUERY_LIMITED_INFORMATION`) and poll `GetExitCodeProcess` / + `WaitForSingleObject(h, 0)` each tick. When the process is gone, switch to + Terminated, gray out / disable the per-subsystem controls and mirror toggles, and + show a clear banner; the Video and Audio panels should drop to idle (their hook + channels are stale) rather than freezing on the last live frame/state. + +3. **Re-attach to a relaunched target.** A killed-and-relaunched game gets a new pid, + but the UI still holds the stale one. In the Terminated state (task 2), remember + the target's image name (the panel already keeps the selected exe name) and offer a + **Re-attach** button that injects only if a live process with that *same name* + exists, rebinding the IPC server to the new pid via the existing `inject_dll` path. + *Open decision (will confirm at build time):* if several processes share the name, + default to the most-recently-started one with a note, or fall back to the picker. + +4. **Select targets by window, not just process.** A flat process list is fine as an + advanced/debug view, but the default should be a **window list** — there are far + fewer top-level windows than processes, and a window directly yields the HWND the + WGC capturer and focus spoof already want. *How:* add a window enumerator + (`EnumWindows`, keeping visible, titled, non-tool top-level windows — + `IsWindowVisible`, `GetWindowTextLength > 0`, exclude `WS_EX_TOOLWINDOW` and our own + HWND, resolve to the root owner) and map each via `GetWindowThreadProcessId` → pid → + image name. Show **title + process name + pid** with a filter box like the process + list; injecting by window injects into its pid and hands the HWND straight to + capture. Keep the process list behind "Debug details" as the advanced path. + +5. **Auto-size and lay out the overlay windows so none need manual resizing.** Panels + currently `Begin` at default cascade positions, so they overlap and clip. Give each + `ImGuiWindowFlags_AlwaysAutoResize` and an initial position computed from + `ImGui::GetMainViewport()->WorkPos/WorkSize`, applied with `ImGuiCond_FirstUseEver` + (still movable), plus a **View → Reset layout** menu item that re-applies it. Target + layout: **Injection** left/top (room to grow downward for hook diagnostics); + **Controllers** top-center; **Video mirror** center, below Controllers; **Audio + mirror** below Video; **Log** right edge, full height (most room for the log + stream). Auto-resize fits these because they're all control/debug panels — the live + mirror image is drawn to the whole host window *behind* the overlay, not inside a + panel. + +6. **Fix the Audio panel "live" column.** It overlays a green dot and grey "idle" + because liveness is recomputed each frame from the per-stream `frames_rendered` + delta, which is zero on most frames (buffers release in bursts), so it flickers. + Replace it with a **debounced activity indicator**: keep a per-stream "last + advanced" timestamp (the panel already stores the previous frame counts) and show + **live** if frames advanced within the last ~300–500 ms, else **idle** — optionally + a small frames/s or activity bar so multi-stream games read clearly. + +7. **Move the synthetic-input toggle to the Controllers panel, under Debug details.** + The "Forward synthetic test input" checkbox is a controller-debugging aid, so move + it out of the Injection panel into `ControllersPanel` and gate it behind + `debug_details`. The publish path is unchanged (the Injection panel already + substitutes the synthetic pattern in `publish`); the flag just moves with it (or is + passed from Controllers into the publish call). + +8. **Mouse & keyboard forwarding (messages + polling-state hooks).** Forward guest + clicks and keystrokes into the unfocused game via a new **MKB hook subsystem** in + `coop_hook.dll` — the toggle *is* the hook (not installed → no forwarding). + *Delivery:* `PostMessage` window-message input (`WM_KEYDOWN`/`WM_KEYUP`/`WM_CHAR`, + `WM_*BUTTONDOWN`/`UP`, `WM_MOUSEWHEEL`) to the game HWND, **plus** hook + `GetAsyncKeyState` / `GetKeyboardState` / `GetCursorPos` in the DLL so polling games + see the synthesized keyboard/cursor state (RawInput and DirectInput games are out of + scope for this version). *Keyboard* is always forwarded; *mouse* only while video is + mirrored (otherwise the operator can't see where they click), and only **clicks + + wheel, not movement** (one cursor can't be in two places). *Coordinate mapping* (the + part that must be exact): WGC + decorated windowed → translate by the window + decoration / client-area offset; hooked capture → relative to the mirrored viewport + only (decorations aren't mirrored); borderless → the same under both backends. + *Critical gating:* forward only when the host's main window is focused **and** ImGui + doesn't want the event (`ImGuiIO::WantCaptureMouse` / `WantCaptureKeyboard`), so + interacting with the overlay's own windows never leaks input into the game. The host + sends MKB events to the hook over a new (or extended) IPC region. + +9. **Rumble / haptics forwarding (both backends).** Currently unsupported — the XInput + hook swallows `XInputSetState`. Add a reverse path: the hook captures the game's + `XInputSetState` (left/right motor) and publishes it over a hook→host channel (the + back-channel already exists), and the host drives the guest's actuators per backend — + **XInput:** call `XInputSetState` on the guest's slot (the viability unknown is + whether Steam's RPT virtual pad accepts vibration and routes it to the guest); + **Steam Input:** `SteamInput()->TriggerVibration` / `Legacy_TriggerHapticPulse`. + Map each guest slot to the right actuator. + +10. **Per-backend input debug visualization.** To separate "wrong input *into* the + tool" from "wrong input *out to* the game", show three distinct views in the + Controllers panel (under Debug details): (a) **received via XInput** (raw + `XInputSource` state), (b) **received via Steam Input** (raw `SteamInputSource` + action values) — so it's obvious which backend delivered what — and (c) + **forwarded to the game** (the `PadInfo` we write to shared memory, alongside what + the game actually read back via the hook's per-slot channel). The hook already + reports per-slot poll counts; extend it to echo the last state the game read so (c) + is a true round-trip. + ### Future work - **Vulkan video hook.** Vulkan games present via `vkQueuePresentKHR`; hooking @@ -120,8 +236,6 @@ milestone with its own tests and commit. we inject is never seen at `Initialize`, so the hook assumes the device **mix format**. A stream initialized in shared mode at a different format would come out wrong-pitched. Detecting the real per-stream format would remove that guess. -- **Rumble / haptics forwarding.** `XInputSetState` is currently swallowed; routing - it back to the guest is a later phase. ## Building