Recover a guessed stream's channels + bit depth by correlation too (step b)

Extends the two-path correlation from rate-only to the full layout, removing the
"channels/bit-depth assumed = device" limitation. correlate_format tries each
candidate de-interleaving (float32 / int16; mono..7.1) of the hook capture, runs
the rate correlation per layout, and keeps whichever aligns with the loopback;
a wrong de-interleaving is noise and won't.

The catch: the hook can't know a guessed stream's real frame size, so its verify
tap pads each render buffer to the device block -- which over-reads stale staging
bytes for a stream with fewer channels/bits, scrambling the audio. So the tap is
now self-describing: it prefixes each buffer with its frame count
([count][count*device_block bytes]), and the host strips the padding per candidate
layout (take the real count*real_block of each chunk) before de-interleaving.

- audio_correlate.hpp: ChunkedCapture + chunk-aware correlate_format + candidate
  layouts; absolute-margin confidence gate (the true layout scores ~1.0, a truly
  ambiguous alternative within ~0.001 -- 2ch@R == 1ch@2R for identical channels --
  is correctly left unconfident).
- audio_hook.cpp: chunked verify tap (free-space-checked so framing can't tear).
- audio_format_verifier: parse chunks; recover_layout path. AudioMirror now corrects
  the full format.
- audio_correlation_test: layout recovery from padded chunks (stereo float, 16-bit
  PCM, 5.1, mono). audio_verify_test gains scenario (b): 2ch on a multichannel
  endpoint with distinct per-channel content (new env-gated ToneSource mode) ->
  recovers ch=2/32-bit float end-to-end.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

README.md

@@ -77,29 +77,30 @@ default** and covers anything the hooked path doesn't.
   back to process-loopback capture, which does *not* mute the game — so the local
   machine hears the audio twice (guests hear it once). The Audio panel shows which
   path is active.
-- **Hooked audio can only recover a pre-existing stream's *sample rate*, not its
+- **A pre-existing stream's format is *recovered*, not known — by measurement and
-  channels/bit-depth.** The tool injects into an already-running game, so the audio
+  cross-correlation.** The tool injects into an already-running game, so the audio
-  render-hook usually never saw the game's `IAudioClient::Initialize`. It recovers the
+  render-hook usually never saw the game's `IAudioClient::Initialize`, and `AUTOCONVERTPCM`
-  true **sample rate** by measuring the render cadence (so playback pitch is correct,
+  hides the buffer stride (WASAPI exposes no API for a pre-existing client's format). The
-  e.g. Godot/Brotato's 44100 Hz on a 48000 Hz endpoint), but **channels and bit-depth
+  hook first recovers the **sample rate** by measuring the render cadence (so playback pitch
-  can't be detected** — with `AUTOCONVERTPCM` `GetBuffer` returns a fixed staging
+  is correct, e.g. Godot/Brotato's 44100 Hz on a 48000 Hz endpoint), assuming the device's
-  buffer (no buffer stride to measure) and WASAPI exposes no API for a pre-existing
+  channels/bit-depth. When the game is still audible (the measurement window), the host then
-  client's format — so they're *assumed* to match the device mix format. That's correct
+  **cross-correlates the two capture paths** — the hook (pre-mix) against a parallel
-  for the common case (engines render stereo float, matching the endpoint, differing
+  process-loopback (post-mix, the known device format) — to verify/correct the rate and to
-  only in rate). A game rendering a *different* channel count or bit depth than the
+  **recover the channels + bit depth** by trying candidate de-interleavings and keeping the one
-  device is mirrored with the wrong layout (garbled audio) on the hooked path, but never
+  that aligns (`audio_format_verifier` → `coop/audio_correlate.hpp`). The recovered format feeds
-  an over-read/crash: the capture copy is clamped to the readable region (`VirtualQuery`),
+  the existing override channel. The one case it can't resolve is a *genuinely ambiguous* layout
-  and the local **mute** is done with `AUDCLNT_BUFFERFLAGS_SILENT` (which makes WASAPI
+  (a stream whose channels carry identical content looks the same as one channel at double the
-  ignore the buffer's contents), so it never *writes* the wrongly-sized buffer either. So
+  rate); there it stays the device assumption and the correlation reports low confidence rather
-  every captured stream is silenced locally — guessed or exact — and there is no echo on
+  than guess. A wrong/assumed layout is mirrored with the wrong de-interleaving (garbled) but
-  the hooked path. The loopback fallback is always format-correct. The Audio panel shows
+  never an over-read/crash: the capture copy is clamped to the readable region (`VirtualQuery`),
-  each stream's
+  and the local **mute** uses `AUDCLNT_BUFFERFLAGS_SILENT` (WASAPI ignores the buffer contents),
-  format provenance (*known* / *measuring* / *measured rate* / *low-confidence* /
+  so it never *writes* the wrongly-sized buffer either. Every captured stream is silenced locally
-  *override*) so the assumption is visible, and (under Debug details) lets the operator
+  — guessed or exact — so there's no echo on the hooked path, and the loopback fallback is always
-  **re-measure** the rate or **override** the format when the guess is wrong. Overrides
+  format-correct. The Audio panel shows each stream's format provenance (*known* / *measuring* /
-  are **remembered per game** (and a format caught exactly at `Initialize` is auto-saved
+  *measured rate* / *low-confidence* / *override*), and (under Debug details) lets the operator
-  as that game's override), so a known-bad game is corrected automatically next launch.
+  **re-measure** or **override** the format when needed. Overrides are **remembered per game** (and
-  Streams created *after* injection are captured exactly.
+  a format caught exactly at `Initialize` is auto-saved), so a known-bad game is corrected
+  automatically next launch. Streams created *after* injection are captured exactly.
 - **Debug-oriented UI:** the ImGui overlay is laid out for diagnosing the
   pipeline, not for end use. F1 hides it entirely so the window is a clean mirror
   for RPT; F2 frees the operator cursor; **F10 saves a PNG screenshot** (back buffer,
@@ -109,26 +110,6 @@ default** and covers anything the hooked path doesn't.
 
 ### Current Tasks
 
-- **Determine a pre-existing stream's audio format by *correlating* the two capture
-  paths, instead of guessing.** When we attach to an already-running game we never saw its
-  `IAudioClient::Initialize`, so the render-hook assumes the device mix format and measures
-  only the sample rate from the render cadence — which can be wrong on a jittery game
-  (intermittent pitch shift) and can't recover channels/bit-depth at all. But during the
-  measurement window the game is still audible, so we already have *both* signals of the
-  same audio: the **process-loopback** capture (post-mix, at the **known** device format)
-  and the **render-hook** capture (pre-mix, at the unknown format). Cross-correlating them
-  pins the real format from ground truth rather than a guess. Two tasks:
-  - **(a) Rate verification/correction.** Resample the hook stream by each candidate standard
-    rate and cross-correlate against the loopback; the rate that holds alignment with no drift
-    over the window is the truth. Robust where cadence measurement is noisy — directly hardens
-    the intermittent pitch-shift symptom. Lands as a verify-and-correct step feeding the
-    existing rate path (the operator override stays as the manual escape hatch).
-  - **(b) Channels + bit-depth recovery.** Extend the correlation to the layout the cadence
-    method *can't* recover: interpret the hook bytes under candidate layouts (float32 vs
-    int16; mono/stereo/…) and keep whichever de-interleaving correlates with the loopback (a
-    wrong interpretation is noise and won't). Removes the "channels/bit-depth assumed = device"
-    limitation, so the garbled-layout case stops being undetectable.
-
 - **Mouse + keyboard forwarding for Raw Input / DirectInput games.** The MKB
   subsystem forwards via window messages (`PostMessage`) plus synthesized
   `GetAsyncKeyState` / `GetKeyboardState` / `GetCursorPos`, which covers message-loop
@@ -235,15 +216,20 @@ ctest --test-dir build -C Debug --output-on-failure
   skew + noise — exactly the hook-vs-loopback situation) and asserts `correlate_rate()` recovers the
   true rate, scoring the right candidate ≈1.0 and the wrong ones ≈0 (incl. the hard 44100-vs-48000
   case the cadence method can misread), and that unrelated signals are *not* confidently matched.
-  Pure header logic, no device.
+  Also covers **layout recovery** (`correlate_format`): from a self-describing *chunked* capture
+  (each render buffer padded to the device block, as the hook's verify tap produces it) it strips
+  the padding per candidate de-interleaving and recovers the true channels + bit depth + rate
+  (stereo float, 16-bit PCM, 5.1, mono), and leaves a genuinely-ambiguous identical-channel layout
+  unconfident. Pure header logic, no device.
 - **`audio_verify_test`** — integration test of the host's two-path verifier
   ([`host/src/audio/audio_format_verifier.cpp`](host/src/audio/audio_format_verifier.cpp)). Launches
   `coop_mock_game` rendering a tone at a non-device rate (matching the device's *channel* count, so
-  this rate test isn't perturbed by a channel mismatch — that's the next task), injects the hook
+  this rate test isn't perturbed by a channel mismatch), injects the hook late (a guessed stream),
-  late (a guessed stream), and runs the real `verify_stream_format()`: it co-captures the hook
+  and runs the real `verify_stream_format()`: it co-captures the hook (pre-mix, via the ring's
-  (pre-mix, via the ring's `verify_capture` tap) and a parallel process-loopback (post-mix) of the
+  `verify_capture` tap) and a parallel process-loopback (post-mix) of the same audio and correlates
-  same audio and correlates them. Asserts it recovers the game's true rate, not the device guess.
+  them. Scenario (a) asserts it recovers the true rate; scenario (b) renders a *different* channel
-  Skips cleanly without an audio endpoint.
+  count than the device with distinct per-channel content and asserts it recovers the full layout
+  (channels + bit depth + rate). Skips cleanly without an audio endpoint.
 - **`render_pacer_test`** — unit test of the mirror's render-feed pacing policy
   (`host/src/audio/render_pacer.hpp`). Simulates a producer/consumer device timeline and asserts
   the shipping `RenderPacer` rides producer jitter that makes the old re-prime-on-partial-fill
@@ -551,7 +537,15 @@ Non-obvious things that cost time and constrain the design:
   the *waveform* needs the hook bytes de-interleaved at the right channel count, so the rate step
   assumes the hook layout matches the device (true for the common stereo-on-stereo case); recovering a
   *different* channel count / bit depth is the layout step, which tries candidate de-interleavings and
-  keeps whichever correlates.
+  keeps whichever correlates. **The layout step needs a self-describing tap**: the hook can't know a
+  guessed stream's real frame size, so it pads each render buffer to the *device* block — which
+  over-reads stale staging bytes for a stream with fewer channels/bits. The raw padded bytes are
+  un-decodable (the stale tail scrambles the audio), so the tap prefixes each buffer with its frame
+  *count* (`[count][count*device_block bytes]`); the host strips the padding per candidate layout
+  (take the real `count*real_block` of each chunk) before de-interleaving. **The genuinely-ambiguous
+  case stays unresolved**: a stream whose channels carry identical content is indistinguishable from
+  one channel at double the rate (`2ch@R` == `1ch@2R` byte-for-byte), so the correlator reports low
+  confidence and the format stays the device assumption rather than guessing wrong.
 - **Re-priming the render feed on a *partial* fill manufactures the gap it's avoiding.** The
   mirror re-renders the captured ring to the output device. The original feed loop re-primed
   (withheld the feed until ~30 ms had rebuffered) whenever it couldn't completely fill the free

common/include/coop/audio_correlate.hpp

@@ -254,4 +254,177 @@ inline RateCorrelation correlate_rate(const std::vector<float>& hook_mono, const
 	return result;
 }
 
+// --- Step (b): channels + bit-depth recovery --------------------------------------------------
+//
+// The rate step assumes the hook bytes are de-interleaved at the device channel/bit layout. When a
+// game renders a DIFFERENT layout than the device (e.g. stereo float on a 7.1 endpoint, or 16-bit
+// PCM), that assumption garbles the waveform and the rate won't lock. We can't measure the layout
+// (AUTOCONVERTPCM hides the stride), but we can RECOVER it: interpret the raw hook bytes under each
+// candidate layout, run the rate correlation, and keep whichever (layout, rate) aligns with the
+// loopback -- a wrong de-interleaving is noise and won't correlate.
+
+inline constexpr unsigned kWaveFormatPcm = 1;   // WAVE_FORMAT_PCM
+inline constexpr unsigned kWaveFormatFloat = 3; // WAVE_FORMAT_IEEE_FLOAT
+
+struct LayoutCandidate
+{
+	unsigned channels;
+	unsigned bits;
+	unsigned tag; // kWaveFormatPcm / kWaveFormatFloat
+};
+
+// Candidate de-interleavings a shared-mode WASAPI render stream realistically uses: float32 and
+// 16-bit PCM, across the common channel counts. Ordered most-likely-first.
+inline const std::vector<LayoutCandidate>& standard_audio_layouts()
+{
+	static const std::vector<LayoutCandidate> v = {
+	    {2, 32, kWaveFormatFloat}, {1, 32, kWaveFormatFloat}, {6, 32, kWaveFormatFloat},
+	    {8, 32, kWaveFormatFloat}, {4, 32, kWaveFormatFloat}, {2, 16, kWaveFormatPcm},
+	    {1, 16, kWaveFormatPcm},   {6, 16, kWaveFormatPcm},   {8, 16, kWaveFormatPcm},
+	    {4, 16, kWaveFormatPcm},
+	};
+	return v;
+}
+
+struct FormatCorrelation
+{
+	bool ok = false;
+	unsigned rate = 0;
+	unsigned channels = 0;
+	unsigned bits = 0;
+	unsigned tag = 0;
+	double score = 0.0;
+	double runner_up = 0.0;
+};
+
+// The hook can't know a guessed stream's real frame size, so its verify tap pushes each render
+// buffer padded to the device block (`stride`) and prefixes it with the real frame `count`. That
+// padding is stale staging-buffer bytes, so the host must extract the real `count*real_block` bytes
+// per buffer (and concatenate) before de-interleaving -- otherwise the padding scrambles the audio.
+// This carries that self-describing capture: `bytes` holds counts[i]*stride bytes per chunk.
+struct ChunkedCapture
+{
+	unsigned stride = 0;               // bytes per frame as pushed (the guessed/device block_align)
+	std::vector<std::uint32_t> counts; // real frame count of each chunk
+	std::vector<std::uint8_t> bytes;   // concatenated, counts[i]*stride bytes per chunk
+};
+
+namespace correlate_detail
+{
+// De-interleave raw bytes under (channels/bits/tag) and average to mono float.
+inline void decode_layout(const std::uint8_t* bytes, std::size_t n, const LayoutCandidate& fmt,
+                          std::vector<float>& mono)
+{
+	mono.clear();
+	const unsigned ch = fmt.channels == 0 ? 1 : fmt.channels;
+	const unsigned bps = fmt.bits / 8;
+	if (bps == 0)
+	{
+		return;
+	}
+	const std::size_t frame = static_cast<std::size_t>(ch) * bps;
+	const std::size_t frames = n / frame;
+	mono.resize(frames);
+	const bool is_float = fmt.tag == kWaveFormatFloat;
+	for (std::size_t i = 0; i < frames; ++i)
+	{
+		double sum = 0.0;
+		for (unsigned c = 0; c < ch; ++c)
+		{
+			const std::uint8_t* p = bytes + i * frame + static_cast<std::size_t>(c) * bps;
+			float s = 0.0f;
+			if (is_float && fmt.bits == 32)
+			{
+				std::memcpy(&s, p, 4);
+			}
+			else if (fmt.bits == 16)
+			{
+				std::int16_t v;
+				std::memcpy(&v, p, 2);
+				s = v / 32768.0f;
+			}
+			else if (fmt.bits == 32)
+			{
+				std::int32_t v;
+				std::memcpy(&v, p, 4);
+				s = static_cast<float>(v / 2147483648.0);
+			}
+			sum += s;
+		}
+		mono[i] = static_cast<float>(sum / ch);
+	}
+}
+} // namespace correlate_detail
+
+// Recover BOTH the layout and the rate of a guessed stream from its (padded, self-describing) hook
+// capture: for each candidate layout, extract the real count*real_block bytes from each padded
+// chunk, de-interleave to mono, and run the rate correlation against the known-format loopback,
+// keeping the (layout, rate) that aligns best. `ok` when the winner clears the alignment floor and
+// clearly beats the runner-up (so a coincidental partial match is rejected).
+//
+// `min_margin` is an ABSOLUTE gap (not a ratio): the true layout scores near-perfectly while a
+// truly-ambiguous alternative (e.g. 1ch@2R vs 2ch@R when the channels carry the same content)
+// scores within ~0.001, so requiring the winner to clear the runner-up by a fixed margin cleanly
+// separates "recovered" from "genuinely ambiguous, don't guess".
+inline FormatCorrelation correlate_format(const ChunkedCapture& hook, const std::vector<float>& loop_mono,
+                                          unsigned device_rate, const std::vector<unsigned>& rates,
+                                          const std::vector<LayoutCandidate>& layouts, double min_score = 0.55,
+                                          double min_margin = 0.04)
+{
+	FormatCorrelation result;
+	if (hook.stride == 0 || hook.counts.empty() || loop_mono.empty() || device_rate == 0)
+	{
+		return result;
+	}
+	double best = -1.0, second = -1.0;
+	std::vector<std::uint8_t> clean;
+	std::vector<float> hook_mono;
+	for (const LayoutCandidate& layout : layouts)
+	{
+		const unsigned real_block = layout.channels * (layout.bits / 8);
+		if (real_block == 0 || real_block > hook.stride)
+		{
+			continue; // can't extract a frame larger than what was pushed (the guess is the max)
+		}
+		// Pull the real count*real_block bytes out of each padded chunk and concatenate -> contiguous
+		// audio for this candidate layout (the padding, which is stale staging bytes, is dropped).
+		clean.clear();
+		std::size_t off = 0;
+		for (std::uint32_t count : hook.counts)
+		{
+			const std::size_t chunk_bytes = static_cast<std::size_t>(count) * hook.stride;
+			const std::size_t take = static_cast<std::size_t>(count) * real_block;
+			if (off + chunk_bytes <= hook.bytes.size())
+			{
+				clean.insert(clean.end(), hook.bytes.begin() + off, hook.bytes.begin() + off + take);
+			}
+			off += chunk_bytes;
+		}
+		correlate_detail::decode_layout(clean.data(), clean.size(), layout, hook_mono);
+		if (hook_mono.size() < device_rate / 5)
+		{
+			continue; // this layout yields too little audio to judge
+		}
+		const RateCorrelation rc = correlate_rate(hook_mono, loop_mono, device_rate, rates, /*min_score=*/0.0,
+		                                           /*separation=*/1.0);
+		if (rc.score > best)
+		{
+			second = best;
+			best = rc.score;
+			result.rate = rc.rate;
+			result.channels = layout.channels;
+			result.bits = layout.bits;
+			result.tag = layout.tag;
+		}
+		else if (rc.score > second)
+		{
+			second = rc.score;
+		}
+	}
+	result.score = best < 0.0 ? 0.0 : best;
+	result.runner_up = second < 0.0 ? 0.0 : second;
+	result.ok = result.score >= min_score && (result.runner_up <= 1e-6 || result.score - result.runner_up >= min_margin);
+	return result;
+}
+
 } // namespace coop

common/include/coop/audio_ring.hpp

@@ -203,6 +203,15 @@ inline std::uint32_t audio_ring_available(const AudioRingHeader& h)
 	return static_cast<std::uint32_t>(w - r);
 }
 
+// Producer: bytes currently free (so a multi-part packet can be checked to fit before any of it is
+// written -- keeps a self-describing [header][payload] framing from tearing on a full ring).
+inline std::uint32_t audio_ring_free_space(const AudioRingHeader& h)
+{
+	const std::uint64_t w = h.write_pos.load(std::memory_order_relaxed);
+	const std::uint64_t r = h.read_pos.load(std::memory_order_acquire);
+	return h.capacity - static_cast<std::uint32_t>(w - r);
+}
+
 // Consumer: copy up to `bytes` into `dst`; returns the number actually popped.
 inline std::uint32_t audio_ring_pop(AudioRingHeader& h, void* dst, std::uint32_t bytes)
 {

hook/src/audio_hook.cpp

@@ -410,8 +410,19 @@ HRESULT STDMETHODCALLTYPE hk_ReleaseBuffer(IAudioRenderClient* self, UINT32 num_
 				const std::uint32_t block = g_streams[i].block_align.load(std::memory_order_relaxed);
 				if (block != 0)
 				{
-					audio_ring_push(*vring, t_gb_data, readable_bytes(t_gb_data, num_frames * block),
+					// Self-describing chunk: [u32 frame-count][num_frames*block bytes]. The host can't
-					                num_frames);
+					// know the real frame size of a guessed stream, so it recovers the layout by
+					// trying candidate de-interleavings -- but it needs the frame count to strip the
+					// per-buffer padding (the guessed/device block over-reads a stream with fewer
+					// channels/bits). Push both parts only if both fit and the payload is fully
+					// readable, so a full ring or a short buffer can never tear the framing.
+					const std::uint32_t want = num_frames * block;
+					if (readable_bytes(t_gb_data, want) == want &&
+					    audio_ring_free_space(*vring) >= static_cast<std::uint32_t>(sizeof(num_frames)) + want)
+					{
+						audio_ring_push(*vring, &num_frames, sizeof(num_frames), 0);
+						audio_ring_push(*vring, t_gb_data, want, num_frames);
+					}
 				}
 			}
 		}

host/src/audio/audio_format_verifier.cpp

@@ -99,6 +99,35 @@ void drain_ring(AudioRingHeader& ring, std::vector<BYTE>& scratch)
 	}
 }
 
+// Parse the hook's self-describing verify stream -- a sequence of [u32 frame-count][count*stride
+// bytes] chunks -- into a ChunkedCapture. `stride` is the device block_align (what the hook padded
+// each buffer to). Stops at the first truncated/garbled record.
+ChunkedCapture parse_chunks(const std::vector<BYTE>& raw, unsigned stride)
+{
+	ChunkedCapture cap;
+	cap.stride = stride;
+	if (stride == 0)
+	{
+		return cap;
+	}
+	std::size_t off = 0;
+	while (off + sizeof(std::uint32_t) <= raw.size())
+	{
+		std::uint32_t count = 0;
+		std::memcpy(&count, raw.data() + off, sizeof(count));
+		off += sizeof(count);
+		const std::size_t payload = static_cast<std::size_t>(count) * stride;
+		if (count == 0 || off + payload > raw.size())
+		{
+			break; // truncated or garbled -> stop
+		}
+		cap.counts.push_back(count);
+		cap.bytes.insert(cap.bytes.end(), raw.data() + off, raw.data() + off + payload);
+		off += payload;
+	}
+	return cap;
+}
+
 } // namespace
 
 FormatVerification verify_stream_format(DWORD pid, AudioRingHeader* ring, unsigned window_ms, bool recover_layout)
@@ -156,31 +185,53 @@ FormatVerification verify_stream_format(DWORD pid, AudioRingHeader* ring, unsign
 	ring->verify_capture.store(0, std::memory_order_release);
 	drain_ring(*ring, scratch); // leave the ring clean for the real capture that follows
 
-	// The hook bytes are at the guessed layout = the device channels/bits (the assumption the
+	// The loopback is at the KNOWN device layout. Decode it to mono ground truth. The hook stream
-	// cadence path also makes). Decode both captures with that layout and correlate.
+	// is self-describing chunks ([count][padded payload]); parse them at the device block.
-	const std::vector<float> hook_mono = to_mono(hook_bytes, dev);
 	const std::vector<float> loop_mono = to_mono(loop_bytes, dev);
+	const unsigned dev_block = dev_wfx->nBlockAlign;
+	const ChunkedCapture cap = parse_chunks(hook_bytes, dev_block);
+	const std::size_t need = dev.rate / 5; // require >= ~200 ms of usable audio on both sides
+	const std::size_t hook_frames = dev_block != 0 ? cap.bytes.size() / dev_block : 0;
 	CoTaskMemFree(dev_wfx);
 
 	if (const char* dbg = std::getenv("COOP_VERIFY_DEBUG"); dbg != nullptr && dbg[0] == '1')
 	{
-		std::fprintf(stderr, "[verify] dev=%uHz/%uch/%ubit tag=%u hook_frames=%zu loop_frames=%zu\n", dev.rate,
+		std::fprintf(stderr, "[verify] dev=%uHz/%uch/%ubit blk=%u chunks=%zu hook_frames=%zu loop=%zu layout=%d\n",
-		             dev.channels, dev.bits, dev.tag, hook_mono.size(), loop_mono.size());
+		             dev.rate, dev.channels, dev.bits, dev_block, cap.counts.size(), hook_frames, loop_mono.size(),
+		             recover_layout ? 1 : 0);
 	}
-	const std::size_t need = dev.rate / 5; // require >= ~200 ms of usable audio on both sides
+	if (loop_mono.size() < need || hook_frames < need)
-	if (hook_mono.size() < need || loop_mono.size() < need)
 	{
 		return result; // not enough non-silent audio captured (game quiet, or stream wasn't a guess)
 	}
 
-	const RateCorrelation rc = correlate_rate(hook_mono, loop_mono, dev.rate, standard_audio_rates());
+	if (recover_layout)
-	result.ok = rc.ok;
+	{
-	result.rate = rc.rate;
+		// Step (b): recover channels + bit depth too, by trying candidate de-interleavings of the
-	result.score = rc.score;
+		// (de-padded) hook bytes and keeping whichever (layout, rate) correlates with the loopback.
-	// Channels/bit-depth stay the device assumption here; step (b) recovers them.
+		const FormatCorrelation fc =
-	result.channels = dev.channels;
+		    correlate_format(cap, loop_mono, dev.rate, standard_audio_rates(), standard_audio_layouts());
-	result.bits = dev.bits;
+		result.ok = fc.ok;
-	result.format_tag = dev.tag;
+		result.rate = fc.rate;
+		result.score = fc.score;
+		result.layout_ok = fc.ok;
+		result.channels = fc.channels;
+		result.bits = fc.bits;
+		result.format_tag = fc.tag;
+	}
+	else
+	{
+		// Step (a): rate only, assuming the hook layout matches the device (common stereo case), so
+		// the de-padded payload is already clean device-layout audio.
+		const std::vector<float> hook_mono = to_mono(cap.bytes, dev);
+		const RateCorrelation rc = correlate_rate(hook_mono, loop_mono, dev.rate, standard_audio_rates());
+		result.ok = rc.ok;
+		result.rate = rc.rate;
+		result.score = rc.score;
+		result.channels = dev.channels;
+		result.bits = dev.bits;
+		result.format_tag = dev.tag;
+	}
 	return result;
 }
 

host/src/audio/audio_loopback.cpp

@@ -323,10 +323,15 @@ void AudioMirror::thread_main(DWORD pid)
 				if (!format_verified)
 				{
 					format_verified = true;
-					const FormatVerification fv = verify_stream_format(pid, rings[0]);
+					// recover_layout: correlate the full format (rate AND channels/bit-depth), so a
+					// game rendering a different layout than the device is corrected too, not just
+					// the rate. A no-op when nothing correlates confidently (e.g. an exact stream,
+					// a silent game, or a genuinely ambiguous identical-channel layout).
+					const FormatVerification fv = verify_stream_format(pid, rings[0], /*window_ms=*/900,
+					                                                   /*recover_layout=*/true);
 					if (fv.ok)
 					{
-						set_status("Verified render-hook sample rate by correlation.");
+						set_status("Verified render-hook format by correlation.");
 						audio_ring_post_op(*rings[0], AudioRingOp_Override, fv.rate, fv.channels, fv.bits,
 						                   fv.format_tag);
 					}

tests/audio_correlation_test.cpp

@@ -6,6 +6,7 @@
 // rates (plus a capture-latency skew and a little noise), then assert correlate_rate() recovers the
 // true rate -- including the hard 44100-vs-48000 case the cadence method can misread. Pure header
 // logic, no device.
+#include <algorithm>
 #include <cmath>
 #include <cstdint>
 #include <cstdio>
@@ -55,6 +56,118 @@ std::vector<float> capture(unsigned rate, double seconds, double t0, double nois
 	return out;
 }
 
+// Per-channel continuous signal: each channel carries genuinely different content (its own
+// frequency set), like a real stereo/surround stream. This is what disambiguates the channel
+// count -- with identical channels, 2ch@R and 1ch@2R produce the same bytes and are truly
+// indistinguishable (the confidence gate correctly rejects that case).
+double multi(double t, unsigned channel)
+{
+	const double k = 1.0 + 0.37 * static_cast<double>(channel); // distinct frequency scale per channel
+	const double chirp = std::sin(2.0 * kPi * (300.0 * k * t + 140.0 * t * t));
+	return 0.5 * std::sin(2.0 * kPi * 221.0 * k * t) + 0.28 * std::sin(2.0 * kPi * 437.0 * k * t + 0.6) +
+	       0.22 * chirp;
+}
+
+double multi_mono(double t, unsigned channels)
+{
+	double sum = 0.0;
+	for (unsigned c = 0; c < channels; ++c)
+	{
+		sum += multi(t, c);
+	}
+	return sum / channels;
+}
+
+// Encode the multichannel signal to raw interleaved PCM bytes at the given layout/rate.
+std::vector<std::uint8_t> encode(unsigned rate, unsigned channels, unsigned bits, unsigned tag, double seconds)
+{
+	const unsigned bps = bits / 8;
+	const std::size_t frames = static_cast<std::size_t>(rate * seconds);
+	std::vector<std::uint8_t> out(frames * channels * bps);
+	for (std::size_t i = 0; i < frames; ++i)
+	{
+		const double t = static_cast<double>(i) / rate;
+		for (unsigned c = 0; c < channels; ++c)
+		{
+			const double s = multi(t, c);
+			std::uint8_t* p = out.data() + (i * channels + c) * bps;
+			if (tag == coop::kWaveFormatFloat)
+			{
+				const float f = static_cast<float>(s);
+				std::memcpy(p, &f, 4);
+			}
+			else
+			{
+				const std::int16_t v = static_cast<std::int16_t>(s * 30000.0);
+				std::memcpy(p, &v, 2);
+			}
+		}
+	}
+	return out;
+}
+
+// Build the hook's self-describing chunked capture from clean audio: split into ~480-frame chunks
+// and pad each frame from real_block up to `stride` with GARBAGE -- exactly what the hook's verify
+// tap produces (it over-reads a guessed stream whose real layout has fewer channels/bits than the
+// device block). correlate_format must strip the padding per candidate layout.
+coop::ChunkedCapture make_chunks(unsigned rate, unsigned ch, unsigned bits, unsigned tag, unsigned stride,
+                                 double seconds)
+{
+	coop::ChunkedCapture cap;
+	cap.stride = stride;
+	const std::vector<std::uint8_t> clean = encode(rate, ch, bits, tag, seconds);
+	const unsigned real_block = ch * (bits / 8);
+	const std::size_t frames = clean.size() / real_block;
+	std::mt19937 rng(123);
+	std::uniform_int_distribution<int> garbage(0, 255);
+	std::size_t f = 0;
+	while (f < frames)
+	{
+		const unsigned count = static_cast<unsigned>(std::min<std::size_t>(480, frames - f));
+		cap.counts.push_back(count);
+		// The hook reads count*stride contiguous bytes: the count real frames first
+		// (count*real_block bytes), then count*(stride-real_block) bytes of stale over-read.
+		const std::uint8_t* src = clean.data() + f * real_block;
+		cap.bytes.insert(cap.bytes.end(), src, src + static_cast<std::size_t>(count) * real_block);
+		for (std::size_t p = 0; p < static_cast<std::size_t>(count) * (stride - real_block); ++p)
+		{
+			cap.bytes.push_back(static_cast<std::uint8_t>(garbage(rng)));
+		}
+		f += count;
+	}
+	return cap;
+}
+
+// One layout scenario: the hook bytes are at (true_*) and still being measured; the loopback is the
+// post-mix mono of the same audio at device_rate. Assert correlate_format recovers the full layout.
+void test_layout(unsigned true_rate, unsigned true_ch, unsigned true_bits, unsigned true_tag,
+                 unsigned device_rate, const char* label)
+{
+	std::printf("== layout: %s (%u Hz / %u ch / %u-bit %s -> device %u Hz) ==\n", label, true_rate, true_ch,
+	            true_bits, true_tag == coop::kWaveFormatFloat ? "float" : "pcm", device_rate);
+	// Device block 32 (8ch float) is the largest stride; every test layout's real block is <= 32.
+	const coop::ChunkedCapture hook = make_chunks(true_rate, true_ch, true_bits, true_tag, /*stride=*/32, 0.6);
+	// Loopback: the post-mix mono of the same audio, at the device rate, started ~18 ms later + noise.
+	const std::size_t loop_frames = static_cast<std::size_t>(device_rate * 0.6);
+	std::vector<float> loop(loop_frames);
+	std::mt19937 rng(5);
+	std::uniform_real_distribution<float> j(-1.0f, 1.0f);
+	for (std::size_t m = 0; m < loop_frames; ++m)
+	{
+		loop[m] = static_cast<float>(multi_mono(0.018 + static_cast<double>(m) / device_rate, true_ch)) +
+		          0.02f * j(rng);
+	}
+
+	const FormatCorrelation r =
+	    correlate_format(hook, loop, device_rate, standard_audio_rates(), standard_audio_layouts());
+	std::printf("   picked %u Hz / %u ch / %u-bit %s score=%.3f runner_up=%.3f ok=%d\n", r.rate, r.channels,
+	            r.bits, r.tag == coop::kWaveFormatFloat ? "float" : "pcm", r.score, r.runner_up, r.ok ? 1 : 0);
+	check(r.ok, "layout pick is confident");
+	check(r.rate == true_rate, "recovered the true rate");
+	check(r.channels == true_ch, "recovered the true channel count");
+	check(r.bits == true_bits && r.tag == true_tag, "recovered the true bit depth / sample format");
+}
+
 // One scenario: true hook rate `true_rate` mixed to `device_rate`. Assert the correlator picks
 // true_rate confidently and that the runner-up is clearly behind.
 void test_case(unsigned true_rate, unsigned device_rate, const char* label)
@@ -84,6 +197,13 @@ int main()
 	test_case(32000, 44100, "low-rate stream on a 44100 endpoint");
 	test_case(48000, 44100, "48000 stream on a 44100 endpoint");
 
+	// Step (b): recover the full layout (channels + bit depth) when it differs from the device, by
+	// trying candidate de-interleavings -- the case the rate-only step can't handle.
+	test_layout(44100, 2, 32, coop::kWaveFormatFloat, 48000, "stereo float, wrong rate");
+	test_layout(44100, 2, 16, coop::kWaveFormatPcm, 48000, "stereo 16-bit PCM (bit depth differs)");
+	test_layout(48000, 6, 32, coop::kWaveFormatFloat, 48000, "5.1 float (channels differ)");
+	test_layout(44100, 1, 32, coop::kWaveFormatFloat, 48000, "mono"); // 2ch@22050 isn't a candidate -> unambiguous
+
 	// Downmix sanity: a stereo interleaved buffer collapses to the same mono the scalar path uses.
 	{
 		std::printf("== downmix stereo -> mono ==\n");

tests/audio_verify_test.cpp

@@ -1,22 +1,23 @@
 // Integration test for the two-path audio-format verifier (host/src/audio/audio_format_verifier).
 //
-// Launches coop_mock_game rendering a tone at a NON-device rate (44100 on a typical 48000 endpoint,
+// Launches coop_mock_game rendering a tone via WASAPI AUTOCONVERTPCM, injects coop_hook.dll late
-// the Godot/Brotato case), injects coop_hook.dll late (so the stream is a *guess*), then runs the
+// (so the stream is a *guess*), and runs the real verify_stream_format(): it co-captures the hook
-// real verify_stream_format(): it co-captures the hook (pre-mix, via the ring's verify tap) and a
+// (pre-mix, via the ring's verify tap) and a parallel process-loopback (post-mix, device format)
-// parallel process-loopback (post-mix, device format) and cross-correlates them. Asserts it
+// and cross-correlates them. Two scenarios:
-// recovers the true 44100 Hz rate -- the cadence method's hard case. Skips cleanly without an audio
+//   (a) rate: render at the device's channel count but a different rate -> recover the rate.
-// endpoint / if Vulkan-free... (only needs WASAPI + a D3D11-capable mock, which the mock always is).
+//   (b) layout: render a DIFFERENT channel count than the device, with distinct per-channel content
+//       -> recover channels + bit depth + rate.
+// Skips cleanly without an audio endpoint.
 #include <cstdint>
 #include <cstdio>
 #include <string>
 
 #include <windows.h>
 
+#include <mmreg.h>
 #include <objbase.h>
 #include <tlhelp32.h>
 
-#include <mmreg.h>
-
 #include "audio/audio_format_verifier.hpp"
 #include "audio/process_loopback_capture.hpp" // default_render_format
 #include "coop/audio_ring.hpp"
@@ -111,42 +112,42 @@ bool inject_retry(unsigned long pid)
 	}
 	return false;
 }
-} // namespace
 
-int main()
+struct Scenario
 {
+	unsigned rate, channels, bits;
+	bool distinct;       // distinct per-channel content (so the channel count is recoverable)
+	bool recover_layout; // false = rate only (step a); true = full layout (step b)
+};
+
+// Launch the mock at the scenario's format, inject the hook late, and run the verifier. `ran` is
+// set false when the environment can't support the test (launch/inject failed) so the caller skips.
+FormatVerification run(const Scenario& sc, bool& ran)
+{
+	ran = false;
+	FormatVerification fv;
 	kill_stray_mock_games();
 
-	const bool com = SUCCEEDED(CoInitializeEx(nullptr, COINIT_MULTITHREADED));
+	if (sc.distinct)
-
-	// Render the mock at the device's CHANNEL count (so this step-(a) rate test isn't perturbed by
-	// a channel mismatch -- that's step (b)'s job) but at a DIFFERENT standard rate than the device,
-	// so the verifier has a real rate to recover. Default to 48000/2ch if we can't read the device.
-	unsigned dev_rate = 48000, dev_channels = 2;
-	if (WAVEFORMATEX* dev = default_render_format())
 	{
-		dev_rate = dev->nSamplesPerSec;
+		SetEnvironmentVariableW(L"COOP_TONE_DISTINCT_CH", L"1");
-		dev_channels = dev->nChannels;
-		CoTaskMemFree(dev);
 	}
-	const unsigned game_rate = (dev_rate == 44100) ? 48000u : 44100u; // guarantee a rate mismatch
-	std::printf("   device %u Hz / %u ch -> rendering the mock at %u Hz / %u ch (rate mismatch)\n", dev_rate,
-	            dev_channels, game_rate, dev_channels);
-
 	const std::wstring exe = exe_directory() + L"coop_mock_game.exe";
-	std::wstring cmd = L"\"" + exe + L"\" dx11 30 " + std::to_wstring(game_rate) + L" " +
+	std::wstring cmd = L"\"" + exe + L"\" dx11 30 " + std::to_wstring(sc.rate) + L" " +
-	                   std::to_wstring(dev_channels) + L" 32 float";
+	                   std::to_wstring(sc.channels) + L" " + std::to_wstring(sc.bits) + L" " +
+	                   (sc.bits == 16 ? L"pcm" : L"float");
 	STARTUPINFOW si{};
 	si.cb = sizeof(si);
 	PROCESS_INFORMATION pi{};
-	if (!CreateProcessW(exe.c_str(), cmd.data(), nullptr, nullptr, FALSE, 0, nullptr, nullptr, &si, &pi))
+	const BOOL launched = CreateProcessW(exe.c_str(), cmd.data(), nullptr, nullptr, FALSE, 0, nullptr, nullptr,
+	                                     &si, &pi);
+	if (sc.distinct)
 	{
-		std::printf("Could not launch coop_mock_game -- skipping audio_verify_test.\n");
+		SetEnvironmentVariableW(L"COOP_TONE_DISTINCT_CH", nullptr);
-		if (com)
+	}
-		{
+	if (!launched)
-			CoUninitialize();
+	{
-		}
+		return fv;
-		return 0;
 	}
 	auto cleanup = [&] {
 		TerminateProcess(pi.hProcess, 0);
@@ -155,73 +156,104 @@ int main()
 		CloseHandle(pi.hProcess);
 		kill_stray_mock_games();
 	};
-	Sleep(800); // window + audio client up
+	Sleep(800);
 
-	// IPC + the primary audio ring the hook produces into.
 	SharedMemory shm;
-	if (!shm.create(shared_memory_name(pi.dwProcessId), sizeof(SharedBlock)))
+	SharedMemory ring_shm;
+	if (!shm.create(shared_memory_name(pi.dwProcessId), sizeof(SharedBlock)) ||
+	    !ring_shm.create(audio_ring_name(pi.dwProcessId), audio_ring_total_size(kAudioRingCapacity)))
 	{
-		std::printf("Could not create IPC block -- skipping.\n");
 		cleanup();
-		return 0;
+		return fv;
 	}
 	auto* block = shm.as<SharedBlock>();
 	block->version = kProtocolVersion;
 	block->pad_count = 0;
 	block->sequence.store(0, std::memory_order_relaxed);
-	// Only the audio subsystem.
+	for (std::uint32_t s = 0; s < HookSubsys_Count; ++s) // audio subsystem only
-	for (std::uint32_t s = 0; s < HookSubsys_Count; ++s)
 	{
-		const bool off = s != HookSubsys_Audio;
+		block->control.subsystem_disabled[s].store(s != HookSubsys_Audio ? 1u : 0u, std::memory_order_release);
-		block->control.subsystem_disabled[s].store(off ? 1u : 0u, std::memory_order_release);
 	}
 	block->magic = kProtocolMagic;
-
-	SharedMemory ring_shm;
-	if (!ring_shm.create(audio_ring_name(pi.dwProcessId), audio_ring_total_size(kAudioRingCapacity)))
-	{
-		std::printf("Could not create audio ring -- skipping.\n");
-		cleanup();
-		return 0;
-	}
 	auto* ring = ring_shm.as<AudioRingHeader>();
-	audio_ring_init(*ring, kAudioRingCapacity);
+	audio_ring_init(*ring, kAudioRingCapacity); // capture_enabled stays 0: audible + still measuring
-	// Leave capture_enabled = 0: we want the stream audible (so loopback hears it) and still being
-	// MEASURED (so the verify tap fires), exactly the window verify_stream_format targets.
 
 	if (!inject_retry(pi.dwProcessId))
 	{
-		std::printf("Could not inject coop_hook.dll -- skipping.\n");
 		cleanup();
-		return 0;
+		return fv;
 	}
-	Sleep(500); // let the hook attach + the pre-existing render client register as a guess
+	Sleep(500); // hook attaches + the pre-existing render client registers as a guess
 
-	// Run the real verifier: co-capture hook (pre-mix) + loopback (post-mix) and correlate.
+	fv = verify_stream_format(pi.dwProcessId, ring, /*window_ms=*/1400, sc.recover_layout);
-	const FormatVerification fv = verify_stream_format(pi.dwProcessId, ring, /*window_ms=*/1400);
+	ran = true;
-	std::printf("   verify: ok=%d rate=%u score=%.3f\n", fv.ok ? 1 : 0, fv.rate, fv.score);
+	cleanup();
+	return fv;
+}
+} // namespace
 
-	if (!fv.ok && fv.rate == 0 && fv.score == 0.0)
+int main()
+{
+	const bool com = SUCCEEDED(CoInitializeEx(nullptr, COINIT_MULTITHREADED));
+
+	unsigned dev_rate = 48000, dev_channels = 2;
+	if (WAVEFORMATEX* dev = default_render_format())
 	{
-		// No audio endpoint, or no usable audio captured (e.g. the mock's WASAPI client never
+		dev_rate = dev->nSamplesPerSec;
-		// started on this machine) -> treat as a skip rather than a failure.
+		dev_channels = dev->nChannels;
-		std::printf("   no usable co-capture (no endpoint / silent) -- skipping audio_verify_test.\n");
+		CoTaskMemFree(dev);
+	}
+	const unsigned mismatched = (dev_rate == 44100) ? 48000u : 44100u; // guarantee a rate mismatch
+	std::printf("device: %u Hz / %u ch\n", dev_rate, dev_channels);
+
+	// (a) Rate: render at the device's channel count (no layout mismatch) but a different rate.
+	std::printf("== (a) rate recovery: %u Hz / %u ch ==\n", mismatched, dev_channels);
+	bool ran = false;
+	FormatVerification a = run({mismatched, dev_channels, 32, /*distinct=*/false, /*recover_layout=*/false}, ran);
+	if (!ran)
+	{
+		std::printf("   environment can't run the mock+inject -- skipping audio_verify_test.\n");
 		if (com)
 		{
 			CoUninitialize();
 		}
-		cleanup();
 		return 0;
 	}
+	std::printf("   ok=%d rate=%u score=%.3f\n", a.ok ? 1 : 0, a.rate, a.score);
+	if (!a.ok && a.rate == 0 && a.score == 0.0)
+	{
+		std::printf("   no usable co-capture (no endpoint / silent) -- skipping.\n");
+		if (com)
+		{
+			CoUninitialize();
+		}
+		return 0;
+	}
+	check(a.ok, "(a) verifier confidently correlated the two capture paths");
+	check(a.rate == mismatched, "(a) recovered the game's true rate (not the device rate)");
 
-	check(fv.ok, "verifier confidently correlated the two capture paths");
+	// (b) Layout: render 2ch with distinct per-channel content -- a layout that differs from a
-	check(fv.rate == game_rate, "verifier recovered the game's true rate (not the device rate)");
+	// multichannel device -- and recover channels + bit depth + rate.
+	std::printf("== (b) layout recovery: 44100 Hz / 2 ch / 32-bit float (distinct channels) ==\n");
+	FormatVerification b = run({44100, 2, 32, /*distinct=*/true, /*recover_layout=*/true}, ran);
+	std::printf("   ok=%d rate=%u ch=%u bits=%u tag=%u score=%.3f\n", b.ok ? 1 : 0, b.rate, b.channels, b.bits,
+	            b.format_tag, b.score);
+	if (b.ok || b.score > 0.0)
+	{
+		check(b.layout_ok, "(b) verifier confidently recovered the layout");
+		check(b.rate == 44100, "(b) recovered the true rate");
+		check(b.channels == 2, "(b) recovered the true channel count (2, not the device's)");
+		check(b.bits == 32 && b.format_tag == 3, "(b) recovered 32-bit float");
+	}
+	else
+	{
+		std::printf("   no usable co-capture for (b) -- skipping that scenario.\n");
+	}
 
 	if (com)
 	{
 		CoUninitialize();
 	}
-	cleanup();
 
 	if (g_failures == 0)
 	{

tools/audio_tone/tone_source.hpp

@@ -7,6 +7,7 @@
 
 #include <cmath>
 #include <cstdint>
+#include <cstdlib>
 
 #include <windows.h>
 
@@ -87,6 +88,13 @@ public:
 		client_->GetBufferSize(&buffer_frames_);
 
 		step_ = kTwoPi * freq_hz / static_cast<double>(fmt_.rate);
+		// Optional: give each channel genuinely different content (a per-channel frequency scale),
+		// so a downstream test can *recover* the channel count by correlation (identical channels
+		// are ambiguous: 2ch@R looks like 1ch@2R). Off by default -> the usual single-tone source.
+		if (const char* d = std::getenv("COOP_TONE_DISTINCT_CH"); d != nullptr && d[0] == '1')
+		{
+			distinct_ = true;
+		}
 		write(buffer_frames_); // pre-roll
 		client_->Start();
 		return true;
@@ -220,13 +228,24 @@ private:
 			}
 			for (unsigned c = 0; c < fmt_.channels; ++c)
 			{
+				double sc = s;
+				if (distinct_ && c < 8)
+				{
+					// Each channel at its own frequency scale -> genuinely different content.
+					sc = std::sin(phase_c_[c]) * 0.25;
+					phase_c_[c] += step_ * (1.0 + 0.37 * static_cast<double>(c));
+					if (phase_c_[c] > kTwoPi)
+					{
+						phase_c_[c] -= kTwoPi;
+					}
+				}
 				if (float_)
 				{
-					reinterpret_cast<float*>(data)[i * fmt_.channels + c] = static_cast<float>(s);
+					reinterpret_cast<float*>(data)[i * fmt_.channels + c] = static_cast<float>(sc);
 				}
 				else
 				{
-					reinterpret_cast<INT16*>(data)[i * fmt_.channels + c] = static_cast<INT16>(s * 32767.0);
+					reinterpret_cast<INT16*>(data)[i * fmt_.channels + c] = static_cast<INT16>(sc * 32767.0);
 				}
 			}
 		}
@@ -243,6 +262,8 @@ private:
 	bool float_ = false;
 	double phase_ = 0.0;
 	double step_ = 0.0;
+	bool distinct_ = false;     // per-channel distinct content (recoverable channel count)
+	double phase_c_[8] = {};    // per-channel phase when distinct_
 };
 
 } // namespace coop::tone