Audio: detect a pre-existing render stream's true sample rate (fix pitch)

Hooked audio mirroring played back pitch-shifted on games we inject into
that render at a non-device sample rate (e.g. Godot/Brotato render 44100 Hz
on a 48000 Hz endpoint via WASAPI AUTOCONVERTPCM). We attach to an
already-running game, so the render-hook never saw its IAudioClient::
Initialize and assumed the device mix format -- right channels/bits, wrong
rate -- so 44100 audio was rendered as 48000 (+~1.5 semitones).

Fix: treat a pre-existing client's format as a guess and measure its true
sample rate from the render cadence (frames/sec over a steady-state window,
snapped to the nearest standard rate) before publishing it, deferring
capture until verified. Discard the first measurement window so the
buffer-fill burst at attach time doesn't over-count. Streams created after
we inject still carry their exact Initialize format.

Channels/bit-depth genuinely can't be recovered for a pre-existing client:
AUTOCONVERTPCM hands GetBuffer a fixed staging buffer (no buffer stride to
measure -- confirmed empirically) and WASAPI exposes no API for the format.
They stay the device-mix guess, which is correct for the common case
(engines render stereo float, matching the endpoint). To keep a wrong guess
safe, a VirtualQuery clamp stops the capture copy from ever over-reading the
source buffer when the guessed bytes/frame is too large.

Surface all of this: a per-stream AudioFormatState (known / measuring /
measured rate (ch/bits assumed)) in HookStatus, shown in the Audio panel for
the hooked path and as "device endpoint (known)" for loopback; clear hook
logs; and enriched mirror status strings. Documented in README (Limitations
+ Lessons learned). The loopback fallback was always correct (post-mix at
the device format).

Tests: extract a shared, configurable ToneSource (used by coop_tone and the
hook self-test); coop_tone takes rate/channels/bits/format args. Rewrite
audio_hook_test to a format matrix x both code paths -- see-init (exact) and
guess (rate measured) -- plus a byte-incompatible guess that asserts the
clamp keeps capture safe. The matrix caught the attach-burst over-count.
audio_loopback_test now spawns coop_tone at several source formats to
confirm loopback is format-agnostic. 11/11 x64 + 3/3 x86 pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-21 23:41:57 +02:00
parent f15f5cdb36
commit 7264cb2ef4
11 changed files with 970 additions and 445 deletions

View File

@@ -14,6 +14,10 @@ namespace coop
namespace
{
const ImVec4 kGreen(0.4f, 1.0f, 0.4f, 1.0f);
const ImVec4 kAmber(1.0f, 0.8f, 0.3f, 1.0f);
const ImVec4 kRed(1.0f, 0.45f, 0.4f, 1.0f);
const char* format_tag_name(std::uint32_t tag)
{
switch (tag)
@@ -29,6 +33,36 @@ const char* format_tag_name(std::uint32_t tag)
}
}
// How the hooked backend learned a stream's format (drives the pitch correctness).
const char* audio_format_state_name(std::uint32_t state)
{
switch (state)
{
case AudioFormat_Exact:
return "known (from game)";
case AudioFormat_Measuring:
return "measuring rate...";
case AudioFormat_Measured:
return "measured rate (ch/bits assumed)";
default:
return "unknown";
}
}
ImVec4 audio_format_state_color(std::uint32_t state)
{
switch (state)
{
case AudioFormat_Exact:
case AudioFormat_Measured:
return kGreen; // format trustworthy -> correct pitch
case AudioFormat_Measuring:
return kAmber; // still verifying the rate
default:
return kRed; // unknown
}
}
} // namespace
void AudioPanel::draw_ui(const HookStatusView& status, bool debug_details)
@@ -71,12 +105,25 @@ void AudioPanel::draw_ui(const HookStatusView& status, bool debug_details)
{
const AudioMirror::Source src = mirror_.source();
const bool hooked = src == AudioMirror::Source::Hooked;
ImGui::TextColored(ImVec4(0.4f, 1.0f, 0.4f, 1.0f), "Mirroring %u Hz, %u ch",
mirror_.sample_rate(), mirror_.channels());
ImGui::TextColored(kGreen, "Mirroring %u Hz, %u ch", mirror_.sample_rate(), mirror_.channels());
ImGui::Text("Source:");
ImGui::SameLine();
ImGui::TextColored(hooked ? ImVec4(0.4f, 1.0f, 0.4f, 1.0f) : ImVec4(1.0f, 0.8f, 0.3f, 1.0f), "%s",
mirror_.source_name());
ImGui::TextColored(hooked ? kGreen : kAmber, "%s", mirror_.source_name());
// Where the rendered format came from -- so it's clear the playback pitch is right.
// Hooked: the primary stream's provenance (exact / measured). Loopback: the audio is
// captured post-mix at the device endpoint format, so it's always known-correct.
ImGui::Text("Format:");
ImGui::SameLine();
if (hooked)
{
const std::uint32_t st = status.audio_streams[0].format_state; // [0] is the primary
ImGui::TextColored(audio_format_state_color(st), "%s", audio_format_state_name(st));
}
else
{
ImGui::TextColored(kGreen, "device endpoint (known, post-mix)");
}
ImGui::Text("Buffered: %4u ms", mirror_.buffered_ms());
}
const std::string mirror_status = mirror_.status();
@@ -132,11 +179,12 @@ void AudioPanel::draw_ui(const HookStatusView& status, bool debug_details)
const double now = ImGui::GetTime();
const bool resample = (now - rate_base_time_) >= 0.5; // recompute frames/s ~2x a second
if (rows > 0 &&
ImGui::BeginTable("audio_streams", 5, ImGuiTableFlags_Borders | ImGuiTableFlags_SizingFixedFit))
ImGui::BeginTable("audio_streams", 6, ImGuiTableFlags_Borders | ImGuiTableFlags_SizingFixedFit))
{
ImGui::TableSetupColumn("#");
ImGui::TableSetupColumn("role");
ImGui::TableSetupColumn("format");
ImGui::TableSetupColumn("source");
ImGui::TableSetupColumn("frames");
ImGui::TableSetupColumn("live");
ImGui::TableHeadersRow();
@@ -170,6 +218,9 @@ void AudioPanel::draw_ui(const HookStatusView& status, bool debug_details)
ImGui::Text("%u Hz %uch %u-bit %s", s.sample_rate, s.channels, s.bits,
format_tag_name(s.format_tag));
ImGui::TableNextColumn();
ImGui::TextColored(audio_format_state_color(s.format_state), "%s",
audio_format_state_name(s.format_state));
ImGui::TableNextColumn();
ImGui::Text("%llu", static_cast<unsigned long long>(s.frames_rendered));
ImGui::TableNextColumn();
if (live)