BlackMark/CoopAllTheThings
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Audio: detect a pre-existing render stream's true sample rate (fix pitch)

Browse Source 
Hooked audio mirroring played back pitch-shifted on games we inject into
that render at a non-device sample rate (e.g. Godot/Brotato render 44100 Hz
on a 48000 Hz endpoint via WASAPI AUTOCONVERTPCM). We attach to an
already-running game, so the render-hook never saw its IAudioClient::
Initialize and assumed the device mix format -- right channels/bits, wrong
rate -- so 44100 audio was rendered as 48000 (+~1.5 semitones).

Fix: treat a pre-existing client's format as a guess and measure its true
sample rate from the render cadence (frames/sec over a steady-state window,
snapped to the nearest standard rate) before publishing it, deferring
capture until verified. Discard the first measurement window so the
buffer-fill burst at attach time doesn't over-count. Streams created after
we inject still carry their exact Initialize format.

Channels/bit-depth genuinely can't be recovered for a pre-existing client:
AUTOCONVERTPCM hands GetBuffer a fixed staging buffer (no buffer stride to
measure -- confirmed empirically) and WASAPI exposes no API for the format.
They stay the device-mix guess, which is correct for the common case
(engines render stereo float, matching the endpoint). To keep a wrong guess
safe, a VirtualQuery clamp stops the capture copy from ever over-reading the
source buffer when the guessed bytes/frame is too large.

Surface all of this: a per-stream AudioFormatState (known / measuring /
measured rate (ch/bits assumed)) in HookStatus, shown in the Audio panel for
the hooked path and as "device endpoint (known)" for loopback; clear hook
logs; and enriched mirror status strings. Documented in README (Limitations
+ Lessons learned). The loopback fallback was always correct (post-mix at
the device format).

Tests: extract a shared, configurable ToneSource (used by coop_tone and the
hook self-test); coop_tone takes rate/channels/bits/format args. Rewrite
audio_hook_test to a format matrix x both code paths -- see-init (exact) and
guess (rate measured) -- plus a byte-incompatible guess that asserts the
clamp keeps capture safe. The matrix caught the attach-burst over-count.
audio_loopback_test now spawns coop_tone at several source formats to
confirm loopback is format-agnostic. 11/11 x64 + 3/3 x86 pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
BlackMark
2026-06-21 23:41:57 +02:00
parent f15f5cdb36
commit 7264cb2ef4
11 changed files with 970 additions and 445 deletions
Download Patch File Download Diff File Expand all files Collapse all files

249
hook/src/audio_hook.cpp
View File

@@ -163,7 +163,8 @@ struct TrackedStream
{
std::atomic<IAudioRenderClient*> client{nullptr};
std::atomic<std::uint64_t> frames{0};
std::atomic<std::uint32_t> block_align{0}; // hot-path frame size for this stream
std::atomic<std::uint32_t> block_align{0}; // hot-path frame size for this stream
std::atomic<std::uint32_t> assumed_format{0}; // 1 = channels/bits guessed -> clamp copies safely
};
TrackedStream g_streams[kMaxAudioStreams];
std::uint32_t g_registered = 0; // slots filled (<= kMaxAudioStreams), under mutex
@@ -171,6 +172,37 @@ std::atomic<std::uint32_t> g_streams_seen{0}; // total distinct clients ever see
std::atomic<std::uint64_t> g_frames_captured{0}; // total frames captured across streams
// When we attach to an already-running game we never saw its IAudioClient::Initialize,
// so a render client discovered on the hot path gets the device mix format as a best
// guess. That guess is wrong for games that render at a non-device rate via WASAPI
// AUTOCONVERTPCM -- e.g. Godot/Brotato render 44100 while the device mixes at 48000, so
// playing the captured 44100 audio back as 48000 shifts the pitch up. For such streams we
// verify (and correct) the guessed sample rate by measuring the real render cadence
// before publishing the format. g_stream_rate_guess marks a guessed stream; g_rate_measure
// is its measurement window. Both guarded by g_setup_mutex.
bool g_stream_rate_guess[kMaxAudioStreams] = {};
struct RateMeasure
{
std::int64_t window_qpc = 0;
std::uint64_t window_frames = 0;
bool primed = false; // first full window discarded (attach/startup burst)
};
RateMeasure g_rate_measure[kMaxAudioStreams] = {};
// We can measure a guessed stream's sample rate, but channels/bits aren't recoverable for a
// client we never saw Initialize -- they stay the device-mix guess. That guess is right for
// the common case (games render stereo float, matching the device, just at a different
// rate), but if a game renders a different channel/bit layout the guessed bytes-per-frame is
// too large and the capture copy would over-read the game's buffer. We can't detect the true
// layout (AUTOCONVERTPCM hands back a fixed staging buffer, so there's no buffer-stride to
// measure, and WASAPI exposes no API for a pre-existing client's format), so we instead clamp
// every guessed-stream copy to the source buffer's committed region (copy_bound_locked /
// readable_bytes) -- the audio may be misinterpreted, but it can never read past the
// allocation. See README "Lessons learned".
// Per-stream AudioFormatState (how its format was determined), mirrored to the host UI.
std::uint32_t g_stream_format_state[kMaxAudioStreams] = {};
// GetBuffer/ReleaseBuffer are paired on one thread, never nested: stash the
// pointer the game just got so ReleaseBuffer can copy it before releasing.
thread_local IAudioRenderClient* t_gb_client = nullptr;
@@ -209,6 +241,26 @@ CapturedFormat capture_format(const WAVEFORMATEX* wfx)
bool stream_tracked(IAudioRenderClient* rc);
void try_register_lazy(IAudioRenderClient* rc);
// Bytes safely readable from `ptr` within its committed region. Used to cap a guessed
// stream's copy: if its channels/bits differ from the device guess the assumed block is too
// large, and this stops the capture copy from reading past the source buffer's allocation
// (the data is then misinterpreted, but it can never AV). A no-op when the guess is right.
std::uint32_t readable_bytes(const void* ptr, std::uint32_t want)
{
MEMORY_BASIC_INFORMATION mbi{};
if (VirtualQuery(ptr, &mbi, sizeof(mbi)) == sizeof(mbi) && mbi.State == MEM_COMMIT)
{
const auto* base = static_cast<const std::uint8_t*>(mbi.BaseAddress);
const auto avail = static_cast<std::uintptr_t>((base + mbi.RegionSize) -
static_cast<const std::uint8_t*>(ptr));
if (avail < want)
{
return static_cast<std::uint32_t>(avail);
}
}
return want;
}
HRESULT STDMETHODCALLTYPE hk_GetBuffer(IAudioRenderClient* self, UINT32 num_frames, BYTE** data)
{
hook_note_call(g_id_getbuffer);
@@ -252,11 +304,22 @@ HRESULT STDMETHODCALLTYPE hk_ReleaseBuffer(IAudioRenderClient* self, UINT32 num_
if (num_frames > 0 && (flags & AUDCLNT_BUFFERFLAGS_SILENT) == 0)
{
AudioRingHeader* ring = g_rings[i].load(std::memory_order_acquire);
// Only capture once the format is published -- for a guessed-rate stream that's
// after the true rate is measured, so we never capture/silence audio we'd
// mis-rate (and don't build a backlog while measuring; the game stays audible).
if (ring != nullptr && ring->capture_enabled.load(std::memory_order_relaxed) != 0 &&
t_gb_client == self && t_gb_data != nullptr && t_gb_frames == num_frames)
audio_ring_format_ready(*ring) && t_gb_client == self && t_gb_data != nullptr &&
t_gb_frames == num_frames)
{
const std::uint32_t block = g_streams[i].block_align.load(std::memory_order_relaxed);
const std::uint32_t bytes = num_frames * block;
std::uint32_t bytes = num_frames * block;
// If this stream's channels/bits were guessed (pre-existing client), the block
// may be too large for the real buffer; clamp to what's actually readable so the
// copy can never over-read the game's buffer (no-op when the guess is right).
if (g_streams[i].assumed_format.load(std::memory_order_relaxed) != 0)
{
bytes = readable_bytes(t_gb_data, bytes);
}
// Only silence if the frames made it into the ring; if the host has
// stalled (ring full) keep playing locally rather than going dead
// silent — degrades to today's echo, never to silence.
@@ -274,10 +337,128 @@ HRESULT STDMETHODCALLTYPE hk_ReleaseBuffer(IAudioRenderClient* self, UINT32 num_
return g_vh_releasebuffer.original<ReleaseBufferFn>()(self, num_frames, flags);
}
// Publish a stream's format + state to the host's per-stream debug channel. Caller holds
// g_setup_mutex.
void publish_stream_info_locked(std::uint32_t slot, const CapturedFormat& cf, std::uint32_t state,
std::uint64_t frames)
{
if (g_ipc == nullptr)
{
return;
}
AudioStreamInfo info{};
info.is_primary = (slot == 0) ? 1u : 0u;
info.sample_rate = cf.rate;
info.channels = static_cast<std::uint16_t>(cf.channels);
info.bits = static_cast<std::uint16_t>(cf.bits);
info.format_tag = cf.tag;
info.frames_rendered = frames;
info.format_state = state;
g_ipc->publish_audio_stream(slot, info);
}
// Snap a measured sample rate to the nearest standard rate when it's close (absorbing
// measurement jitter); standard rates are far enough apart that a 2% window is
// unambiguous. An unusual measured rate is taken as-is (rounded).
std::uint32_t snap_sample_rate(double measured)
{
static constexpr std::uint32_t kStd[] = {8000, 11025, 16000, 22050, 32000, 44100,
48000, 88200, 96000, 176400, 192000};
for (std::uint32_t s : kStd)
{
if (measured >= s * 0.98 && measured <= s * 1.02)
{
return s;
}
}
return static_cast<std::uint32_t>(measured + 0.5);
}
// Measure a stream's true sample rate from its render cadence over a >=200 ms active
// window. Returns 0 until a window has accumulated (the caller retries each tick), so a
// momentarily idle stream doesn't yield a bogus low rate. Caller holds g_setup_mutex.
std::uint32_t measured_stream_rate(std::uint32_t slot)
{
LARGE_INTEGER now{}, freq{};
QueryPerformanceCounter(&now);
QueryPerformanceFrequency(&freq);
const std::uint64_t frames = g_streams[slot].frames.load(std::memory_order_relaxed);
RateMeasure& m = g_rate_measure[slot];
if (m.window_qpc == 0)
{
m.window_qpc = now.QuadPart; // begin a fresh window
m.window_frames = frames;
return 0;
}
const std::int64_t dt = now.QuadPart - m.window_qpc;
if (freq.QuadPart <= 0 || dt < freq.QuadPart / 5) // < 200 ms -> keep accumulating
{
return 0;
}
const std::uint64_t df = frames - m.window_frames;
m.window_qpc = now.QuadPart; // restart the window for the next attempt
m.window_frames = frames;
if (df < 1000) // stream idle/near-silent this window -> can't trust it; re-stabilize
{
m.primed = false;
return 0;
}
if (!m.primed)
{
// Discard the first complete window. When we attach to a stream its already-queued
// buffers can be delivered in a burst (the app filling its WASAPI buffer), which
// over-counts frames; measure the next, steady-state window instead.
m.primed = true;
return 0;
}
return snap_sample_rate(static_cast<double>(df) /
(static_cast<double>(dt) / static_cast<double>(freq.QuadPart)));
}
// Publish stream `slot`'s format to its ring, first correcting a guessed sample rate by
// measurement. Returns true once published (false = no ring yet, or a guess still being
// measured, in which case the caller retries next tick). Caller holds g_setup_mutex.
bool publish_stream_format_locked(std::uint32_t slot)
{
AudioRingHeader* ring = g_rings[slot].load(std::memory_order_acquire);
if (ring == nullptr || g_stream_formats[slot].rate == 0)
{
return false; // no ring attached yet, or no stream in this slot
}
if (audio_ring_format_ready(*ring))
{
return true; // already published
}
CapturedFormat cf = g_stream_formats[slot];
if (g_stream_rate_guess[slot])
{
const std::uint32_t measured = measured_stream_rate(slot);
if (measured == 0)
{
return false; // wait for enough rendered audio to measure the true rate
}
if (measured != cf.rate)
{
logf("audio stream %u: corrected guessed rate %uHz -> measured %uHz", slot, cf.rate, measured);
}
cf.rate = measured;
g_stream_formats[slot].rate = measured; // reflect the correction in the debug/UI snapshot
g_stream_rate_guess[slot] = false; // rate verified; channels/bits stay the device assumption
g_stream_format_state[slot] = AudioFormat_Measured;
publish_stream_info_locked(slot, cf, AudioFormat_Measured,
g_streams[slot].frames.load(std::memory_order_relaxed));
}
audio_ring_set_format(*ring, cf.rate, cf.channels, cf.bits, cf.tag, cf.block_align);
logf("audio stream %u: format %uHz/%uch/%ubit -> ring %p", slot, cf.rate, cf.channels, cf.bits, ring);
return true;
}
// Registers a newly created render client: assigns it a debug slot, marks the
// first as primary (the one we capture), publishes it to HookStatus, and hooks
// the render-client vtable on first sight. Caller holds g_setup_mutex.
void register_render_client_locked(IAudioRenderClient* rc, const CapturedFormat& cf)
// the render-client vtable on first sight. `rate_is_guess` is true when `cf` is the
// device mix format assumed for a pre-existing client (its rate is then measured before
// the format is published). Caller holds g_setup_mutex.
void register_render_client_locked(IAudioRenderClient* rc, const CapturedFormat& cf, bool rate_is_guess)
{
for (std::uint32_t i = 0; i < kMaxAudioStreams; ++i)
{
@@ -302,31 +483,33 @@ void register_render_client_locked(IAudioRenderClient* rc, const CapturedFormat&
}
g_registered = slot + 1;
const std::uint32_t state = rate_is_guess ? AudioFormat_Measuring : AudioFormat_Exact;
g_stream_formats[slot] = cf;
g_stream_rate_guess[slot] = rate_is_guess;
g_stream_format_state[slot] = state;
g_rate_measure[slot] = RateMeasure{}; // fresh measurement window (used only for a guess)
g_streams[slot].frames.store(0, std::memory_order_relaxed);
g_streams[slot].assumed_format.store(rate_is_guess ? 1u : 0u, std::memory_order_relaxed);
g_streams[slot].block_align.store(cf.block_align, std::memory_order_relaxed); // before client (hot path)
g_streams[slot].client.store(rc, std::memory_order_release);
AudioStreamInfo info{};
info.is_primary = (slot == 0) ? 1u : 0u;
info.sample_rate = cf.rate;
info.channels = static_cast<std::uint16_t>(cf.channels);
info.bits = static_cast<std::uint16_t>(cf.bits);
info.format_tag = cf.tag;
info.frames_rendered = 0;
if (g_ipc != nullptr)
if (rate_is_guess)
{
g_ipc->publish_audio_stream(slot, info);
logf("audio stream %u: format unknown (pre-existing client) -> assuming device mix %uHz/%uch/%ubit; "
"measuring true rate; channels/bits assumed (verified byte-compatible before capture)",
slot, cf.rate, cf.channels, cf.bits);
}
else
{
logf("audio stream %u: exact format %uHz/%uch/%ubit from the game's Initialize", slot, cf.rate,
cf.channels, cf.bits);
}
publish_stream_info_locked(slot, cf, state, 0);
// Publish this stream's format to its own ring if the host has attached one yet.
AudioRingHeader* ring = g_rings[slot].load(std::memory_order_acquire);
logf("stream %u set: rc=%p ring=%p fmt=%uHz/%uch/%ubit (%s)", slot, rc, ring, cf.rate, cf.channels, cf.bits,
ring ? "published" : "no ring yet");
if (ring != nullptr)
{
audio_ring_set_format(*ring, cf.rate, cf.channels, cf.bits, cf.tag, cf.block_align);
}
// Publish the format to the stream's ring (if the host has attached one). A guessed
// rate is measured/corrected inside the helper first, so this may defer until enough
// audio has rendered to measure -- republish_audio_format retries each worker tick.
publish_stream_format_locked(slot);
// GetBuffer/ReleaseBuffer are hooked proactively at install time (the shared
// vtable covers every render client), so nothing to install per-stream here.
}
@@ -363,7 +546,7 @@ void try_register_lazy(IAudioRenderClient* rc)
return; // a concurrent path registered it first
}
logf("try_register_lazy: discovered pre-existing render client rc=%p", rc);
register_render_client_locked(rc, g_mix_format);
register_render_client_locked(rc, g_mix_format, /*rate_is_guess=*/true);
}
HRESULT STDMETHODCALLTYPE hk_Initialize(IAudioClient* self, AUDCLNT_SHAREMODE mode, DWORD flags,
@@ -418,7 +601,9 @@ HRESULT STDMETHODCALLTYPE hk_GetService(IAudioClient* self, REFIID riid, void**
if (have)
{
std::scoped_lock lock(g_setup_mutex);
register_render_client_locked(static_cast<IAudioRenderClient*>(*ppv), cf);
// We saw this client's Initialize (or its shared-mode mix format), so the rate
// is exact, not a guess.
register_render_client_locked(static_cast<IAudioRenderClient*>(*ppv), cf, /*rate_is_guess=*/false);
}
}
return hr;
@@ -582,15 +767,9 @@ void republish_audio_format()
std::scoped_lock lock(g_setup_mutex);
for (std::uint32_t i = 0; i < kMaxAudioStreams; ++i)
{
AudioRingHeader* ring = g_rings[i].load(std::memory_order_acquire);
if (ring == nullptr || audio_ring_format_ready(*ring) || g_stream_formats[i].rate == 0)
{
continue; // no ring, already published, or this slot has no stream yet
}
const CapturedFormat& cf = g_stream_formats[i];
audio_ring_set_format(*ring, cf.rate, cf.channels, cf.bits, cf.tag, cf.block_align);
logf("republish_audio_format: stream %u -> %uHz/%uch/%ubit ring %p", i, cf.rate, cf.channels, cf.bits,
ring);
// Publishes an exact format immediately; a guessed rate is measured first and
// published once a measurement window completes (retried on the next tick).
publish_stream_format_locked(i);
}
}
@@ -649,7 +828,11 @@ void remove_audio_hooks()
g_streams[i].client.store(nullptr, std::memory_order_relaxed);
g_streams[i].frames.store(0, std::memory_order_relaxed);
g_streams[i].block_align.store(0, std::memory_order_relaxed);
g_streams[i].assumed_format.store(0, std::memory_order_relaxed);
g_stream_formats[i] = CapturedFormat{};
g_stream_rate_guess[i] = false;
g_stream_format_state[i] = AudioFormat_Unknown;
g_rate_measure[i] = RateMeasure{};
g_rings[i].store(nullptr, std::memory_order_release);
}
g_client_formats.clear();