BlackMark/CoopAllTheThings
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Capture every audio stream into its own ring and mix them on the host

Browse Source 
Games with several concurrent WASAPI render streams (e.g. Spider-Man: Miles
Morales) only had their first ("primary") stream mirrored; the rest kept playing
locally and never reached the guest. Now the render-hook captures + silences EVERY
tracked stream into its own ring (coop_audio_<pid>[_<index>]), each published with
that stream's own detected format (Initialize when caught, else GetMixFormat -- the
per-stream format detection, now actually used per ring rather than only for the
primary). The host creates a ring per stream and mixes the same-format streams with
a soft clip (host/src/audio/audio_mix.hpp); streams whose format differs from the
primary are still silenced (no echo) but skipped from the mix (would need
resampling).

The single-stream case is byte-for-byte unchanged: when only one stream is active
the host passes it through without the mixer, so the common path has no overhead or
fidelity change.

Verified: new audio_mix_test covers the decode/sum/soft-clip/encode math (float32 +
int16); audio_hook_test (x64 + x86) still passes, guarding the primary
capture+silence path against regression; full build x64 + x86 clean; ctest x64
11/11, x86 3/3. Multi-stream mixing against a real multi-stream game needs a live
session to fully confirm.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
BlackMark
2026-06-21 05:45:47 +02:00
parent dfd2be8c17
commit 784c31a9b5
10 changed files with 412 additions and 147 deletions
Download Patch File Download Diff File Expand all files Collapse all files

164
hook/src/audio_hook.cpp
View File

@@ -113,7 +113,9 @@ struct CapturedFormat
// --- Global hook state -----------------------------------------------------
IpcClient* g_ipc = nullptr;
std::atomic<AudioRingHeader*> g_ring{nullptr};
// One ring per tracked stream (index = the stream's debug slot). Stream 0 is the
// primary; the host creates a ring per stream and mixes them.
std::atomic<AudioRingHeader*> g_rings[kMaxAudioStreams]{};
std::mutex g_setup_mutex; // guards installs + the format map + stream registration
@@ -146,32 +148,28 @@ std::atomic<IAudioRenderClient*> g_self_render{nullptr};
CapturedFormat g_mix_format;
std::atomic<std::uint32_t> g_have_mix_format{0};
// The primary stream's actual format, captured when it's registered. The host
// creates the ring only when audio mirroring is toggled on — typically *after*
// the primary stream was already registered — so the format must be (re)published
// to the ring whenever it attaches. (The in-process probe creates the ring before
// injecting, so it never exercises this ordering; the full app always does.)
// Guarded by g_setup_mutex.
CapturedFormat g_primary_format;
// Each tracked stream's actual format, captured when it's registered. The host
// creates the rings only when audio mirroring is toggled on — typically *after* a
// stream was already registered — so the format must be (re)published to a ring
// whenever it attaches. Guarded by g_setup_mutex.
CapturedFormat g_stream_formats[kMaxAudioStreams];
// Per IAudioClient, the format captured at Initialize, looked up when its render
// client is created. Setup-path only (never touched on the audio thread).
std::unordered_map<IAudioClient*, CapturedFormat> g_client_formats;
// Streams we track for the debug view (frame counting). Index 0 is primary.
// Streams we track (frame counting + per-stream capture). Index 0 is primary.
struct TrackedStream
{
std::atomic<IAudioRenderClient*> client{nullptr};
std::atomic<std::uint64_t> frames{0};
std::atomic<std::uint32_t> block_align{0}; // hot-path frame size for this stream
};
TrackedStream g_streams[kMaxAudioStreams];
std::uint32_t g_registered = 0; // slots filled (<= kMaxAudioStreams), under mutex
std::atomic<std::uint32_t> g_streams_seen{0}; // total distinct clients ever seen
// Hot-path primary identity + frame size (avoids touching the map/mutex).
std::atomic<IAudioRenderClient*> g_primary{nullptr};
std::atomic<std::uint32_t> g_primary_block_align{0};
std::atomic<std::uint64_t> g_frames_captured{0};
std::atomic<std::uint64_t> g_frames_captured{0}; // total frames captured across streams
// GetBuffer/ReleaseBuffer are paired on one thread, never nested: stash the
// pointer the game just got so ReleaseBuffer can copy it before releasing.
@@ -235,42 +233,43 @@ HRESULT STDMETHODCALLTYPE hk_ReleaseBuffer(IAudioRenderClient* self, UINT32 num_
try_register_lazy(self);
}
// Frame counting for any tracked stream (drives the live/idle debug view).
// Per tracked stream: count frames (debug view) and, into the stream's own ring,
// capture + silence its buffer while capture is enabled. Every stream is captured
// into its own ring; the host mixes them.
for (std::uint32_t i = 0; i < kMaxAudioStreams; ++i)
{
if (g_streams[i].client.load(std::memory_order_acquire) == self)
if (g_streams[i].client.load(std::memory_order_acquire) != self)
{
const std::uint64_t total =
g_streams[i].frames.fetch_add(num_frames, std::memory_order_relaxed) + num_frames;
if (g_ipc != nullptr)
{
g_ipc->note_audio_frames(i, total);
}
break;
continue;
}
const std::uint64_t total =
g_streams[i].frames.fetch_add(num_frames, std::memory_order_relaxed) + num_frames;
if (g_ipc != nullptr)
{
g_ipc->note_audio_frames(i, total);
}
}
// Capture + silence only the primary stream, only while enabled.
if (self == g_primary.load(std::memory_order_acquire) && num_frames > 0 &&
(flags & AUDCLNT_BUFFERFLAGS_SILENT) == 0)
{
AudioRingHeader* ring = g_ring.load(std::memory_order_acquire);
if (ring != nullptr && ring->capture_enabled.load(std::memory_order_relaxed) != 0 &&
t_gb_client == self && t_gb_data != nullptr && t_gb_frames == num_frames)
if (num_frames > 0 && (flags & AUDCLNT_BUFFERFLAGS_SILENT) == 0)
{
const std::uint32_t block = g_primary_block_align.load(std::memory_order_relaxed);
const std::uint32_t bytes = num_frames * block;
// Only silence if the frames made it into the ring; if the host has
// stalled (ring full) keep playing locally rather than going dead
// silent — degrades to today's echo, never to silence.
if (audio_ring_push(*ring, t_gb_data, bytes, num_frames))
AudioRingHeader* ring = g_rings[i].load(std::memory_order_acquire);
if (ring != nullptr && ring->capture_enabled.load(std::memory_order_relaxed) != 0 &&
t_gb_client == self && t_gb_data != nullptr && t_gb_frames == num_frames)
{
std::memset(t_gb_data, 0, bytes); // belt-and-suspenders vs a driver ignoring SILENT
g_frames_captured.fetch_add(num_frames, std::memory_order_relaxed);
return g_vh_releasebuffer.original<ReleaseBufferFn>()(
self, num_frames, flags | AUDCLNT_BUFFERFLAGS_SILENT);
const std::uint32_t block = g_streams[i].block_align.load(std::memory_order_relaxed);
const std::uint32_t bytes = num_frames * block;
// Only silence if the frames made it into the ring; if the host has
// stalled (ring full) keep playing locally rather than going dead
// silent — degrades to today's echo, never to silence.
if (block != 0 && audio_ring_push(*ring, t_gb_data, bytes, num_frames))
{
std::memset(t_gb_data, 0, bytes); // belt-and-suspenders vs a driver ignoring SILENT
g_frames_captured.fetch_add(num_frames, std::memory_order_relaxed);
return g_vh_releasebuffer.original<ReleaseBufferFn>()(
self, num_frames, flags | AUDCLNT_BUFFERFLAGS_SILENT);
}
}
}
break;
}
return g_vh_releasebuffer.original<ReleaseBufferFn>()(self, num_frames, flags);
}
@@ -303,7 +302,9 @@ void register_render_client_locked(IAudioRenderClient* rc, const CapturedFormat&
}
g_registered = slot + 1;
g_stream_formats[slot] = cf;
g_streams[slot].frames.store(0, std::memory_order_relaxed);
g_streams[slot].block_align.store(cf.block_align, std::memory_order_relaxed); // before client (hot path)
g_streams[slot].client.store(rc, std::memory_order_release);
AudioStreamInfo info{};
@@ -318,18 +319,13 @@ void register_render_client_locked(IAudioRenderClient* rc, const CapturedFormat&
g_ipc->publish_audio_stream(slot, info);
}
if (slot == 0)
// Publish this stream's format to its own ring if the host has attached one yet.
AudioRingHeader* ring = g_rings[slot].load(std::memory_order_acquire);
logf("stream %u set: rc=%p ring=%p fmt=%uHz/%uch/%ubit (%s)", slot, rc, ring, cf.rate, cf.channels, cf.bits,
ring ? "published" : "no ring yet");
if (ring != nullptr)
{
g_primary_format = cf;
g_primary_block_align.store(cf.block_align, std::memory_order_relaxed);
g_primary.store(rc, std::memory_order_release);
AudioRingHeader* ring = g_ring.load(std::memory_order_acquire);
logf("primary stream set: rc=%p ring=%p (format %s)", rc, ring,
ring ? "published" : "no ring yet");
if (ring)
{
audio_ring_set_format(*ring, cf.rate, cf.channels, cf.bits, cf.tag, cf.block_align);
}
audio_ring_set_format(*ring, cf.rate, cf.channels, cf.bits, cf.tag, cf.block_align);
}
// GetBuffer/ReleaseBuffer are hooked proactively at install time (the shared
// vtable covers every render client), so nothing to install per-stream here.
@@ -464,7 +460,7 @@ bool install_audio_hooks(IpcClient& ipc, AudioRingHeader* ring)
{
std::scoped_lock lock(g_setup_mutex);
g_ipc = &ipc;
g_ring.store(ring, std::memory_order_release);
g_rings[0].store(ring, std::memory_order_release);
if (g_vh_activate)
{
return true; // anchor already installed
@@ -568,33 +564,48 @@ bool install_audio_hooks(IpcClient& ipc, AudioRingHeader* ring)
void republish_audio_format()
{
AudioRingHeader* ring = g_ring.load(std::memory_order_acquire);
if (ring == nullptr || audio_ring_format_ready(*ring))
// Nothing to do if no ring needs a format yet (cheap pre-check, no lock).
bool any_pending = false;
for (std::uint32_t i = 0; i < kMaxAudioStreams; ++i)
{
return; // no ring yet, or the format is already published
AudioRingHeader* ring = g_rings[i].load(std::memory_order_acquire);
if (ring != nullptr && !audio_ring_format_ready(*ring))
{
any_pending = true;
break;
}
}
if (!any_pending)
{
return;
}
std::scoped_lock lock(g_setup_mutex);
if (audio_ring_format_ready(*ring))
for (std::uint32_t i = 0; i < kMaxAudioStreams; ++i)
{
return; // raced with another publisher
}
if (g_primary.load(std::memory_order_acquire) != nullptr && g_primary_format.rate != 0)
{
audio_ring_set_format(*ring, g_primary_format.rate, g_primary_format.channels, g_primary_format.bits,
g_primary_format.tag, g_primary_format.block_align);
logf("republish_audio_format: published %uHz/%uch/%ubit to ring %p", g_primary_format.rate,
g_primary_format.channels, g_primary_format.bits, ring);
AudioRingHeader* ring = g_rings[i].load(std::memory_order_acquire);
if (ring == nullptr || audio_ring_format_ready(*ring) || g_stream_formats[i].rate == 0)
{
continue; // no ring, already published, or this slot has no stream yet
}
const CapturedFormat& cf = g_stream_formats[i];
audio_ring_set_format(*ring, cf.rate, cf.channels, cf.bits, cf.tag, cf.block_align);
logf("republish_audio_format: stream %u -> %uHz/%uch/%ubit ring %p", i, cf.rate, cf.channels, cf.bits,
ring);
}
}
void set_audio_ring(AudioRingHeader* ring)
void set_audio_ring(unsigned index, AudioRingHeader* ring)
{
g_ring.store(ring, std::memory_order_release);
logf("set_audio_ring: ring=%p capture_enabled=%u", ring,
if (index >= kMaxAudioStreams)
{
return;
}
g_rings[index].store(ring, std::memory_order_release);
logf("set_audio_ring: index=%u ring=%p capture_enabled=%u", index, ring,
ring ? ring->capture_enabled.load(std::memory_order_relaxed) : 0u);
// The primary may already be registered (game was playing before we injected
// and before the host created the ring); publish its format so the host stops
// waiting and consumes the ring instead of falling back to loopback.
// The stream may already be registered (game was playing before we injected and
// before the host created the ring); publish its format so the host stops waiting
// and consumes the ring instead of falling back to loopback.
republish_audio_format();
}
@@ -624,20 +635,19 @@ void remove_audio_hooks()
g_self_client = nullptr;
}
g_have_mix_format.store(0, std::memory_order_relaxed);
g_primary_format = CapturedFormat{};
g_registered = 0;
g_streams_seen.store(0, std::memory_order_relaxed);
g_primary.store(nullptr, std::memory_order_release);
g_primary_block_align.store(0, std::memory_order_relaxed);
g_frames_captured.store(0, std::memory_order_relaxed);
for (auto& s : g_streams)
for (std::uint32_t i = 0; i < kMaxAudioStreams; ++i)
{
s.client.store(nullptr, std::memory_order_relaxed);
s.frames.store(0, std::memory_order_relaxed);
g_streams[i].client.store(nullptr, std::memory_order_relaxed);
g_streams[i].frames.store(0, std::memory_order_relaxed);
g_streams[i].block_align.store(0, std::memory_order_relaxed);
g_stream_formats[i] = CapturedFormat{};
g_rings[i].store(nullptr, std::memory_order_release);
}
g_client_formats.clear();
g_ring.store(nullptr, std::memory_order_release);
g_ipc = nullptr;
}