BlackMark/CoopAllTheThings
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Capture every audio stream into its own ring and mix them on the host

Browse Source 
Games with several concurrent WASAPI render streams (e.g. Spider-Man: Miles
Morales) only had their first ("primary") stream mirrored; the rest kept playing
locally and never reached the guest. Now the render-hook captures + silences EVERY
tracked stream into its own ring (coop_audio_<pid>[_<index>]), each published with
that stream's own detected format (Initialize when caught, else GetMixFormat -- the
per-stream format detection, now actually used per ring rather than only for the
primary). The host creates a ring per stream and mixes the same-format streams with
a soft clip (host/src/audio/audio_mix.hpp); streams whose format differs from the
primary are still silenced (no echo) but skipped from the mix (would need
resampling).

The single-stream case is byte-for-byte unchanged: when only one stream is active
the host passes it through without the mixer, so the common path has no overhead or
fidelity change.

Verified: new audio_mix_test covers the decode/sum/soft-clip/encode math (float32 +
int16); audio_hook_test (x64 + x86) still passes, guarding the primary
capture+silence path against regression; full build x64 + x86 clean; ctest x64
11/11, x86 3/3. Multi-stream mixing against a real multi-stream game needs a live
session to fully confirm.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
BlackMark
2026-06-21 05:45:47 +02:00
parent dfd2be8c17
commit 784c31a9b5
10 changed files with 412 additions and 147 deletions
Download Patch File Download Diff File Expand all files Collapse all files

164
hook/src/audio_hook.cpp
View File

@@ -113,7 +113,9 @@ struct CapturedFormat
// --- Global hook state -----------------------------------------------------
IpcClient* g_ipc = nullptr;
std::atomic<AudioRingHeader*> g_ring{nullptr};
// One ring per tracked stream (index = the stream's debug slot). Stream 0 is the
// primary; the host creates a ring per stream and mixes them.
std::atomic<AudioRingHeader*> g_rings[kMaxAudioStreams]{};
std::mutex g_setup_mutex; // guards installs + the format map + stream registration
@@ -146,32 +148,28 @@ std::atomic<IAudioRenderClient*> g_self_render{nullptr};
CapturedFormat g_mix_format;
std::atomic<std::uint32_t> g_have_mix_format{0};
// The primary stream's actual format, captured when it's registered. The host
// creates the ring only when audio mirroring is toggled on — typically *after*
// the primary stream was already registered — so the format must be (re)published
// to the ring whenever it attaches. (The in-process probe creates the ring before
// injecting, so it never exercises this ordering; the full app always does.)
// Guarded by g_setup_mutex.
CapturedFormat g_primary_format;
// Each tracked stream's actual format, captured when it's registered. The host
// creates the rings only when audio mirroring is toggled on — typically *after* a
// stream was already registered — so the format must be (re)published to a ring
// whenever it attaches. Guarded by g_setup_mutex.
CapturedFormat g_stream_formats[kMaxAudioStreams];
// Per IAudioClient, the format captured at Initialize, looked up when its render
// client is created. Setup-path only (never touched on the audio thread).
std::unordered_map<IAudioClient*, CapturedFormat> g_client_formats;
// Streams we track for the debug view (frame counting). Index 0 is primary.
// Streams we track (frame counting + per-stream capture). Index 0 is primary.
struct TrackedStream
{
std::atomic<IAudioRenderClient*> client{nullptr};
std::atomic<std::uint64_t> frames{0};
std::atomic<std::uint32_t> block_align{0}; // hot-path frame size for this stream
};
TrackedStream g_streams[kMaxAudioStreams];
std::uint32_t g_registered = 0; // slots filled (<= kMaxAudioStreams), under mutex
std::atomic<std::uint32_t> g_streams_seen{0}; // total distinct clients ever seen
// Hot-path primary identity + frame size (avoids touching the map/mutex).
std::atomic<IAudioRenderClient*> g_primary{nullptr};
std::atomic<std::uint32_t> g_primary_block_align{0};
std::atomic<std::uint64_t> g_frames_captured{0};
std::atomic<std::uint64_t> g_frames_captured{0}; // total frames captured across streams
// GetBuffer/ReleaseBuffer are paired on one thread, never nested: stash the
// pointer the game just got so ReleaseBuffer can copy it before releasing.
@@ -235,42 +233,43 @@ HRESULT STDMETHODCALLTYPE hk_ReleaseBuffer(IAudioRenderClient* self, UINT32 num_
try_register_lazy(self);
}
// Frame counting for any tracked stream (drives the live/idle debug view).
// Per tracked stream: count frames (debug view) and, into the stream's own ring,
// capture + silence its buffer while capture is enabled. Every stream is captured
// into its own ring; the host mixes them.
for (std::uint32_t i = 0; i < kMaxAudioStreams; ++i)
{
if (g_streams[i].client.load(std::memory_order_acquire) == self)
if (g_streams[i].client.load(std::memory_order_acquire) != self)
{
const std::uint64_t total =
g_streams[i].frames.fetch_add(num_frames, std::memory_order_relaxed) + num_frames;
if (g_ipc != nullptr)
{
g_ipc->note_audio_frames(i, total);
}
break;
continue;
}
const std::uint64_t total =
g_streams[i].frames.fetch_add(num_frames, std::memory_order_relaxed) + num_frames;
if (g_ipc != nullptr)
{
g_ipc->note_audio_frames(i, total);
}
}
// Capture + silence only the primary stream, only while enabled.
if (self == g_primary.load(std::memory_order_acquire) && num_frames > 0 &&
(flags & AUDCLNT_BUFFERFLAGS_SILENT) == 0)
{
AudioRingHeader* ring = g_ring.load(std::memory_order_acquire);
if (ring != nullptr && ring->capture_enabled.load(std::memory_order_relaxed) != 0 &&
t_gb_client == self && t_gb_data != nullptr && t_gb_frames == num_frames)
if (num_frames > 0 && (flags & AUDCLNT_BUFFERFLAGS_SILENT) == 0)
{
const std::uint32_t block = g_primary_block_align.load(std::memory_order_relaxed);
const std::uint32_t bytes = num_frames * block;
// Only silence if the frames made it into the ring; if the host has
// stalled (ring full) keep playing locally rather than going dead
// silent — degrades to today's echo, never to silence.
if (audio_ring_push(*ring, t_gb_data, bytes, num_frames))
AudioRingHeader* ring = g_rings[i].load(std::memory_order_acquire);
if (ring != nullptr && ring->capture_enabled.load(std::memory_order_relaxed) != 0 &&
t_gb_client == self && t_gb_data != nullptr && t_gb_frames == num_frames)
{
std::memset(t_gb_data, 0, bytes); // belt-and-suspenders vs a driver ignoring SILENT
g_frames_captured.fetch_add(num_frames, std::memory_order_relaxed);
return g_vh_releasebuffer.original<ReleaseBufferFn>()(
self, num_frames, flags | AUDCLNT_BUFFERFLAGS_SILENT);
const std::uint32_t block = g_streams[i].block_align.load(std::memory_order_relaxed);
const std::uint32_t bytes = num_frames * block;
// Only silence if the frames made it into the ring; if the host has
// stalled (ring full) keep playing locally rather than going dead
// silent — degrades to today's echo, never to silence.
if (block != 0 && audio_ring_push(*ring, t_gb_data, bytes, num_frames))
{
std::memset(t_gb_data, 0, bytes); // belt-and-suspenders vs a driver ignoring SILENT
g_frames_captured.fetch_add(num_frames, std::memory_order_relaxed);
return g_vh_releasebuffer.original<ReleaseBufferFn>()(
self, num_frames, flags | AUDCLNT_BUFFERFLAGS_SILENT);
}
}
}
break;
}
return g_vh_releasebuffer.original<ReleaseBufferFn>()(self, num_frames, flags);
}
@@ -303,7 +302,9 @@ void register_render_client_locked(IAudioRenderClient* rc, const CapturedFormat&
}
g_registered = slot + 1;
g_stream_formats[slot] = cf;
g_streams[slot].frames.store(0, std::memory_order_relaxed);
g_streams[slot].block_align.store(cf.block_align, std::memory_order_relaxed); // before client (hot path)
g_streams[slot].client.store(rc, std::memory_order_release);
AudioStreamInfo info{};
@@ -318,18 +319,13 @@ void register_render_client_locked(IAudioRenderClient* rc, const CapturedFormat&
g_ipc->publish_audio_stream(slot, info);
}
if (slot == 0)
// Publish this stream's format to its own ring if the host has attached one yet.
AudioRingHeader* ring = g_rings[slot].load(std::memory_order_acquire);
logf("stream %u set: rc=%p ring=%p fmt=%uHz/%uch/%ubit (%s)", slot, rc, ring, cf.rate, cf.channels, cf.bits,
ring ? "published" : "no ring yet");
if (ring != nullptr)
{
g_primary_format = cf;
g_primary_block_align.store(cf.block_align, std::memory_order_relaxed);
g_primary.store(rc, std::memory_order_release);
AudioRingHeader* ring = g_ring.load(std::memory_order_acquire);
logf("primary stream set: rc=%p ring=%p (format %s)", rc, ring,
ring ? "published" : "no ring yet");
if (ring)
{
audio_ring_set_format(*ring, cf.rate, cf.channels, cf.bits, cf.tag, cf.block_align);
}
audio_ring_set_format(*ring, cf.rate, cf.channels, cf.bits, cf.tag, cf.block_align);
}
// GetBuffer/ReleaseBuffer are hooked proactively at install time (the shared
// vtable covers every render client), so nothing to install per-stream here.
@@ -464,7 +460,7 @@ bool install_audio_hooks(IpcClient& ipc, AudioRingHeader* ring)
{
std::scoped_lock lock(g_setup_mutex);
g_ipc = &ipc;
g_ring.store(ring, std::memory_order_release);
g_rings[0].store(ring, std::memory_order_release);
if (g_vh_activate)
{
return true; // anchor already installed
@@ -568,33 +564,48 @@ bool install_audio_hooks(IpcClient& ipc, AudioRingHeader* ring)
void republish_audio_format()
{
AudioRingHeader* ring = g_ring.load(std::memory_order_acquire);
if (ring == nullptr || audio_ring_format_ready(*ring))
// Nothing to do if no ring needs a format yet (cheap pre-check, no lock).
bool any_pending = false;
for (std::uint32_t i = 0; i < kMaxAudioStreams; ++i)
{
return; // no ring yet, or the format is already published
AudioRingHeader* ring = g_rings[i].load(std::memory_order_acquire);
if (ring != nullptr && !audio_ring_format_ready(*ring))
{
any_pending = true;
break;
}
}
if (!any_pending)
{
return;
}
std::scoped_lock lock(g_setup_mutex);
if (audio_ring_format_ready(*ring))
for (std::uint32_t i = 0; i < kMaxAudioStreams; ++i)
{
return; // raced with another publisher
}
if (g_primary.load(std::memory_order_acquire) != nullptr && g_primary_format.rate != 0)
{
audio_ring_set_format(*ring, g_primary_format.rate, g_primary_format.channels, g_primary_format.bits,
g_primary_format.tag, g_primary_format.block_align);
logf("republish_audio_format: published %uHz/%uch/%ubit to ring %p", g_primary_format.rate,
g_primary_format.channels, g_primary_format.bits, ring);
AudioRingHeader* ring = g_rings[i].load(std::memory_order_acquire);
if (ring == nullptr || audio_ring_format_ready(*ring) || g_stream_formats[i].rate == 0)
{
continue; // no ring, already published, or this slot has no stream yet
}
const CapturedFormat& cf = g_stream_formats[i];
audio_ring_set_format(*ring, cf.rate, cf.channels, cf.bits, cf.tag, cf.block_align);
logf("republish_audio_format: stream %u -> %uHz/%uch/%ubit ring %p", i, cf.rate, cf.channels, cf.bits,
ring);
}
}
void set_audio_ring(AudioRingHeader* ring)
void set_audio_ring(unsigned index, AudioRingHeader* ring)
{
g_ring.store(ring, std::memory_order_release);
logf("set_audio_ring: ring=%p capture_enabled=%u", ring,
if (index >= kMaxAudioStreams)
{
return;
}
g_rings[index].store(ring, std::memory_order_release);
logf("set_audio_ring: index=%u ring=%p capture_enabled=%u", index, ring,
ring ? ring->capture_enabled.load(std::memory_order_relaxed) : 0u);
// The primary may already be registered (game was playing before we injected
// and before the host created the ring); publish its format so the host stops
// waiting and consumes the ring instead of falling back to loopback.
// The stream may already be registered (game was playing before we injected and
// before the host created the ring); publish its format so the host stops waiting
// and consumes the ring instead of falling back to loopback.
republish_audio_format();
}
@@ -624,20 +635,19 @@ void remove_audio_hooks()
g_self_client = nullptr;
}
g_have_mix_format.store(0, std::memory_order_relaxed);
g_primary_format = CapturedFormat{};
g_registered = 0;
g_streams_seen.store(0, std::memory_order_relaxed);
g_primary.store(nullptr, std::memory_order_release);
g_primary_block_align.store(0, std::memory_order_relaxed);
g_frames_captured.store(0, std::memory_order_relaxed);
for (auto& s : g_streams)
for (std::uint32_t i = 0; i < kMaxAudioStreams; ++i)
{
s.client.store(nullptr, std::memory_order_relaxed);
s.frames.store(0, std::memory_order_relaxed);
g_streams[i].client.store(nullptr, std::memory_order_relaxed);
g_streams[i].frames.store(0, std::memory_order_relaxed);
g_streams[i].block_align.store(0, std::memory_order_relaxed);
g_stream_formats[i] = CapturedFormat{};
g_rings[i].store(nullptr, std::memory_order_release);
}
g_client_formats.clear();
g_ring.store(nullptr, std::memory_order_release);
g_ipc = nullptr;
}

14
hook/src/audio_hook.hpp
View File

@@ -23,13 +23,15 @@ namespace coop::hook
// anchor hook is in place; safe to call repeatedly (install-once internally).
bool install_audio_hooks(IpcClient& ipc, AudioRingHeader* ring);
// Attach/replace the producer ring after install (e.g. host created it late).
void set_audio_ring(AudioRingHeader* ring);
// Attach/replace the producer ring for stream `index` after install (e.g. host
// created it late). Index 0 is the primary stream; the host creates one ring per
// tracked stream (coop_audio_<pid>[_<index>]) and mixes them.
void set_audio_ring(unsigned index, AudioRingHeader* ring);
// Publish the registered primary stream's format to the attached ring if it
// isn't published yet. Idempotent; call periodically so a ring the host attaches
// (or re-initializes on a mirror re-toggle) gets the format even though the
// stream was registered earlier. No-op if there's no ring / no primary yet.
// Publish each registered stream's format to its attached ring if not published
// yet. Idempotent; call periodically so a ring the host attaches (or re-initializes
// on a mirror re-toggle) gets the format even though the stream was registered
// earlier. No-op for rings that have no stream yet.
void republish_audio_format();
// Removes all installed render hooks (best effort; used on DLL detach).

47
hook/src/dllmain.cpp
View File

@@ -30,8 +30,8 @@ namespace
coop::hook::IpcClient g_ipc;
std::atomic<bool> g_running{true};
coop::SharedMemory g_audio_shm; // the host's audio ring, opened when present
coop::SharedMemory g_log_shm; // the host's log ring, opened when present
coop::SharedMemory g_audio_shm[coop::kMaxAudioStreams]; // per-stream audio rings, opened when present
coop::SharedMemory g_log_shm; // the host's log ring, opened when present
DWORD WINAPI worker_thread(LPVOID)
{
@@ -168,31 +168,40 @@ DWORD WINAPI worker_thread(LPVOID)
coop::hook::logf("worker_thread: MKB hooks removed (host request)");
}
if (audio_installed && !audio_ring_open)
// Attach a ring per stream. The host creates up to kMaxAudioStreams rings
// (coop_audio_<pid>[_<index>]); we open each as it appears and (re)attach it so
// every stream is captured + silenced into its own ring for the host to mix.
if (audio_installed)
{
const std::wstring name = coop::audio_ring_name(GetCurrentProcessId());
if (!g_audio_shm.valid())
for (unsigned i = 0; i < coop::kMaxAudioStreams; ++i)
{
g_audio_shm.open(name, coop::audio_ring_total_size(coop::kAudioRingCapacity));
}
if (g_audio_shm.valid())
{
auto* ring = g_audio_shm.as<coop::AudioRingHeader>();
if (coop::audio_ring_valid(*ring))
if (!g_audio_shm[i].valid())
{
coop::hook::set_audio_ring(ring);
audio_ring_open = true;
coop::hook::logf("worker_thread: audio ring opened");
g_audio_shm[i].open(coop::audio_ring_name(GetCurrentProcessId(), i),
coop::audio_ring_total_size(coop::kAudioRingCapacity));
}
else
if (g_audio_shm[i].valid())
{
g_audio_shm.reset(); // present but not our contract; retry
auto* ring = g_audio_shm[i].as<coop::AudioRingHeader>();
if (coop::audio_ring_valid(*ring))
{
coop::hook::set_audio_ring(i, ring); // idempotent re-attach
if (i == 0 && !audio_ring_open)
{
audio_ring_open = true;
coop::hook::logf("worker_thread: audio ring 0 opened");
}
}
else
{
g_audio_shm[i].reset(); // present but not our contract; retry
}
}
}
}
// The primary stream is often registered before the ring is attached (or the
// host re-inits the ring on a mirror re-toggle, clearing its format); keep
// the format published so the host consumes the ring instead of falling back.
// A stream is often registered before its ring is attached (or the host re-inits
// a ring on a mirror re-toggle, clearing its format); keep formats published so
// the host consumes the rings instead of falling back to loopback.
if (audio_ring_open)
{
coop::hook::republish_audio_format();