Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion .claude/skills/audio-nodes/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -351,7 +351,14 @@ Callback IDs are stored as `std::atomic<uint64_t>` on the node. `0` means no lis
All graph mutations are queued via `AudioGraphManager` using its own SPSC channel (`addPendingNodeConnection`, `addPendingParamConnection`). The audio thread calls `graphManager_->preProcessGraph()` before each render pass to apply pending changes.

### Settable channel attributes (channelCount / channelCountMode / channelInterpretation)
These are mutable after construction. `AudioNode` (core) exposes virtual `setChannelCount` / `setChannelCountMode` / `setChannelInterpretation`. `channelCount` and `channelCountMode` are read only on the host thread during negotiation, so the JSI setter updates the core field directly then calls `HostNode::renegotiate()` → `Graph::renegotiateNodeChannels()` → `HostGraph::renegotiateNodeChannels()` (reuses `collectNegotiations` + an `AGEvent` buffer swap, self-drain aware when there is no audio/render consumer — offline construction/suspend and realtime suspended/stopped windows). When `AudioBufferSourceNode` `setBuffer` changes channel width, update `channelCount_` on the host thread then `renegotiate()` so MAX/CLAMPED_MAX downstream nodes update; the audio event still installs the prebuilt buffer (no audio-thread alloc). `channelInterpretation` is read on the audio thread in `processInputs` (`getInputBuffer()->sum(*input, channelInterpretation_)`), so it MUST be applied via `scheduleAudioEvent`, not mutated directly.
These are mutable after construction. `AudioNode` (core) exposes virtual `setChannelCount` / `setChannelCountMode` / `setChannelInterpretation`. `channelCount` and `channelCountMode` are read only on the host thread during negotiation, so the JSI setter updates the core field directly then calls `HostNode::renegotiate()` → `Graph::renegotiateNodeChannels()` → `HostGraph::renegotiateNodeChannels()` (reuses `collectNegotiations` + an `AGEvent` buffer swap, self-drain aware when there is no audio/render consumer — offline construction/suspend and realtime suspended/stopped windows). When `AudioBufferSourceNode` `setBuffer` changes channel width, update `outputChannelNumber_` together with the buffer then `renegotiate()` so MAX/CLAMPED_MAX downstream nodes update; the audio event still installs the prebuilt buffer (no audio-thread alloc). `channelInterpretation` is read on the audio thread in `processInputs` (`getInputBuffer()->sum(*input, channelInterpretation_)`), so it MUST be applied via `scheduleAudioEvent`, not mutated directly.

### Output channel number vs `channelCount` (sources)
`channelCount` is the spec's input-mixing attribute and never decides how many channels a node emits. `AudioNode` keeps a separate atomic `outputChannelNumber_` (`getOutputChannelNumber()`), initialised from `AudioNodeOptions::outputChannelNumber` and falling back to `channelCount` for nodes whose output follows their negotiated input layout. `HostGraph` negotiation reads the output channel number for source inputs (`getUpstreamChannelCount` returns it when `numberOfInputs_ == 0`), so a spec-default `channelCount = 2` oscillator still negotiates a mono downstream buffer. Rules:
- A node that always emits one channel (Oscillator, ConstantSource) derives its options from `MonoSourceNodeOptions`; do not lower `channelCount` or override `setChannelCount` to fake it.
- A source that learns its width later (AudioBufferSource, AudioFileSource, AudioBufferQueueSource, RecorderAdapter) writes `outputChannelNumber_` in the same step it swaps `audioBuffer_`, never `channelCount_`.
- When the host object schedules that swap (AudioBufferSourceNode `setBuffer`), it must first call `setOutputChannelNumber()` on the host thread and `renegotiate()`, otherwise downstream MAX / CLAMPED_MAX nodes negotiate against the stale width until the audio event lands (this broke the StereoPanner WPT `stereopannernode-panning` test when it lived on `updateChannelCount`). Never route this through `updateChannelCount`: that changes the JS-visible `channelCount` attribute.
- Read the emitted width from `getOutputChannelNumber()` on any thread; the buffer pointer itself is audio-thread only.

### Idle-node stale-buffer zeroing (settleProcessableState)
`AudioGraph::iter()` filters to `isProcessable()` nodes, so a node that has gone idle (e.g. a finished source) is skipped and its output buffer is NOT refreshed — it keeps the samples from an earlier quantum. Downstream consumers still read that buffer via `getOutput()` when collecting inputs, which would re-sum ghost echoes every quantum (this broke the `audionode-channel-rules` ~170-node WPT test).
Expand Down
4 changes: 4 additions & 0 deletions packages/audiodocs/docs/effects/convolver-node.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,10 @@ Inherits all properties from [`AudioNode`](../core/audio-node.mdx#properties).
| `buffer` | [`AudioBuffer`](../sources/audio-buffer.mdx) | Associated AudioBuffer. Setting it throws `NotSupportedError` unless the buffer has 1, 2 or 4 channels and the same sample rate as the context. |
| `normalize` | `boolean` | Whether the impulse response from the buffer will be scaled by an equal-power normalization when the buffer attribute is set. |

:::info Channel constraints
The input is mixed down to mono or stereo before convolution. `channelCount` cannot be set above `2` and `channelCountMode` cannot be set to `'max'`; either attempt, in the constructor options or via the setter, throws `NotSupportedError`.
:::

:::caution
Linear convolution is a heavy computational process, so if your audio has some weird artefacts that should not be there, try to decrease the duration of impulse response buffer.
:::
Expand Down
2 changes: 1 addition & 1 deletion packages/audiodocs/docs/effects/delay-node.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Inherits all properties from [`AudioNodeOptions`](../core/audio-node.mdx#audiono

| Parameter | Type | Default | |
| :---: | :---: | :----: | :---- |
| `maxDelayTime` <Optional /> | `number` | `1.0` | Maximum amount of time, in seconds, to buffer delayed values. |
| `maxDelayTime` <Optional /> | `number` | `1.0` | Maximum amount of time, in seconds, to buffer delayed values. Must be greater than `0` and less than `180`, otherwise the constructor throws `NotSupportedError`. |
| `delayTime` <Optional /> | `number` | `0.0` | Initial value for [`delayTime`](./delay-node.mdx#properties). |

You can also create a `DelayNode` via the [`BaseAudioContext.createDelay(maxDelayTime?: number)`](../core/base-audio-context.mdx#createdelay) factory method, which uses default values when called without arguments.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,6 @@ DelayNodeHostObject::DelayNodeHostObject(
delayTimeParam_ =
std::make_shared<AudioParamHostObject>(graph_, node_, delayNode_->getDelayTimeParam());

auto delayBuffer = std::make_shared<AudioBuffer>(
static_cast<size_t>(options.maxDelayTime * context->getSampleRate() + 1),
channelCount_,
context->getSampleRate());

// order has to be preserved because adding cycle would not change their order in the graph
delayReaderHostNode_ =
std::make_shared<DelayReaderHostNode>(graph_, std::move(delayNode_->delayReader_));
Expand All @@ -47,24 +42,22 @@ DelayNodeHostObject::DelayNodeHostObject(
addGetters(JSI_EXPORT_PROPERTY_GETTER(DelayNodeHostObject, delayTime));
}

std::shared_ptr<utils::graph::HostNode> DelayNodeHostObject::getInput(int /*outputIndex*/) {
return delayReaderHostNode_;
std::shared_ptr<utils::graph::HostNode> DelayNodeHostObject::getInput(int /*inputIndex*/) {
return delayWriterHostNode_;
}

std::shared_ptr<utils::graph::HostNode> DelayNodeHostObject::getOutput(int /*inputIndex*/) {
return delayWriterHostNode_;
std::shared_ptr<utils::graph::HostNode> DelayNodeHostObject::getOutput(int /*outputIndex*/) {
return delayReaderHostNode_;
}

JSI_PROPERTY_GETTER_IMPL(DelayNodeHostObject, delayTime) {
return jsi::Object::createFromHostObject(runtime, delayTimeParam_);
}

size_t DelayNodeHostObject::getMemoryPressure() const {
const float maxDelaySeconds = delayNode_->getDelayTimeParam()->getMaxValue();
const float sampleRate = delayNode_->getContextSampleRate();
// The delay line ring buffer dominates: (maxDelay * sr + 1) frames * channels * float.
// The delay line ring buffer dominates: ring frames * channels * float.
const size_t ringBytes =
static_cast<size_t>(maxDelaySeconds * sampleRate + 1) * channelCount_ * sizeof(float);
delayNode_->delayLine_->getBuffer()->getSize() * channelCount_ * sizeof(float);
// Base `audioBuffer_` from AudioNodeHostObject::getMemoryPressure(), plus
// the reader/writer AudioNode sub-nodes (each owns its own RQ audioBuffer_)
// and the delayTime AudioParam.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -82,16 +82,15 @@ JSI_HOST_FUNCTION_IMPL(AudioBufferQueueSourceNodeHostObject, enqueueBuffer) {
// buffer modification is not allowed on JS thread

auto swapBuffer = false; // whether to swap internal node buffer with the new buffer
if (!channelCountSet_) {
channelCount_ = static_cast<int>(audioBufferHostObject->audioBuffer_->getNumberOfChannels());
channelCountSet_ = true;
if (outputChannelNumber_ == 0) {
outputChannelNumber_ = audioBufferHostObject->audioBuffer_->getNumberOfChannels();
swapBuffer = true;
}

// first buffer defines channel count, rest of them is mixed to channel count of the first buffer
// first buffer defines the output channel number, the rest are mixed to it
auto copiedBuffer = std::make_shared<AudioBuffer>(
audioBufferHostObject->audioBuffer_->getSize(),
channelCount_,
outputChannelNumber_,
audioBufferHostObject->audioBuffer_->getSampleRate());

copiedBuffer->sum(*audioBufferHostObject->audioBuffer_);
Expand All @@ -114,9 +113,9 @@ JSI_HOST_FUNCTION_IMPL(AudioBufferQueueSourceNodeHostObject, enqueueBuffer) {
bufferId = bufferId_,
tailBuffer,
swapBuffer,
channelCount = channelCount_](BaseAudioContext &) {
outputChannelNumber = outputChannelNumber_](BaseAudioContext &) {
if (swapBuffer) {
node->setChannelCount(static_cast<int>(channelCount));
node->resizeOutputBuffer(static_cast<int>(outputChannelNumber));
}
node->enqueueBuffer(copiedBuffer, bufferId, tailBuffer);
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,8 @@ class AudioBufferQueueSourceNodeHostObject : public AudioBufferBaseSourceNodeHos

size_t bufferId_ = 0;
bool stretchHasBeenInit_ = false;
bool channelCountSet_ = false;
/// Width of the first enqueued buffer; later buffers are mixed to it. 0 until then.
size_t outputChannelNumber_ = 0;
};

} // namespace audioapi
Original file line number Diff line number Diff line change
Expand Up @@ -161,14 +161,15 @@ void AudioBufferSourceNodeHostObject::setBuffer(const std::shared_ptr<AudioBuffe

std::shared_ptr<AudioBuffer> copiedBuffer;
std::shared_ptr<DSPAudioBuffer> audioBuffer;
const size_t newChannelCount = buffer == nullptr ? AudioBufferSourceOptions::kDefaultChannelCount
: buffer->getNumberOfChannels();
const size_t newOutputChannelNumber = buffer == nullptr
? AudioBufferSourceOptions::kDefaultOutputChannelNumber
: buffer->getNumberOfChannels();

if (buffer == nullptr) {
copiedBuffer = nullptr;
audioBuffer = std::make_shared<DSPAudioBuffer>(
RENDER_QUANTUM_SIZE,
AudioBufferSourceOptions::kDefaultChannelCount,
AudioBufferSourceOptions::kDefaultOutputChannelNumber,
audioBufferSourceNode_->getContextSampleRate());
} else {
if (pitchCorrection_) {
Expand All @@ -190,9 +191,12 @@ void AudioBufferSourceNodeHostObject::setBuffer(const std::shared_ptr<AudioBuffe
audioBufferSourceNode_->getContextSampleRate());
}

// Update channelCount on the host thread before renegotiation so MAX /
// CLAMPED_MAX downstream nodes see the new width immediately.
updateChannelCount(newChannelCount);
// Publish the new output width on the host thread before renegotiation so
// MAX / CLAMPED_MAX downstream nodes see it immediately
if (newOutputChannelNumber != audioBufferSourceNode_->getOutputChannelNumber()) {
audioBufferSourceNode_->setOutputChannelNumber(newOutputChannelNumber);
renegotiate();
}

auto event =
[handle, node = audioBufferSourceNode_, copiedBuffer, audioBuffer](BaseAudioContext &) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ class MediaElementAudioSourceNodeHostObject : public AudioNodeHostObject {
std::make_unique<MediaElementAudioSourceNode>(
context,
fileSource,
MediaElementAudioSourceOptions(static_cast<int>(fileSource->getChannelCount())))) {}
MediaElementAudioSourceOptions(
static_cast<int>(fileSource->getOutputChannelNumber().value())))) {}
};

} // namespace audioapi
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,12 @@ AudioNode::AudioNode(
numberOfInputs_(options.numberOfInputs),
numberOfOutputs_(options.numberOfOutputs),
channelCount_(options.channelCount),
outputChannelNumber_(options.outputChannelNumber.value_or(options.channelCount)),
channelCountMode_(options.channelCountMode),
channelInterpretation_(options.channelInterpretation),
requiresTailProcessing_(options.requiresTailProcessing) {
audioBuffer_ = std::make_shared<DSPAudioBuffer>(
RENDER_QUANTUM_SIZE, channelCount_, context->getSampleRate());
RENDER_QUANTUM_SIZE, outputChannelNumber_.load(), context->getSampleRate());
}

bool AudioNode::canBeDestructed() const {
Expand All @@ -44,7 +45,7 @@ bool AudioNode::isProcessable() const {
}

size_t AudioNode::getChannelCount() const {
return channelCount_.load(std::memory_order_acquire);
return static_cast<size_t>(channelCount_);
}

bool AudioNode::requiresTailProcessing() const {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
#include <cstddef>
#include <cstdint>
#include <memory>
#include <optional>
#include <utility>
#include <vector>

Expand All @@ -29,14 +30,18 @@ class AudioNode : public utils::graph::GraphObject, public std::enable_shared_fr
~AudioNode() override = default;
DELETE_COPY_AND_MOVE(AudioNode);

/// @brief Returns this node's `channelCount` attribute.
/// @note Safe to call from any thread: `channelCount_` is atomic because
/// source subclasses update it on the audio thread while the JS thread reads
/// it during channel-count negotiation.
/// @brief Returns this node's `channelCount` attribute: the width inputs are
/// mixed to.
/// @note Read only on the host thread (channel negotiation), so
/// `setChannelCount` from the JS thread is race-free with audio processing.
[[nodiscard]] size_t getChannelCount() const;

void setOutputChannelNumber(size_t outputChannelNumber) {
outputChannelNumber_.store(static_cast<int>(outputChannelNumber), std::memory_order_release);
}

/// @brief Returns this node's `channelCountMode` attribute.
/// @note Read only on the host thread (channel-count negotiation) — never on
/// @note Read only on the host thread (channel negotiation) — never on
/// the audio thread — so mutating it from the JS thread via
/// `setChannelCountMode` is race-free with audio processing.
[[nodiscard]] ChannelCountMode getChannelCountMode() const {
Expand All @@ -47,7 +52,7 @@ class AudioNode : public utils::graph::GraphObject, public std::enable_shared_fr
return channelInterpretation_;
}

/// @brief Sets `channelCount`. Drives channel-count negotiation, which reads
/// @brief Sets `channelCount`. Drives channel negotiation, which reads
/// this value on the host thread. Callers must trigger a renegotiation so
/// the change propagates to buffer layouts.
/// @note Host (JS) thread only. Overridable for node-specific constraints.
Expand Down Expand Up @@ -122,11 +127,12 @@ class AudioNode : public utils::graph::GraphObject, public std::enable_shared_fr
setOutputBuffer(buffer);
}

/// @brief Channel count this node presents on upstream connections (toward
/// AudioDestinationNode) after negotiation. Default: the negotiated channel
/// count. StereoPanner always outputs stereo.
[[nodiscard]] virtual size_t getUpstreamChannelCount(size_t negotiatedChannelCount) const {
return negotiatedChannelCount;
/// @brief Number of channels this node emits toward AudioDestinationNode
/// when that does not follow its `computedNumberOfChannels`. Default:
/// nullopt, the output is as wide as the inputs are mixed to. Sources
/// present `outputChannelNumber_`; StereoPanner always outputs stereo.
[[nodiscard]] virtual std::optional<size_t> getOutputChannelNumber() const {
return std::nullopt;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't overrides of this method solve the problem of outputChannelCount?

/// @note JS Thread only
Expand Down Expand Up @@ -206,14 +212,16 @@ class AudioNode : public utils::graph::GraphObject, public std::enable_shared_fr

const int numberOfInputs_ = 1;
const int numberOfOutputs_ = 1;
/// @brief Number of channels this node presents.
/// @brief The `channelCount` attribute (input mixing width). Host thread only.
int channelCount_ = 2;
/// @brief Number of channels this node emits; the width of `audioBuffer_`
///
/// Atomic because it is read on the JS thread during channel-count
/// negotiation (`HostGraph`/`getChannelCount`) while source subclasses
/// (AudioBufferSource, Streamer, AudioFileSource, RecorderAdapter,
/// AudioBufferQueueSource) write it on the audio thread once they learn the
/// decoded/buffer channel count. Plain reads/writes here would race.
std::atomic<int> channelCount_ = 2;
/// Atomic because it is read on the JS thread during channel negotiation
/// (`getOutputChannelNumber` overrides) while AudioBufferQueueSource and
/// RecorderAdapter write it from other threads once they learn their
/// channel count. Every other writer is the host thread, which must
/// renegotiate after a change.
std::atomic<int> outputChannelNumber_ = 2;
ChannelCountMode channelCountMode_ = ChannelCountMode::MAX;
ChannelInterpretation channelInterpretation_ = ChannelInterpretation::SPEAKERS;
const bool requiresTailProcessing_;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ PannerNode::PannerNode(
outputBuffer_(
std::make_shared<DSPAudioBuffer>(
RENDER_QUANTUM_SIZE,
channelCount_,
kOutputChannelNumber,
context->getSampleRate())) {}

void PannerNode::processNode(int framesToProcess) {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#pragma once

#include <memory>
#include <optional>

#include <audioapi/core/AudioNode.h>
#include <audioapi/core/AudioParam.h>
Expand All @@ -18,9 +19,6 @@ class PannerNode : public AudioNode {
const std::shared_ptr<BaseAudioContext> &context,
AudioListener *listener,
const PannerOptions &options);

~PannerNode() override = default;

[[nodiscard]] std::shared_ptr<AudioParam> getPositionXParam() const {
return positionXParam_;
}
Expand Down Expand Up @@ -105,8 +103,8 @@ class PannerNode : public AudioNode {
void setNegotiatedBuffer(const std::shared_ptr<DSPAudioBuffer> &buffer) override {
audioBuffer_ = buffer;
}
[[nodiscard]] size_t getUpstreamChannelCount(size_t /*negotiatedChannelCount*/) const override {
return outputBuffer_->getNumberOfChannels();
[[nodiscard]] std::optional<size_t> getOutputChannelNumber() const override {
return kOutputChannelNumber;
}

protected:
Expand All @@ -116,6 +114,8 @@ class PannerNode : public AudioNode {
}

private:
static constexpr size_t kOutputChannelNumber = 2;

AudioListener *listener_ = nullptr;

const std::shared_ptr<AudioParam> positionXParam_;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ StereoPannerNode::StereoPannerNode(
outputBuffer_(
std::make_shared<DSPAudioBuffer>(
RENDER_QUANTUM_SIZE,
channelCount_,
kOutputChannelNumber,
context->getSampleRate())) {}

std::shared_ptr<AudioParam> StereoPannerNode::getPanParam() const {
Expand All @@ -37,8 +37,8 @@ void StereoPannerNode::setNegotiatedBuffer(const std::shared_ptr<DSPAudioBuffer>
audioBuffer_ = buffer;
}

size_t StereoPannerNode::getUpstreamChannelCount(size_t /*negotiatedChannelCount*/) const {
return outputBuffer_->getNumberOfChannels();
std::optional<size_t> StereoPannerNode::getOutputChannelNumber() const {
return kOutputChannelNumber;
}

void StereoPannerNode::processNode(int framesToProcess) {
Expand Down
Loading
Loading