Audio

The audio subsystem is a standalone Web Audio engine plus two scene components. It gives you mixing buses, one-shot and spatial sound effects, looping music with crossfade, an insertable effects rack, microphone capture, and FFT/level analysis — all on top of a single AudioContext.

Source lives in src/audio/. The AudioEngine is usable on its own (no Engine required); the AudioSource and AudioListener components glue it to the scene graph when you want sound to follow GameObjects.

import {
  AudioEngine,
  createReverb,
  type AudioClip,
  type SoundHandle,
} from 'taos/audio/index.js';

The shape of it#

                       ┌── 'sfx'   ──┐
sources ──connect──►  ├── 'music' ──┤──► 'master' ──► destination (speakers)
                       ├── 'ui'    ──┤
                       └── 'voice' ──┘

• AudioEngine owns the AudioContext, the bus tree, the loader, and the pool of live voices.
• A bus (AudioBus) is a named mixing group: gain → effects chain → parent. All buses route into master, and master into the speakers.
• A voice (SoundHandle) is one playing sound. play() / playAt() return a handle you can control or stop; it disposes itself when it ends.
• A clip (AudioClip) is a decoded buffer ready to play.

Lifecycle: unlock the context first#

Browsers will not start an AudioContext until the user interacts with the page. Until then play() / playAt() return null. Two ways to handle it:

const audio = new AudioEngine();

// Option A — arm one-shot listeners that resume on the first interaction.
audio.resumeOnGesture();              // defaults to window; pass a target to scope it

// Option B — resume explicitly from inside a click/keydown handler.
button.addEventListener('click', () => { void audio.resume(); });

audio.ready is true once the context exists and is running. Tear everything down with audio.dispose() (stops all voices and closes the context).

Two per-frame calls keep spatial audio correct (the components below do these for you):

engine.afterFrame(() => {
  audio.updateListener(cam.position, cam.forward, cam.up); // 3D "ears"
  audio.update();                                          // prune + advance follow()
});

Loading clips#

const clip   = await audio.load('/sfx/blip.ogg');          // fetch + decode, cached
const clips  = await audio.loadAll(['/a.ogg', '/b.ogg']);  // parallel
const decoded = await audio.decode('boom', arrayBuffer);   // decode a buffer you already have

Loads are cached and deduped by URL — the same URL always yields the same AudioClip. Reach the cache via audio.loader (evict(url?) drops one entry, or all). An AudioClip exposes duration, channels, and sampleRate.

Playback#

Non-spatial one-shots — play()#

const handle = audio.play(clip, {
  bus: 'sfx',          // 'master' | 'sfx' | 'music' | 'ui' | 'voice' (default 'sfx')
  volume: 0.9,         // per-voice gain (default 1)
  playbackRate: 1.0,   // pitch multiplier
  loop: false,
  fadeIn: 0,           // seconds
  offset: 0,           // start offset, seconds
});

Spatial one-shots — playAt()#

const emitter = { position: new Vec3(0, 0, -3) };

const voice = audio.playAt(clip, emitter.position, {
  bus: 'sfx',
  loop: true,
  volume: 0.8,
  spatial: { refDistance: 2, maxDistance: 30, rolloffFactor: 1 }, // HRTF panner tuning
});
voice?.follow(emitter);   // bind to a { position } object; engine syncs it each frame

spatial options (SpatialOptions): panningModel (default 'HRTF'), distanceModel (default 'inverse'), refDistance (5), maxDistance (50), rolloffFactor (1).

Both calls return a SoundHandle | null (null if the context isn't unlocked). See the full options in sound_handle.ts.

Controlling a voice — SoundHandle#

Music#

Only one music track plays at a time. playMusic() accepts a URL or a preloaded clip and loops by default.

await audio.resume();
await audio.playMusic('/music/ambient.ogg', { fade: 1.5, volume: 0.8, loop: true });
// ... later
audio.fadeOutMusic(2);     // fade over 2s, then stop
audio.stopMusic(0.5);      // stop with a short fade
audio.musicVolume = 0.3;   // the music bus level, [0, 1]

Real example — samples/grassy_hills.ts:224:

const audio = new AudioEngine();
audio.musicVolume = 0.3;
audio.resumeOnGesture();
void audio.playMusic(ambienteUrl);

Buses, volume, and muting#

const sfx = audio.bus('sfx');     // get-or-create; built-ins are lazily made
sfx.volume = 0.8;                 // linear, 1 = unity
sfx.fade(0.0, 0.5);               // ramp to 0 over half a second
sfx.muted = true;                 // silence without losing the volume value

audio.masterVolume = 0.9;         // shortcut for audio.master.volume
audio.muted = true;               // global mute

Gain compounds along the path: a voice on sfx is scaled by its own volume × sfx.volume × master.volume.

Effects rack#

Effects are insertable sub-graphs (a single input → output) added to a bus in signal order. Factories live in audio_effects.ts:

const sfx = audio.bus('sfx');
const reverb = sfx.addEffect(createReverb(audio.context, { seconds: 2.5, decay: 2, wet: 0.4 }));
const lp     = sfx.addEffect(createFilter(audio.context, { type: 'lowpass', frequency: 1200 }));

sfx.removeEffect(reverb);   // remove + dispose one
sfx.clearEffects();         // remove + dispose all

Microphone & analysis#

let mic = null;
try {
  mic = await audio.requestMicrophone();   // prompts; rejects if denied / no device
} catch (e) { /* user declined */ }

if (mic) {
  const level = mic.getLevel();      // [0, 1] RMS
  // mic.connect(node) to route it somewhere — it is NOT sent to speakers by default
  mic.muted = true;                  // disable the stream's tracks
  mic.stop();                        // release the device
}

Tap any bus (or the master) with a non-destructive analyser:

const analyser = audio.master.analyser({ fftSize: 2048, smoothing: 0.82 });
const wave     = new Float32Array(analyser.node.fftSize);
const spectrum = new Float32Array(analyser.binCount);

engine.afterFrame(() => {
  analyser.getWaveform(wave);        // time domain, [-1, 1]
  analyser.getFrequencies(spectrum); // dB spectrum, ~[-100, 0]
  const loud = analyser.getLevel();  // cheap RMS estimate
});

The full audio test bench — buses, effects, spatial orbit, mic, and a waveform/spectrum visualizer — is samples/audio_test.ts.

Scene components#

When sound should follow scene objects, skip the manual updateListener / follow plumbing and attach components. They call into the same AudioEngine.

import { AudioSource } from 'taos/engine/components/audio_source.js';
import { AudioListener } from 'taos/engine/components/audio_listener.js';

// One AudioListener, usually on the camera — drives the 3D listener each frame.
cameraGO.addComponent(new AudioListener(audio));

// An AudioSource plays a clip that tracks its GameObject's world position.
const src = beeGO.addComponent(new AudioSource(audio, {
  clip: buzzClip,
  loop: true,
  spatial: true,
  bus: 'sfx',
  autoplay: false,
}));
src.play();             // -> SoundHandle | null
src.isPlaying;          // getter
src.stop(0.25);         // fade out

AudioSource options: clip, bus, loop, volume, playbackRate, spatial (default true), spatialOptions, autoplay.

Gotchas#

• Unlock before you play. No user gesture → context suspended → play() returns null. Call resumeOnGesture() early, or resume() from a handler.
• Drive the listener and update() every frame for spatial audio, or use the AudioListener / AudioSource components which do it for you.
• Mic isn't routed to speakers by default (feedback). Call mic.connect() to send it somewhere; the built-in mic.analyser is already wired.
• One music track at a time — playMusic() replaces the current track.
• Effect order is insertion order. Filters early, reverb/delay late, is the usual arrangement.
• Handles self-dispose when playback ends and the engine prunes them in update(); manual dispose() is safe but unnecessary.

See also#

• Quick-Start Guide — the engine and component model these build on
• samples/audio_test.ts — the full audio bench
• Chapter 14 — Audio and Chapter 22 — Crafty Audio — the theory