Skip to content

feat(audio): sound effects, BGM, volume and procedural synth#91

Draft
xinyao27 wants to merge 9 commits into
pocket-stack:mainfrom
xinyao27:feat/audio
Draft

feat(audio): sound effects, BGM, volume and procedural synth#91
xinyao27 wants to merge 9 commits into
pocket-stack:mainfrom
xinyao27:feat/audio

Conversation

@xinyao27

Copy link
Copy Markdown

Summary

Adds a complete audio feature to PocketJS as a new host surface globalThis.audio (spec ops 26–31) — deliberately outside the deterministic core, mirroring the DevTools-ops contract so goldens and input tapes are untouched byte-for-byte.

  • Public API (@pocketjs/framework/audio, AUDIO.md): playSfx / defineSfx / playSynth (8-bit synth descriptors: square/pulse/triangle/saw/sine/noise + sweep + envelope), playBgm / pauseBgm / resumeBgm / stopBgm (loop, cross-fade), setVolume / setMuted (master/sfx/bgm buses).
  • Asset pipeline: <appDir>/sounds.json bakes WAVs → mono s16 SND pak entries (audio:sfx.* / audio:bgm.*), resampled at build time so hosts do zero decoding; BGM budget warning at 1 MB/track.
  • Web host: WebAudio graph (master→sfx/bgm buses), gesture unlock (deferred BGM, dropped SFX pre-unlock), synth descriptors pre-rendered into cached AudioBuffers.
  • PSP host: one hardware channel + an allocation-free, integer-only (Q15/16.16) software mixer thread (priority 16), fixed 8-voice pool with oldest-of-kind stealing, SPSC command ring, cross-fade via a borrowed dying voice; PCM plays straight from .rodata.
  • Sim host: records the audio command stream into Trace.audio; the music-demo journey asserts ops land on exact frames, chaos-equal.
  • Demos: music plays 3 generated chiptune loops (deterministic gen-sounds.ts) with cross-fading track switches and a synth blip; settings' SOUND EFFECTS toggle is now real.
  • Docs: new AUDIO.md (spec + usage), touchpoints in README/DESIGN/RUNTIMES.

Verification

  • bun run test — 168 + 9 + 4 pass, 0 fail (incl. new audio.test.ts ×22, sound-bake.test.ts ×20, music sim journey)
  • bun run golden — 35/35 unchanged; bun run tape:check — 180 frames match (audio provably outside the replay world)
  • bunx tsc --noEmit clean; music / settings / hero builds green
  • Review fixes applied on top of the implementation: PCM cursor widened to u64 (BGM > 65535 frames), click-free fade rounding, stop-while-paused zombie voice

Not verified here

The PSP native mixer (native/src/audio.rs) compiles against the vendored rust-psp checkout, which is absent on this machine — needs a cargo check + PPSSPP/hardware smoke in a PSP-toolchain environment before ready-for-review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant