diff --git a/.claude/skills/audio-nodes/SKILL.md b/.claude/skills/audio-nodes/SKILL.md index da108b451..fda5aaf42 100644 --- a/.claude/skills/audio-nodes/SKILL.md +++ b/.claude/skills/audio-nodes/SKILL.md @@ -141,7 +141,6 @@ classDiagram AudioNode <|-- WorkletNode AudioNode <|-- AnalyserNode AudioNode <|-- AudioDestinationNode - AudioNode <|-- AudioRecorder AudioScheduledSourceNode <|-- AudioBufferBaseSourceNode AudioScheduledSourceNode <|-- OscillatorNode @@ -152,6 +151,25 @@ classDiagram AudioBufferBaseSourceNode <|-- AudioBufferQueueSourceNode ``` +### AudioRecorder (not an AudioNode) + +`core/inputs/AudioRecorder` is a standalone base class, not part of the `AudioNode` hierarchy — it +feeds the graph through a `RecorderAdapterNode` instead of being processed by it. + +The split between it and `IOSAudioRecorder` / `AndroidAudioRecorder` is: the base owns everything +that happens to recorded frames (file writer, JS callback, adapter node — `enableFileOutput`, +`setupFileWriter`, `setOnAudioReadyCallback`, `connect`, `detachOutputs`/`finalizeOutputs`), the +subclasses own only the platform input stream. The one thing the base needs from the platform is +`resolveStreamFormat()`, returning sample rate, channel count and max frames per buffer; iOS reads +it from `NativeAudioRecorder` on every call (a route change invalidates it), Android returns values +cached when the Oboe stream opened. Add shared recorder behavior to the base, not to one platform. + +Pitfall: never redeclare a base member (`deinterleavingBuffer_`, `streamSampleRate_`, +`recordingSegmentPaths_`) in a platform recorder. The shadowing copy compiles fine, but the base's +audio-thread fan-out reads its own member and silently drops that output. + +--- + ### AudioScheduledSourceNode (internal only — not exposed to JS directly) Base class for source nodes that have a scheduled start and stop time. **Not instantiated directly.** diff --git a/.claude/skills/build-compilation-dependencies/SKILL.md b/.claude/skills/build-compilation-dependencies/SKILL.md index 38bf3ebbe..604fd9228 100644 --- a/.claude/skills/build-compilation-dependencies/SKILL.md +++ b/.claude/skills/build-compilation-dependencies/SKILL.md @@ -30,6 +30,7 @@ react-native-audio-api/ │ │ └── CMakeLists.txt # Actual Android C++ build target │ ├── common/cpp/audioapi/ # Shared C++ (used by all platforms) │ │ ├── decoding/ # Decoder factory, backends, SeekDecoderDaemon, AudioDecoding, AudioFileConcatenator +│ │ ├── encoding/ # AudioEncoder interface, EncoderCapabilities, OS encoder/remux selector headers │ │ ├── libs/ # Third-party wrappers (FFmpeg, miniaudio, pffft, …) │ │ └── external/ # Prebuilt binaries per platform │ │ ├── android/ # .a static libs (Opus, Ogg, Vorbis, OpenSSL) @@ -274,6 +275,7 @@ CI runs a parallel `cpp-coverage` job via `.github/workflows/cpp-coverage-job.ym - Compile definitions: `RN_AUDIO_API_ENABLE_WORKLETS=0`, `RN_AUDIO_API_TEST=1`, `RN_AUDIO_API_FFMPEG_DISABLED=1` - Google Test auto-fetched via `FetchContent` if not installed locally - New test files in `test/src/**/*.cpp` are picked up automatically by glob — no CMakeLists edit needed +- `jsi.cpp` is compiled into the static lib so library members that reference JSI symbols (e.g. `AudioFileProperties::CreateFromJSIValue`) link when a test first pulls them in; a static-lib member costs nothing unless demanded. If a new test triggers `Undefined symbols: facebook::jsi::...`, the referenced runtime source is missing from the lib — add it there rather than stubbing the symbol For `MockAudioEventHandlerRegistry`, `TestableXxx` pattern, and full CMakeLists analysis see [build-details.md](build-details.md#c-test-build--commoncpptestcmakeliststxt--detailed-analysis). @@ -312,6 +314,8 @@ Resolution pitfalls learned the hard way (both handled inside `package-root.js`) | `HAVE_ACCELERATE` | Not set | `GCC_PREPROCESSOR_DEFINITIONS` | Not set | | `RN_AUDIO_API_TEST` | Not set | Not set | Always set to 1 | +**OS-API selector headers** (`decoding/OSDecoding.h`, `encoding/OSEncoding.h`, `encoding/OSRemux.h`, `encoding/OSFilePath.h`): common code reaches platform implementations through `#if defined(__ANDROID__)` / `#elif defined(__APPLE__) && !defined(RN_AUDIO_API_TEST) && !defined(RN_AUDIO_API_NODE)` dispatch. The Apple branch must exclude **both** desktop defines: the gtest build (`RN_AUDIO_API_TEST`) and the WPT node addon (`RN_AUDIO_API_NODE`) run on macOS (where `__APPLE__` is defined) but do not compile or link the `ios/` ObjC++ sources. The node build cannot borrow `RN_AUDIO_API_TEST` instead — that flag also switches on gtest-only code (`gtest_prod.h` includes, test `ArrayBuffer` shims). When adding a new OS-selector header, copy the full three-clause guard; an incremental `wpt_tests/build` dir can mask a missing clause for a long time, so verify with a clean `yarn node:build`. Platform glue selected this way lives in `android/src/main/cpp/audioapi/android/` (e.g. `AndroidDecoding`, `AndroidEncoder`, `AndroidRemux`) and `ios/audioapi/ios/core/utils/` (e.g. `IOSDecoding`, `IOSEncoder`, `IOSRemux`) — both picked up automatically by the CMake glob / podspec glob, no build-file edits needed. + --- ## Common Build Failure Patterns diff --git a/.claude/skills/post-work-checks/SKILL.md b/.claude/skills/post-work-checks/SKILL.md index 951f994fc..40cc02bec 100644 --- a/.claude/skills/post-work-checks/SKILL.md +++ b/.claude/skills/post-work-checks/SKILL.md @@ -119,13 +119,13 @@ yarn test # from monorepo root — runs test:js + test:cpp **When**: after any change to C++ files or TypeScript files in `src/`. Prefer this for a quick local test loop covering both TS and C++ logic; run `yarn validate:fast` before opening a PR. -### AudioEvent enum sync check +### Enum sync check ```bash yarn check-audio-enum-sync ``` -**When**: only when you modify the `AudioEvent` enum or any file that maps event names across C++/Kotlin/TypeScript. Skip this step if you already ran `validate:fast` (it includes enum sync). +**When**: when you modify `AudioEvent`, `FileFormat` / `AudioFileProperties::Format`, or other JSI-crossing recorder enums (`FileDirectory`, `BitDepth`, `IOSAudioQuality`). Skip if you already ran `validate:fast` (it includes enum sync). --- diff --git a/.claude/skills/post-work-checks/maintenance.md b/.claude/skills/post-work-checks/maintenance.md index 1073cb524..3a866f18a 100644 --- a/.claude/skills/post-work-checks/maintenance.md +++ b/.claude/skills/post-work-checks/maintenance.md @@ -10,5 +10,5 @@ Review this skill when `pre-push-update` reports changes in: | `packages/react-native-audio-api/package.json` scripts | Package-level command changes (including per-language lint/format) | | `lefthook.yml` | Pre-commit / commit-msg hook changes | | `scripts/validate.sh` | Tier behavior (`--fast` / `--graph` / `--android` / `--ios` / `--full`), skip rules | -| `scripts/check-audio-enum-sync*` or `packages/react-native-audio-api/scripts/check-audio-events-sync.sh` | Enum sync check details | +| `scripts/check-audio-enum-sync*` or `packages/react-native-audio-api/scripts/check-*-enum-sync.sh` / `check-enum-sync.sh` | Enum sync check details (AudioEvent + AudioFileProperties) | | `.github/workflows/ci.yml`, `tests.yml`, `graph-tests.yml` | What CI covers vs local validation tiers | diff --git a/.claude/skills/thread-safety-itc/SKILL.md b/.claude/skills/thread-safety-itc/SKILL.md index 1e13b88b4..f9339f655 100644 --- a/.claude/skills/thread-safety-itc/SKILL.md +++ b/.claude/skills/thread-safety-itc/SKILL.md @@ -151,6 +151,8 @@ See the `utilities` skill for full API. **Pitfall — file writer / recorder shutdown:** `TaskOffloader::shutdown()` drains the SPSC queue before joining the worker. Call it (or destroy the offloader) only after `isFileOpen_` is cleared so the audio thread stops enqueueing. Otherwise rotated or closed M4A segments lose seconds of buffered audio. Types with a `.slot` member use `slot == size_t max` as the shutdown sentinel. +**Pitfall — the task type cannot be a nested struct.** `TaskOffloader` constrains `T` with `std::default_initializable`. A task struct carrying default member initializers (which the `.slot` sentinel requires) does *not* satisfy that constraint while its enclosing class is still incomplete, so `using Offloader = TaskOffloader;` inside the class fails to compile with "constraints not satisfied". Making the struct `public` does not help — it is not an access problem. Declare the task type at **namespace scope** instead (`PendingFileWrite`, `PendingCallbackFrames`). Dropping the initializers to satisfy the constraint is worse: `T{}` would then produce `slot == 0`, a valid slot index, making the shutdown sentinel indistinguishable from real work. + --- ## Driver synchronization (layered model) @@ -207,6 +209,7 @@ back-to-back). - **Copying `shared_ptr` inside `processNode()`** — increments atomic refcount; capture before entering hot path. - **Locking `initialize()` or graph factory methods** — `initialize()` runs synchronously during HostObject construction on the JS thread; node factories and `createMediaElementSource()` are synchronous JS calls. Only lifecycle methods that touch the driver or offline render thread need `driverMutex_`. - **Locking only `AudioContext`** — iOS recorder, session, and interruption paths mutate the shared `AVAudioEngine` outside `AudioContext`; keep the `AudioEngine` mutex on those entry points. Offline render uses the same `driverMutex_` on `BaseAudioContext`. +- **Duplicating recorder fan-out in platform code** — `AudioRecorder::onAudioFrames(interleavedFrames, numFrames)` (base class, `common/cpp/audioapi/core/inputs/`) is the single audio-thread fan-out to file writer, JS callback, and adapter node, using tryLock-and-drop per consumer. Platform recorders (e.g. `IOSAudioRecorder`) only normalize the platform buffer to interleaved float32 and call it — adding per-consumer writes in the platform receiver block double-writes every buffer. The interleave config (`inputChannelCount_`, scratch buffer) is read unlocked by the audio thread, so it may only be mutated while the input is disarmed (start/stop/input-format-change paths). - **Re-entering `driverMutex_` or the `AudioEngine` mutex on the same thread** — call `tryStartDriver()` directly from `resume()` instead of `start()`; use lock-free `isStreamRunning()` from `isDriverRunning()`. `AudioContext::start()` does not acquire `driverMutex_`; it asserts the lock is already held when the driver is not initialized (via `scheduleAudioEvent` synchronous path). When already initialized, `start()` is a lock-free no-op so `source.start()` on the audio thread does not take the mutex. --- diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3051bc4ff..bb45c5ad8 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -34,7 +34,7 @@ jobs: check-audio-enum-sync: uses: ./.github/workflows/ci-check.yml with: - name: Check AudioEvent enum sync + name: Check enum sync run: yarn check-audio-enum-sync build-audio-api: diff --git a/apps/common-app/src/demos/Record/Record.tsx b/apps/common-app/src/demos/Record/Record.tsx index 6bcb7df73..657c6d291 100644 --- a/apps/common-app/src/demos/Record/Record.tsx +++ b/apps/common-app/src/demos/Record/Record.tsx @@ -20,6 +20,16 @@ import RecordingVisualization from './RecordingVisualization'; import Status from './Status'; import { RecordingState } from './types'; +// concatAudioFiles supports WAV, M4A, and FLAC — the formats recordable on +// both iOS and Android. +const RECORDING_EXTENSION = FileFormat.M4A; +const ROTATING_SIZE = 250_000; + +const RECORDING_EXTENSION_NAME_MAP = { + [FileFormat.Wav]: 'wav', + [FileFormat.M4A]: 'm4a', + [FileFormat.Flac]: 'flac', +}; const Record: FC = () => { const [state, setState] = useState(RecordingState.Idle); const [hasPermissions, setHasPermissions] = useState(false); @@ -130,9 +140,14 @@ const Record: FC = () => { return; } - const outputPath = info.paths[0].replace(/[^/]+$/, 'recording.m4a'); + const extension = RECORDING_EXTENSION_NAME_MAP[RECORDING_EXTENSION]; + const outputPath = info.paths[0].replace( + /[^/]+$/, + `recording.${extension}` + ); const finalPath = await concatAudioFiles(info.paths, outputPath); + // const finalPath = info.paths[0]; const audioBuffer = await audioContext.decodeAudioData(finalPath); setRecordedBuffer(audioBuffer); @@ -262,7 +277,10 @@ const Record: FC = () => { }, [onPauseRecording, onResumeRecording]); useEffect(() => { - Recorder.enableFileOutput({ rotateIntervalBytes: 1_000_000, format: FileFormat.M4A }); + Recorder.enableFileOutput({ + rotateIntervalBytes: ROTATING_SIZE, + format: RECORDING_EXTENSION, + }); return () => { stopPlayback(); diff --git a/package.json b/package.json index c3519acbb..e33c75407 100644 --- a/package.json +++ b/package.json @@ -19,7 +19,7 @@ "clean": "del-cli packages/**/android/build apps/**/android/build apps/**/android/app/build apps/**/ios/build packages/**/lib node_modules apps/**/node_modules packages/**/node_modules", "typecheck": "yarn workspaces foreach -A -p run typecheck", "test": "yarn workspace react-native-audio-api run test", - "check-audio-enum-sync": "bash packages/react-native-audio-api/scripts/check-audio-events-sync.sh", + "check-audio-enum-sync": "bash packages/react-native-audio-api/scripts/check-enum-sync.sh", "validate:fast": "bash scripts/validate.sh --fast", "validate:graph": "bash scripts/validate.sh --graph", "validate:android": "bash scripts/validate.sh --android", diff --git a/packages/audiodocs/docs/inputs/audio-recorder.mdx b/packages/audiodocs/docs/inputs/audio-recorder.mdx index 1c7c5b8b2..6ccccf0e0 100644 --- a/packages/audiodocs/docs/inputs/audio-recorder.mdx +++ b/packages/audiodocs/docs/inputs/audio-recorder.mdx @@ -698,17 +698,42 @@ interface AudioRecorderFileOptions { Describes desired file extension as well as codecs, containers (and muxers!) used to encode the file. +All encoding is done with platform system APIs — iOS AVFoundation and Android MediaCodec/MediaMuxer. Because each platform exposes a different set of system encoders, format support is platform-specific. + ```tsx enum FileFormat { Wav, Caf, M4A, Flac, + Aiff, + Alac, + OpusOgg, + OpusWebm, + VorbisWebm, + Ulaw, + Alaw, } ``` -:::caution Android + FFmpeg -On Android, encoded file output for `M4A`, `FLAC`, and `CAF` uses FFmpeg. When FFmpeg is disabled in the build, only **WAV** recording to file is supported. iOS uses system AVFoundation for all listed formats. See [Runtime flags](../other/runtime-flags.mdx#where-ffmpeg-is-used). +The table below lists which formats each platform can encode with its system APIs: + +| `FileFormat` | Container / codec | iOS | Android | +| :--- | :--- | :---: | :---: | +| `Wav` | WAV / PCM | ✅ | ✅ | +| `M4A` | M4A / AAC-LC | ✅ | ✅ | +| `Flac` | FLAC / FLAC | ✅ | ✅ | +| `Caf` | CAF / PCM | ✅ | ❌ | +| `Aiff` | AIFF / PCM | ✅ | ❌ | +| `Alac` | M4A / Apple Lossless | ✅ | ❌ | +| `Ulaw` | WAV / µ-law | ✅ | ❌ | +| `Alaw` | WAV / a-law | ✅ | ❌ | +| `OpusOgg` | OGG / Opus | ❌ | ✅ | +| `OpusWebm` | WebM / Opus | ❌ | ✅ | +| `VorbisWebm` | WebM / Vorbis | ❌ | ✅ | + +:::caution Platform support +Selecting a format the current platform cannot encode (for example `Caf` on Android or `OpusOgg` on iOS) fails when file output is enabled, with a descriptive error. Some Android formats also depend on the device OS version (Opus/OGG muxing requires newer releases); if a device lacks a system encoder for the requested format, recording start returns an error. Use `Wav`, `M4A`, or `Flac` for the widest cross-platform support. ::: #### FileInfo diff --git a/packages/audiodocs/docs/other/disabling-prebuilt-libraries.mdx b/packages/audiodocs/docs/other/disabling-prebuilt-libraries.mdx index 6cd9faa22..39d986252 100644 --- a/packages/audiodocs/docs/other/disabling-prebuilt-libraries.mdx +++ b/packages/audiodocs/docs/other/disabling-prebuilt-libraries.mdx @@ -17,7 +17,7 @@ The available flags are independent and can be combined: | Flag | What it removes | What stops working | | :---: | :---- | :---- | -| `disableFFmpeg` | FFmpeg shared libraries (`libavcodec`, `libavformat`, `libavutil`, `libswresample`) | Remote URL streaming / HLS, remote URL metadata, M4A concat, **Android** non-WAV recording — see [Runtime flags](./runtime-flags.mdx#where-ffmpeg-is-used) | +| `disableFFmpeg` | FFmpeg shared libraries (`libavcodec`, `libavformat`, `libavutil`, `libswresample`) | Remote URL streaming / HLS, remote URL metadata — see [Runtime flags](./runtime-flags#where-ffmpeg-is-used). | | `disableStaticExternalLibs` | Static libs: `libopus`, `libopusfile`, `libogg`, `libvorbis`, `libvorbisenc`, `libvorbisfile` | Decoding `ogg`, `opus`, `oga` files | :::info diff --git a/packages/audiodocs/docs/other/runtime-flags.mdx b/packages/audiodocs/docs/other/runtime-flags.mdx index c65709891..6aa079f5b 100644 --- a/packages/audiodocs/docs/other/runtime-flags.mdx +++ b/packages/audiodocs/docs/other/runtime-flags.mdx @@ -2,7 +2,7 @@ These helpers let you check at runtime which optional native features are compiled into your app. They are synchronous and safe to call from JavaScript after the library has been installed. -Use them to branch UI or loading logic — for example, skip remote metadata preload when FFmpeg is disabled, disable hls streaming, or offer only WAV recording on Android. +Use them to branch UI or loading logic — for example, skip remote metadata preload when FFmpeg is disabled or disable hls streaming. :::info Build-time vs runtime To **disable** optional libraries at build time (and reduce app size), see [Disabling prebuilt libraries](./disabling-prebuilt-libraries.mdx). Runtime flags only tell you what ended up in the binary you are running. @@ -20,7 +20,7 @@ Returns whether the native build includes [`FFmpeg`](https://github.com/FFmpeg/F import { isFfmpegEnabled } from 'react-native-audio-api'; if (!isFfmpegEnabled()) { - console.warn('Remote URL metadata, streaming, and Android M4A recording require an FFmpeg build.'); + console.warn('Remote URL metadata and streaming require an FFmpeg build.'); } ``` @@ -29,14 +29,12 @@ if (!isFfmpegEnabled()) { | Area | API | Requires FFmpeg for | | :---: | :---: | :---- | -| Streaming | [`Audio tag`](../sources/audio-tag.mdx), `createFileSource` | Remote URL streaming (HTTP byte ranges) and HLS (`.m3u8`) | -| Metadata | [`getAudioDuration`](../utils/decoding.mdx#getaudioduration), [`