WebAudioRecorder Project - Comprehensive Analysis

Project Overview

WebAudioRecorder is a doctor-patient clinical audio recording platform built as a browser-only application. It's a Vite + React + TypeScript application with no backend - all data is stored locally in IndexedDB and Origin Private File System (OPFS).

Key Architecture & Performance Features

For a high-performance clinical recorder handling 4-hour sessions on 2GB RAM devices, the architectural choice between these APIs is the difference between a stable application and a guaranteed crash. Below is the detailed breakdown with performance numbers and clinical-grade calculations.

MediaRecorder vs AudioWorklet

Feature	MediaRecorder	AudioWorklet (Chosen)
Primary Job	""Black Box"" encoding to a file.	Raw PCM capture and processing
Latency	Variable/High. Managed by the browser's internal buffers.	Ultra-Low (~2.9ms). Operates on 128-sample quantums.
Raw Access	None. Only provides compressed blobs (VP8/Opus).	Full. Access to every individual Float32 sample
Thread	Main Thread / Browser Internals.	Dedicated Audio Thread. Isolated from UI ""jank"".
Pros	Simple API; low CPU for basic tasks.	High precision; real-time visualizers; no audio gaps.
Cons	No visualization; gaps in sequential data.	High complexity; requires separate Worker file.

IndexedDB vs OPFS (SyncAccessHandle)

Feature	IndexedDB	OPFS (SyncAccessHandle)
Primary Use	Metadata, WAL, & small state	High-performance binary storage
Performance	Slower; triggers UI ""jank"" with large blobs	Near-native synchronous I/O in workers
Pros	Widely supported; easy queryability	Scalable to GBs; atomic writes; isolated sandbox
Cons	RAM spikes during serialization	Requires Web Worker context for sync access

Decision: OPFS was selected for binary storage to meet the 4-hour / 2GB RAM requirement, while IndexedDB handles metadata and the Write-Ahead Log.

1. Audio Pipeline Architecture

Microphone → AudioWorklet (pcm-processor.js) → Main Thread → MP3 Web Worker → OPFS (encrypted) → IndexedDB

Zero-copy PCM processing: Uses transferable ArrayBuffers to avoid copying PCM data between threads.

2. 1MB Slab Accumulation System

Instead of encrypting/writing each MP3 chunk (4096 samples ≈ 93ms), the system accumulates MP3 frames in a 1MB RAM slab:

• Performance gain: Reduces encryption/write operations by ~1000x for typical recordings
• Crypto overhead: ~28 bytes per slab (12B IV + 16B GCM tag) = 0.0027% for 1MB
• WAL journaling: Each slab write is journaled for crash recovery

3. Write-Ahead Log (WAL) Journaling

Prevents data corruption on crashes:

• Before each slab write: Record {recordingId, chunkIndex, offset, status:'pending'} to IndexedDB
• After successful flush: Update status to 'committed'
• Crash recovery: Detect status:'in_progress' recordings on app restart, truncate OPFS to last committed offset before decrypt

4. Lazy Export & Persistent OPFS

• Initial save: Recording meta data saved in IndexedDB
• OPFS persistence: OPFS files stay alive indefinitely until explicit deletion
• On-demand decryption: lazyDecryptAndExport() fetches key, decrypts OPFS, creates blob
• History operations: Play/download/transcribe all trigger lazy decryption from persistent OPFS

Core Components

1. Audio Processing Pipeline

AudioWorkletProcessor (public/worklets/pcm-processor.js)

• Captures PCM from microphone at 44.1kHz
• Batches into 4096-sample chunks (93ms each)
• Transfers chunks via zero-copy (transferable ArrayBuffers)
• Handles pause/resume/flush operations

RecorderController (src/audio/RecorderController.ts)

• Manages AudioContext lifecycle
• Sets up audio graph: Mic → Analyser → Worklet → Muted Gain
• Handles microphone permissions and errors
• Provides pause/resume/stop functionality

EncoderBridge (src/audio/EncoderBridge.ts)

• Manages MP3 encoder Web Worker
• Creates sessions/recordings in IndexedDB
• Generates and stores AES-GCM encryption keys
• Handles periodic auto-save (every 30s)
• Manages lazy decryption and download

MP3 Encoder Worker (src/workers/mp3-encoder.worker.ts)

• Loads lamejs MP3 encoder via CDN bundle
• Converts PCM → MP3 at 128kbps
• Implements 1MB slab accumulation
• Handles OPFS file writes with encryption
• Manages WAL entries (worker-side IDB access)

2. Data Storage System

IndexedDB Schema (DB_VERSION = 3)

// Core entities
patients: {id, name, dateOfBirth?, notes?, createdAt}
sessions: {id, patientId, createdAt, notes?}
recordings: {id, sessionId, patientId, blob: Blob|null, durationMs, title, transcript, status, createdAt}

// Security & recovery
recording-keys: {id: recordingId, value: CryptoKey}
recording-wal: {id, recordingId, chunkIndex, offset, status:'pending'|'committed'}

OPFS Encryption (src/audio/opfs-crypto.ts)

• AES-GCM encryption with 256-bit keys
• File format: [4B blockLen][12B IV][ciphertext+16B tag]
• Additional authenticated data (AAD): chunkIndex as 8-byte LE
• Persistent until explicit deletion

3. State Management

Zustand Store (src/state/useRecorderStore.ts)

• Finite state machine for recorder lifecycle
• States: IDLE → REQUESTING_MIC → RECORDING → PAUSED → STOPPED → ENCODING → ENCODED → TRANSCRIBING → TRANSCRIBED
• Timing management with pause/resume support
• Patient selection and transcript storage

Recording Lifecycle

1. Start Recording

1. User selects patient, clicks "Start Recording"
2. Auto-create session with timestamp
3. Generate title: "Recording N – HH:MM"
4. Pre-create recording in IDB with status:'in_progress'
5. Generate AES-GCM key, save to recording-keys store
6. Initialize MP3 worker with sampleRate, bitRate, key, recordingId
7. Start 30s auto-save interval

2. During Recording

• Every 93ms: 4096 PCM samples → MP3 encoder → 1MB slab accumulation
• Every 30s: partial-flush → update duration in IDB (no slab flush)
• WAL entries: Created before each slab write, committed after flush

3. Stop Recording

1. Flush remaining PCM from AudioWorklet
2. Encode final MP3 tail
3. flushSlab(true) → encrypt/write final partial slab
4. Close OPFS sync handle
5. Update recording: status:'complete', final duration
6. Keep OPFS file + key persistent
7. Delete WAL entries (all committed)

4. Export/Playback

1. lazyDecryptAndExport(recordingId)
2. Fetch key from recording-keys store
3. Decrypt OPFS file → create Blob
4. For download: create object URL, trigger download
5. OPFS file remains alive for future access

5. Crash Recovery

1. App restart: detect status:'in_progress' recordings
2. Show CrashRecoveryModal with affected recordings
3. rollbackPendingWal(): truncate OPFS to last committed offset
4. Decrypt remaining valid data
5. Finalize recording as usual

6. Deletion

1. deleteRecordingFull(id): Delete OPFS file + key + WAL entries + IDB row
2. Cascade delete: Patient/session deletion cleans up all related recordings

Performance Optimizations

Memory Efficiency

• Zero-copy PCM: Transferable ArrayBuffers between AudioWorklet ↔ Main Thread ↔ Worker
• 1MB slab accumulation: Reduces crypto ops from per-chunk to per-slab
• Lazy blob creation: Blobs only created on demand, not stored in IDB

Storage Efficiency

• OPFS as source of truth: Encrypted files in OPFS, metadata in IDB
• WAL minimalism: ~40 bytes per slab entry
• Cascade cleanup: Full cleanup on delete operations

Crash Safety

• WAL journaling: Atomic write semantics
• Pending detection: Identify and recover interrupted recordings
• Partial flush safety: Auto-save updates duration without risking data

UI Components

Main Application

• Two tabs: Recorder (default) and Patients
• Crash recovery modal: Auto-shows on app restart with in-progress recordings

Recorder Page Components

• PatientSelector: Select/create patients
• Controls: Start/pause/stop buttons with state transitions
• Timer: Real-time recording duration display
• Waveform: Visual audio level display via AnalyserNode
• ExportButton: Trigger download of completed recordings
• TranscriptPanel: Display/edit transcript (mock API integration)

Patients Page Components

• PatientsPage: List all patients with creation/editing
• PatientCard: Compact patient display
• PatientDetail: Detailed patient view with sessions
• SessionCard: Session details with recordings
• RecordingRow: Individual recording with play/download/delete
• PatientForm: Create/edit patient data

Development & Build

Tech Stack

• Frontend: React 19 + TypeScript + Vite
• State: Zustand
• Audio: Web Audio API + lamejs (MP3 encoding)
• Storage: IndexedDB + OPFS
• Crypto: Web Crypto API (AES-GCM)

Build Process

• npm run dev: Vite dev server at http://localhost:5173
• npm run build: TypeScript compilation + Vite production build
• Bundle size: ~233KB gzip main bundle

Critical Implementation Details

AudioWorklet Requirements

• Must be plain JavaScript in public/worklets/ (not bundled by Vite)
• Loaded via audioContext.audioWorklet.addModule('/worklets/pcm-processor.js')

IndexedDB Transaction Safety

• Gather IDs BEFORE write transactions (no nested async in transactions)
• Transactions auto-commit when no pending requests
• Worker-side IDB access separate from main thread

Security & Privacy

Data Encryption

• AES-GCM 256-bit: Military-grade encryption
• Unique keys: Per-recording key generation
• IV(Initialisation Vector) per slab: Random 12-byte IV for each 1MB slab
• AAD integrity: chunkIndex authenticated but not encrypted

Local-Only Storage

• No backend API calls (except mock transcription)
• No data leaves browser unless user exports
• Full patient data privacy by design

Cleanup Guarantees

• Explicit deletion only: OPFS files persist until user deletes
• Cascade deletes: Patient → sessions → recordings → OPFS+keys+WAL
• No auto-cleanup: User controls all data lifecycle

Use Cases & Scalability

Clinical Recording Scenarios

• Short consultations: 5-15 minute recordings
• Long sessions: Up to 4+ hours (240 slabs, manageable WAL)
• Batch operations: Multiple patients/sessions per day

Storage Estimates

• MP3 bitrate: 128kbps = 16KB/s
• 1MB slab: ~64 seconds of audio
• 4-hour recording: ~225MB raw → ~240 slabs + WAL
• WAL overhead: ~9.6KB for 240 entries

Browser Compatibility

• Required APIs: Web Audio, IndexedDB, OPFS, Web Workers, Crypto
• Modern browsers: Chrome/Edge 122+, Firefox 120+, Safari 17+
• Mobile considerations: AudioWorklet support varies

Summary

WebAudioRecorder is a sophisticated browser-based clinical audio recording system that prioritizes:

1. Performance: Zero-copy audio, slab accumulation, lazy operations
2. Reliability: WAL journaling, crash recovery, atomic writes
3. Privacy: Local-only storage, AES-GCM encryption, explicit cleanup
4. Usability: Simple UI, automatic session creation, easy export

The architecture represents a modern approach to web audio applications, leveraging cutting-edge browser APIs while maintaining robustness and user privacy.

🧠 Thinking Process: Future Roadmap

Due to the time constraints of this assessment, the following features were deferred but are fully planned as part of the architectural vision for a production-ready medical tool.

The Sync (Resilient Cloud Synchronization)

For a 4-hour file (~230MB at 128kbps), standard uploads are unreliable. My planned approach for synchronization includes:

• Background Fetch API: To ensure uploads continue even if the clinician closes the browser tab or the device enters a Wi-Fi "dead zone".
• Multipart Uploads: Splitting the encrypted OPFS file into 5MB parts for S3-compatible cloud storage. This allows for granular retries—if one part fails, only that 5MB chunk is re-sent, not the entire 4-hour recording.
• Sync Queue: Extending the IndexedDB schema to track which 1MB slabs have been successfully acknowledged by the cloud before purging them from local storage.

The Guard (Clinical Key Management)

To transition from "Local Security" to "Cloud Security," the key management strategy must be upgraded:

• Envelope Encryption: The local AES-256 session key would be wrapped (encrypted) using an RSA-OAEP public key from a Cloud Key Management Service (KMS). This ensures that the platform provider can never decrypt the audio without a hardware-secured private key.
• Identity-Bound Decryption: Expanding the AAD to include DoctorID and PatientID to cryptographically bind the audio data to a specific consultation record.

The Transcriber (On-Device Speech-to-Text)

The architectural challenge is balancing high-fidelity medical accuracy with the strict Zero-Knowledge and 2GB RAM constraints of the device. The implementation would split into three distinct pipelines: Live (Streaming), On-Demand (Cloud-Batch), and On-Device (WASM).

Live Transcription (Streaming WebSocket)

Goal: Provide the clinician with real-time "draft" text during the consultation.

• The "Tap" Mechanism: I would add a second MessagePort to the existing AudioWorkletNode. While one port sends 1MB slabs to the MP3 Encoder, the second port sends smaller 100ms PCM chunks to a dedicated TranscriptionWorker.

• The WebSocket Relay: The TranscriptionWorker maintains a persistent WSS (Secure WebSocket) connection to a transcription engine (e.g., Whisper, Google Text To Speech).

• UI Integration: Results are pushed to the useRecorderStore and displayed in a scrolling LiveTranscript component.

• Security: Since this is "Live," the audio is streamed unencrypted in transit (via TLS). To maintain Zero-Knowledge, the provider must be configured with a "No-Store" policy, where audio is processed in RAM and immediately discarded.

On-Demand Transcription (Cloud-Batch)

Goal: High-accuracy, speaker-diarized, and punctuated clinical reports generated after the session.

• The Process: This utilizes Sync architecture. Once the encrypted MP3 is successfully uploaded to the cloud, the server triggers a batch job.
• Decryption at the Edge: If using a Zero-Knowledge model, the server cannot decrypt the file. The "On-Demand" request must be sent from the browser after local decryption.
• Implementation:
  • Call lazyDecryptAndExport(recordingId) to get the plaintext Blob.
  • Stream the Blob to a secure transcription endpoint.
  • Store the resulting JSON transcript in the recordings IndexedDB store.

On-Device Transcription (WASM / WebGPU)

Goal: Maximum privacy. No audio data ever leaves the browser, satisfying the strictest interpretation of BSI TR-03161 and § 203 StGB.

• The Tech Stack: Use a Whisper.cpp or Transformers.js build compiled to WebAssembly (WASM).

• The 2GB RAM Challenge: This is the most difficult task. A standard Whisper "base" model requires ~250MB of RAM, but the "medium" model (required for medical accuracy) requires >1.5GB.

• The Strategy:

  • Quantization: I would use a 4-bit quantized (INT4) model to reduce the memory footprint by 75%.
  • WebGPU Acceleration: Instead of CPU-bound WASM, use WebGPU to offload the heavy matrix multiplication to the GPU, which has its own memory pool, keeping the main 2GB RAM free for the OS and Browser.
  • Chunked Decoding: Feed the model 30-second windows of audio retrieved from OPFS to prevent the memory heap from spiking.

Feature	Live (Streaming)	On-Demand (Cloud)	On-Device (WASM)
Accuracy	Moderate (Draft)	Highest (Diarized)	High (Model dependent)
Privacy	TLS-Only	Zero-Knowledge (Cloud)	Absolute (Local-Only)
RAM Usage	Minimal (<10MB)	Moderate (~50MB)	High (250MB - 1GB)
Latency	<500ms	30s - 2m (Post-session)	Variable (CPU speed)

📉 Known Limitations & Trade-offs

• Export Memory Pressure: The current lazyDecryptAndExport implementation loads the entire recording into memory for decryption. While safe for small files, a 4-hour recording on a 2GB RAM device would require a Streaming Decryptor pattern via a Service Worker to stay under 10MB during export.
• Local-Only Persistence: Currently, if a user clears their browser's site data, all recordings and non-extractable keys are lost, as they are strictly sandboxed within IndexedDB and OPFS.
• Browser Compatibility: The solution relies on modern APIs like OPFS SyncAccessHandle and AudioWorklet, making it officially compatible only with recent versions of Chrome, Edge, and Safari (15.2+).

🇩🇪 Security & Compliance Alignment The architecture is designed to align with German medical standards:

• BSI TR-03161: Guidelines for secure medical applications (DiGA/DiPA).
• § 203 StGB: Protecting professional secrecy through End-to-End Encryption (E2EE) logic.
• GDPR/DSGVO: Local-first processing ensures no ePHI is transmitted unencrypted.