Release 2.9.0-dev.0#1119
Draft
hiroshihorie wants to merge 10 commits into
Draft
Conversation
…1106) ## Summary - Align Flutter default local-video degradation behavior with the Swift SDK. - Default unset `VideoPublishOptions.degradationPreference` to `maintainResolution` for camera and screen-share publishing. - Keep explicit degradation preferences overrideable by apps. ## Context Related to #1097, which explores preserving video quality through a live-streaming option. This PR takes the smaller SDK-default approach instead: use maintain-resolution by default, matching Swift, without adding a separate app-facing toggle for this behavior. ## Testing - `dart analyze` - `flutter test test/core/room_e2e_test.dart`
## Summary Migrates the Android plugin to AGP 9's built-in Kotlin while keeping older toolchains building (same pattern as flutter-webrtc/flutter-webrtc#2075): - Apply the **Kotlin Gradle Plugin only when built-in Kotlin is inactive** — AGP < 9, or AGP 9 with `android.builtInKotlin=false` (the configuration Flutter currently ships by default while the ecosystem migrates). When AGP 9's built-in Kotlin is active it registers the `kotlin` extension itself and rejects KGP, so applying it is skipped. - Set the JVM target through the `kotlin { compilerOptions {} }` DSL **when the extension supports it** (KGP 1.9+ / AGP 9 built-in Kotlin), falling back to the legacy `kotlinOptions` DSL for apps still on KGP 1.8.x. - Bump the standalone buildscript fallback KGP to 2.1.0 so it is self-consistent. - Add a changeset entry for the generated release notes. ## Context AGP 9 uses built-in Kotlin support and rejects Android plugins that still apply KGP directly. This follows the Flutter compatibility migration path instead of raising the minimum supported toolchain. ## Verification - Example app builds (`flutter build apk --debug`) on the current stable toolchain (AGP 8.x + modern KGP path). - The AGP 9 built-in path mirrors the reviewed and merged flutter-webrtc implementation.
## Summary - Add `deployment` field to `RoomAgentDispatch` for targeting specific agent deployments - Add `agentDeployment` to `TokenRequestOptions` to pass deployment through token requests - Update generated JSON serialization code The `deployment` field allows targeting a specific agent deployment (e.g., "staging"). Leave empty to target the production deployment. Related PRs: - node-sdks: livekit/node-sdks#675 - python-sdks: livekit/python-sdks#722 - rust-sdks: livekit/rust-sdks#1176 - client-sdk-swift: livekit/client-sdk-swift#1043 - client-sdk-js: livekit/client-sdk-js#1971 ## Usage ```dart final options = TokenRequestOptions( roomName: 'my-room', agentName: 'my-agent', agentDeployment: 'staging', // Optional: target specific deployment ); ``` Or directly via `RoomAgentDispatch`: ```dart final dispatch = RoomAgentDispatch( agentName: 'my-agent', metadata: 'my-metadata', deployment: 'staging', ); ``` ## Test plan ### Unit Tests ```bash flutter test flutter test test/token/token_source_test.dart -v ``` ### Manual Verification **1. Verify JSON serialization includes deployment:** ```dart final dispatch = RoomAgentDispatch( agentName: 'my-agent', deployment: 'staging', ); final json = dispatch.toJson(); print(json); // Should include 'deployment': 'staging' ``` **2. Verify TokenRequestOptions converts to request correctly:** ```dart final options = TokenRequestOptions( roomName: 'test-room', agentName: 'my-agent', agentDeployment: 'staging', ); final request = options.toRequest(); print(request.roomConfiguration?.agents?.first?.deployment); // Should print 'staging' ``` **3. Verify JSON round-trip:** ```dart final original = RoomAgentDispatch( agentName: 'my-agent', deployment: 'staging', ); final json = original.toJson(); final restored = RoomAgentDispatch.fromJson(json); assert(restored.deployment == 'staging'); ``` ### End-to-End Verification 1. Use TokenSource to get credentials with agentDeployment set 2. Connect to room - agent with matching deployment should join 3. Verify only staging agent receives the dispatch 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
…ead-back (#1107) ## What Runtime control of audio processing (AEC / NS / AGC / HPF) for local audio tracks, plus an engine-wide diagnostic read-back, built on the WebRTC-SDK audio processing options API (webrtc-sdk/webrtc#247 + webrtc-sdk/webrtc#254). ## API **Set** — `AudioProcessingOptions` with per-component enabled flags and modes (`automatic` / `platform` / `software`), applied either at capture time via `AudioCaptureOptions` or at runtime: ```dart final result = await localAudioTrack.setAudioProcessingOptions(options); ``` Caller bugs (invalid combination, remote track) throw `AudioProcessingException`; legitimate outcomes return a typed `AudioProcessingApplyResult` (`applied` / `stored` / rejections). **Read** — the audio processing module is owned by the native peer connection factory and shared engine-wide, so the snapshot lives on `AudioManager`: ```dart final state = await AudioManager.instance.getAudioProcessingState(); ``` Per component: `requested` (nullable — null means nothing was ever applied), `isSoftwareResolved` / `isSoftwareActive`, `isPlatformAvailable` / `isPlatformResolved` / `isPlatformActive`, and `effective` as the merged verdict. Same requested → resolved → active → effective vocabulary as the native SDKs. ## Commits Bottom-up, each builds standalone: 1. `chore(deps)`: WebRTC-SDK pin bump 2. Dart `AudioProcessingOptions` for `LocalAudioTrack` 3. Routing through the LiveKit native plugin (iOS + Android handlers) 4. Typed apply results 5. Engine-wide v2 state read-back on `AudioManager` ## Dependencies / not yet done - **Lib pin is still `144.7559.08`** — will bump to `144.7559.09` (which carries the state v2 API from webrtc-sdk/webrtc#254) once published. The state read-back native code requires `.09` to compile. - **Android requires `FlutterWebRTCPlugin.getPeerConnectionFactory()`** — flutter-webrtc/flutter-webrtc#2077. - Device smoke test on iOS + Android pending the `.09` artifacts.
## What Adds first-class, process-wide audio session and routing control through `AudioManager` on iOS and Android. LiveKit owns the platform audio session by default, while apps that need exact platform behavior can switch to manual mode and apply typed session configs. ## API and behavior - **Automatic by default**: calls need no setup. LiveKit applies a managed communication policy. - **iOS automatic mode**: the native WebRTC audio-engine delegate drives `AVAudioSession` from engine lifecycle events. Listen-only playout uses `playback`; recording uses `playAndRecord`. - **Android automatic mode**: LiveKit uses a communication session through the new AudioSwitch-backed `LKAudioSwitchManager`. - **Manual mode**: `setAudioSessionOptions(...)` and `deactivateAudioSession()` switch `AudioManager` to manual mode. `setAudioSessionManagementMode(AudioSessionManagementMode.automatic)` hands lifecycle control back to LiveKit. - **Typed options**: `AudioSessionOptions.communication()` and `AudioSessionOptions.media()` pre-fill Apple and Android configs, with per-platform overrides applied verbatim in manual mode. - **Speaker routing**: `AudioManager.instance.setSpeakerOutputPreferred(...)` owns speaker preference and forced speaker routing. Wired and Bluetooth devices still win unless `force: true`. ## Compatibility - Existing calls keep working without audio-session setup. - `Hardware` audio members and `Room.setSpeakerOn(...)` are deprecated forwarders to `AudioManager`. - `flutter_webrtc` native audio-session management is disabled so LiveKit has one owner for the session. - `bypassVoiceProcessing` now only controls WebRTC voice processing; it no longer changes the session intent. ## Docs and tests - Adds `docs/audio.md` and updates the README audio sections. - Adds coverage for session options, automatic/manual mode transitions, Apple/Android policy resolution, routing serialization, and engine-state observation. - Verified locally with `dart analyze`, `flutter test test/audio/audio_session_test.dart`, and `flutter test --reporter compact`.
## Summary - move create-time audio processing setup into the local capture start path so publish/preconnect prepares platform audio processing before WebRTC opens the microphone, matching the Swift SDK flow - keep `LocalAudioTrack.setAudioProcessingOptions` as a command-style runtime API that returns on success and throws `AudioProcessingException` on failure - expose structured failure reasons for invalid combinations, unavailable platform support, native apply failures, and unknown failures - clean up local audio tracks when capture/publish startup fails and keep cleanup failures from masking the original error - bump `flutter_webrtc` to `1.5.2` for Android audio device module access ## Behavior - `LocalAudioTrack.create(...)` stores requested audio processing options; it no longer attempts to apply them immediately. - `track.start()` starts local capture and applies stored processing options before the microphone opens on supported platforms. - Publish and preconnect paths fail during capture start if the exposed native platform API reports audio processing setup failure. - Runtime `setAudioProcessingOptions(...)` still applies immediately for active local audio tracks and throws on failed native apply/store. - Android uses `JavaAudioDeviceModule.prewarmRecording(options)`. That WebRTC API returns `void`, so this PR can surface thrown failures, but clean internal `false` returns from `initRecordingIfNeeded()` / `prewarmRecordingIfNeeded()` are not observable until WebRTC/flutter_webrtc exposes a resultful API. ## Testing - `flutter pub get` - `flutter pub get` in `example` - `flutter analyze --no-pub` - `flutter test --no-pub` - `flutter build apk --debug --no-pub` in `example` - Android emulator smoke test: example joined a room, published local microphone audio, muted/unpublished, and showed no ADM/platform-unavailable errors
## Summary - Add `LiveKitClient.initialize(initialAudioSessionOptions: ...)` for Android WebRTC audio-device initialization. - Seed Android automatic audio-session policy from the same initial options so playback-first apps do not need to immediately call `AudioManager.setAudioSessionOptions(...)` with the same preset. - Rename the playback preset to `AudioSessionOptions.mediaPlayback()` for clearer intent. - Avoid sticky Android speaker routing when updating speaker preference. - Update audio session docs and change entry. ## Runtime behavior Android uses `AudioSessionOptions` in two places with different timing: | API | Timing | What changes | | --- | --- | --- | | `LiveKitClient.initialize(initialAudioSessionOptions: ...)` | Before WebRTC initializes. | Passes Android audio configuration into flutter_webrtc/WebRTC ADM creation. This is where WebRTC playout `AudioAttributes` such as `usageType` and `contentType` are set. It also seeds LiveKit's automatic Android session policy. | | `AudioManager.instance.setAudioSessionOptions(...)` | Runtime. | Replaces LiveKit's stored session policy, switches to manual management, and applies LiveKit's platform session behavior: Android audio mode, audio focus mode, stream type, focus ownership, routing handler policy, and iOS category/options/mode. | The important Android limitation is that WebRTC playout `AudioAttributes` are still initialization-time configuration. Calling `AudioManager.setAudioSessionOptions(...)` at runtime applies LiveKit's platform session policy, but it does not mutate the `AudioAttributes` on an already-created WebRTC audio device module. For playback-first Android apps that need media volume/routing from WebRTC playout, pass `AudioSessionOptions.mediaPlayback()` to `LiveKitClient.initialize(...)` before connecting. ## Validation - `flutter test --no-pub test/audio/audio_session_test.dart` - `flutter analyze --no-pub --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Validation
Notes