Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .changes/android-media-routing
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
patch type="fixed" "Apply Android media audio attributes during WebRTC initialization"
patch type="fixed" "Use initialization audio options as the default Android session policy"
patch type="fixed" "Avoid sticky Android speaker routing when updating route preference"
Original file line number Diff line number Diff line change
Expand Up @@ -195,20 +195,14 @@ internal class LKAudioSwitchManager(private val context: Context) {
}

private fun applySpeakerRouting(switch: AbstractAudioSwitch, speakerRouting: SpeakerRouting) {
// AudioSwitch treats selectDevice(null) as "select no device"; it does not
// recompute the best route from the preferred-device list. Keep routing
// automatic here so normal preference and forced-speaker priority both
// follow device hot-plug changes without leaving a sticky selected device.
switch.setPreferredDeviceList(speakerRouting.preferredDeviceList)
val forcedSpeaker = if (speakerRouting.speakerOutputForced) {
switch.availableAudioDevices.firstOrNull { it is AudioDevice.Speakerphone }
} else {
null
}
// AudioSwitch selections are sticky. Use them only for forced speaker output.
// Clearing the selection lets the preferred-device list handle normal routing
// and headset hot-plug priority.
switch.selectDevice(forcedSpeaker)
}

private fun speakerRoutingSnapshot() = SpeakerRouting(
speakerOutputForced = speakerOutputForced,
preferredDeviceList = preferredDeviceList(
speakerOutputPreferred = speakerOutputPreferred,
speakerOutputForced = speakerOutputForced,
Expand Down Expand Up @@ -253,7 +247,6 @@ internal class LKAudioSwitchManager(private val context: Context) {
)

private data class SpeakerRouting(
val speakerOutputForced: Boolean,
val preferredDeviceList: List<Class<out AudioDevice>>,
)
}
Expand Down
41 changes: 33 additions & 8 deletions docs/audio.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ LiveKit disables flutter_webrtc's own native audio management automatically when

Speaker output is preferred by default, but a wired or Bluetooth headset still wins over the speaker. Forced speaker output is off, so the speaker is never forced over a connected headset unless you ask for it.

The media playback preset (`AudioSessionOptions.mediaPlayback()`) is for playback-first experiences such as viewer-only live streams. On Android, pass it to `LiveKitClient.initialize` before WebRTC initializes when you need the WebRTC audio device module to use media mode and media volume. This also seeds LiveKit's automatic runtime session policy until you explicitly replace it with `AudioManager.instance.setAudioSessionOptions(...)`. Runtime session updates apply LiveKit's platform session policy, but WebRTC playout `AudioAttributes` are currently initialized when the audio device module is created.

The default audio capture options apply standard voice processing, so echo cancellation, noise suppression, and auto gain control are on and the high pass filter is off. You can change this per track with `AudioProcessingOptions`.

On macOS the audio engine state is reported but no `AVAudioSession` is configured. On web, Windows, and Linux the session APIs do not configure native audio. Speaker switching is available only on iOS and Android, where `AudioManager.instance.canSwitchSpeakerphone` is true.
Expand All @@ -27,24 +29,45 @@ import 'package:livekit_client/livekit_client.dart';

// Take manual control and apply a media playback session.
await AudioManager.instance.setAudioSessionOptions(
const AudioSessionOptions.media(),
const AudioSessionOptions.mediaPlayback(),
);
```

On Android, media output type has one WebRTC initialization-time piece and one LiveKit runtime-session piece. For playback-first apps, initialize WebRTC with the media intent before connecting:

```dart
await LiveKitClient.initialize(
initialAudioSessionOptions: const AudioSessionOptions.mediaPlayback(),
);
```

See the next section for the full rule. Apply options before connecting when you can.
This seeds both WebRTC's initialization-time playout attributes and LiveKit's automatic runtime session policy. See the next section for the full rule. Apply options before connecting when you can.

## Configuration timing

`AudioSessionOptions` are used in two places on Android:

| API | Timing | What it can update |
| --- | --- | --- |
| `LiveKitClient.initialize(initialAudioSessionOptions: ...)` | Before WebRTC initializes. | WebRTC audio device module playout `AudioAttributes`, and LiveKit's initial automatic runtime session policy. Today WebRTC uses the Android `usageType` and `contentType` fields, for example `USAGE_MEDIA` and `CONTENT_TYPE_UNKNOWN` from `AudioSessionOptions.mediaPlayback()`. |
| `AudioManager.instance.setAudioSessionOptions(...)` | Runtime. | Explicitly replaces the stored session options, switches to manual management, and applies LiveKit's platform session policy: Android audio mode, audio focus mode, stream type, focus ownership, routing handler policy, and iOS category/options/mode. |

Most fields in `AudioSessionOptions` are runtime session policy and can be applied again with `AudioManager.instance.setAudioSessionOptions(...)`. The exception is Android WebRTC playout attributes: changing `AndroidAudioSessionConfiguration.usageType` or `contentType` at runtime updates LiveKit's session/focus handler, but it does not change the `AudioAttributes` of an already-created WebRTC audio device module. Pass those options to `LiveKitClient.initialize(...)` before WebRTC initializes when they must affect playout volume/routing. You do not need to call `setAudioSessionOptions(...)` with the same options just to make LiveKit use them during automatic session management.

We plan to make Android WebRTC playout attributes runtime-updatable in a future SDK/WebRTC integration if the native layer can safely update the stored attributes and recreate playout with acceptable behavior. Until then, treat WebRTC playout `AudioAttributes` as initialization-time configuration.

## Automatic vs manual mode

The two modes differ in who owns the session lifecycle.

In automatic mode (the default) LiveKit manages the session from room, connect, and engine lifecycle and chooses the configuration for you. It does not take session options in this mode.
In automatic mode (the default) LiveKit manages the session from room, connect, and engine lifecycle. On iOS it derives the active category/mode from the current audio engine state. On Android it uses the current session intent, defaulting to communication and optionally seeded by `LiveKitClient.initialize(initialAudioSessionOptions: ...)`.

In manual mode LiveKit does not touch the session on its own, and your app owns it. Enter manual mode when you need to apply a fixed platform configuration or deactivate the session yourself.

```dart
// Apply a fixed config. This enters manual mode.
await AudioManager.instance.setAudioSessionOptions(
const AudioSessionOptions.media(),
const AudioSessionOptions.mediaPlayback(),
);

// Later, hand control back to LiveKit.
Expand Down Expand Up @@ -84,7 +107,9 @@ await AudioManager.instance.setSpeakerOutputPreferred(true, force: true);
await AudioManager.instance.setSpeakerOutputPreferred(false);
```

Speaker routing is independent of the management mode and does not switch it. On Android and in iOS automatic mode, LiveKit applies the preference through its managed route policy. In iOS manual mode, the fixed Apple config you apply owns non-forced receiver vs speaker behavior. `force: true` still uses Apple's speaker override when the active category is `playAndRecord`. Read the current preference through `AudioManager.instance.isSpeakerOutputPreferred` and `AudioManager.instance.isSpeakerOutputForced`. `AudioManager.instance.canSwitchSpeakerphone` is true on iOS and Android.
Speaker routing is independent of the management mode and does not switch it. In Android communication/call sessions and in iOS automatic mode, LiveKit applies the preference through its managed route policy. In iOS manual mode, the fixed Apple config you apply owns non-forced receiver vs speaker behavior. `force: true` still uses Apple's speaker override when the active category is `playAndRecord`. Read the current preference through `AudioManager.instance.isSpeakerOutputPreferred` and `AudioManager.instance.isSpeakerOutputForced`. `AudioManager.instance.canSwitchSpeakerphone` is true on iOS and Android.

On Android, LiveKit speaker routing is a communication/call routing policy. Media sessions use Android's normal media routing. Pass `AudioSessionOptions.mediaPlayback()` to `LiveKitClient.initialize` before WebRTC initializes when the WebRTC audio device module should use media attributes.

`Room.setSpeakerOn(...)` is deprecated and forwards to `AudioManager.instance.setSpeakerOutputPreferred`. You can also set an initial preference through `RoomOptions` (`defaultAudioOutputOptions.speakerOn`) before connecting, which LiveKit applies when the session starts.

Expand Down Expand Up @@ -148,7 +173,7 @@ print('echo cancellation in effect: ${state?.echoCancellation.effective}');

## Per platform overrides

When the preset constructors are not enough you can pin exact platform values. Supplying options through `setAudioSessionOptions` switches to manual mode, so these configs are a manual-mode tool. `AudioSessionOptions.communication()` and `AudioSessionOptions.media()` pre-fill Apple and Android configs. Passing `apple` or `android` replaces that platform config rather than merging with the preset.
When the preset constructors are not enough you can pin exact platform values. Supplying options through `setAudioSessionOptions` switches to manual mode, so these configs are a manual-mode tool. `AudioSessionOptions.communication()` and `AudioSessionOptions.mediaPlayback()` pre-fill Apple and Android configs. Passing `apple` or `android` replaces that platform config rather than merging with the preset.

```dart
await AudioManager.instance.setAudioSessionOptions(
Expand Down Expand Up @@ -188,15 +213,15 @@ final clearedMode = updated.copyWith(
);
```

Create a new `AudioSessionOptions.communication()` or `AudioSessionOptions.media()` when you want to start from a different preset config.
Create a new `AudioSessionOptions.communication()` or `AudioSessionOptions.mediaPlayback()` when you want to start from a different preset config.

## Platform support

| Platform | Audio session | Speaker routing | Engine state |
| --- | --- | --- | --- |
| iOS | Automatic mode follows live engine state. Manual mode applies your Apple config verbatim. | Yes. Normal preference respects wired and Bluetooth devices. Forced speaker uses Apple's speaker override while the active category is `playAndRecord`. | Yes, from native WebRTC engine events. |
| macOS | Not configured. There is no `AVAudioSession`. | No. `canSwitchSpeakerphone` is false. | Yes, the same engine events are reported. |
| Android | Automatic mode uses the communication session (in-communication mode, voice call stream). A media session is available in manual mode. Managed through LiveKit's AudioSwitch handler. | Yes. Normal preference orders headsets before the speaker. Forced speaker selects the speaker device. | Not reported, the Dart state stays idle. |
| Android | Automatic mode uses the current session intent, seeded by `LiveKitClient.initialize(initialAudioSessionOptions: ...)` and defaulting to communication. Pass media options before WebRTC initializes when the WebRTC audio device module should use media mode/volume. | Yes for communication/call routing. Media playback follows Android media routing. | Not reported, the Dart state stays idle. |
| Web, Windows, Linux | Not configured. | No. `canSwitchSpeakerphone` is false. | Not reported. |

On iOS automatic mode, listen only playout uses `playback`. When recording starts, LiveKit reapplies the session as `playAndRecord`. In manual mode, non-forced receiver vs speaker behavior comes from the Apple config you applied.
Expand Down
15 changes: 15 additions & 0 deletions lib/src/audio/audio_manager.dart
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,21 @@ class AudioManager {
_audioEngineStateController.add(nextState);
}

/// Seeds the initial session intent without taking over the session lifecycle.
///
/// `LiveKitClient.initialize(initialAudioSessionOptions: ...)` uses this so the
/// WebRTC initialization-time Android audio attributes and LiveKit's automatic
/// runtime session policy start from the same intent. Unlike
/// [setAudioSessionOptions], this keeps automatic management enabled and does
/// not apply native session changes immediately.
@internal
void setInitialAudioSessionOptions(AudioSessionOptions options) {
if (_managementMode != AudioSessionManagementMode.automatic) {
return;
}
_options = options;
}

/// Applies an explicit audio session configuration and switches to manual mode.
///
/// Calling this puts [AudioManager] in [AudioSessionManagementMode.manual]:
Expand Down
11 changes: 6 additions & 5 deletions lib/src/audio/audio_session.dart
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,12 @@ class AudioSessionOptions {

/// One-way media playback preset.
///
/// This pre-fills playback-oriented platform policies. Apple playback policy
/// leaves routing to the platform, while Android speaker routing remains a
/// runtime preference. Override [apple] or [android] for exact platform
/// behavior.
const AudioSessionOptions.media({
/// This pre-fills playback-oriented platform policies. Apple and Android
/// media routing are platform-owned. On Android, pass this to
/// `LiveKitClient.initialize` before WebRTC initializes when WebRTC playout
/// should use media `AudioAttributes`; the same value seeds LiveKit's initial
/// automatic runtime media session policy.
const AudioSessionOptions.mediaPlayback({
AppleAudioSessionConfiguration apple = AppleAudioSessionConfiguration.media,
AndroidAudioSessionConfiguration android = AndroidAudioSessionConfiguration.media,
}) : this._(apple: apple, android: android);
Comment thread
hiroshihorie marked this conversation as resolved.
Expand Down
7 changes: 4 additions & 3 deletions lib/src/audio/audio_session_policy.dart
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,10 @@ class ResolvedAudioSessionPolicy {
}

AndroidAudioSessionConfiguration get androidConfiguration {
if (automatic) {
return AndroidAudioSessionConfiguration.communication;
}
// In automatic mode LiveKit still owns activation timing, focus/routing
// lifecycle, and speaker preference. The Android session intent itself is
// the current AudioSessionOptions value, seeded by LiveKitClient.initialize
// or replaced explicitly by AudioManager.setAudioSessionOptions.
return options.android;
}
}
40 changes: 31 additions & 9 deletions lib/src/livekit.dart
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,10 @@
import 'package:flutter_webrtc/flutter_webrtc.dart' as rtc;

import 'audio/audio_manager.dart';
import 'audio/audio_session.dart';
import 'support/native.dart';
import 'support/platform.dart' show lkPlatformIsMobile;
import 'support/platform.dart' show PlatformType, lkPlatformIs, lkPlatformIsMobile;
import 'support/webrtc_initialize_options.dart';

/// Main entry point to connect to a room.
/// {@category Room}
Expand All @@ -26,25 +28,45 @@ class LiveKitClient {
/// Initialize the WebRTC plugin.
///
/// Optional: call once at startup to enable [bypassVoiceProcessing] before
/// connecting. Otherwise WebRTC initializes lazily with defaults.
/// connecting, or to apply Android [initialAudioSessionOptions] before WebRTC
/// creates its audio device module. Otherwise WebRTC initializes lazily with
/// defaults.
///
/// LiveKit owns the platform audio session, and flutter_webrtc's own native
/// audio management is disabled automatically when the LiveKit plugin loads
/// (done natively at registration), so that does not depend on this call.
///
/// Configure audio-session behavior through [AudioManager] before connecting,
/// e.g. `await AudioManager.instance.setAudioSessionManagementMode(...)` and
/// Configure explicit runtime audio-session behavior through [AudioManager]
/// before connecting, e.g.
/// `await AudioManager.instance.setAudioSessionManagementMode(...)` and
/// `await AudioManager.instance.setAudioSessionOptions(...)`.
///
/// [initialAudioSessionOptions] currently affects Android's WebRTC
/// initialization-time playout attributes, such as media vs voice
/// communication usage. It also seeds [AudioManager]'s initial automatic
/// runtime session policy until the app explicitly replaces it with
/// [AudioManager.setAudioSessionOptions]. A future SDK/WebRTC integration may
/// make those Android playout attributes runtime-updatable; for now, pass them
/// here before WebRTC initializes.
static Future<void> initialize({
bool bypassVoiceProcessing = false,
AudioSessionOptions? initialAudioSessionOptions,
}) async {
if (lkPlatformIsMobile()) {
// bypassVoiceProcessing controls only WebRTC voice processing, not the
// session intent. The audio session is owned by AudioManager.
// bypassVoiceProcessing controls only WebRTC voice processing. Android
// playout attributes are passed here because WebRTC reads them when it
// creates the audio device module.
Native.bypassVoiceProcessing = bypassVoiceProcessing;
await rtc.WebRTC.initialize(options: {
if (bypassVoiceProcessing) 'bypassVoiceProcessing': bypassVoiceProcessing,
});
await rtc.WebRTC.initialize(
options: liveKitWebRTCInitializeOptions(
bypassVoiceProcessing: bypassVoiceProcessing,
initialAudioSessionOptions: initialAudioSessionOptions,
includeAndroidAudioConfiguration: lkPlatformIs(PlatformType.android),
),
);
if (lkPlatformIs(PlatformType.android) && initialAudioSessionOptions != null) {
AudioManager.instance.setInitialAudioSessionOptions(initialAudioSessionOptions);
}
}
}
}
30 changes: 30 additions & 0 deletions lib/src/support/webrtc_initialize_options.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
// Copyright 2026 LiveKit, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

import 'package:meta/meta.dart';

import '../audio/android_audio_session_adapter.dart';
import '../audio/audio_session.dart';

@internal
Map<String, dynamic> liveKitWebRTCInitializeOptions({
required bool bypassVoiceProcessing,
required AudioSessionOptions? initialAudioSessionOptions,
required bool includeAndroidAudioConfiguration,
}) =>
{
if (bypassVoiceProcessing) 'bypassVoiceProcessing': bypassVoiceProcessing,
if (includeAndroidAudioConfiguration && initialAudioSessionOptions != null)
'androidAudioConfiguration': androidAudioSessionConfigurationToMap(initialAudioSessionOptions.android),
};
Loading
Loading