Add iOS session states, speaker preference, and route events (PAR-021) - #372
Draft
MaxHeimbrock wants to merge 4 commits into
Draft
Add iOS session states, speaker preference, and route events (PAR-021)#372MaxHeimbrock wants to merge 4 commits into
MaxHeimbrock wants to merge 4 commits into
Conversation
Define the phase-A routing API on PlatformAudio, purely in C# and shaped identically to the planned FFI-backed implementation so app code written against it survives the plumbing swap: - AudioOutputKind enum (values mirror the planned FFI proto enum 1:1) - AudioDevice.Kind / AudioDevice.IsSelected - OutputPreference ranked policy (default BT > wired > speaker > earpiece) - IsSpeakerOutputPreferred as documented sugar over the list order - SelectOutput / ClearOutputOverride sticky override - DevicesChanged event (playout, recording) on the Unity main thread - internal IRouteController seam with desktop (FFI enumeration/GUID selection) and unsupported-mobile implementations; Android/iOS backends plug into the seam in follow-up work Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… into max/par-021-unity-ios-session-features
The audio session config is now derived from a small state machine (idle / playout-only / recording) driven from C#, where PlatformAudio knows the recording state, plus a speaker-vs-earpiece preference expressed through the session mode (VideoChat/VoiceChat) — never overrideOutputAudioPort, so external devices always win. Non-recording states drop to mode Default + MixWithOthers to keep Unity audio unprocessed between calls; PlayAndRecord stays because the fork's ADM supports playout-only via an input-disabled VPIO unit but nothing guarantees VPIO under the Playback category. Every apply is mirrored into WebRTC's RTCAudioSessionConfiguration snapshot (reflected, no link-time dependency) so ADM-driven restarts re-apply the same config. Route changes are observed via AVAudioSessionRouteChangeNotification and forwarded to the new IosRouteController, which implements the PAR-019 seam: the playout list is the session's current output route with real Kind/IsSelected, DevicesChanged is raised on route changes (marshalled to the Unity main thread by PlatformAudio), OutputPreference reduces to the Speaker/Earpiece relative order, and SelectOutput throws the documented NotSupportedException pointing at AVRoutePickerView. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The ADM skips the VPIO rebuild on route changes that keep the hardware sample rate (HandleValidRouteChange -> HandleSampleRateChange no-ops), so a live VoiceChat -> VideoChat switch left the unit calibrated for the receiver — device-observed as an attenuated loudspeaker after an earpiece -> speaker toggle. Cycle isAudioEnabled after a mode change (when call audio is wanted) to force a clean rebuild against the new route, the same mechanism the foreground recovery already uses. The recovery and configure paths pass NO: the former cycles itself, the latter runs before the unit exists. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
PR #346 made iOS playout stable by putting the app in charge of the audio session, but it left the session in a single static config (PlayAndRecord + VideoChat) for the whole PlatformAudio lifetime — so Unity music plays at call volume even when the user is just listening — and PAR-019's routing API (#370) had no iOS backend:
IsSpeakerOutputPreferredround-tripped without effect andDevicesChangednever fired. This PR is Option A's iOS half: speaker-vs-earpiece preference, session state transitions, and route-change observation, kept behind the C#↔.mm seam so phase B can move the semantics native (PAR-008/PAR-017) and shrink the plugin mechanically.Stacked on #369 (which is on #346), and the branch has
max/par-019-routing-api-surface(#370) merged in for theIRouteControllerseam — the diff shows PAR-019's commit until #370 merges. Merge order: #346 → #369 → #370 → this.Changes
LiveKitAudioSession.mm, driven from C# (PlatformAudioknows recording state): ctor → playout-only,StartRecording/StopRecording→ recording/playout-only,SetSessionAudioEnabled(false)→ idle. State table (documented in the .mm header):PlayAndRecord+ modeDefault+MixWithOthers(+DefaultToSpeakeriff speaker preferred) — music-friendly between/before callsPlayAndRecord+ modeVideoChat(speaker) /VoiceChat(earpiece), noDefaultToSpeaker— the preference is expressed by the mode alone, neveroverrideOutputAudioPort, so headsets always winInitPlayout → InitPlayOrRecord(false) → Initialize(sr, enable_input=false)), but nothing establishes VPIO under categoryPlayback— hence playout-only keepsPlayAndRecordwith modeDefault+MixWithOthers. Music volume was measured on device, not assumed.RTCAudioSessionConfigurationsnapshot via the reflectedsetWebRTCConfiguration:(stillNSClassFromStringonly, no link-time dependency), so ADM-driven reconfigurations re-apply our config instead of the+initialize-frozen snapshot. The Recover iOS platform audio after backgrounding (Unity 6 focus race) #369 foreground recovery now re-asserts the current state's config instead of hardcoded VideoChat, and logs expected vs. actual.IosRouteControllerimplements the PAR-019 seam: playout list = the session's current output route with realKind/IsSelected(port-type mapping lives in the .mm),DevicesChangedraised fromAVAudioSessionRouteChangeNotification(marshalled to the Unity main thread byPlatformAudio),OutputPreferencereduces to the Speaker/Earpiece relative order,SelectOutputthrows the documentedNotSupportedExceptionpointing atAVRoutePickerView.HandleValidRouteChange → HandleSampleRateChangeno-ops), which left the loudspeaker attenuated by receiver-calibrated echo control after an earpiece → speaker toggle (device-observed).LiveKit_ApplySessionConfignow cyclesisAudioEnabledafter a mode change while call audio is wanted — the same rebuild the foreground recovery uses — at the cost of a brief audio gap per toggle.AllowBluetooth→AllowBluetoothHFPbehind the same version guard the WebRTC fork uses; no option is load-bearing for the in-call speaker preference.SelectOutputthrow). No public API surface changes (PAR-019's surface is frozen).Testing
clang -fsyntax-onlyagainst the iphoneos SDK under ARC and MRCDevicesChangedfires on BT connect/disconnect and mode switches, on the Unity main thread; interruption (Siri/call) and background/foreground recovery work with who-won logs; music survives hang-up; session restores on dispose. The quiet-loudspeaker-after-toggle issue found in the first Unity 6 round is fixed by the rebuild commit and was re-verified on device.🤖 Generated with Claude Code