You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Move macOS display capture to ScreenCaptureKit when available, with AVFoundation kept as fallback.
This is a backport of macOS capture work from Lumen, which appears to be an abandoned fork of Sunshine. The old AVFoundation capture path is still present and used if ScreenCaptureKit is unavailable or fails to initialize.
The main benefit is more reliable cursor capture. With the current AVFoundation path, some macOS apps can hide the pointer while typing and Sunshine does not see it reappear until the pointer returns to the Dock. This seems to be an Apple bug in AVFoundation, with ScreenCaptureKit being the most efficient fix.
Other benefits include improving handling for virtual or newly-created displays by refreshing ScreenCaptureKit shareable content when the selected display is not immediately available.
Tested manually on macOS with Moonlight/Sunshine streaming. This fixes the cursor-hidden-until-leaving-window behavior I was seeing with apps that hide the pointer during text input, which is pretty much everything from Terminal to XCode.
I did one follow-up pass on the ScreenCaptureKit path after testing in macOS VM environments.
Two changes seemed worth folding in:
The stream callback now only accepts complete ScreenCaptureKit frames with usable image data. Previously we could process update notifications that did not actually carry a screen buffer, which was harmless but unnecessary work.
There is now a 60 Hz fallback poll for an on-demand sample when the SCKit stream is idle. In Apple virtualization environments, some visible guest-side updates do not appear to reliably produce stream frames, which could leave Sunshine displaying stale content until a later input or screen event. The poll bounds that delay to roughly one frame interval.
In testing this fixed the delayed-update behavior without measurable CPU impact. The encoder path is already running at the target frame cadence, and the fallback still feeds it a zero-copy sample buffer rather than introducing a software copy path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Move macOS display capture to ScreenCaptureKit when available, with AVFoundation kept as fallback.
This is a backport of macOS capture work from Lumen, which appears to be an abandoned fork of Sunshine. The old AVFoundation capture path is still present and used if ScreenCaptureKit is unavailable or fails to initialize.
The main benefit is more reliable cursor capture. With the current AVFoundation path, some macOS apps can hide the pointer while typing and Sunshine does not see it reappear until the pointer returns to the Dock. This seems to be an Apple bug in AVFoundation, with ScreenCaptureKit being the most efficient fix.
Other benefits include improving handling for virtual or newly-created displays by refreshing ScreenCaptureKit shareable content when the selected display is not immediately available.
Tested manually on macOS with Moonlight/Sunshine streaming. This fixes the cursor-hidden-until-leaving-window behavior I was seeing with apps that hide the pointer during text input, which is pretty much everything from Terminal to XCode.
Issues Fixed or Closed
Resolves #3433.
Type of Change
Checklist
AI Usage