-
Notifications
You must be signed in to change notification settings - Fork 3.5k
Cleaning up the Hosting native Android views page #13356
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -31,18 +31,25 @@ visit [Hosting native macOS views][]. | |
| Platform Views on Android have several implementations. | ||
| They come with tradeoffs both in terms of performance and fidelity. | ||
|
|
||
| ## Hybrid composition {: #hybrid-composition } | ||
| ### Choosing an implementation | ||
|
|
||
| The following matrix summarizes the different implementations and their trade-offs: | ||
|
|
||
| | Mode | Benefits | Considerations | Enabler | | ||
| | :--- | :--- | :--- | :--- | | ||
| | **Texture layer** | • Best Flutter performance<br>• Full widget transforms work | • Janky during quick scrolling<br>• SurfaceViews lose accessibility and text magnifier breaks | Default behavior or standard `AndroidView` | | ||
| | **Hybrid composition** | • Full native fidelity<br>• Correct accessibility and SurfaceView support | • Causes thread merging, which degrades Flutter FPS | `PlatformViewLink` with `AndroidViewSurface` | | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Given that flutter's thread merge of (ui and platform) threads is probably what people will think of, we should clarify here that HC's thread merge is (raster and platform)
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also, on the enabler: I think what matters here is the onCreatePlatformView: (PlatformViewCreationParams params) {
print('useHybridComposition=$useHybridComposition');
late AndroidViewController controller;
if (useHybridComposition) {
controller = PlatformViewsService.initExpensiveAndroidView(
id: params.id,
viewType: params.viewType,
layoutDirection: TextDirection.ltr,
);
} else {
controller = PlatformViewsService.initSurfaceAndroidView(
id: params.id,
viewType: params.viewType,
layoutDirection: TextDirection.ltr,
);
} |
||
| | **HCPP** (Experimental) | • Full fidelity and performance<br>• Solves original sync overhead | • Requires Android API 34+ and Vulkan support | • `<meta-data>` in `AndroidManifest.xml`<br>• `--enable-hcpp` local flag | | ||
|
|
||
| {:.table .table-striped} | ||
|
|
||
|
|
||
| ## Hybrid Composition {: #hybrid-composition } | ||
|
|
||
| Platform Views are rendered as they are normally. | ||
| Flutter content is rendered into a texture. | ||
| SurfaceFlinger composes the Flutter content and the platform views. | ||
|
|
||
| * `+` best performance and fidelity of Android views. | ||
| * `-` Flutter performance suffers. | ||
| * `-` FPS of application will be lower. | ||
| * `-` Certain transformations that can be applied to Flutter widgets | ||
| won't work when applied to platform views. | ||
|
|
||
| ## Hybrid Composition++ (HCPP) {: #hcpp } | ||
|
|
||
| :::note | ||
|
|
@@ -58,7 +65,7 @@ It is currently available as an opt-in feature. | |
|
|
||
| * **Android API 34 or later**: Required for native transaction | ||
| synchronization capabilities. | ||
| * **Vulkan Rendering**: The device must be capable of rendering with Vulkan. | ||
| * **Vulkan rendering**: The device must be capable of rendering with Vulkan. | ||
|
|
||
| If these requirements are not met on the end-user device, | ||
| Flutter will automatically fall back to the existing platform view strategy | ||
|
|
@@ -72,7 +79,7 @@ it's enabled through configuration rather than standard Dart initialization meth | |
|
|
||
| You can enable HCPP using one of the following methods: | ||
|
|
||
| 1. **Command Line Flag (Run/Test)**: | ||
| 1. **Command line flag (run/test)**: | ||
| Pass the `--enable-hcpp` flag to your `flutter run` or `flutter test` command: | ||
|
|
||
| ```bash | ||
|
|
@@ -81,7 +88,7 @@ You can enable HCPP using one of the following methods: | |
|
|
||
| :::note | ||
| This flag is intended for local execution and testing. | ||
| It **can't** be passed to the `flutter build` commands. | ||
| **It can't be passed to the `flutter build` commands.** | ||
| For release builds, use the manifest configuration | ||
| as shown in the next step. | ||
| ::: | ||
|
|
@@ -98,34 +105,38 @@ You can enable HCPP using one of the following methods: | |
|
|
||
| ### Limitations and known issues | ||
|
|
||
| * **Complex Overlay Stacking**: | ||
| * **Complex overlay stacking**: | ||
| Transparent platform views won't display correctly | ||
| in layout stacks structured as: | ||
| Flutter canvas -> Platform View -> Overlay -> Transparent Platform View, | ||
| **Flutter canvas -> Platform View -> Overlay -> Transparent Platform View**, | ||
| when all four of these layers intersect. | ||
|
|
||
| To create a platform view on Android, use the following steps. | ||
|
|
||
| ## Texture layer { #texturelayerhybridcomposition } | ||
| ## Texture layer {: #texture-layer } | ||
|
|
||
| Platform Views are rendered into a texture. | ||
| Flutter draws the platform views (using the texture). | ||
| Flutter content is rendered directly into a Surface. | ||
|
|
||
| * `+` good performance for Android Views | ||
| * `+` best performance for Flutter rendering. | ||
| * `+` all transformations work correctly. | ||
| * `-` quick scrolling (such as a web view) will be janky | ||
| * `-` SurfaceViews are problematic in this mode and will be moved into a virtual | ||
| display (breaking a11y) | ||
| * `-` Text magnifier will break unless Flutter is rendered into a TextureView. | ||
| This approach provides: | ||
|
|
||
| * good performance for Android Views | ||
| * best performance for Flutter rendering | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. comment about best performance also applies here |
||
| * all transformations work correctly | ||
|
|
||
| However, this approach might cause: | ||
|
|
||
| * jankiness on quick scrolling (such as a web view) | ||
| * broken accessibility for `SurfaceView`s | ||
| * broken text magnification unless Flutter is rendered | ||
| into a `TextureView` | ||
|
|
||
| ## On the Dart side | ||
|
|
||
| On the Dart side, create a `Widget` | ||
| and add one of the following build implementations. | ||
| To create a platform view on Android, use the following steps. | ||
| First, on the Dart side, create a `Widget` and add one of the | ||
| following build implementations depending on your chosen strategy. | ||
|
|
||
| ### Hybrid composition | ||
| ### Hybrid Composition | ||
|
|
||
| In your Dart file, | ||
| for example `native_view_example.dart`, | ||
|
|
@@ -142,7 +153,7 @@ use the following instructions: | |
| import 'package:flutter/services.dart'; | ||
| ``` | ||
|
|
||
| 2. Implement a `build()` method: | ||
| 2. Implement a `build` method: | ||
|
|
||
| <?code-excerpt "lib/native_view_example_1.dart (hybrid-composition)"?> | ||
| ```dart | ||
|
|
@@ -179,7 +190,7 @@ use the following instructions: | |
| } | ||
| ``` | ||
|
|
||
| For more information, visit the API docs for: | ||
| For more information, visit the following API docs: | ||
|
|
||
| * [`PlatformViewLink`][] | ||
| * [`AndroidViewSurface`][] | ||
|
|
@@ -203,7 +214,7 @@ use the following instructions: | |
| import 'package:flutter/services.dart'; | ||
| ``` | ||
|
|
||
| 2. Implement a `build()` method: | ||
| 2. Implement a `build` method: | ||
|
|
||
| <?code-excerpt "lib/native_view_example_2.dart (virtual-display)"?> | ||
| ```dart | ||
|
|
@@ -222,9 +233,7 @@ use the following instructions: | |
| } | ||
| ``` | ||
|
|
||
| For more information, visit the API docs for: | ||
|
|
||
| * [`AndroidView`][] | ||
| For more information, visit the [`AndroidView`][] API page. | ||
|
|
||
| [`AndroidView`]: {{site.api}}/flutter/widgets/AndroidView-class.html | ||
|
|
||
|
|
@@ -485,19 +494,70 @@ android { | |
| ### Manual view invalidation | ||
|
|
||
| Certain Android Views don't invalidate themselves when their content changes. | ||
| Some example views include `SurfaceView` and `SurfaceTexture`. | ||
| When your Platform View includes these views, you are required to | ||
| manually invalidate the view after they have been drawn to | ||
| (or more specifically: after the swap chain is flipped). | ||
| Manual view invalidation is done by calling `invalidate` on the View | ||
| or one of its parent views. | ||
| Some examples include `SurfaceView` and `SurfaceTexture`. | ||
| When your Platform View includes these views, | ||
| you must manually invalidate it after it has been drawn | ||
| (or, more specifically, after the swap chain is flipped). | ||
| Invalidate the view by calling `invalidate` on it or on one of its parents. | ||
|
|
||
| [`AndroidViewSurface`]: {{site.api}}/flutter/widgets/AndroidViewSurface-class.html | ||
|
|
||
| ### Issues | ||
|
|
||
| [Existing Platform View issues][] | ||
| Check out the [existing Platform View issues][] on GitHub. | ||
|
|
||
| [existing Platform View issues]: {{site.github}}/flutter/flutter/issues?q=is%3Aopen+is%3Aissue+label%3A%22a%3A+platform-views | ||
|
|
||
| ## Performance | ||
|
|
||
| Platform views in Flutter come with performance trade-offs. | ||
|
|
||
| In a typical Flutter app, | ||
| the Flutter UI is composed on a dedicated raster thread, | ||
| while platform code runs on the UI/platform thread. | ||
| This separation keeps Flutter rendering fast and fluid. | ||
|
|
||
| However, when a platform view is rendered on Android using hybrid | ||
| composition, Flutter merges the raster and UI threads into a single thread to | ||
| ensure correct synchronization between the native Android views and the Flutter canvas. | ||
| Because of this thread merging, rendering complex Flutter widgets | ||
| alongside a platform view can compete with OS messages and plugin interactions, | ||
| potentially causing lower application FPS and frame drops. | ||
|
|
||
| Also, prior to Android 10, hybrid composition copied each Flutter frame | ||
| out of the graphic memory into main memory, | ||
| and then copied it back to a GPU texture. As this copy happens per frame, | ||
| the performance of the entire Flutter UI might be impacted. | ||
| In Android 10 or above, the graphics memory is copied only once. | ||
|
|
||
| Hybrid Composition++ (HCPP) minimizes this overhead by using native | ||
| transaction synchronization on supported devices (Android API 34+ with Vulkan), | ||
| allowing superior performance without the heavy costs of original hybrid | ||
| composition. | ||
|
|
||
| Virtual display, on the other hand, makes each pixel of the native view flow | ||
| through additional intermediate graphic buffers, | ||
| which cost graphic memory and drawing performance. | ||
| This can cause jank during high-frequency updates like fast scrolling. | ||
|
|
||
| For complex cases, there are some techniques that | ||
| can be used to mitigate these issues. | ||
|
|
||
| For example, you could use a placeholder texture | ||
| while an animation is happening in Dart. | ||
| In other words, if an animation is slow while a | ||
| platform view is rendered, | ||
| then consider taking a screenshot of the | ||
| native view and rendering it as a texture. | ||
|
|
||
| For more information, visit the following API pages: | ||
|
|
||
| {% render "docs/platform-view-perf.md", site: site %} | ||
| * [`TextureLayer`][] | ||
| * [`TextureRegistry`][] | ||
| * [`FlutterTextureRegistry`][] | ||
| * [`FlutterImageView`][] | ||
|
|
||
| [Existing Platform View issues]: {{site.github}}/flutter/flutter/issues?q=is%3Aopen+is%3Aissue+label%3A%22a%3A+platform-views | ||
| [`FlutterImageView`]: {{site.api}}/javadoc/io/flutter/embedding/android/FlutterImageView.html | ||
| [`FlutterTextureRegistry`]: {{site.api}}/ios-embedder/protocol_flutter_texture_registry-p.html | ||
| [`TextureLayer`]: {{site.api}}/flutter/rendering/TextureLayer-class.html | ||
| [`TextureRegistry`]: {{site.api}}/javadoc/io/flutter/view/TextureRegistry.html | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should weaken this to "good" flutter performance. I don't have results from benchmarking here, but abstractly based on the architecture
TLHC: PV -> Renders to Texture -> Uploads to Impeller -> Impeller composites Flutter Content & PV Content
HCPP: PV -> Renders to native android Surface, Impeller renders to native Android Surface, SurfaceFlinger composites the two together
there should be more synchronization work for TLHC to do, so I imagine HCPP would have better flutter performance (though TLHC's flutter performance is still good).