Skip to content

Commit b36cd77

Browse files
rolfbjarnerolfbjarne
committed
[tools/msbuild] Add a trimmer step to inline calls to ObjCRuntime.Dlfcn.
Add a new custom linker step that replaces runtime `Dlfcn.dlsym` calls with direct native symbol lookups via P/Invokes to `__Internal`. This improves startup performance and app size, and ensures the native linker can see symbol references that would otherwise only be resolved at runtime. Two modes are supported, controlled by the `InlineDlfcnMethods` MSBuild property: * `strict`: inlines all calls to `ObjCRuntime.Dlfcn` APIs. * `compatibility`: only inlines calls that reference symbols from `[Field]` attributes. Post-trimming MSBuild targets ensure native code is only generated for symbols that survive trimming (ILTrim or NativeAOT): * `_CollectPostILTrimInformation`: scans trimmed assemblies for surviving inlined dlfcn P/Invokes, with per-assembly caching for incremental builds. * `_CollectPostNativeAOTTrimInformation`: reads unresolved symbols from the NativeAOT object file or static library and filters for inlined dlfcn symbols. * `_PostTrimmingProcessing`: generates native C code for surviving symbols and adds it to the native link inputs. Extends `MachO.cs` with LC_SYMTAB parsing to read unresolved symbols from both Mach-O object files and static libraries. Contributes towards #17693 (even though this adds a *new* custom trimmer step, it's a pre-marking step that's easy to move out of the linker later on, and it's a partial replacement for the ListExportedSymbols step, which can't be moved out of the trimmer).
1 parent 4096e3d commit b36cd77

27 files changed

Lines changed: 1840 additions & 51 deletions

‎.github/copilot-instructions.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,12 @@ The build system supports multiple platforms simultaneously:
4444

4545
The `configure` script is used to select which platforms to build. This script must be run before `make`.
4646

47+
## MSBuild code
48+
49+
Non-standard patterns in .targets files:
50+
51+
- Always use `$(DeviceSpecificIntermediateOutputPath)` instead of `$(IntermediateOutputPath)`.
52+
4753
## Binding System
4854

4955
### bgen (Binding Generator)
@@ -331,4 +337,4 @@ try {
331337
} catch (Exception e) {
332338
// Code here
333339
}
334-
```
340+
```

‎docs/building-apps/build-properties.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -561,6 +561,28 @@ See also:
561561
* The [AlternateAppIcon](build-items.md#alternateappicon) item group.
562562
* The [AppIcon](#appicon) property.
563563

564+
## InlineDlfcnMethods
565+
566+
Controls whether the build system replaces runtime calls to `ObjCRuntime.Dlfcn` methods with direct native symbol lookups at build time, eliminating the overhead of `dlsym` at runtime.
567+
568+
The valid options are:
569+
570+
* `compatibility`: Inlines dlfcn method calls but only creates native references for symbols used in `[Field]` attributes. This is more conservative and avoids link errors for symbols that don't exist at build time.
571+
* `strict`: Inlines dlfcn method calls and creates native references for all symbols. This is more aggressive and may cause link errors if referenced native symbols don't exist.
572+
* (empty): Disables inlining of dlfcn method calls.
573+
574+
Default value:
575+
* .NET 11+: `strict` when using NativeAOT (`PublishAot=true`), `compatibility` otherwise.
576+
* .NET 10 and earlier: not set (disabled).
577+
578+
Example:
579+
580+
```xml
581+
<PropertyGroup>
582+
<InlineDlfcnMethods>compatibility</InlineDlfcnMethods>
583+
</PropertyGroup>
584+
```
585+
564586
## iOSMinimumVersion
565587

566588
Specifies the minimum iOS version the app can run on.

‎docs/code/native-symbols.md‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# Native symbols
2+
3+
Native symbols can be referenced from managed code in several ways:
4+
5+
* P/Invokes (DllImports)
6+
* Calls to `dlsym`, which can happen through:
7+
* The various APIs in `ObjCRuntime.Dlfcn`
8+
* The various APIs in `System.Runtime.InteropServices.NativeLibrary`
9+
* A P/Invoke directly into `dlsym`
10+
11+
It's highly desirable to use a direct native reference to native symbols when building a mobile app, for a few reasons:
12+
13+
* It's faster at runtime, and the app is smaller.
14+
* If the referenced native symbol comes from a third-party static library, the
15+
native linker can remove it if it's configured to remove unused code
16+
(because the native linker can't see that the native symbol is in fact used
17+
at runtime) unless there's a direct native reference to the symbol.
18+
19+
On the other hand there's one scenario when a direct native reference is not desirable: when the native symbol does not exist.
20+
21+
In order to create a direct native reference to native symbols, we need to know the names of those native symbols.
22+
23+
## The `InlineDlfcnMethods` property
24+
25+
This behavior is controlled by the `InlineDlfcnMethods` MSBuild property, which
26+
has two modes:
27+
28+
* `strict`: all calls to `ObjCRuntime.Dlfcn` APIs are inlined.
29+
* `compatibility`: only calls that reference symbols from `[Field]` attributes are inlined.
30+
31+
See the [build properties documentation](../building-apps/build-properties.md) for default values.
32+
33+
## How it works
34+
35+
During the build we try to collect the following:
36+
37+
* Any property or field with the `[Foundation.Field]` attribute: we collect the symbol name.
38+
* Any calls to the `ObjCRuntime.Dlfcn` APIs: we try to collect the symbol name (this might not always succeed, if the symbol name is not a constant).
39+
* We don't process calls to `System.Runtime.InteropServices.NativeLibrary` at the moment (this may change in the future, if there's need).
40+
41+
This is further complicated by the fact that we only want to create native
42+
references for symbols that survive trimming.
43+
44+
So we do the following:
45+
46+
1. Before or during trimming, we run the inlining steps:
47+
48+
* `ProcessExportedFields`: collects all members with `[Field]` attributes.
49+
50+
* `InlineDlfcnMethodsStep`: inspects all calls to `ObjCRuntime.Dlfcn`, and
51+
inlines them depending on the selected mode. If inlined, the step creates
52+
a P/Invoke to a native method that will return the address for that symbol
53+
(using a direct native reference), and modifies the code that fetches that
54+
symbol to call said P/Invoke.
55+
56+
2. After trimming, we figure out which of those symbols survived:
57+
58+
* For ILTrim: the `_CollectPostILTrimInformation` MSBuild target inspects
59+
the trimmed assemblies and collects all the inlined dlfcn P/Invokes that
60+
survived. Per-assembly results are cached to speed up incremental builds.
61+
62+
* For NativeAOT: the `_CollectPostNativeAOTTrimInformation` MSBuild target
63+
inspects the native object file (or static library) produced by NativeAOT,
64+
collects all unresolved native references, and filters them against the
65+
inlined dlfcn symbols to determine which survived.
66+
67+
3. The `_PostTrimmingProcessing` MSBuild target takes the surviving symbols
68+
from either path, generates the corresponding native C code, and adds it to
69+
the list of files to compile and link into the final executable.

‎dotnet/targets/Xamarin.Shared.Sdk.props‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,12 @@
101101
<AccelerateBuildsInVisualStudio Condition="'$(AccelerateBuildsInVisualStudio)' == ''">false</AccelerateBuildsInVisualStudio>
102102
</PropertyGroup>
103103

104+
<!-- Set default value for InlineDlfcnMethods based on .NET version and NativeAOT -->
105+
<PropertyGroup Condition="'$(InlineDlfcnMethods)' == '' And $([MSBuild]::VersionGreaterThanOrEquals($(TargetFrameworkVersion), '11.0'))">
106+
<InlineDlfcnMethods Condition="'$(_UseNativeAot)' == 'true'">strict</InlineDlfcnMethods>
107+
<InlineDlfcnMethods Condition="'$(InlineDlfcnMethods)' == ''">compatibility</InlineDlfcnMethods>
108+
</PropertyGroup>
109+
104110
<!-- Set the default RuntimeIdentifier if not already specified. -->
105111
<PropertyGroup Condition="'$(_RuntimeIdentifierIsRequired)' == 'true' And '$(RuntimeIdentifier)' == '' And '$(RuntimeIdentifiers)' == '' ">
106112
<!-- The _<platform>RuntimeIdentifier values are set from the IDE -->

‎dotnet/targets/Xamarin.Shared.Sdk.targets‎

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,13 +18,17 @@
1818
</PropertyGroup>
1919

2020
<UsingTask Runtime="$(_TaskRuntime)" TaskName="Xamarin.MacDev.Tasks.CompileNativeCode" AssemblyFile="$(_TaskAssemblyName)" />
21+
<UsingTask Runtime="$(_TaskRuntime)" TaskName="Xamarin.MacDev.Tasks.CollectPostILTrimInformation" AssemblyFile="$(_TaskAssemblyName)" />
22+
<UsingTask Runtime="$(_TaskRuntime)" TaskName="Xamarin.MacDev.Tasks.CollectUnresolvedNativeSymbols" AssemblyFile="$(_TaskAssemblyName)" />
23+
<UsingTask Runtime="$(_TaskRuntime)" TaskName="Xamarin.MacDev.Tasks.ComputeNativeAOTSurvivingNativeSymbols" AssemblyFile="$(_TaskAssemblyName)" />
2124
<UsingTask Runtime="$(_TaskRuntime)" TaskName="Xamarin.MacDev.Tasks.FindAotCompiler" AssemblyFile="$(_TaskAssemblyName)" />
2225
<UsingTask Runtime="$(_TaskRuntime)" TaskName="Xamarin.MacDev.Tasks.GetFullPaths" AssemblyFile="$(_TaskAssemblyName)" />
2326
<UsingTask Runtime="$(_TaskRuntime)" TaskName="Xamarin.MacDev.Tasks.InstallNameTool" AssemblyFile="$(_TaskAssemblyName)" />
2427
<UsingTask Runtime="$(_TaskRuntime)" TaskName="Xamarin.MacDev.Tasks.LinkNativeCode" AssemblyFile="$(_TaskAssemblyName)" />
2528
<UsingTask Runtime="$(_TaskRuntime)" TaskName="Xamarin.MacDev.Tasks.MergeAppBundles" AssemblyFile="$(_TaskAssemblyName)" />
2629
<UsingTask Runtime="$(_TaskRuntime)" TaskName="Xamarin.MacDev.Tasks.MobileILStrip" AssemblyFile="$(_TaskAssemblyName)" />
2730
<UsingTask Runtime="$(_TaskRuntime)" TaskName="Xamarin.MacDev.Tasks.MacDevMessage" AssemblyFile="$(_TaskAssemblyName)" />
31+
<UsingTask Runtime="$(_TaskRuntime)" TaskName="Xamarin.MacDev.Tasks.PostTrimmingProcessing" AssemblyFile="$(_TaskAssemblyName)" />
2832

2933
<!-- Project types and how do we distinguish between them
3034
@@ -612,7 +616,9 @@
612616
@(_BundlerEnvironmentVariables -> 'EnvironmentVariable=Overwrite=%(Overwrite)|%(Identity)=%(Value)')
613617
@(_XamarinFrameworkAssemblies -> 'FrameworkAssembly=%(Filename)')
614618
Interpreter=$(MtouchInterpreter)
619+
InlineDlfcnMethods=$(InlineDlfcnMethods)
615620
IntermediateLinkDir=$(IntermediateLinkDir)
621+
IntermediateOutputPath=$(DeviceSpecificIntermediateOutputPath)
616622
InvariantGlobalization=$(InvariantGlobalization)
617623
HybridGlobalization=$(HybridGlobalization)
618624
ItemsDirectory=$(_LinkerItemsDirectory)
@@ -750,6 +756,7 @@
750756
<_TrimmerCustomSteps Include="$(_AdditionalTaskAssembly)" BeforeStep="MarkStep" Type="MonoTouch.Tuner.CoreTypeMapStep" />
751757
<_TrimmerCustomSteps Include="$(_AdditionalTaskAssembly)" BeforeStep="MarkStep" Type="MonoTouch.Tuner.ProcessExportedFields" />
752758
<_TrimmerCustomSteps Include="$(_AdditionalTaskAssembly)" BeforeStep="MarkStep" Type="Xamarin.Linker.PreserveProtocolsStep" Condition="'$(_AreAnyAssembliesTrimmed)' == 'true' And '$(_UseDynamicDependenciesForProtocolPreservation)' == 'true'" />
759+
<_TrimmerCustomSteps Include="$(_AdditionalTaskAssembly)" BeforeStep="MarkStep" Condition="'$(InlineDlfcnMethods)' != ''" Type="Xamarin.Linker.Steps.InlineDlfcnMethodsStep" />
753760
<!-- The final decision to remove/keep the dynamic registrar must be done before the linking step -->
754761
<_TrimmerCustomSteps Include="$(_AdditionalTaskAssembly)" BeforeStep="MarkStep" Type="MonoTouch.Tuner.RegistrarRemovalTrackingStep" />
755762
<!-- TODO: these steps should probably run after mark. -->
@@ -1608,6 +1615,87 @@
16081615
</ItemGroup>
16091616
</Target>
16101617

1618+
<Target Name="_ComputeTrimmedAssemblies"
1619+
DependsOnTargets="_LoadLinkerOutput"
1620+
>
1621+
<ItemGroup>
1622+
<_TrimmedAssembly Include="$(IntermediateLinkDir)*.dll" />
1623+
</ItemGroup>
1624+
</Target>
1625+
1626+
<Target Name="_ComputePostTrimmingPaths">
1627+
<PropertyGroup>
1628+
<_ILTrimSurvivingNativeSymbolsFile>$(DeviceSpecificIntermediateOutputPath)inlined-dlfcn\iltrim-surviving-native-symbols.txt</_ILTrimSurvivingNativeSymbolsFile>
1629+
<_NativeAOTUnresolvedSymbolsFile>$(DeviceSpecificIntermediateOutputPath)nativeaot-unresolved-symbols.txt</_NativeAOTUnresolvedSymbolsFile>
1630+
<_NativeAOTSurvivingNativeSymbolsFile>$(DeviceSpecificIntermediateOutputPath)nativeaot-surviving-native-symbols.txt</_NativeAOTSurvivingNativeSymbolsFile>
1631+
</PropertyGroup>
1632+
</Target>
1633+
1634+
<!-- See docs/code/native-symbols.md for an overview of native symbol handling. -->
1635+
<Target Name="_CollectPostILTrimInformation"
1636+
Condition="'$(InlineDlfcnMethods)' != ''"
1637+
DependsOnTargets="_ComputeTrimmedAssemblies;_ComputePostTrimmingPaths"
1638+
Inputs="@(_TrimmedAssembly)"
1639+
Outputs="$(_ILTrimSurvivingNativeSymbolsFile)"
1640+
>
1641+
<CollectPostILTrimInformation
1642+
TrimmedAssemblies="@(_TrimmedAssembly)"
1643+
SurvivingNativeSymbolsFile="$(_ILTrimSurvivingNativeSymbolsFile)"
1644+
CacheDirectory="$(DeviceSpecificIntermediateOutputPath)posttrim-info\cache"
1645+
/>
1646+
<ReadLinesFromFile File="$(_ILTrimSurvivingNativeSymbolsFile)">
1647+
<Output TaskParameter="Lines" ItemName="_AllExecutableSymbols" />
1648+
</ReadLinesFromFile>
1649+
<ItemGroup>
1650+
<FileWrites Include="$(_ILTrimSurvivingNativeSymbolsFile)" />
1651+
</ItemGroup>
1652+
</Target>
1653+
1654+
<!-- See docs/code/native-symbols.md for an overview of native symbol handling. -->
1655+
<Target Name="_PostTrimmingProcessing"
1656+
DependsOnTargets="_CollectPostILTrimInformation;_CollectPostNativeAOTTrimInformation;_ComputeTargetArchitectures;_ComputePostTrimmingPaths"
1657+
>
1658+
<ItemGroup>
1659+
<_SurvivingNativeSymbolsFile Include="$(_ILTrimSurvivingNativeSymbolsFile)" Condition="Exists('$(_ILTrimSurvivingNativeSymbolsFile)')" />
1660+
<_SurvivingNativeSymbolsFile Include="$(_NativeAOTSurvivingNativeSymbolsFile)" Condition="Exists('$(_NativeAOTSurvivingNativeSymbolsFile)')" />
1661+
</ItemGroup>
1662+
<PostTrimmingProcessing
1663+
Architecture="$(TargetArchitectures)"
1664+
OutputDirectory="$(DeviceSpecificIntermediateOutputPath)inlined-dlfcn"
1665+
ReferenceNativeSymbol="@(ReferenceNativeSymbol)"
1666+
SurvivingNativeSymbolsFiles="@(_SurvivingNativeSymbolsFile)"
1667+
>
1668+
<Output TaskParameter="NativeSourceFiles" ItemName="_PostTrimmingSourceFiles" />
1669+
</PostTrimmingProcessing>
1670+
1671+
<ItemGroup>
1672+
<FileWrites Include="@(_PostTrimmingSourceFiles)" />
1673+
<_PostTrimmingSourceFiles>
1674+
<OutputFile>$(DeviceSpecificIntermediateOutputPath)posttrim-info-compiled/%(Filename).o</OutputFile>
1675+
</_PostTrimmingSourceFiles>
1676+
</ItemGroup>
1677+
1678+
<CompileNativeCode
1679+
SessionId="$(BuildSessionId)"
1680+
Condition="'$(IsMacEnabled)' == 'true'"
1681+
CompileInfo="@(_PostTrimmingSourceFiles)"
1682+
DotNetRoot="$(_DotNetRoot)"
1683+
MinimumOSVersion="$(_MinimumOSVersion)"
1684+
IncludeDirectories="@(_XamarinMainIncludeDirectory)"
1685+
SdkDevPath="$(_SdkDevPath)"
1686+
SdkIsSimulator="$(_SdkIsSimulator)"
1687+
SdkRoot="$(_SdkRoot)"
1688+
TargetFrameworkMoniker="$(_ComputedTargetFrameworkMoniker)"
1689+
>
1690+
</CompileNativeCode>
1691+
1692+
<ItemGroup>
1693+
<_CompiledPostTrimmingFiles Include="@(_PostTrimmingSourceFiles -> '%(OutputFile)')" />
1694+
<_NativeExecutableObjectFiles Include="@(_CompiledPostTrimmingFiles)" />
1695+
<FileWrites Include="@(_CompiledPostTrimmingFiles)" />
1696+
</ItemGroup>
1697+
</Target>
1698+
16111699
<PropertyGroup>
16121700
<_CompileNativeExecutableDependsOn>
16131701
$(_CompileNativeExecutableDependsOn);
@@ -1664,13 +1752,33 @@
16641752
_ReadAppManifest;
16651753
_WriteAppManifest;
16661754
_CompileNativeExecutable;
1755+
_PostTrimmingProcessing;
16671756
_ReidentifyDynamicLibraries;
16681757
_AddSwiftLinkerFlags;
16691758
_ComputeLinkNativeExecutableInputs;
16701759
_ForceLinkNativeExecutable;
16711760
</_LinkNativeExecutableDependsOn>
16721761
</PropertyGroup>
16731762

1763+
<!-- See docs/code/native-symbols.md for an overview of native symbol handling. -->
1764+
<Target Name="_CollectPostNativeAOTTrimInformation"
1765+
Condition="'$(_UseNativeAot)' == 'true'"
1766+
DependsOnTargets="_ComputePostTrimmingPaths;_CompileNativeExecutable"
1767+
Inputs="$(NativeObject)"
1768+
Outputs="$(_NativeAOTSurvivingNativeSymbolsFile)"
1769+
>
1770+
<CollectUnresolvedNativeSymbols
1771+
SessionId="$(BuildSessionId)"
1772+
StaticLibrary="$(NativeObject)"
1773+
OutputFile="$(_NativeAOTUnresolvedSymbolsFile)"
1774+
/>
1775+
<ComputeNativeAOTSurvivingNativeSymbols
1776+
SessionId="$(BuildSessionId)"
1777+
UnresolvedSymbolsFile="$(_NativeAOTUnresolvedSymbolsFile)"
1778+
SurvivingNativeSymbolsFile="$(_NativeAOTSurvivingNativeSymbolsFile)"
1779+
/>
1780+
</Target>
1781+
16741782
<Target Name="_AddSwiftLinkerFlags" DependsOnTargets="_DetectSdkLocations">
16751783
<PropertyGroup>
16761784
<!-- Runtime now requires swift https://github.com/dotnet/runtime/commit/2c70e36356e8dfb50e6b32c8b7c9ce1a8e9f1331 -->

0 commit comments

Comments
 (0)