[wasm-split] Split module elements early#8688
Conversation
Before WebAssembly#8443, we scanned `ref.func`s in global initializers early in `indirectReferencesToSecondaryFunctions` and created trampolines for them and replaced `ref.func $func`s with `ref.func $trampoline_func` if `func` was set to move to a secondary module. But in case the global containing `ref.func $trampoline_func` also ends up moving to the same secondary module, creating trampoline and using it was not necessary, because the global can simply use `ref.func $func` because `func` is in the same secondary module. To fix this, in WebAssembly#8443, we postponed creating trampolines for `ref.func`s in global initializers until `shareImportableItems`. This had a problem, because we end up creating new trampolines late in `shareImportableItems`. But trampolines were designed to go through `indirectCallsToSecondaryFunctions` and `setupTablePatching`, so those late trampolines were invalid, like ```wast (func $trampoline_foo (call $foo) ) ``` when `foo` was in a secondary module. This was supposed to be converted to a `call_indirect` in `indirectCallsToSecondaryFunctions` and the table elements were supposed to set up in `setupTablePatching`. --- This moves `shareImportableItems` before `indirectReferencesToSecondaryFunctions`. Turns out, except for the active table and its base global, we can do all splitting before we make most of the changes related to splitting. This also simplifies `shareImportableItems` because we can now delete code handling the consequences of various transformations. Because the active table and its base global may not be registered as "used" in `shareImportableItems` before `setupTablePatching`, we make sure they are correctly shared with secondary modules in `setupTablePatching`. `active-table-base-global-used-elsewhere.wast` has some edge cases that I feel easy to miss, because now we specially handle the active table and the base global in `setupTablePatching`. `global-reffunc.wast` is the same but just renamed expectation rewritten. `global-reffunc2.wast` is the failing case simplified from WebAssembly#8510. Other test changes are just the changes in the creation order of module elements and not meaningful. Replaces WebAssembly#8542 and fixes WebAssembly#8510.
| return exports; | ||
| } | ||
|
|
||
| void ModuleSplitter::makeImportExport(Importable& primaryItem, |
There was a problem hiding this comment.
This is extracted from the same lambda method that used to be in shareImportableItems
| } | ||
|
|
||
| std::unordered_map<std::pair<ExternalKind, Name>, Name> | ||
| ModuleSplitter::initExportedPrimaryItems(const Module& primary) { |
There was a problem hiding this comment.
This is necessary to keep track of item exports because we call makeImportExport from multiple functions
| // Generate the call and the function. We generate a direct call here, but | ||
| // this will be converted to a call_indirect in | ||
| // indirectCallsToSecondaryFunctions. |
There was a problem hiding this comment.
Drive-by comment improvement
| void ModuleSplitter::exportImportCalledPrimaryFunctions() { | ||
| // Find primary functions called/referred to from the secondary modules. | ||
| using CalledPrimaryToModules = std::map<Name, std::set<Module*>>; | ||
| struct CallCollector : PostWalker<CallCollector> { |
There was a problem hiding this comment.
This was just taken out of ParallelFunctionAnalysis to be reused
| @@ -0,0 +1,52 @@ | |||
| ;; RUN: wasm-split %s -all -g -o1 %t.1.wasm -o2 %t.2.wasm --split-funcs=split1,split2 | |||
| @@ -0,0 +1,63 @@ | |||
| ;; NOTE: Assertions have been generated by update_lit_checks.py --all-items and should not be edited. | |||
There was a problem hiding this comment.
These are some edge cases that I feel easy to miss, because now we specially handle the active table and the base global in setupTablePatching
| @@ -0,0 +1,45 @@ | |||
| ;; RUN: wasm-split %s -all -g -o1 %t.1.wasm -o2 %t.2.wasm --keep-funcs=keep | |||
There was a problem hiding this comment.
This was renamed from global-funcref.wast (but Github fails to recognize it) and expectation rewritten
| } | ||
|
|
||
| CallCollector collector(primaryFuncs, calledPrimaryToModules); | ||
| collector.walkModuleCode(secondary); |
There was a problem hiding this comment.
Previously, we handled ref.funcs in global initializers later in shareImportableItems in a little hacky way:
binaryen/src/ir/module-splitting.cpp
Lines 851 to 886 in d9fd5da
Now we move globals early, we just can walk all module code normally here, and the hack above is not necessary anymore.
Before #8443, we scanned
ref.funcs in global initializers early inindirectReferencesToSecondaryFunctionsand created trampolines for them and replacedref.func $funcs withref.func $trampoline_funciffuncwas set to move to a secondary module. But in case the global containingref.func $trampoline_funcalso ends up moving to the same secondary module, creating trampoline and using it was not necessary, because the global can simply useref.func $funcbecausefuncis in the same secondary module.To fix this, in #8443, we postponed creating trampolines for
ref.funcs in global initializers untilshareImportableItems. This had a problem, because we end up creating new trampolines late inshareImportableItems. But trampolines were designed to go throughindirectCallsToSecondaryFunctionsandsetupTablePatching, so those late trampolines were invalid, likewhen
foowas in a secondary module. This was supposed to be converted to acall_indirectinindirectCallsToSecondaryFunctionsand the table elements were supposed to set up insetupTablePatching.This moves
shareImportableItemsbeforeindirectReferencesToSecondaryFunctions. Turns out, except for the active table and its base global, we can do all splitting before we make most of the changes related to splitting. This also simplifiesshareImportableItemsbecause we can now delete code handling the consequences of various transformations.Because the active table and its base global may not be registered as "used" in
shareImportableItemsbeforesetupTablePatching, we make sure they are correctly shared with secondary modules insetupTablePatching.active-table-base-global-used-elsewhere.wasthas some edge cases that I feel easy to miss, because now we specially handle the active table and the base global insetupTablePatching.global-reffunc.wastis the same but just renamed expectation rewritten.global-reffunc2.wastis the failing case simplified from #8510. Other test changes are just the changes in the creation order of module elements and not meaningful.Replaces #8542 and fixes #8510.