Copying collector: Store most tracing info inline in the object header

[fitzgen]

We have a number of available bits inside the VMGcHeader. This commit uses them to bitpack the following type:

pub enum InlineTraceInfo {
    /// The trace info is not encoded inline. Look it up from `TraceInfos`.
    OutOfLine,

    /// Inline trace info for an array type.
    Array {
        /// Whether the array's elements are GC references that need tracing.
        elems_are_gc_refs: bool,
    },

    /// Inline trace info for a struct, exception, or externref type.
    Struct {
        /// A bitmap where the `i`th bit is set iff the `i`th `u32` in the
        /// object's data (after the header) is a `VMGcRef` that needs
        /// tracing. Only the lower 23 bits are meaningful.
        gc_ref_bitmap: u32,
    },
}

This allows us to encode the majority of objects' GC edges inline in the GC object header, avoiding the need to perform expensive hash map lookups in the common case.

This results in a 7-8% speed up on splay.wasm in Sightglass:

execution :: cycles :: benchmarks/splay/splay.wasm

  Δ = 5858240.88 ± 445613.91 (confidence = 99%)

  inline-trace-info.dylib (-Ccollector=copying -Wgc,function-references) is 1.07x to 1.08x faster than main.dylib (-Ccollector=copying -Wgc,function-references)!

  [77193983 79604574.70 88386948] inline-trace-info.dylib (-Ccollector=copying -Wgc,function-references)
  [83057364 85462815.58 92545612] main.dylib (-Ccollector=copying -Wgc,function-references)

[github-actions]

Subscribe to Label Action

cc @fitzgen

Details

This issue or pull request has been labeled: "wasmtime:api", "wasmtime:ref-types"

Thus the following users have been cc'd because of the following labels:

• fitzgen: wasmtime:ref-types

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.