Release to OCI registry

[andreaTP]

This should enable tools such as wkg to consume wasm-tools.

I've only been exploring this area for a few days, so please point out anything I may have overlooked.

[alexcrichton]

Thanks! If you don't mind though I'd like to get some mroe background on this and ask some questions as well. I'm not familiar what's being added here myself, and I've for example never used oras, cosign, or OCI artifacts with wkg much which may explain a bit if these questions seem ignorant.

• Could you expand a bit more on the motivation here? This looks like it's publishing wasm-tools.wasm, and you say for use within wkg, but could you expand on that? For example is the intention to do wkg install some:thing/wasm-tools or something like that?
• This is publishing the wasm32-wasip1 binary, where for something with wkg/registries/etc I'd expect a component to be published rather than a core module. Is it intentional that wasm32-wasip1 is used? This touches on my previous question -- I'm not sure of the use case here so I'm not sure what the intention is.
• Currently this repo disallows all third-party github actions unless they're allow-listed, so as-is this won't pass CI because oras-project/setup-oras and sigstore/cosign-installer aren't allowed actions. I can add them, but is it possible to do what you're doing here with bulit-in tools like the gh CLI perhaps?

[andreaTP]

Thanks for getting back, happy to have this conversation now to validate (or not) my plan before I streamline inlay.

The bigger problem:

Consuming wasm payloads doesn't have a standard. Or rather, consuming wasm as a final user is fine when it ships inside the host package (npm, jars, etc.), but consuming wasm as an intermediate artifact seems mostly left as an exercise for the reader.

We do it a lot across multiple projects, and I've ended up with a variety of mechanisms depending on the circumstances:

• For wasm-tools we download the tar.gz from the GitHub release on the fly and use it
• When the wasm payload is small enough we simply commit it in git
• When it's too big we rely on on-the-fly builds
• When the payload is too big and the compilation takes too long we use GitHub releases or similar mechanisms

All of those have downsides: they clutter git history, make reproducibility harder, and don't integrate with version control tooling (e.g. dependabot won't work). The worst part is that having a different approach per project is starting to cause friction.

I looked around for alternatives and the most promising and mature approach seems to be what they're doing at Microsoft and wasmCloud (for example), which is using OCI Artifacts.

Using OCI artifacts has a number of benefits:

• They are largely available
• You can pin versions using SHAs
• They can sign artifacts to improve supply chain security
• They handle large blobs of data well

This is where my effort on inlay started. In endive we are not yet ready for components, but I thought I could use the same distribution mechanism without requiring the CM parts of the tooling. I took wkg as it provides the building blocks for a cross-ecosystem compatible lock file, wrapped it with the standard tooling for dealing with Sigstore artifacts (the oras/cosign dance), and made it available in Java.

This mechanism has the advantage that it can be applied in all of my repositories consuming wasm, overcoming the shortcomings of the bespoke solutions mentioned above. wasm-tools happened to be the first one, and I think this is a good conversation to have.

Regarding wasip1: you're right, to follow the approach 100% we should publish a component. I'm not ready for it, the CM implementation is ongoing , and I'd love to use a solution that is not gated by the CM requirement. Uploading the wasip1 artifact would let me immediately consume it in the endive build with minimal changes.

Regarding oras and cosign: these are the standard tools used to deal with OCI artifacts and signing (e.g. https://www.systemshardening.com/articles/wasm/wasm-oci-signing/, but you can easily find more resources). The gh CLI doesn't support pushing OCI artifacts, so it's not an alternative here. The actions used (oras-project/setup-oras and sigstore/cosign-installer) are the official ones maintained by the upstream projects; I think allowlisting them is the cleaner path, but happy to explore alternatives.

I hope I've answered all the questions. Maybe we can try to tag wkg developers as they might have additional thoughts around this(or @ricochet). Let me know what you think!

[alexcrichton]

Ok that all sounds reasonable enough, yeah. I'll admit I'm a bit murky on the vision here insofar as it sounds like "push things to OCI and hope everything settles around that", but the cost here in this repo isn't really much and it's just a bit of CI configuration.

Another question, though, it looks like this is going to publish ghcr.io/bytecodealliance/wasm-tools-wasm:$version, but wasm-tools-wasm isn't actually a repo. Are we allowed to do that? I would naively expect that there'd be some sort of restriction that bytecodealliance/wasm-tools would be the only name that could be updated/modified in this workflow.

Additionally, do you know how the wasip1->wasip2/3 transition will work in the future? The name here doesn't have target information or anything like that. Is this something where we'd assemble an OCI artifact with both the core wasm and component? Or would the releases just switch to being a component one day?

[andreaTP]

I'll give a full answer and follow up early next week.

Meanwhile I'd like to react to this:

push things to OCI and hope everything settles around that

I don't disagree, at the same time I don't know if I have any better option as of today.
We identified a gap and any guidance on improving this aspect more structurally is very welcome.

[alexcrichton]

Oh sure yeah, and to be clear I'm not opposed to this nor do I have a better idea. I mostly want to ideally make sure this repo isn't boxed into a corner where it can't do anything else by accident. If there's a natural way to evolve this that's totally fine