OCI: Define MediaTypes and decompression logic for nupkg/zip

[JustinGrote]

Summary of the new feature / enhancement

As a publisher, I want to publish PowerShell Modules packaged in nupkg as ORAS artifacts to consume in their native zip format rather than having to convert them to tar+gzip.

If I am hosting a large registry (e.g. https://pwsh.gallery), I can reference the same storage/packages for both nugetv2/v3 and oras packages, otherwise I have to hold and maintain duplicate copies/storage of every package.

Proposed technical implementation details (optional)

OCI allows definitions of both a media type and an image type. Rather then be generic, we should define types for both of these and implement them for nupkg/zip:

Nupkg MediaType: application/vnd.dotnet.nupkg.v1+zip

Zip Layer MediaType: application/vnd.oci.image.layer.v1.zip

When pulling packages, apply proper unzip to these layers and then handle the same as the tar+gzip. Potentially support a strict option to only accept items that were strictly published with the dotnet.nupkg mediatype.

This behavior would only occur on the specified image type, so it would not break any compatibility.

Registries could choose to then host powershell modules in either native zip, tar, tar+gzip, or tar+zstd.

This implementation would not require any registry-side changes, as they are designed for this and do not need to know or understand the zip format to work for both publishing and retrieval, unless a registry does some sort of non-standard validation of the artifact.

It may make sense to define these formats or the transfer mechanism in general in a PowerShell RFC so that all OCI-based PowerShell tools have a standard set of instructions to work from.

[JustinGrote]

This is exacerbated by the fact that the current Az uploads to MAR are actually zip blobs (good catch @jborean93), but have the wrong mediatype as tar, so that also needs to be fixed as it currently violates the OCI images spec . Hopefully the ORAS PR can make it easier to get that correct.

[jborean93]

It looks like the current client logic just selects the first layer that contains the digest key so from a backwards compatibility perspective it should be fine to change the layer media type to the correct one.

PSResourceGet/src/code/ContainerRegistryServerAPICalls.cs

Lines 638 to 645 in 9c8bd26

	foreach (JObject item in layers)
	{
	if (item.ContainsKey("digest"))
	{
	digest = item.GetValue("digest").ToString();
	break;
	}
	}

It also looks like the logic for retrieving the psd1 metadata stored on the package also just selects the first layer

PSResourceGet/src/code/ContainerRegistryServerAPICalls.cs

Line 864 in 9c8bd26

JToken annotations = layers[0]["annotations"];

We will have to be careful if more layers are added in the future to ensure that the first layer is always the nupkg until we can be sure that any of the clients with this logic are no longer used or are out of support.

If does look like the logic in the ORAS PR uses the same logic in that it just assumes the first layer is the nupkg without checking the media type

PSResourceGet/src/code/ContainerRegistryServerAPICalls.cs

Lines 336 to 337 in 0d82ebd

	var nupkgLayer = manifest.Layers[0];
	_cmdletPassedIn.WriteVerbose($"Downloading blob for {packageNameLowercase} - {packageVersion} (digest: {nupkgLayer.Digest})");

PSResourceGet/src/code/ContainerRegistryServerAPICalls.cs

Line 610 in 0d82ebd

var annotations = manifest.Layers[0].Annotations;

If the change to media type is going to be made it should be done as early as possible to avoid having a 3rd party client rely on the existing logic in any way.