Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions doc/concept/layer/hang.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,9 @@ Unfortunately, the raw codec bitstream lacks timestamp information so we need so
Containers can support additional features and configuration.
For example, `CMAF` specifies a timescale instead of hard-coding it to microseconds like `legacy`.

The `kind` field selects the framing and new kinds can be added over time.
A consumer ignores any rendition whose `kind` it doesn't recognize, keeping the rest of the catalog usable, and carries the unrecognized entry through untouched when it republishes the catalog.

### Legacy

This is a lightweight container with no frills attached.
Expand Down
3 changes: 3 additions & 0 deletions doc/lib/rs/env/native.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,9 +185,12 @@ Check the `container` field for each rendition:

- **`legacy`** — Each frame is a varint timestamp (microseconds) followed by the codec payload. This is the common case.
- **`cmaf`** — Each frame is a `moof` + `mdat` pair (fragmented MP4). Used for HLS compatibility.
- **`loc`** — Low Overhead Container: each frame is a small property block followed by the codec payload.

`OrderedConsumer` decodes legacy timestamps for you automatically.

Anything else decodes as `Container::Unknown`, which preserves the original JSON so you can republish the catalog unchanged. Skip those renditions: their frames can't be parsed.

Comment thread
kixelated marked this conversation as resolved.
## Next Steps

- [hang format](/concept/layer/hang) — Catalog schema and container details
Expand Down
64 changes: 64 additions & 0 deletions js/hang/src/catalog/container.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
import { expect, test } from "bun:test";
import { type Container, ContainerSchema, containerSupported } from "./container.ts";
import { RootSchema } from "./root.ts";

test("known containers round-trip", () => {
const known: Container[] = [{ kind: "legacy" }, { kind: "cmaf", init: "AAEC" }, { kind: "loc" }];
for (const container of known) {
const parsed = ContainerSchema.parse(container);
expect(parsed).toEqual(container);
expect(containerSupported(parsed)).toBe(true);
}
});

test("unknown container is preserved instead of throwing", () => {
const container = { kind: "future", extra: { nested: [1, 2] }, flag: true };
const parsed = ContainerSchema.parse(container);
// Tagged with a literal `kind` so the union stays discriminated; `raw` is the original.
expect(parsed).toEqual({ kind: "unknown", raw: container });
expect(containerSupported(parsed)).toBe(false);
});

test("catalog with an unknown container keeps its other renditions", () => {
const catalog = {
video: {
renditions: {
future: {
codec: "avc1.64001f",
container: { kind: "future", magic: 7 },
},
legacy: {
codec: "avc1.64001f",
codedWidth: 1280,
codedHeight: 720,
container: { kind: "legacy" },
},
},
},
};

const parsed = RootSchema.parse(catalog);
if (!parsed.video || !("renditions" in parsed.video)) throw new Error("missing video section");

const known = parsed.video.renditions.legacy;
expect(known?.container.kind).toBe("legacy");
expect(Number(known?.codedWidth)).toBe(1280);

// The unknown rendition keeps its original JSON verbatim under `raw`.
expect(parsed.video.renditions.future?.container).toEqual({
kind: "unknown",
raw: { kind: "future", magic: 7 },
});
});

test("a malformed known container errors instead of degrading to passthrough", () => {
// `cmaf` without `init` fails its own schema. It must NOT fall through to the passthrough
// arm, which would still report kind "cmaf" and hand decoders an undefined init segment.
expect(() => ContainerSchema.parse({ kind: "cmaf" })).toThrow();

// A genuinely unrecognized kind still parses, so one future rendition can't fail the catalog.
expect(ContainerSchema.parse({ kind: "future", magic: 7 })).toEqual({
kind: "unknown",
raw: { kind: "future", magic: 7 },
});
});
69 changes: 58 additions & 11 deletions js/hang/src/catalog/container.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,32 @@
import * as z from "zod/mini";

/** The container kinds this build knows how to decode. */
const KNOWN_KINDS = ["legacy", "cmaf", "loc"];

/**
* A container this build does not recognize, preserved verbatim.
*
* The original object is kept verbatim under `raw`, so nothing about the rendition is lost and a
* republisher can write it back out unchanged. Such a rendition must be ignored, not decoded.
*
* Recognized kinds are rejected here so they can only ever parse through their own strict
* schema. Without that, a malformed known container (`{"kind":"cmaf"}` with no `init`) would
* fall through to this arm, still report as CMAF, and hand decoders an undefined init segment.
*/
export const UnknownContainerSchema = z.pipe(
z.looseObject({
kind: z.string().check(
z.refine((kind) => !KNOWN_KINDS.includes(kind), {
message: "recognized container kind must match its own schema",
}),
),
}),
// Map to a literal `kind` so {@link Container} stays a discriminated union: a bare
// `kind: string` arm would widen the discriminant and stop `kind === "cmaf"` from
// narrowing. `raw` keeps the original object, including its real `kind`.
z.transform((raw) => ({ kind: "unknown" as const, raw })),
);

/**
* Container format for frame timestamp encoding and frame payload structure.
*
Expand All @@ -9,21 +36,41 @@ import * as z from "zod/mini";
* The init segment (ftyp+moov) is base64-encoded in the catalog.
* - "loc": Low Overhead Container (draft-ietf-moq-loc). Each frame has a small
* property block followed by the codec payload.
*
* Anything else parses as {@link UnknownContainerSchema} instead of throwing, so one rendition
* using a future container does not take down the rest of the catalog.
*/
export const ContainerSchema = z._default(
z.discriminatedUnion("kind", [
// The default hang container
z.object({ kind: z.literal("legacy") }),
// CMAF container with base64-encoded init segment (ftyp+moov).
z.object({
kind: z.literal("cmaf"),
init: z.base64(),
}),
// Low Overhead Container.
z.object({ kind: z.literal("loc") }),
z.union([
z.discriminatedUnion("kind", [
// The default hang container
z.object({ kind: z.literal("legacy") }),
// CMAF container with base64-encoded init segment (ftyp+moov).
z.object({
kind: z.literal("cmaf"),
init: z.base64(),
}),
// Low Overhead Container.
z.object({ kind: z.literal("loc") }),
]),
UnknownContainerSchema,
]),
{ kind: "legacy" },
);

/** The per-frame container format declared in the catalog. */
/**
* The per-frame container format declared in the catalog.
*
* A discriminated union: `container.kind === "cmaf"` narrows and gives you `init`. An
* unrecognized container arrives as `{ kind: "unknown", raw }` rather than widening `kind`,
* so tolerating a future container costs no type safety here.
*/
export type Container = z.infer<typeof ContainerSchema>;

/** The CMAF variant of {@link Container}, carrying the base64 init segment. */
export type CmafContainer = Extract<Container, { kind: "cmaf" }>;

/** Whether a container can be decoded by this build, i.e. its `kind` is recognized. */
export function containerSupported(container: Container): boolean {
return container.kind !== "unknown";
}
7 changes: 7 additions & 0 deletions js/watch/src/audio/decoder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -513,6 +513,13 @@ export class Decoder {
}

async function supported(config: Catalog.AudioConfig): Promise<boolean> {
if (!Catalog.containerSupported(config.container)) {
// `kind` is the literal "unknown" tag; the container the publisher actually named is in `raw`.
const kind = config.container.kind === "unknown" ? config.container.raw.kind : config.container.kind;
console.warn(`audio: ignoring rendition with unknown container: ${kind}`);
return false;
}

// Opus only runs at its native rates, so a catalog advertising anything else is wrong and Safari
// refuses to decode it. Warn rather than reject: Chrome and Firefox ignore the configured rate and
// play these streams fine, so rejecting would silence them for a publisher they handle today.
Expand Down
12 changes: 10 additions & 2 deletions js/watch/src/video/decoder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -413,9 +413,10 @@ class DecoderTrack {
}

#runCmaf(effect: Effect, sub: Moq.Track.Subscriber, decoder: VideoDecoder): void {
if (this.config.container.kind !== "cmaf") return;
const container = this.config.container;
if (container.kind !== "cmaf") return;

const initSegment = base64ToBytes(this.config.container.init);
const initSegment = base64ToBytes(container.init);
const init = Container.Cmaf.decodeInitSegment(initSegment);
const description = this.config.description ? Util.Hex.toBytes(this.config.description) : init.description;

Expand Down Expand Up @@ -555,6 +556,13 @@ class DecoderTrack {
}

async function supported(config: Catalog.VideoConfig): Promise<boolean> {
if (!Catalog.containerSupported(config.container)) {
// `kind` is the literal "unknown" tag; the container the publisher actually named is in `raw`.
const kind = config.container.kind === "unknown" ? config.container.raw.kind : config.container.kind;
console.warn(`video: ignoring rendition with unknown container: ${kind}`);
return false;
}

let description: Uint8Array | undefined;
if (config.description) {
description = Util.Hex.toBytes(config.description);
Expand Down
2 changes: 1 addition & 1 deletion rs/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Layered roughly transport -> container/format -> media -> apps/bindings.
- `moq-hls` (lib): HLS / LL-HLS gateway (import + export, playlists + fMP4 via `moq-mux`).
- `moq-bench` (bin): relay load generator. `JoinSet`-spawned staggered connections, rand sampling.
- `moq-boy` (bin): crowd-controlled Game Boy emulator publisher (blocking emulator thread + async monitor tasks).
- `moq-token` (lib) / `moq-token` (bin from the `moq-token-cli` crate): JWT auth. `Claims`, `Algorithm`, `KeyType` (EC/RSA/OCT/OKP), JWKS. CLI does generate/sign/verify.
- `moq-token` (lib) / `moq-token` (bin from the `moq-token-cli` crate): JWT auth. `Claims`, `Algorithm`, `KeyMaterial` (EC/RSA/OCT/OKP), JWKS. CLI does generate/sign/verify.

**Bindings**

Expand Down
Loading
Loading