Compositions documentation by Gawor270 · Pull Request #278 · fishjam-cloud/documentation

Gawor270 · 2026-07-23T13:45:02Z

Closes FCE-2924, FCE-2925 and FCE-3137 by publishing the API specification here.

Documents compositions as a core Fishjam feature, integrated into the existing docs structure instead of a standalone section:

Concepts: explanation/compositions (what compositions are, core concepts, the scene model).
Tutorials: tutorials/compositions (create a composition end to end with curl).
How-to: how-to/compositions/ (write and deploy a template, compose a Fishjam room, drive a template with events, choose inputs and outputs).
API hub (api/reference): a Compositions section covering REST, the WebSocket event stream, authentication, and errors.

New top-level docs section for the hosted composition service: concept overview (disambiguated from the client-side Smelter library), quick start, template authoring with the composition SDK and CLI, room composition, runtime events, an input/output guide, a scene-model explainer, and an API reference. The REST reference renders a sanitized preview OpenAPI spec via Scalar; the auth model and WebSocket event stream are documented in prose.

linear · 2026-07-23T13:45:06Z

FCE-2924

FCE-2925

FCE-3137

Replace the hand-written preview spec with one generated from the service source, copied by scripts/update_api.sh (the source repo is private, so it is cloned on demand instead of being a submodule) and sanitized of internal-only fields. The committed spec comes from the template-workers branch with the oneOf variant-title improvement applied, matching what main will contain after the pending merge. Align the docs with the actual API surface: templated outputs are registered in a single multipart request to POST .../output/{output_id}/template (config + template), not registered first and uploaded after; rescaler takes a single child. Template code samples are now type-checked by the build, with @fishjam-cloud/composition linked from the js-server-sdk submodule.

CI's SUBMODULES_TOKEN covers the private source repo (same as the fishjam and room-manager submodules), so clone-on-demand is not needed. The repo has no semver release tags yet, so the submodule is checked out at a branch instead of the latest tag.

Copilot

Pull request overview

Adds a new beta Smelter Cloud documentation section to the Docusaurus site, including concept docs, how-to guides, and a published OpenAPI/Scalar REST API reference.

Changes:

Introduces new docs/smelter/ docs tree (overview, quick start, scenes, how-tos, API reference) and sidebar categories.
Publishes Smelter Cloud REST API docs via Scalar at /api/smelter-cloud, backed by a committed OpenAPI JSON spec.
Updates tooling to fetch/copy the Smelter Cloud spec from a new submodule and links the local @fishjam-cloud/composition package for type-checkable examples.

Reviewed changes

Copilot reviewed 16 out of 18 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
yarn.lock	Adds workspace link entry for `@fishjam-cloud/composition`.
static/api/smelter-cloud-openapi.json	Adds committed Smelter Cloud OpenAPI 3.1 JSON spec for publishing.
spelling.txt	Adds `rtmp` to spellcheck allowlist.
scripts/update_api.sh	Adds branch-based submodule checkout and JSON spec sanitization/copy step.
package.json	Adds linked dependency on `@fishjam-cloud/composition` for examples/typechecking.
docusaurus.config.ts	Adds a Scalar instance to render the Smelter Cloud REST API reference page.
docs/smelter/what-is-smelter-cloud.mdx	Adds Smelter Cloud concept overview and navigation pointers.
docs/smelter/scenes.mdx	Documents video/audio scene model and live updates.
docs/smelter/quick-start.mdx	Adds end-to-end tutorial for creating/registering inputs/outputs and publishing.
docs/smelter/how-to/write-and-deploy-a-template.mdx	Adds template authoring/build/deploy guide including multipart upload endpoint.
docs/smelter/how-to/inputs-and-outputs.mdx	Summarizes supported input/output protocols and key fields.
docs/smelter/how-to/drive-a-template-with-events.mdx	Documents event delivery to templates via `/event` endpoint and `eventBus`.
docs/smelter/how-to/compose-a-fishjam-room.mdx	Documents linking a Fishjam room and templating room-driven layouts.
docs/smelter/how-to/category.json	Adds How-to Guides generated index under Smelter Cloud section.
docs/smelter/api-reference.mdx	Adds entry-point page for REST API, WebSocket auth, and error format.
docs/smelter/category.json	Adds top-level “Smelter Cloud” sidebar category at position 1.
.gitmodules	Adds `api/smelter-cloud` submodule entry.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Guard the jq dependency in update_api.sh, use optional chaining on peer metadata in the template example, and fix quick-start wording: a whip_client output publishes to a WHIP endpoint; playback is WHEP on the receiving service.

The reference page now lives at /api/smelter-cloud next to the other API docs, the Scalar page moves to /api/smelter-cloud/rest (mirroring the /api + /api/rest pair of the server API), and the API hub page links both. Smelter section links updated.

Replace the navbar dropdown with a single API Reference link landing on the /api index, where the Composition API (Smelter Cloud) sits as a subsection next to the SDK references and the hub page. The prose page is renamed Composition API to match the spec title shown by Scalar.

Split the single autogenerated sidebar in two: the Docs sidebar drops the api category, and a new apiSidebar (docs and versioned) holds the API index, the SDK references, and the Composition API page. Landing on API Reference now shows only API navigation instead of the full docs tree.

Dissolve the standalone Smelter Cloud section: the concept page (with the scene model) lives in explanation/, the quick start in tutorials/, and the guides in how-to/compositions/. All content now presents compositions as a Fishjam feature named Compositions, without beta notes. The API Reference returns to its earlier shape. The Composition API is published like the other APIs: a Scalar page at /api/compositions, a second document on /api/rest, a navbar dropdown listing both APIs, and a Compositions section (REST, WebSocket, auth, errors) on the API hub page. A presentation overlay (scripts/composition_openapi_overlay.jq) adds operation summaries, tag groups, request examples, and titles for anonymous oneOf variants. The submodule and spec are renamed to api/composition and composition-openapi.json. The room guide now documents the real flow: a single Fishjam track_forwardings call forwards the room, links the composition, and registers inputs automatically. The template guide starts from a simple static example, and the room-aware hooks live in the room guide.

Copilot

Pull request overview

Copilot reviewed 15 out of 17 changed files in this pull request and generated 4 comments.

The submodule was moved to api/composition earlier but kept its old api/smelter-cloud section name in .gitmodules.

Bump the api/composition submodule to the template-workers merge of fishjam-cloud/foundry#133 and refresh the published spec. The source spec now carries all presentation metadata itself: operation summaries, human tag groups with a new Room forwarding tag, request body examples for every operation, titled union variants, and tagged unions whose examples validate against their schemas.

The overlay existed to add summaries, tags, examples, and variant titles the generated spec lacked, and to strip internal fields and relative links. All of it now lives in the source generator (fishjam-cloud/foundry#133), and the overlay verifiably produces byte-identical output to a plain copy, so update_api.sh copies the spec directly and no longer needs jq. Also quote the paths in checkout_submodule_branch.

Copilot

Pull request overview

Copilot reviewed 14 out of 16 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (7)

docs/how-to/compositions/compose-a-fishjam-room.mdx:114

This curl example uses a placeholder Authorization: ****** header, which is not a valid HTTP Authorization value and omits the required Bearer prefix for the Management Token.

  -H "Authorization: Bearer $TOKEN" \

docusaurus.config.ts:390

The /api/rest Scalar config declares a sources entry for the Composition API, but the page-level authentication.preferredSecurityScheme is set to management_token, which does not exist in composition-openapi.json (it uses BearerAuth). This can break or misconfigure the auth UI when switching to the Composition source, and it’s also redundant given the dedicated /api/compositions Scalar page below.

              title: "Composition API",
              slug: "composition",
              url: "/docs/api/composition-openapi.json",
            },
          ],

docs/how-to/compositions/compose-a-fishjam-room.mdx:97

This curl example uses a placeholder Authorization: ****** header, which is not a valid HTTP Authorization value and omits the required Bearer prefix for the Management Token.

  -H "Authorization: Bearer $TOKEN" \

docs/how-to/compositions/compose-a-fishjam-room.mdx:125

This curl example uses a placeholder Authorization: ****** header, which is not a valid HTTP Authorization value and omits the required Bearer prefix for the Management Token.

  -H "Authorization: Bearer $TOKEN"

docs/how-to/compositions/compose-a-fishjam-room.mdx:136

This curl example uses a placeholder Authorization: ****** header, which is not a valid HTTP Authorization value and omits the required Bearer prefix for the Management Token.

  -H "Authorization: Bearer $TOKEN"

docs/api/reference.md:93

The Composition authentication table contains broken/placeholder header values (e.g. missing Bearer, missing closing backticks) which will render incorrectly and gives an invalid auth example.

| Control-plane calls (composition, inputs, outputs, renderers, events, room) | `Authorization: Bearer <token>`: your Fishjam **Management Token**, the same token used across Fishjam.                                                 |
| Publishing to an input (`/whip/{input_id}`)                                 | The **input's** own bearer token, returned when you register a `whip_server` input (or the one you supplied). This is distinct from your account token. |
| Playing back an output (`/whep/{output_id}`)                                | Optional. Public playback is allowed; add a Bearer token only if you want to require one.                                                               |

docs/how-to/compositions/compose-a-fishjam-room.mdx:88

This curl example uses a placeholder Authorization: ****** header, which is not a valid HTTP Authorization value and omits the required Bearer prefix for the Management Token.

This issue also appears in the following locations of the same file:

line 97
line 114
line 125
line 136

  -H "Authorization: Bearer $TOKEN" \

Verified the compositions docs against foundry, fishjam, and the SFU, and fixed the places where they diverged from the implementation: - Drop the claim that a track forwarding can be removed from a live room. Fishjam routes track_forwardings with only: [:create], and the SFU only tears a forwarding down on room stop or endpoint crash. - Describe cleanup as it works: after five minutes with no media on any input, rather than as soon as the composition has no inputs. - Explain cleanup_without_inputs: false as widening the cleanup condition to need inputs and outputs quiet, not as switching cleanup off. - Note that a room forwards to one composition at a time, with the repeat call being a no-op and a different target returning 409. - Cover the start, reset, and delete lifecycle calls, and list every endpoint that can answer 503, not just composition creation.

PiotrWodecki

postponing review until internal things are live

Ran every documented call against rtc.fishjam.io and built every template snippet with the published composition-cli. Fixes what did not hold up: - Scene updates must mirror the sides the output was registered with. The docs claimed the opposite, that omitted sides were left untouched, and the example they gave returns 400 for any output that has audio. - Drop the WHEP playback row from the auth table. Outputs cannot be played back from the Composition API, since whep_server outputs are rejected, so viewers connect to the destination the output pushes to instead. - Show endpoint_route in the input register response, which the service returns but neither the docs nor the OpenAPI document mentioned, and note that the input token is the only one the publish endpoint accepts. - Warn that registering an output connects to endpoint_url right away, so the destination has to exist first. Also point the composition submodule and update_api.sh at foundry main, now that template workers have landed there. The generated spec is unchanged, confirming the published document already matched main.

The diagram sent the composition's WHIP output straight to a WHEP viewer, which skipped the service in between. Nothing plays back from a composition itself: the output publishes to a destination over WHIP, and viewers watch from that destination over WHEP. Name that destination in the diagram, drop the layout hint next to the composition node, and fix the two places in the surrounding prose that described playing back from the composition. Also list the destination as a prerequisite, since registering the output fails unless it exists.

Updating a scene lived in the concepts page, curl and all, which left the tutorial ending on a layout the reader could only set once and never change. Split it by what each page is for: - The tutorial gains a step that swaps the grid for a single full-frame camera while the composition runs, so the reader sees a scene change, and closes by pointing at templates as the way to stop sending those updates by hand. - Choose inputs and outputs gains an Update a scene section holding the mechanics: the request, the rule that an update must carry the same sides the output was registered with, and scheduled updates. - The concepts page keeps only the idea, that a scene can change over time and that you either drive it yourself or hand it to a template. Also trim the wording of the note about registering an output against an unreachable endpoint, and say what the Scenes link actually leads to.

Every link to smelter.dev landed on the home page, which left the reader to find the reference themselves, and the two pages need different ones. The concepts page describes the JSON scene, so it now points at the HTTP API reference, while the template guide describes the React components and points at the TypeScript SDK reference. Sending someone writing JSON to the TypeScript docs shows them the wrong property names. Both stay at section level rather than deep linking each component, since those paths can move and nothing here would catch it if they did. Also shorten the section on changing a scene over time, which had grown past what a concepts page needs to say.

PiotrWodecki · 2026-08-06T12:11:27Z

+{ "message": "Composition not found", "http_status_code": 404 }
+```
+
+Common statuses are `400` (bad request), `401` (unauthorized), `404` (not found), `500` (server error), and `503` (no capacity, returned by composition creation, templated output registration, events, and the WebSocket stream).


issue: an invalid token returns 502 instead of 401, should be fixed internally

PiotrWodecki · 2026-08-06T12:25:24Z

+Every non-2xx response of the Composition API is a JSON object:
+
+```json
+{ "message": "Composition not found", "http_status_code": 404 }
+```


issue: /event + /room return raw valibot dumps {"typed":false,"success":false,"issues":[…]} should be fixed internally

PiotrWodecki · 2026-08-06T12:26:16Z

+{ "message": "Composition not found", "http_status_code": 404 }
+```
+
+Common statuses are `400` (bad request), `401` (unauthorized), `404` (not found), `500` (server error), and `503` (no capacity, returned by composition creation, templated output registration, events, and the WebSocket stream).


nitpick: 422 is also possible

PiotrWodecki · 2026-08-06T12:28:33Z

+
+## Step 2: Create the composition and register the templated output
+
+Create the composition with auto-start off, because the room's inputs only appear once forwarding starts. Setting `cleanup_without_inputs` to `false` widens the cleanup condition to require both the inputs and the outputs to go quiet, so the composition survives the wait for the first peer to publish.


issue: this seems wrong, I think it should be widens --> thightens

PiotrWodecki · 2026-08-06T12:29:10Z

+
+## Step 5: Clean up
+
+Delete the composition when you are done. Forwarding stops on the Fishjam side when the room itself stops, so delete the room too once you no longer need it. There is no separate call to remove a forwarding from a live room.


question: what about DELETE /composition/{id}/room

PiotrWodecki · 2026-08-06T12:31:06Z

+Everything else happens automatically: Fishjam links the room to the composition, registers an input for every forwarded track, and streams the media in. Your template's `usePeers()` fills with the room's peers as their media starts flowing. You never register room inputs by hand.
+
+A room forwards to one composition at a time. Repeating the call with the same `compositionURL` is a no-op, while pointing the room at a different composition fails with `409 Conflict`.


nitpick: same happens for adding a second room to a composition

PiotrWodecki · 2026-08-06T12:31:59Z

    "@scalar/docusaurus": "^0.8.5",
    "@shikijs/rehype": "^3.6.0",
    "@shikijs/transformers": "^3.6.0",
    "@swmansion/smelter": "^0.2.1",


nitpick: should be bumped to 0.3.0 for twoslash I think

PiotrWodecki · 2026-08-06T12:33:10Z

+);
+```
+
+Messages are JSON text frames, each with a `type` field identifying the event.


question: are the event names avaialable anywhere?

Gawor270 added 2 commits July 24, 2026 14:21

Gawor270 requested a review from Copilot July 24, 2026 12:32

Copilot started reviewing on behalf of Gawor270 July 24, 2026 12:32 View session

Copilot AI reviewed Jul 24, 2026

View reviewed changes

Comment thread scripts/update_api.sh Outdated

Comment thread docs/smelter/how-to/write-and-deploy-a-template.mdx Outdated

Comment thread docs/smelter/quick-start.mdx Outdated

Address review comments

6fd1611

Guard the jq dependency in update_api.sh, use optional chaining on peer metadata in the template example, and fix quick-start wording: a whip_client output publishes to a WHIP endpoint; playback is WHEP on the receiving service.

Gawor270 marked this pull request as ready for review July 24, 2026 14:37

Gawor270 added 6 commits July 24, 2026 16:53

Remove em dashes from the Smelter Cloud docs

6f781f3

Add Smelter Cloud REST API to the API Reference navbar dropdown

31d5153

Gawor270 changed the title ~~[FCE-2924][FCE-2925] Smelter Cloud documentation~~ Compositions documentation Jul 27, 2026

Gawor270 requested a review from Copilot July 27, 2026 16:15

Merge branch 'main' into fce-2924-2925-smelter-cloud-docs

cb9d0c6

Copilot started reviewing on behalf of Gawor270 July 27, 2026 16:16 View session

Copilot AI reviewed Jul 27, 2026

View reviewed changes

Comment thread .gitmodules Outdated

Comment thread scripts/update_api.sh Outdated

Comment thread docusaurus.config.ts

Comment thread scripts/composition_openapi_overlay.jq Outdated

Gawor270 added 3 commits July 30, 2026 11:47

Rename the composition submodule key to match its path

4f3f938

The submodule was moved to api/composition earlier but kept its old api/smelter-cloud section name in .gitmodules.

Gawor270 requested a review from Copilot July 30, 2026 10:02

Copilot started reviewing on behalf of Gawor270 July 30, 2026 10:02 View session

Copilot AI reviewed Jul 30, 2026

View reviewed changes

Gawor270 requested review from PiotrWodecki and czerwiukk July 30, 2026 14:59

PiotrWodecki reviewed Aug 3, 2026

View reviewed changes

PiotrWodecki self-requested a review August 6, 2026 09:59

Gawor270 and others added 7 commits August 6, 2026 12:05

Merge branch 'main' into fce-2924-2925-smelter-cloud-docs

86e0307

make main tutorial easier

4a1b8da

improve livestream input introduction

b8e9492

PiotrWodecki reviewed Aug 6, 2026

View reviewed changes


		## Step 2: Create the composition and register the templated output

		Create the composition with auto-start off, because the room's inputs only appear once forwarding starts. Setting `cleanup_without_inputs` to `false` widens the cleanup condition to require both the inputs and the outputs to go quiet, so the composition survives the wait for the first peer to publish.


		## Step 5: Clean up

		Delete the composition when you are done. Forwarding stops on the Fishjam side when the room itself stops, so delete the room too once you no longer need it. There is no separate call to remove a forwarding from a live room.

		Everything else happens automatically: Fishjam links the room to the composition, registers an input for every forwarded track, and streams the media in. Your template's `usePeers()` fills with the room's peers as their media starts flowing. You never register room inputs by hand.

		A room forwards to one composition at a time. Repeating the call with the same `compositionURL` is a no-op, while pointing the room at a different composition fails with `409 Conflict`.

Uh oh!

Conversation

Gawor270 commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

linear Bot commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

PiotrWodecki left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Gawor270 commented Jul 23, 2026 •

edited

Loading

linear Bot commented Jul 23, 2026 •

edited

Loading