Skip to content

Commit fa8d07b

Browse files
dazzatronusdazzatronus
committed
refactor(skill): extract universal authoring conventions to shared/agent-core.md (ENG-567)
1 parent 013ac86 commit fa8d07b

2 files changed

Lines changed: 173 additions & 152 deletions

File tree

‎SKILL.md‎

Lines changed: 4 additions & 152 deletions
Original file line numberDiff line numberDiff line change
@@ -71,160 +71,11 @@ Use `render` only when you're confident the JSON is final, or there's no human t
7171

7272
## Authoring Edit JSON
7373

74-
These are the conventions agents most often get wrong. Read this section before generating any Edit JSON.
74+
The conventions agents most often get wrong (track order is reversed, asset type names, font allowlist, smart strings) live in the shared agent-core guide:
7575

76-
### Before composing JSON: check the schema
76+
**See [`shared/agent-core.md`](shared/agent-core.md)** before composing any Edit JSON.
7777

78-
Don't invent property names or enum values. The Shotstack schema is published — fetch one of these before composing JSON from scratch:
79-
80-
- <https://shotstack.io/docs/api/api.edit.json> — single-file OpenAPI Schema. Machine-validatable; load it once and validate locally instead of round-tripping the API.
81-
- <https://shotstack.io/docs/api/> — interactive HTML reference. Fastest for human scanning.
82-
- <https://shotstack.io/docs/guide/llms-full.txt> — single-file LLM-friendly version of the full guide + reference.
83-
- <https://github.com/shotstack/oas-api-definition/tree/main/schemas> — raw OpenAPI YAML, source of truth.
84-
85-
CSS naming conventions (`alignment`, `vertical: "center"`) **do not** apply. The spec uses precise names that often differ from web/CSS instincts:
86-
87-
| You'd guess (wrong) | API uses (right) |
88-
|---|---|
89-
| `alignment` | `align` |
90-
| `align.vertical: "center"` | `align.vertical: "middle"` |
91-
| `font.name` | `font.family` |
92-
| `duration` | `length` |
93-
| `transitions: [...]` (array) | `transition: { in, out }` (object) |
94-
| `fit: "cover"` (CSS instinct: scale+crop maintaining aspect) | `fit: "crop"` — Shotstack's `cover` STRETCHES without maintaining aspect ratio |
95-
96-
When the API rejects a property, the error message names the field — fix and retry. Don't guess twice.
97-
98-
### Track ordering is REVERSED
99-
100-
`timeline.tracks` is an array. **The first element (`tracks[0]`) is the TOP layer; the last element is the BOTTOM layer.** This is opposite to most z-index conventions.
101-
102-
> "Tracks are layered on top of each other in the same order they are added to the array with the top most track layered over the top of those below it." — [Shotstack docs](https://shotstack.io/docs/guide/architecting-an-application/guidelines/)
103-
104-
Practical rule: **put captions, overlays, and titles in early tracks; put video/image backgrounds in later tracks.**
105-
106-
```json
107-
{
108-
"timeline": {
109-
"tracks": [
110-
{ "clips": [/* TOP — captions */] },
111-
{ "clips": [/* MIDDLE — title overlay */] },
112-
{ "clips": [/* BOTTOM — background video */] }
113-
]
114-
}
115-
}
116-
```
117-
118-
### Smart clip strings
119-
120-
These string values are accepted in addition to numbers:
121-
122-
| Where | Value | Meaning |
123-
|---|---|---|
124-
| `clip.start` | `"auto"` | Start when the previous clip on the same track finishes |
125-
| `clip.start` | `"alias://<name>"` | Inherit start from another clip with `alias: "<name>"` |
126-
| `clip.length` | `"auto"` | Play for the full duration of the asset |
127-
| `clip.length` | `"end"` | Play until the end of the timeline |
128-
| `clip.length` | `"alias://<name>"` | Inherit length from another clip |
129-
130-
The `alias://` protocol is also used in `rich-caption` `src` to auto-transcribe a referenced audio/video clip — see `references/rich-caption.md`.
131-
132-
### Public HTTPS URLs only
133-
134-
All asset `src` URLs must be publicly accessible HTTPS. **No local file paths, no `data:` URIs, no signed URLs that expire mid-render.** The render workers fetch assets from the public internet.
135-
136-
For test renders without your own assets, use the placeholder library at <https://shotstack-assets.s3.amazonaws.com/> — see `references/asset-library.md`.
137-
138-
### Don't overlap clips on the same track
139-
140-
Clips on the same track must not have overlapping `start`/`length` ranges. Overlapping clips flicker because the engine can't decide which to display. Put parallel content on separate tracks.
141-
142-
### Output resolution
143-
144-
Pick `output.resolution` (preset) OR `output.size.width`+`output.size.height` (custom):
145-
146-
| Preset | Pixels @ fps |
147-
|---|---|
148-
| `preview` | 512×288 @ 15 |
149-
| `mobile` | 640×360 @ 25 |
150-
| `sd` | 1024×576 @ 25 |
151-
| `hd` | 1280×720 @ 25 (default) |
152-
| `1080` | 1920×1080 @ 25 |
153-
154-
Custom sizes must be divisible by 2.
155-
156-
## Asset types
157-
158-
The render API supports many asset types. Use only the **current** ones; the deprecated ones still parse but should not be used in new templates.
159-
160-
### Current
161-
162-
| Type | Purpose |
163-
|---|---|
164-
| `video` | Video file (mp4, mov, webm). |
165-
| `image` | Static image (jpg, png). |
166-
| `audio` | Audio clip placed at a specific time on the timeline. |
167-
| `rich-text` | Styled text overlay with full typography control. **Use this instead of `text`/`html`/`title`.** |
168-
| `svg` | Vector graphics from raw SVG markup. See `references/svg.md`. |
169-
| `rich-caption` | Word-level animated captions sourced from audio, video, or subtitle files. See `references/rich-caption.md`. |
170-
| `luma` | Luma matte for masking effects. |
171-
| `image-to-video` | **AI**: animate a still image into a short video clip. Billed per generation. |
172-
| `text-to-image` | **AI**: generate an image from a text prompt. Billed per generation. |
173-
174-
`timeline.soundtrack` is a separate top-level field (not an asset type) for a single background music track. `timeline.fonts[]` is a separate field for custom font URLs.
175-
176-
### Deprecated — do not use
177-
178-
`text`, `title`, `caption`, `html`, `shape`. They still parse but produce inferior output. Replace with:
179-
180-
| If you'd use… | Use instead |
181-
|---|---|
182-
| `text` or `title` | `rich-text` |
183-
| `caption` | `rich-caption` |
184-
| `html` | `rich-text` |
185-
| `shape` | `svg` with `<rect>`, `<circle>`, `<polygon>` etc. |
186-
187-
### AI-generated assets
188-
189-
`image-to-video`, `text-to-speech`, and `text-to-image` are billed per generation **even when invoked through the sandbox stage endpoint** (which is otherwise free). They are async — the render submits the AI job and waits. Renders containing AI assets take longer.
190-
191-
## Fonts
192-
193-
Prefer **custom Google Fonts via `timeline.fonts[]`** over the built-in font list. The Studio SDK exposes ~400 Google Fonts. See `references/fonts.md` for the URL pattern and the built-in fallback list.
194-
195-
Per the [Shotstack docs](https://shotstack.io/docs/guide/architecting-an-application/rich-text/#custom-fonts), custom fonts give you the full Google Fonts catalogue with predictable rendering. System fonts like `Arial`, `Helvetica`, and `Times New Roman` are NOT installed and will fail with "Font not found".
196-
197-
```json
198-
{
199-
"timeline": {
200-
"fonts": [
201-
{ "src": "https://fonts.gstatic.com/s/montserrat/v31/JTUSjIg1_i6t8kCHKm45xW5rygbi49c.ttf" }
202-
],
203-
"tracks": [
204-
{
205-
"clips": [{
206-
"asset": {
207-
"type": "rich-text",
208-
"text": "Hello",
209-
"font": { "family": "JTUSjIg1_i6t8kCHKm45xW5rygbi49c", "size": 60, "color": "#ffffff" }
210-
},
211-
"start": 0, "length": 3
212-
}]
213-
}
214-
]
215-
}
216-
}
217-
```
218-
219-
The `font.family` value is the **font file basename** (without `.ttf`/`.otf`).
220-
221-
## Top 5 mistakes
222-
223-
1. **Reverse track order.** `tracks[0]` is the TOP layer, not the bottom. Captions go in early tracks; backgrounds go in late tracks.
224-
2. **System fonts.** `Arial`, `Helvetica`, `Times New Roman`, etc. are not installed. Use Google Fonts via `timeline.fonts[]` (preferred) or one of the built-in fonts in `references/fonts.md`.
225-
3. **Captions fill the whole frame.** A `rich-caption` clip without `width`, `height`, and `fit: "none"` covers the entire output. Use a named preset from `references/rich-caption.md`.
226-
4. **`<text>` inside an SVG asset.** Raw `<text>` is unsupported. Use a `rich-text` asset for any text content; reserve SVG for shapes only.
227-
5. **Composing custom caption styles when presets exist.** The five named presets (Nico, Kai, Kapow, Lovely Little Lychee, Rizz) cover the common styles. Use one verbatim from `references/rich-caption.md` unless the user asks for something specific.
78+
This file is also delivered to chat-based clients (Claude.ai, ChatGPT) via the Shotstack MCP server, so the conventions stay identical across surfaces.
22879

22980
## Exit codes
23081

@@ -239,6 +90,7 @@ The `font.family` value is the **font file basename** (without `.ttf`/`.otf`).
23990
Authoritative sources, in order of preference:
24091

24192
- `shotstack --help` and `shotstack <command> --help` — current CLI flag listing
93+
- [`shared/agent-core.md`](shared/agent-core.md) — Edit JSON authoring conventions (shared with MCP server)
24294
- <https://shotstack.io/docs/api/api.edit.json> — OpenAPI Schema for the render API (machine-validatable)
24395
- <https://shotstack.io/docs/guide/llms-full.txt> — full API docs in LLM-friendly single file
24496
- <https://github.com/shotstack/oas-api-definition/tree/main/schemas> — raw OpenAPI YAML, source of truth for property names and enums

‎shared/agent-core.md‎

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
# Shotstack Agent Core
2+
3+
Universal Edit JSON authoring conventions. Both the [Shotstack CLI](https://github.com/shotstack/shotstack-cli) skill and the [Shotstack MCP server](https://mcp.shotstack.io) ship this file; phase 3 of [ENG-567](https://linear.app/shotstack/issue/ENG-567) promotes it to `https://shotstack.io/docs/agents/agent-core.md`.
4+
5+
Read this **before composing any Edit JSON**. The conventions agents most often get wrong are listed here once.
6+
7+
## Before composing JSON: check the schema
8+
9+
Don't invent property names or enum values. The Shotstack schema is published — fetch one of these before composing JSON from scratch:
10+
11+
- <https://shotstack.io/docs/api/api.edit.json> — single-file OpenAPI Schema. Machine-validatable; load it once and validate locally instead of round-tripping the API.
12+
- <https://shotstack.io/docs/api/> — interactive HTML reference. Fastest for human scanning.
13+
- <https://shotstack.io/docs/guide/llms-full.txt> — single-file LLM-friendly version of the full guide + reference.
14+
- <https://github.com/shotstack/oas-api-definition/tree/main/schemas> — raw OpenAPI YAML, source of truth.
15+
16+
CSS naming conventions (`alignment`, `vertical: "center"`) **do not** apply. The spec uses precise names that often differ from web/CSS instincts:
17+
18+
| You'd guess (wrong) | API uses (right) |
19+
|---|---|
20+
| `alignment` | `align` |
21+
| `align.vertical: "center"` | `align.vertical: "middle"` |
22+
| `font.name` | `font.family` |
23+
| `duration` | `length` |
24+
| `transitions: [...]` (array) | `transition: { in, out }` (object) |
25+
| `fit: "cover"` (CSS instinct: scale+crop maintaining aspect) | `fit: "crop"` — Shotstack's `cover` STRETCHES without maintaining aspect ratio |
26+
27+
When the API rejects a property, the error message names the field — fix and retry. Don't guess twice.
28+
29+
## Track ordering is REVERSED
30+
31+
`timeline.tracks` is an array. **The first element (`tracks[0]`) is the TOP layer; the last element is the BOTTOM layer.** This is opposite to most z-index conventions.
32+
33+
> "Tracks are layered on top of each other in the same order they are added to the array with the top most track layered over the top of those below it." — [Shotstack docs](https://shotstack.io/docs/guide/architecting-an-application/guidelines/)
34+
35+
Practical rule: **put captions, overlays, and titles in early tracks; put video/image backgrounds in later tracks.**
36+
37+
```json
38+
{
39+
"timeline": {
40+
"tracks": [
41+
{ "clips": [/* TOP — captions */] },
42+
{ "clips": [/* MIDDLE — title overlay */] },
43+
{ "clips": [/* BOTTOM — background video */] }
44+
]
45+
}
46+
}
47+
```
48+
49+
## Smart clip strings
50+
51+
These string values are accepted in addition to numbers:
52+
53+
| Where | Value | Meaning |
54+
|---|---|---|
55+
| `clip.start` | `"auto"` | Start when the previous clip on the same track finishes |
56+
| `clip.start` | `"alias://<name>"` | Inherit start from another clip with `alias: "<name>"` |
57+
| `clip.length` | `"auto"` | Play for the full duration of the asset |
58+
| `clip.length` | `"end"` | Play until the end of the timeline |
59+
| `clip.length` | `"alias://<name>"` | Inherit length from another clip |
60+
61+
The `alias://` protocol is also used in `rich-caption` `src` to auto-transcribe a referenced audio/video clip — see `references/rich-caption.md`.
62+
63+
## Public HTTPS URLs only
64+
65+
All asset `src` URLs must be publicly accessible HTTPS. **No local file paths, no `data:` URIs, no signed URLs that expire mid-render.** The render workers fetch assets from the public internet.
66+
67+
For test renders without your own assets, use the placeholder library at <https://shotstack-assets.s3.amazonaws.com/> — see `references/asset-library.md`.
68+
69+
## Don't overlap clips on the same track
70+
71+
Clips on the same track must not have overlapping `start`/`length` ranges. Overlapping clips flicker because the engine can't decide which to display. Put parallel content on separate tracks.
72+
73+
## Output resolution
74+
75+
Pick `output.resolution` (preset) OR `output.size.width`+`output.size.height` (custom):
76+
77+
| Preset | Pixels @ fps |
78+
|---|---|
79+
| `preview` | 512×288 @ 15 |
80+
| `mobile` | 640×360 @ 25 |
81+
| `sd` | 1024×576 @ 25 |
82+
| `hd` | 1280×720 @ 25 (default) |
83+
| `1080` | 1920×1080 @ 25 |
84+
85+
Custom sizes must be divisible by 2.
86+
87+
## Asset types
88+
89+
Use only the **current** asset types; the deprecated ones still parse but should not be used in new templates.
90+
91+
### Current
92+
93+
| Type | Purpose |
94+
|---|---|
95+
| `video` | Video file (mp4, mov, webm). |
96+
| `image` | Static image (jpg, png). |
97+
| `audio` | Audio clip placed at a specific time on the timeline. |
98+
| `rich-text` | Styled text overlay with full typography control. **Use this instead of `text`/`html`/`title`.** |
99+
| `svg` | Vector graphics from raw SVG markup. See `references/svg.md`. |
100+
| `rich-caption` | Word-level animated captions sourced from audio, video, or subtitle files. See `references/rich-caption.md`. |
101+
| `luma` | Luma matte for masking effects. |
102+
| `image-to-video` | **AI**: animate a still image into a short video clip. Billed per generation. |
103+
| `text-to-image` | **AI**: generate an image from a text prompt. Billed per generation. |
104+
105+
`timeline.soundtrack` is a separate top-level field (not an asset type) for a single background music track. `timeline.fonts[]` is a separate field for custom font URLs.
106+
107+
### Deprecated — do not use
108+
109+
`text`, `title`, `caption`, `html`, `shape`. They still parse but produce inferior output. Replace with:
110+
111+
| If you'd use… | Use instead |
112+
|---|---|
113+
| `text` or `title` | `rich-text` |
114+
| `caption` | `rich-caption` |
115+
| `html` | `rich-text` |
116+
| `shape` | `svg` with `<rect>`, `<circle>`, `<polygon>` etc. |
117+
118+
### AI-generated assets
119+
120+
`image-to-video`, `text-to-speech`, and `text-to-image` are billed per generation **even when invoked through the sandbox stage endpoint** (which is otherwise free). They are async — the render submits the AI job and waits. Renders containing AI assets take longer.
121+
122+
## Fonts
123+
124+
Prefer **custom Google Fonts via `timeline.fonts[]`** over the built-in font list. The Studio SDK exposes ~400 Google Fonts. See `references/fonts.md` for the URL pattern and the built-in fallback list.
125+
126+
Per the [Shotstack docs](https://shotstack.io/docs/guide/architecting-an-application/rich-text/#custom-fonts), custom fonts give you the full Google Fonts catalogue with predictable rendering. System fonts like `Arial`, `Helvetica`, and `Times New Roman` are NOT installed and will fail with "Font not found".
127+
128+
```json
129+
{
130+
"timeline": {
131+
"fonts": [
132+
{ "src": "https://fonts.gstatic.com/s/montserrat/v31/JTUSjIg1_i6t8kCHKm45xW5rygbi49c.ttf" }
133+
],
134+
"tracks": [
135+
{
136+
"clips": [{
137+
"asset": {
138+
"type": "rich-text",
139+
"text": "Hello",
140+
"font": { "family": "JTUSjIg1_i6t8kCHKm45xW5rygbi49c", "size": 60, "color": "#ffffff" }
141+
},
142+
"start": 0, "length": 3
143+
}]
144+
}
145+
]
146+
}
147+
}
148+
```
149+
150+
The `font.family` value is the **font file basename** (without `.ttf`/`.otf`).
151+
152+
## Top 5 mistakes
153+
154+
1. **Reverse track order.** `tracks[0]` is the TOP layer, not the bottom. Captions go in early tracks; backgrounds go in late tracks.
155+
2. **System fonts.** `Arial`, `Helvetica`, `Times New Roman`, etc. are not installed. Use Google Fonts via `timeline.fonts[]` (preferred) or one of the built-in fonts in `references/fonts.md`.
156+
3. **Captions fill the whole frame.** A `rich-caption` clip without `width`, `height`, and `fit: "none"` covers the entire output. Use a named preset from `references/rich-caption.md`.
157+
4. **`<text>` inside an SVG asset.** Raw `<text>` is unsupported. Use a `rich-text` asset for any text content; reserve SVG for shapes only.
158+
5. **Composing custom caption styles when presets exist.** The five named presets (Nico, Kai, Kapow, Lovely Little Lychee, Rizz) cover the common styles. Use one verbatim from `references/rich-caption.md` unless the user asks for something specific.
159+
160+
## Per-topic deep dives
161+
162+
For details beyond this core guide, see:
163+
164+
- `references/timeline.md` — track layering, transitions, soundtrack vs audio
165+
- `references/rich-caption.md` — sizing per resolution, default style, the 5 named presets, alias pattern
166+
- `references/svg.md` — required attrs, supported elements
167+
- `references/fonts.md` — built-in fonts, Google Fonts URL pattern, custom-font workflow
168+
- `references/asset-library.md` — placeholder videos, images, music
169+
- `references/troubleshooting.md` — common errors and fixes

0 commit comments

Comments
 (0)