Skip to content

Commit fd0aefb

Browse files
MattFisherclaudema2za
authored
Add LaTeX, super/subscript, pull quotes, callouts, and Markdown docs (#65)
* Add table and LaTeX block support GFM tables (`| col | col |`) map to Substack's table/table_row/ table_header/table_cell schema. Table node shape was confirmed by probing the live API. LaTeX math blocks ($$...$$) map to latex_block via mdit_py_plugins' dollarmath plugin. Both features are covered by unit tests and the e2e fixture/golden file. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add inline LaTeX support and fix corrupted footnote fixture Inline math ($x$ and $$x$$ via dollarmath's double_inline) now maps to Substack's inline `latex` node, confirmed against the live API. Previously inline math tokens were silently dropped. Also restores the [^unused] and [^listnote] footnote definitions in the e2e fixture, which had been corrupted, dropping footnote #4 from coverage. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Restrict inline LaTeX to standard single-dollar syntax Drop dollarmath's double_inline option so $$...$$ is only ever a display block, matching the common Markdown convention. Inline math stays $...$. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Replace tables with pull quotes, callouts, and sub/superscript Substack has no table renderer or insert UI, so the table node (added earlier) stored but never displayed. Removed it in favor of features Substack actually supports, confirmed by round-tripping editor-authored nodes through the live API: - pullquote and calloutBlock via `:::pullquote` / `:::callout` containers - superscript (`^x^`) and subscript (`~x~`) inline marks Subscript's single-tilde syntax coexists with `~~strikethrough~~`. The sub/superscript plugins require mdit-py-plugins >= 0.5, so the constraint and lockfile are updated (now resolves 0.6.1). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Document supported Markdown features Add docs/markdown.md covering every Markdown construct from_markdown() converts (formatting, headings, lists, images, footnotes, math, pull quotes, callouts) plus what is intentionally unsupported (tables, editor-only widgets). Link it from the README. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix currency false-positives and labeled blocks in dollar-math parsing Address PR #65 review feedback from ma2za: - Configure dollarmath with allow_space=False and allow_digits=False (Pandoc-style delimiters), so paired dollar amounts like "$5 million to $10 million" stay plain text instead of being parsed as inline math. - Render math_block_label tokens ("$$ ... $$ (label)") as latex_block, discarding the label, instead of silently dropping the whole block. - Add both of the reviewer's regression tests and document the delimiter rules in docs/markdown.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JfNdp5q3sjjY62Zd6ZMzg3 --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: ma2za <mazzapaolo2019@gmail.com>
1 parent a243e39 commit fd0aefb

9 files changed

Lines changed: 458 additions & 10 deletions

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -305,7 +305,7 @@ Paragraph with **bold**, *italic*, `code`, [links](https://example.com), and foo
305305
)
306306
```
307307

308-
Supported Markdown includes headings, paragraphs, bold, italic, inline code, strikethrough, links, images, linked images, image captions, code blocks, blockquotes, ordered lists, unordered lists, horizontal rules, and footnotes.
308+
Supported Markdown includes headings, paragraphs, bold, italic, inline code, strikethrough, superscript, subscript, links, images, linked images, image captions, code blocks, blockquotes, ordered lists, unordered lists, horizontal rules, footnotes, LaTeX math, pull quotes, and callouts. See [docs/markdown.md](docs/markdown.md) for the full reference with examples.
309309

310310
When an `Api` instance is passed to `from_markdown`, local image paths are uploaded before the draft is created:
311311

docs/markdown.md

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
# Markdown support
2+
3+
`Post.from_markdown(markdown_content, api=None)` converts a Markdown document
4+
into Substack's document format. Parsing is handled by
5+
[markdown-it-py](https://github.com/executablebooks/markdown-it-py) (CommonMark)
6+
plus a few plugins, so standard CommonMark works as you'd expect. This page
7+
documents everything that maps to a Substack node.
8+
9+
```python
10+
from substack.post import Post
11+
12+
post = Post(title="My Post", subtitle="", user_id=api.get_user_id())
13+
post.from_markdown(open("post.md").read(), api=api)
14+
draft = api.post_draft(post.get_draft())
15+
```
16+
17+
Pass `api=` when your Markdown references local images so they can be uploaded
18+
(see [Images](#images)); it is optional otherwise.
19+
20+
## Text formatting
21+
22+
| Markdown | Result |
23+
|-------------------------------|-------------------|
24+
| `**bold**` | **bold** |
25+
| `*italic*` | *italic* |
26+
| `***bold italic***` | ***bold italic*** |
27+
| `` `inline code` `` | `inline code` |
28+
| `~~strikethrough~~` | ~~strikethrough~~ |
29+
| `^superscript^` | superscript |
30+
| `~subscript~` | subscript |
31+
| `[text](https://example.com)` | link |
32+
| `<https://example.com>` | autolinked URL |
33+
34+
Note that subscript uses a single tilde (`~x~`) and strikethrough uses a double
35+
tilde (`~~x~~`); both work in the same document.
36+
37+
## Headings
38+
39+
Levels 1–6, using `#` through `######`. Headings may contain inline formatting
40+
and links.
41+
42+
```markdown
43+
# Heading level 1
44+
## Heading level 2 with **bold** and a [link](https://example.com)
45+
```
46+
47+
## Paragraphs and line breaks
48+
49+
Blank lines separate paragraphs. A single newline within a paragraph is treated
50+
as a space (soft break), matching CommonMark.
51+
52+
## Lists
53+
54+
Bullet lists (`-`, `*`, or `+`) and ordered lists (`1.`), including nesting:
55+
56+
```markdown
57+
- Bullet one
58+
- Bullet two
59+
- Nested bullet
60+
1. Nested number
61+
62+
1. Ordered one
63+
2. Ordered two
64+
```
65+
66+
## Blockquotes
67+
68+
```markdown
69+
> A blockquote.
70+
>
71+
> With multiple paragraphs.
72+
```
73+
74+
## Code blocks
75+
76+
Fenced code blocks, with an optional language for syntax highlighting, and
77+
indented code blocks:
78+
79+
````markdown
80+
```python
81+
print("hello")
82+
```
83+
````
84+
85+
## Horizontal rule
86+
87+
```markdown
88+
---
89+
```
90+
91+
## Images
92+
93+
A paragraph containing only an image becomes a captioned image.
94+
95+
```markdown
96+
![Alt text](https://example.com/image.png)
97+
```
98+
99+
- **Caption:** use the CommonMark title slot — `![alt](url "caption text")`.
100+
- **Link:** wrap the image in a link — `[![alt](url)](https://target.com)`.
101+
- **Local upload:** if `api=` is passed and the `src` is a local path (not an
102+
`http(s)` URL), the file is uploaded to Substack and the returned URL is used.
103+
Local paths resolve relative to the current working directory.
104+
105+
```markdown
106+
![A chart](chart.png "Figure 1: quarterly results")
107+
```
108+
109+
## Footnotes
110+
111+
References become inline anchors; definitions become footnote blocks at the end,
112+
numbered by order of first appearance. Labels may be numeric or named, and a
113+
definition may contain block content such as lists or multiple paragraphs.
114+
115+
```markdown
116+
A claim that needs support.[^1] Another, with a named label.[^source]
117+
118+
[^1]: The supporting detail, with a [link](https://example.com).
119+
[^source]: Author, *Title* (2025).
120+
```
121+
122+
A reference used more than once is emitted as a separate numbered anchor each
123+
time, mirroring the Substack editor. Definitions that are never referenced are
124+
dropped.
125+
126+
## Math (LaTeX)
127+
128+
Inline math with single dollars, block math with double dollars:
129+
130+
```markdown
131+
Einstein showed $E=mc^2$ inline.
132+
133+
$$
134+
\int_0^\infty e^{-x} \, dx = 1
135+
$$
136+
```
137+
138+
Delimiters follow Pandoc's rules: the opening `$` must not be followed by
139+
whitespace, and the closing `$` must not be preceded by whitespace or followed
140+
by a digit. Ordinary dollar amounts (`$5 million to $10 million`) therefore
141+
stay plain text. A label after a block (`$$ ... $$ (label)`) is accepted but
142+
discarded, since Substack has no equation labels.
143+
144+
## Pull quotes and callouts
145+
146+
These use fenced-container syntax (`:::`), since they have no native Markdown
147+
equivalent:
148+
149+
```markdown
150+
:::pullquote
151+
A highlighted pull quote. **Formatting** works inside.
152+
:::
153+
154+
:::callout
155+
A callout block, e.g. an aside or note.
156+
:::
157+
```
158+
159+
## Not supported
160+
161+
- **Tables** — Substack has no table renderer or editor UI for them, so GFM
162+
table syntax is not converted. To include tabular data, embed a chart (for
163+
example via [Datawrapper](https://support.substack.com/hc/en-us/articles/15722290158100))
164+
and add it in the Substack editor.
165+
- Widgets authored only in the Substack editor (buttons, polls, embeds, etc.)
166+
have no Markdown equivalent.

poetry.lock

Lines changed: 7 additions & 7 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ requests = "^2.32.0"
3939
python-dotenv = "^1.2.1"
4040
PyYAML = "^6.0"
4141
markdown-it-py = "^3.0"
42-
mdit-py-plugins = "^0.4"
42+
mdit-py-plugins = ">=0.5,<0.7"
4343
fastmcp = { version = "^3.1.1", optional = true }
4444

4545
[tool.poetry.extras]

substack/mdrender.py

Lines changed: 39 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,11 @@
2222

2323
from markdown_it import MarkdownIt
2424
from markdown_it.tree import SyntaxTreeNode
25+
from mdit_py_plugins.container import container_plugin
26+
from mdit_py_plugins.dollarmath import dollarmath_plugin
2527
from mdit_py_plugins.footnote import footnote_plugin
28+
from mdit_py_plugins.subscript import sub_plugin
29+
from mdit_py_plugins.superscript import superscript_plugin
2630

2731
from substack import nodes
2832
from substack.nodes import MarkType, NodeType
@@ -31,11 +35,25 @@
3135
"strong": {"type": MarkType.STRONG},
3236
"em": {"type": MarkType.EM},
3337
"s": {"type": MarkType.STRIKETHROUGH},
38+
"sup": {"type": MarkType.SUPERSCRIPT},
39+
"sub": {"type": MarkType.SUBSCRIPT},
3440
}
3541

3642

3743
def _make_parser() -> MarkdownIt:
38-
return MarkdownIt("commonmark").use(footnote_plugin).enable("strikethrough")
44+
return (
45+
MarkdownIt("commonmark")
46+
.use(footnote_plugin)
47+
# Pandoc-style delimiters: no whitespace just inside the dollars and no
48+
# digit just outside them, so paired currency amounts ("$5 ... $10")
49+
# stay plain text instead of becoming math.
50+
.use(dollarmath_plugin, allow_space=False, allow_digits=False)
51+
.use(sub_plugin)
52+
.use(superscript_plugin)
53+
.use(container_plugin, name="pullquote")
54+
.use(container_plugin, name="callout")
55+
.enable("strikethrough")
56+
)
3957

4058

4159
def _coalesce(out_nodes: List[Dict]) -> List[Dict]:
@@ -64,6 +82,8 @@ def _render_inline(node: SyntaxTreeNode, marks: List[Dict], ctx: Dict) -> List[D
6482
out.append(nodes.text(child.content, marks))
6583
elif t == "code_inline":
6684
out.append(nodes.text(child.content, marks + [nodes.code_mark()]))
85+
elif t == "math_inline":
86+
out.append(nodes.latex_inline(child.content.strip()))
6787
elif t in _MARK_FOR:
6888
out.extend(_render_inline(child, marks + [_MARK_FOR[t]], ctx))
6989
elif t == "link":
@@ -159,10 +179,28 @@ def _render_block(node: SyntaxTreeNode, api, ctx: Dict) -> List[Dict]:
159179
if t == "ordered_list":
160180
return [nodes.ordered_list(_render_list_items(node, api, ctx))]
161181

182+
# "$$...$$ (label)" tokenizes as math_block_label; Substack has no equation
183+
# labels, so it renders like an unlabeled block.
184+
if t in ("math_block", "math_block_label"):
185+
return [nodes.latex_block(node.content.strip())]
186+
187+
if t == "container_pullquote":
188+
return [nodes.pullquote(_render_container_body(node, api, ctx))]
189+
190+
if t == "container_callout":
191+
return [nodes.callout_block(_render_container_body(node, api, ctx))]
192+
162193
# footnote_block is handled separately in markdown_to_doc; ignore it here.
163194
return []
164195

165196

197+
def _render_container_body(node: SyntaxTreeNode, api, ctx: Dict) -> List[Dict]:
198+
body: List[Dict] = []
199+
for child in node.children:
200+
body.extend(_render_block(child, api, ctx))
201+
return body
202+
203+
166204
def _render_list_items(list_node: SyntaxTreeNode, api, ctx: Dict) -> List[Dict]:
167205
items = []
168206
for li in list_node.children:

substack/nodes.py

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,13 +33,19 @@ class NodeType:
3333
FOOTNOTE_ANCHOR = "footnoteAnchor"
3434
CAPTIONED_IMAGE = "captionedImage"
3535
CAPTION = "caption"
36+
LATEX_BLOCK = "latex_block"
37+
LATEX_INLINE = "latex"
38+
PULLQUOTE = "pullquote"
39+
CALLOUT_BLOCK = "calloutBlock"
3640

3741

3842
class MarkType:
3943
STRONG = "strong"
4044
EM = "em"
4145
CODE = "code"
4246
STRIKETHROUGH = "strikethrough"
47+
SUPERSCRIPT = "superscript"
48+
SUBSCRIPT = "subscript"
4349
LINK = "link"
4450

4551

@@ -146,3 +152,32 @@ def footnote(number: int, paragraphs: List[Dict]) -> Dict:
146152
"attrs": {"number": number},
147153
"content": paragraphs or [paragraph()],
148154
}
155+
156+
157+
def latex_block(expression: str) -> Dict:
158+
return {
159+
"type": NodeType.LATEX_BLOCK,
160+
"attrs": {"persistentExpression": expression, "dirty": True},
161+
}
162+
163+
164+
def latex_inline(expression: str) -> Dict:
165+
return {
166+
"type": NodeType.LATEX_INLINE,
167+
"attrs": {"expression": expression, "persistentExpression": expression},
168+
}
169+
170+
171+
def pullquote(paragraphs: List[Dict]) -> Dict:
172+
return {
173+
"type": NodeType.PULLQUOTE,
174+
"attrs": {"align": None, "color": None},
175+
"content": paragraphs or [paragraph()],
176+
}
177+
178+
179+
def callout_block(paragraphs: List[Dict]) -> Dict:
180+
return {
181+
"type": NodeType.CALLOUT_BLOCK,
182+
"content": paragraphs or [paragraph()],
183+
}

0 commit comments

Comments
 (0)