LaTeX math support

[photino]

Many large language models output markdown content which embeds LaTeX math formulae using \( ... \) and \[ ... \]. Could we consider support them as aliases of $ and $$?

[kivikakk]

Hi there!

I'll leave this decision up to @digitalmoksha as the originator of the support for math in Comrak, but I consider the rationale here (adapting Comrak for an LLM) in a very dim light, and would suggest instructing the LLM to use the supported syntax instead.

[photino]

We also use comrak to render markdown files which are provided by users.

[digitalmoksha]

I kind of agree with not specifically adapting comrak for an LLM. That being said, we've also had requests to use LaTex math delimiters for normal usage in GitLab, it's just never bubbled up on priority. So I'm open to a PR adding that support.

[nicoburns]

@sepcnt has ported katex to Rust as katex-rs, so the LaTex -> HTML conversion is covered. What would be required to integrate this into comrak?

[kivikakk]

That's very neat! I imagine a great PR might:

• add katex-rs as an optional dependency under a katex feature (and also added to the cli feature),
• which when enabled adds something like a --katex / RenderOptions.katex: bool option¹,
• which in turn causes comrak::html::render_math and possibly comrak::html::render_math_code_block to pipe content through katex-rs.

One could also optionally extend the parser to accept \( … \) and \[ ... \] with a new extension option.

We currently have two extension options for math:

• math_dollars
  • parses $x$ as inline math, puts its content in <span data-math-style="inline">
  • parses $$x$$ as display math, puts its content in <span data-math-style="display">
• math_code
  • parses $`x`$ as inline math, puts its content in <code data-math-style="inline">
  • parses ```math\nx\n```\n as display math, puts its content in <pre><code class="language-math" data-math-style="display"> (or <pre lang="math" data-math-style="display"><code> if github_pre_lang is used)

How the --katex option should interact with these (and/or a new extension option to parse LaTeX-style delimiters) feels a little bit bike-sheddy to me; we should aim for a decent default, keeping in mind that users can override the exact output with a custom formatter.

Footnotes

1. maybe it should be called --latex, or --math-latex? I don't know/mind at all. ↩

[digitalmoksha]

I'm not sure why I didn't mention it before, but the real challenge in using delimiter's \(...\) and \[...\] is that in CommonMark, the \ escapes the following character. This means \( just gets rendered as (.

It also means that while parsing, how do you know if the user wants to use latex \(, or is purposely escaping the character \(.

That's why markdown contorted itself to have its own syntax using $ and $$. That's not even a standard, but is generally recognized as math syntax.

You could use \\(, etc, but then I think that defeats the purpose.

As for katex-rs, very cool. So I assume it would covert into either MATHML or SVG.

Coming from the angle of putting untrusted customer data through comrak, we do extensive sanitization. And while I've never verified this, I've always been hesitant to allow MathML and SVG due to this warning in the sanitize gem:

Sanitize cannot fully sanitize the contents of <math> or <svg> elements. MathML and SVG elements are foreign elements that don't follow normal HTML parsing rules.

By default, Sanitize will remove all MathML and SVG elements. If you add MathML or SVG elements to a custom element allowlist, you may create a security vulnerability in your application.

Therefore GitLab always does the math transformation on the frontend. But I could see how it would be useful in certain environments, where the content can be trusted.

That being the case, it might make more sense to use katex-rs via a custom formatter. If it was added as an option to comrak, I think I would want to see a warning about sanitization, for those that format untrusted data.

[nicoburns]

Thanks for the additional context!

As for katex-rs, very cool. So I assume it would covert into either MATHML or SVG.

I believe there's a MATHML option, but there is also styled plain HTML (try inspecting the rendered example on https://katex.org)

Therefore GitLab always does the math transformation on the frontend

When comrak is used in https://github.com/DioxusLabs/blitz, we are the frontend!

[digitalmoksha]

When comrak is used in https://github.com/DioxusLabs/blitz, we are the frontend!

Love it 😆

I think this still might make sense handled in a wrapper crate, where you can use a custom formatter. I would think you would want to be able to set exactly the katex options you want, as well as security options, which can be very different from other users.

[kivikakk]

I'm not sure why I didn't mention it before, but the real challenge in using delimiter's \(...\) and \[...\] is that in CommonMark, the \ escapes the following character. This means \( just gets rendered as (.

Indeed — we'd want to parse \( and \[ specially, and enter a different state until we find the matching \) or \]. This is tractable!

As for katex-rs, very cool. So I assume it would covert into either MATHML or SVG.

The examples on https://katex.org/ all use pure HTML with inline styles, so I think there's no concerns here.