Skip to content

docs.md

Latest commit

 

History

History
795 lines (653 loc) · 25.6 KB

docs.md

File metadata and controls

795 lines (653 loc) · 25.6 KB
layout page
title Writing documentation
permalink /guides/docs/
nav_order 4
parent Topical Guides

{% include toc.html %}

Writing documentation

Documentation used to require learning reStructuredText (sometimes referred to as reST / rST), but today we have great choices for documentation in markdown, the same format used by GitHub, Wikipedia, and others. This guide covers Sphinx and Mkdocs, and uses the modern MyST plugin to get Markdown support.

{: .note-title }

Popular frameworks

There are other frameworks as well; these often are simpler, but are not as commonly used, and have somewhat fewer examples and plugins. They are:

  • Sphinx: A popular documentation framework for scientific libraries with a history of close usage with scientific tools like LaTeX. Examples include astropy and corner.
  • MkDocs: A from-scratch new documentation system based on markdown and HTML. Less support for man pages & PDFs than Sphinx, since it doesn't use docutils. Has over 200 plugins - they are much easier to write than Sphinx. Example sites include hatch, PDM, cibuildwheel, Textual, pipx, Pydantic, Polars, and FastAPI
  • JupyterBook: A powerful system for rendering a collection of notebooks using Sphinx internally. Can also be used for docs, though, see echopype.

{: .warning-title }

The Future of MkDocs

The creators of mkdocs-material and mkdocstrings have come together to create a new documentation package called Zensical. The framework is still in alpha development, but aims to simplify the documentation process, be blazing fast, and move away from the limitations of MkDocs. This also means MkDocs's future is uncertain, and mkdocs-material will be minimally maintained until late 2026.

What to include

Ideally, software documentation should include:

  • Introductory tutorials, to help new users (or potential users) understand what the software can do and take their first steps.
  • Task-oriented guides, examples that address specific uses.
  • Reference, specifying the detailed inputs and outputs of every public object in the codebase.
  • Explanations to convey deeper understanding of why and how the software operates the way it does.

{: .note-title }

The Diátaxis framework

This overall framework has a name, Diátaxis, and you can read more about it if you are interested.

Hand-written docs

Create docs/ directory within your project (next to src/). From here, Sphinx and MkDocs diverge.

{% tabs %}{% tab sphinx Sphinx %}

pyproject.toml additions

Setting a docs dependency group looks like this:

[dependency-groups]
docs = [
  "furo",
  "myst_parser >=0.13",
  "sphinx >=4.0",
  "sphinx-copybutton",
  "sphinx-autodoc-typehints",
]

You should include the docs group via --group=docs when using uv or pip to install, or install all groups, such as by running uv sync --all-groups.

There is a sphinx-quickstart tool, but it creates unnecessary files (make/bat, we recommend a cross-platform noxfile instead), and uses rST instead of Markdown. Instead, this is our recommended starting point for conf.py:

conf.py

from __future__ import annotations

import importlib.metadata
from typing import Any

project = "package"
copyright = "2026, My Name"
author = "My Name"
version = release = importlib.metadata.version("package")

extensions = [
    "myst_parser",
    "sphinx.ext.autodoc",
    "sphinx.ext.intersphinx",
    "sphinx.ext.mathjax",
    "sphinx.ext.napoleon",
    "sphinx_autodoc_typehints",
    "sphinx_copybutton",
]

source_suffix = [".rst", ".md"]
exclude_patterns = [
    "_build",
    "**.ipynb_checkpoints",
    "Thumbs.db",
    ".DS_Store",
    ".env",
    ".venv",
]

html_theme = "furo"

html_theme_options: dict[str, Any] = {
    "footer_icons": [
        {
            "name": "GitHub",
            "url": "https://github.com/org/package",
            "html": """
                <svg stroke="currentColor" fill="currentColor" stroke-width="0" viewBox="0 0 16 16">
                    <path fill-rule="evenodd" d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0 0 16 8c0-4.42-3.58-8-8-8z"></path>
                </svg>
            """,
            "class": "",
        },
    ],
    "source_repository": "https://github.com/org/package",
    "source_branch": "main",
    "source_directory": "docs/",
}

myst_enable_extensions = [
    "colon_fence",
]

intersphinx_mapping = {
    "python": ("https://docs.python.org/3", None),
}

nitpick_ignore = [
    ("py:class", "_io.StringIO"),
    ("py:class", "_io.BytesIO"),
]

always_document_param_types = True

We start by setting some configuration values, but most notably we are getting the package version from the installed version of your package. We are listing several good extensions:

  • myst_parser is the Markdown parsing engine for Sphinx.
  • sphinx.ext.autodoc will help us build API docs via reStructuredText and dynamic analysis. Also see the package sphinx-autodoc2, which supports Markdown and uses static analysis; it might not be as battle tested at this time, though.
  • sphinx.ext.intersphinx will cross-link to other documentation.
  • sphinx.ext.mathjax allows you to include mathematical formulas.
  • sphinx.ext.napoleon adds support for several common documentation styles like numpydoc.
  • sphinx_autodoc_typehints handles type hints
  • sphinx_copybutton adds a handle little copy button to code snipits.

We are including both possible file extensions. We are also avoiding some common file patterns, just in case.

For theme, many scientific packages choose the pydata-sphinx-theme. The Furo theme is another popular choice. The site sphinx-themes.org can be used to compare options.

We are enabling a useful MyST extension: colon_fence allows you to use three colons for directives, which might be highlighted better if the directive contains text than three backticks. See more built-in extensions in MyST's docs.

One key feature of Sphinx is intersphinx, which allows documentation to cross-reference each other. You can list other projects you are using, but a good minimum is to at least link to the CPython docs. You need to provide the path to the objects.inv file, usually at the main documentation URL.

We are going to be enabling nitpick mode, and when we do, there's a chance some classes will complain if they don't link with intersphinx. A couple of common examples are listed here (StringIO/BytesIO don't point at the right thing) - feel free to add/remove as needed.

Finally, when we have static types, we'll want them always listed in the docstrings, even if the parameter isn't documented yet. Feel free to check sphinx-autodoc-typehints for more options.

index.md

Your index.md file can start out like this:

# package

```{toctree}
:maxdepth: 2
:hidden:

```

```{include} ../README.md
:start-after: <!-- SPHINX-START -->
```

## Indices and tables

- {ref}`genindex`
- {ref}`modindex`
- {ref}`search`

You can put your project name in as the title. The toctree directive houses your table of contents; you'll list each new page you add inside that directive.

If you want to inject a readme, you can use the include directive shown above. You don't want to add the README's title (and probably your badges) to your docs, so you can add a expression to your README (<!-- SPHINX-START --> above) to mark where you want the docs portion to start.

You can add the standard indices and tables at the end.

{% endtab %} {% tab mkdocs MkDocs %}

While the cookie cutter creates a basic structure for your MkDocs (a top level mkdocs.yml file and the docs directory), you can also follow the official Getting started guide instead.

If you selected the mkdocs option when using the template cookie-cutter repository, you will already have this group. Otherwise, add to your pyproject.toml:

[dependency-groups]
docs = [
    "markdown>=3.9",
    "mdx-include>=1.4.2",
    "mkdocs-material>=9.1.19",
    "mkdocs>=1.1.2,",
    "mkdocstrings-python>=1.18.2",
    "pyyaml>=6.0.1",
]

You should include the docs group via --group=docs when using uv or pip to install, or install all groups, such as by running uv sync --all-groups.

These dependencies include several common plugins---such as generating reference API documentation from docstrings---to make life easier.

Similar to Sphinx, MkDocs puts your written documentation into the /docs directory, but also has a top-level mkdocs.yml configuration file. You can see the minimal configuration for the file here, which is only four lines. However, the mkdocs.yml file bundled with the template repository have many options pre-configured. Let's run through an example configuration now.

Here's the whole file for completeness. We'll break it into sections underneath.

site_name: package
site_url: https://package.readthedocs.io/
site_author: "My Name"

repo_name: "org/package"
repo_url: "https://github.com/org/package"

theme:
  name: material
  icon:
    repo: fontawesome/brands/github
  features:
    - search.suggest
    - search.highlight
    - navigation.expand
    - navigation.tracking
    - toc.follow
  palette:
    # See options to customise your color scheme here:
    # https://squidfunk.github.io/mkdocs-material/setup/changing-the-colors/
    - media: "(prefers-color-scheme: light)"
      scheme: default
      toggle:
        icon: material/weather-sunny
        name: Switch to light mode
    - media: "(prefers-color-scheme: dark)"
      scheme: slate
      toggle:
        icon: material/weather-night
        name: Switch to dark mode

plugins:
  autorefs: {}
  mkdocstrings:
    handlers:
      python:
        paths: [.]
        inventories:
          - https://docs.python.org/3/objects.inv
          - https://docs.pydantic.dev/latest/objects.inv
        options:
          members_order: source
          separate_signature: true
          filters: ["!^_"]
          show_root_heading: true
          show_if_no_docstring: true
          show_signature_annotations: true
  search: {}

nav:
  - Home: index.md
  - Python API: api.md

First, the basic site metadata contains authors, repository details, URLs, etc:

site_name: some_project
site_url: https://some_project.readthedocs.io/
site_author: "Bruce Wayne"
repo_name: "wayne_industries/some_project"
repo_url: "https://github.com/wayne_industries/some_project"

After that, we can configure the visual theming for the the site. The repo icon is what appears in the top-right of the site next to the link to your GitHub/GitLab/etc, and you can peruse other FontAwesome icons here if the default GitHub or GitLab brand is unwanted.

Extra search features are documented here, and the three enabled are autocomplete for search suggestions (search.suggest) and highlighting search terms after a user clicks on a search result (search.highlight).

For navigation plugins (documented here), we request the side navigation to be expanded by default (navigation.expand) and that the URL autoupdate to the latest anchor as a user scrolls through the page (navigation.tracking). Finally, we request that the current user section is always shown and highlight in the sidebar via toc.follow.

In the palette section (documented here) you can easily modify the scheme, icons, primary colors, and accents for both light and dark themes.

theme:
  name: material
  icon:
    repo: fontawesome/brands/github
  features:
    - search.suggest
    - search.highlight
    - navigation.expand
    - navigation.tracking
    - toc.follow
  palette:
    # See options to customise your color scheme here:
    # https://squidfunk.github.io/mkdocs-material/setup/changing-the-colors/
    - media: "(prefers-color-scheme: light)"
      scheme: default
      toggle:
        icon: material/weather-sunny
        name: Switch to light mode
    - media: "(prefers-color-scheme: dark)"
      scheme: slate
      toggle:
        icon: material/weather-night
        name: Switch to dark mode

Onto the best part of MkDocs: it's many plugins!

  • search enabled search functionality.
  • autorefs allows easier linking across pages and anchors.
  • mkdocstrings lets you generate reference API documentation from your docstring.
plugins:
  autorefs: {}
  mkdocstrings:
    handlers:
      python:
        paths: [.]
        inventories:
          - https://docs.python.org/3/objects.inv
          - https://docs.pydantic.dev/latest/objects.inv
        options:
          members_order: source
          separate_signature: true
          filters: ["!^_"]
          show_root_heading: true
          show_if_no_docstring: true
          show_signature_annotations: true
  search: {}

Finally, we have to define the actual structure of our site by providing the primary navigation sidebar layout. Here we have three top-level links, one for the home page and one where all the generated API documentation from mkdocstrings will live.

nav:
  - Home: index.md
  - Python API: api.md

{% endtab %} {% endtabs %}

.readthedocs.yaml

In order to use https://readthedocs.org to build, host, and preview your documentation, you must have a .readthedocs.yaml file {% rr RTD100 %} like this:

{% tabs %} {% tab sphinx Sphinx %}

# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

version: 2

build:
  os: ubuntu-24.04
  tools:
    python: "3.13"
  commands:
    - asdf plugin add uv
    - asdf install uv latest
    - asdf global uv latest
    - uv sync --group docs
    - uv run python -m sphinx -T -b html -d docs/_build/doctrees -D language=en
      docs $READTHEDOCS_OUTPUT/html

{% endtab %} {% tab mkdocs MkDocs %}

# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

version: 2

build:
  os: ubuntu-24.04
  tools:
    python: "3.13"
  commands:
    - asdf plugin add uv
    - asdf install uv latest
    - asdf global uv latest
    - uv sync --group docs
    - uv run mkdocs build --site-dir $READTHEDOCS_OUTPUT/html

{% endtab %} {% endtabs %}

This sets the Read the Docs config version (2 is required) {% rr RTD101 %}.

The build table is the modern way to specify a runner. You need an os (a modern Ubuntu should be fine) {% rr RTD102 %} and a tools table (we'll use Python {% rr RTD103 %}, several languages are supported here).

Finally, we have a commands table which describes how to install our dependencies and build the documentation into the ReadTheDocs output directory.

noxfile.py additions

Add a session to your noxfile.py to generate docs:

{% tabs %} {% tab sphinx Sphinx %}

@nox.session(reuse_venv=True, default=False)
def docs(session: nox.Session) -> None:
    """
    Build the docs. Pass --non-interactive to avoid serving. First positional argument is the target directory.
    """

    doc_deps = nox.project.dependency_groups(PROJECT, "docs")
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "-b", dest="builder", default="html", help="Build target (default: html)"
    )
    parser.add_argument("output", nargs="?", help="Output directory")
    args, posargs = parser.parse_known_args(session.posargs)
    serve = args.builder == "html" and session.interactive

    session.install("-e.", *doc_deps, "sphinx-autobuild")

    shared_args = (
        "-n",  # nitpicky mode
        "-T",  # full tracebacks
        f"-b={args.builder}",
        "docs",
        args.output or f"docs/_build/{args.builder}",
        *posargs,
    )

    if serve:
        session.run("sphinx-autobuild", "--open-browser", *shared_args)
    else:
        session.run("sphinx-build", "--keep-going", *shared_args)

This is a more complex Nox job just because it's taking some options (the ability to build and serve instead of just build). The first portion is just setting up argument parsing so we can serve if building html. Then it does some conditional installs based on arguments (sphinx-autobuild is only needed if serving). It does an editable install of your package so that you can skip the install steps with -R and still get updated documentation.

Then there's a dedicated handler for the 'linkcheck' builder, which just checks links, and doesn't really produce output. Finally, we collect some useful args, and run either the autobuild (for --serve) or regular build. We could have just added python -m http.server pointing at the built documentation, but autobuild will rebuild if you change a file while serving.

{% endtab %} {% tab mkdocs MkDocs %}

@nox.session(reuse_venv=True, default=False)
def docs(session: nox.Session) -> None:
    """
    Make or serve the docs. Pass --non-interactive to avoid serving.
    """

    doc_deps = nox.project.dependency_groups(PROJECT, "docs")
    session.install("-e.", *doc_deps)

    if session.interactive:
        session.run("mkdocs", "serve", "--clean", *session.posargs)
    else:
        session.run("mkdocs", "build", "--clean", *session.posargs)

This Nox job will invoke MkDocs to serve a live copy of your documentation under a local endpoint, such as http://localhost:8080 (the link will be in the job output). By requesting a serve instead of a build, any time documentation or the source code is changed, the documentation will automatically update. For documentation on how to configure what directories are watched for changes, consult the MkDocs configuration page.

{% endtab %} {% endtabs %}

API docs

{% tabs %} {% tab sphinx Sphinx %}

To build API docs, you need to add the following Nox job. It will rerun sphinx-apidoc to generate the sphinx autodoc pages for each of your public modules.

noxfile.py additions

@nox.session(default=False)
def build_api_docs(session: nox.Session) -> None:
    """
    Build (regenerate) API docs.
    """

    session.install("sphinx")
    session.run(
        "sphinx-apidoc",
        "-o",
        "docs/api/",
        "--module-first",
        "--no-toc",
        "--force",
        "src/<package-name-here>",
    )

And you'll need this added to your docs/index.md:

```{toctree}
:maxdepth: 2
:hidden:
:caption: API

api/<package-name-here>
```

Note that your docstrings are still parsed as reStructuredText.

{% endtab %} {% tab mkdocs MkDocs %}

API documentation can be built from your docstring using the mkdocstrings plugin, as referenced previously. Unlike with Sphinx, which requires a direct invocation of sphinx-apidoc, MkDocs plugins are integrated into the MkDocs build.

All mkdocstrings requires is your markdown files to specify what module, class, or function you would like documented in said file. See the mkdocstring Usage page for more details, but for a minimal example, if you add an api.md file and set its contents to:

# Documentation for `my_package.my_module`

::: my_package.my_module

Where the triple colon syntax is used to specify what documentation you would like built. In this case, we are asking to document the entire module my_module (and all classes and functions within it) which is located in my_package. You could instead ask for only a single component inside your module by being more specific, like ::: my_package.my_module.MyClass.

{% endtab %} {% endtabs %}

Notebooks in docs

{% tabs %} {% tab sphinx Sphinx %}

You can combine notebooks into your docs. The tool for this is nbsphinx. If you want to use it, add nbsphinx and ipykernel to your documentation requirements, add "nbsphinx" to your conf.py's extensions = list, and add some options for nbsphinx in conf.py:

nbsphinx_execute = "auto"

nbsphinx_execute_arguments = [
    "--InlineBackend.figure_formats={'png2x'}",
    "--InlineBackend.rc=figure.dpi=96",
]

nbsphinx_kernel_name = "python3"

You can set nbsphinx_execute to always, never, or auto - auto will only execute empty notebooks. The execute arguments shown above will produce "retina" images from Matplotlib. You can set the kernel name (make sure you can execute all of your (unexecuted) notebooks).

Note that you will also need the pandoc command line tool installed locally for this to work. CI services like readthedocs usually have it installed.

If you want to use Markdown instead of notebooks, you can use jupytext (see here).

{% endtab %} {% tab mkdocs MkDocs %}

You can combine notebooks into your docs. The plugin for this is mkdocs-jupyter, and configuration is detailed here and you can find examples here.

Once you have a notebook (which has been run and populated with results, as the plugin will not execute your notebooks for you), you simply need to add a link to the notebook in your mkdocs.yml navigation.

nav:
  - Home: index.md
  - Notebook page: notebook.ipynb
  - Python file: python_script.py
plugins:
  - mkdocs-jupyter

Note that the mkdocs-jupyter plugin allows you to include both python scripts and notebooks. If you have a directory of example python files to run, consider mkdocs-gallery as an alternative. For an external example, the ChainConsumer docs show mkdocs-gallery in action.

{% endtab %} {% endtabs %}

<script src="{% link assets/js/tabs.js %}"></script>