Fix CI Divergence (Mostly `mypy`) by jeff-hykin · Pull Request #1293 · dimensionalOS/dimos

jeff-hykin · 2026-02-18T21:32:10Z

READ B4 REVIEW

Why does the new CI workflow have a ARM/MacOS runner?
Why are there modifications to pyproject.toml?

Mypy divergence has been a miserable rabbit hole, so please bear with me
whether mypy fails/passes depends on EVERY version of EVERY dependency (because it inspects their source code)
- Easy fix right? uv sync --freeze <ci stuff>
- no. Because of the <ci stuff>
In principle uv sync <ci stuff> shouldn't fail on any system because we should be adding stuff like add "drake; only if linux" guards in our pyproject.toml. But our current pyproject.toml doesn't because we don't have any checks to prevent it (and not everyone should get a jetson and a mac just to test their changes to pyproject.toml)
This PR should prevent those pyproject.toml issues from getting on dev (and thus prevent CI divergence)

Why is `setup.py` modified?

Turns out, when testing with the macos-15 github runner, there's a MacOS install issue with dimos. When using a python executable that is universal (which is true for the uv python action on github runners) our C-compiler optimization arguments (e.g. -march=native in setup.py) cause uv sync to break. This is because clang can only perform that optimization if it targets specifically arm64 xor x86 MacOS (can't do it when targeting MacOS's universal executable format). The setup.py keeps the optimization whenever possible.

Why is there a `fast-docs.yml`?

Right now when I push doc changes, nothing actually checks the doc links or that diagrams are up to date (see https://github.com/dimensionalOS/dimos/actions/runs/22152132097/job/64046102043?pr=1291 for an example). This should fix that and test diagrams (which I don't think we were checking before)

…M machines

This reverts commit dd90ef4.

…s into jeff/fix/ci

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

…s into jeff/fix/gen-diagrams

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

jeff-hykin · 2026-02-18T22:27:00Z

.github/workflows/fast-docs.yml

+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'


In this workflow (documentation checks) the version shouldn't matter

jeff-hykin · 2026-02-18T22:27:24Z

.github/workflows/fast-docs.yml

+        with:
+          files: 'docs/**/*.md'
+          fail-on-change: true
+          # maybe later we can enforce python, for now just do diagram langs


Previously we weren't checking diagram generation AFAIK, this adds that

jeff-hykin · 2026-02-18T22:28:15Z

.github/workflows/fast-test.yml

+
+      - name: Install System dependencies
+        run: |
+          # these lines should stay exactly in sync with docs/development/README.md


This is an important change: we should have something on CI that matches our docs, and this new test does that

jeff-hykin · 2026-02-18T22:31:03Z

.github/workflows/fast-test.yml

+  pytest_fast:
+    strategy:
+      matrix:
+        os: [ubuntu-24.04, macos-15, ubuntu-24.04-arm]


Discussion needed: do we want to host our own macos / linux-arm testers? (or keep this as-is and run on github)

jeff-hykin · 2026-02-18T22:33:18Z

bin/ci-fast

@@ -0,0 +1,26 @@
+#!/usr/bin/env bash


This file is the overlap between local and CI: this file gets executed by CI and is executed locally (when running bin/check-pr)

jeff-hykin · 2026-02-18T22:35:44Z

bin/ci-fast

+
+echo "Running mypy checks"
+time uv run mypy --python-version 3.10 dimos
+time uv run mypy --python-version 3.12 dimos


This could be faster by excuting in parallel, but I opted for simplicity. No cache ~5min on github runners, so its pretty fast

jeff-hykin · 2026-02-18T22:38:26Z

bin/gen-diagrams

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+set -euo pipefail


Merge #1291 (very small) first and this diff will go away

jeff-hykin · 2026-02-18T22:41:14Z

bin/pr-check

@@ -0,0 +1,99 @@
+#!/usr/bin/env python3


This is what everyone should run locally before making a pull request.
It

checks branch name (you can ignore the error and keep going)

runs only doclink checks if only markdown is edited

runs bin/ci-fast if more than just markdown is edited

A lot of good enhancements could be added, but I wanted to start simple

Why just the fast tests? Shouldn't people run all the tests? (The non-fast ones are more commonly broken on dev especially since a few of them don't run in CI at all.)

jeff-hykin · 2026-02-18T22:44:02Z

bin/pr-name-check

@@ -0,0 +1,68 @@
+#!/usr/bin/env bash


I don't care which branch-naming-scheme we use, I just implemented a check for the one that is listed on our wiki <name>/<type>/<description>

Right now its not a forceful/blocking check it'll just yell at you if it doesnt match

jeff-hykin · 2026-02-18T22:47:17Z

docker/python/Dockerfile

 # Install dependencies with UV (10-100x faster than pip)
 RUN uv pip install --upgrade 'pip>=24' 'setuptools>=70' 'wheel' 'packaging>=24' && \
-    uv pip install '.[misc,cpu,sim,drone,unitree,web,perception,visualization]'
+    uv sync --frozen --extra misc --extra visualization --extra agents --extra web --extra perception --extra unitree --extra manipulation --extra cpu --extra dev --extra sim --extra drone --extra base


Most Important Change

--frozen instead of "close enough" install

An issue I see here is that this forces us to always run with uv run .... This should be fixable by adding some of the variables which activating gives you. This could be enough:

ENV VIRTUAL_ENV=/app/.venv ENV PATH="/app/.venv/bin:$PATH"

Another issue that worries me is that uv sync always installs in .venv. The previous command installed in the system python. By installing in /app it means we can't mount our local dir as /app because it would use the .venv we have locally instead of the one in the docker image.

Not sure what the solution is here. (Maybe copy pyproject.toml to /app-deps and install from that dir so the packages are installed outside /app?

jeff-hykin · 2026-02-18T22:47:52Z

docs/development/writing_docs.md

    - If it only matters to people who contribute to dimos (like this doc), put them in `docs/development`
    - Otherwise put them in `docs/usage`
-2. Run `bin/gen_diagrams` to generate the svg's for your diagrams. We use [pikchr](https://pikchr.org/home/doc/trunk/doc/userman.md) as a diagram language.
+2. Run `bin/gen-diagrams` to generate the svg's for your diagrams. We use [mermaid](https://mermaid.js.org/intro/) (no generation needed) and [pikchr](https://pikchr.org/home/doc/trunk/doc/userman.md) as diagrams languages.


Merge #1291 (very small) first and this diff will go away

jeff-hykin · 2026-02-18T23:12:54Z

.github/workflows/docker.yml

      cmd: "MYPYPATH=/opt/ros/humble/lib/python3.10/site-packages mypy dimos"
      dev-image: ros-dev:${{ (needs.check-changes.outputs.python == 'true' || needs.check-changes.outputs.dev == 'true' || needs.check-changes.outputs.ros == 'true') && needs.ros-dev.result == 'success' && needs.check-changes.outputs.branch-tag || 'dev' }}

-  # Run module tests directly to avoid pytest forking issues


If we need to restore old code, lets use git as it was meant to be (instead of having giant blocks of commented out code

jeff-hykin · 2026-02-18T23:13:27Z

.github/workflows/docker.yml

    with:
-      cmd: "pytest -m heavy"
+      cmd: "uv run pytest -m heavy"
      dev-image: dev:${{ (needs.check-changes.outputs.python == 'true' || needs.check-changes.outputs.dev == 'true') && needs.dev.result == 'success' && needs.check-changes.outputs.branch-tag || 'dev' }}


I think this is a nessary change, based on: https://github.com/dimensionalOS/dimos/actions/runs/22160016061/job/64077594237?pr=1293

jeff-hykin · 2026-02-18T23:20:36Z

pyproject.toml

    # Planning (Drake)
-    "drake>=1.40.0; platform_machine != 'aarch64'",
+    "drake==1.45.0; sys_platform == 'darwin' and platform_machine != 'aarch64'",
+    "drake>=1.40.0; sys_platform != 'darwin' and platform_machine != 'aarch64'",


This is both a fix for MacOS and Arm. On MacOS drake 1.45.0 is needed, on x86 arm anything >1.40 is okay.

The drake team says they're interested in supporting Arm linux so hopefully that'll come soon

jeff-hykin · 2026-02-18T23:21:52Z

pyproject.toml

 cpu = [
    # CPU inference backends
-    "onnxruntime",
+    "onnxruntime<1.24; python_version < '3.11'",


CI caught this which is great: it was impossible for python 3.10 to have the version of onnx we had specified in the lock file. This guard fixes that

jeff-hykin · 2026-02-18T23:23:11Z

setup.py

 from setuptools import find_packages, setup

+
+def python_is_macos_universal_binary(executable: str | None = None) -> bool:


without this change macos-15 github uv python runner can't uv sync

jeff-hykin · 2026-02-18T23:25:34Z

@greptile-apps please re-review. Edit your original message if you are able to

greptile-apps

_{15 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2026-02-18T23:31:57Z

.github/workflows/fast-docs.yml

+on:
+  pull_request:
+      - main
+      - dev
+  paths:
+      - 'docs/**'


Invalid YAML structure — pull_request needs branches: key and paths must be at same indentation level as pull_request:

Suggested change

on:

pull_request:

- main

- dev

paths:

- 'docs/**'

on:

pull_request:

branches:

- main

- dev

paths:

- 'docs/**'

.github/workflows/fast-test.yml

greptile-apps · 2026-02-18T23:31:59Z

.github/workflows/fast-test.yml

+            # install homebrew
+            /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"


Installing Homebrew on every CI run is slow (~5-10 minutes) and unnecessary. GitHub's macos-15 runners come with Homebrew pre-installed. Remove these lines:

Suggested change

# install homebrew

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

elif [ "$(uname)" = "Darwin" ]; then

# install dependencies

huge fantastic

paul-nechifor · 2026-02-19T03:13:01Z

bin/ci-slow

+
+echo
+echo "== dev: pytest -m integration =="
+run_in_image "dev" "pytest -m integration"


If you're running all of them sequentially, you might as well run a single command:

run_in_image "dev" "-m 'not (tool or module or neverending or mujoco)'"

bin/pr-name-check

Co-authored-by: Paul Nechifor <paul@nechifor.net>

paul-nechifor · 2026-02-19T03:50:27Z

.github/workflows/docker.yml

    secrets: inherit
    with:
-      cmd: "pytest && pytest -m ros" # run tests that depend on ros as well
+      cmd: "uv run pytest && uv run pytest -m ros" # run tests that depend on ros as well


By changing environment variables we can make the uv python be the default python so that we don't need to prefix with uv run (mentioned in a comment below)

paul-nechifor · 2026-02-19T03:53:33Z

bin/ci-fast

+echo "# pytest fast"
+echo "# "
+echo "# "
+time uv run bin/pytest-fast


pytest-fast has this:

. .venv/bin/activate exec pytest "$@" dimos

You probably just want to run pytest without any -m, not ./bin/pytest-fast.

paul-nechifor · 2026-02-19T03:59:11Z

bin/ci-fast

+set -euo pipefail
+
+echo "Ensuring pip versions are correct"
+time uv sync --frozen --extra misc --extra visualization --extra agents --extra web --extra perception --extra unitree --extra manipulation --extra cpu --extra dev --extra sim --extra drone --extra base


I'm not sure this should be here. In CI, the packages should have been installed in the python step, no?

And locally this removes packages. (I do uv sync --all-extras locally.)

leshy · 2026-02-19T08:02:52Z

.github/workflows/fast-docs.yml

+        with:
+          python-version: '3.12'
+
+      - name: Run doclinks # Note: doesn't need uv python, just needs some python to run the script


git-commit hooks run this, so this is already executed by our code-cleanup action in CI

jeff-hykin and others added 30 commits February 10, 2026 09:28

add arm support

98725c5

-

8696799

Update dependency specifications for open3d to support ARM and non AR…

5913387

…M machines

Merge branch 'dev' into jeff/arm_support

3470deb

patch for CI

cf097a2

fix mypy errors

17fabf8

Fix: conditionally include xcbuild for Darwin platforms only

87702f4

Fix: conditionally exclude packages without ARM wheels

f6e6794

format

3318e91

try to make mypy CI happy

9c4ef0f

remove unused

37f05dc

Merge branch 'dev' into jeff/arm_support

d4f53ca

Merge branch 'dev' into jeff/arm_support

bf457f1

basic CI test

bd3b49d

only run on python changes

dd90ef4

Revert "only run on python changes"

bda7800

This reverts commit dd90ef4.

clean up

7b2309d

test

5d70fac

-

366599e

Merge branch 'dev' into jeff/fix/ci

5d58aaa

Merge branch 'dev' into jeff/fix/ci

aa763eb

add a fast docs checker

ce0110e

add missing command

a5cff27

Merge branch 'jeff/fix/gen-diagrams' of github.com:dimensionalOS/dimo…

fefcf26

…s into jeff/fix/ci

Apply suggestion from @greptile-apps[bot]

f254a3c

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

fix

aaa4e98

Merge branch 'jeff/fix/gen-diagrams' of github.com:dimensionalOS/dimo…

028429a

…s into jeff/fix/gen-diagrams

Update bin/gen-diagrams

1056faa

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

-

7c29734

Merge branch 'dev' into jeff/arm_support

ba7be91

jeff-hykin commented Feb 18, 2026

View reviewed changes

jeff-hykin changed the title ~~Fix CI Divergence~~ Fix CI Divergence (Mostly mypy) Feb 18, 2026

jeff-hykin added 2 commits February 18, 2026 17:08

fixup minor aspects

e0849db

Merge branch 'dev' of github.com:dimensionalOS/dimos into jeff/fix/ci

7d1cd3e

jeff-hykin commented Feb 18, 2026

View reviewed changes

greptile-apps bot reviewed Feb 18, 2026

View reviewed changes

paul-nechifor reviewed Feb 19, 2026

View reviewed changes

bin/pr-name-check Outdated Show resolved Hide resolved

paul-nechifor reviewed Feb 19, 2026

View reviewed changes

bin/pr-name-check Show resolved Hide resolved

jeff-hykin and others added 2 commits February 18, 2026 21:33

Update bin/pr-name-check

214e748

Co-authored-by: Paul Nechifor <paul@nechifor.net>

Update bin/pr-name-check

25aedba

Co-authored-by: Paul Nechifor <paul@nechifor.net>

paul-nechifor reviewed Feb 19, 2026

View reviewed changes

leshy reviewed Feb 19, 2026

View reviewed changes

		from setuptools import find_packages, setup


		def python_is_macos_universal_binary(executable: str \| None = None) -> bool:

		# install homebrew
		/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Conversation

jeff-hykin commented Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

READ B4 REVIEW

Why does the new CI workflow have a ARM/MacOS runner? Why are there modifications to pyproject.toml?

Why is setup.py modified?

Why is there a fast-docs.yml?

Uh oh!

jeff-hykin Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeff-hykin Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeff-hykin Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeff-hykin Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeff-hykin Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeff-hykin Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paul-nechifor Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeff-hykin Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Most Important Change

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeff-hykin Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeff-hykin commented Feb 18, 2026

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

greptile-apps bot Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paul-nechifor Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

jeff-hykin commented Feb 18, 2026 •

edited

Loading

Why does the new CI workflow have a ARM/MacOS runner?
Why are there modifications to pyproject.toml?

Why is `setup.py` modified?

Why is there a `fast-docs.yml`?

jeff-hykin Feb 18, 2026 •

edited

Loading

jeff-hykin Feb 18, 2026 •

edited

Loading

jeff-hykin Feb 18, 2026 •

edited

Loading

jeff-hykin Feb 18, 2026 •

edited

Loading

jeff-hykin Feb 18, 2026 •

edited

Loading

jeff-hykin Feb 18, 2026 •

edited

Loading

paul-nechifor Feb 19, 2026 •

edited

Loading

jeff-hykin Feb 18, 2026 •

edited

Loading

jeff-hykin Feb 18, 2026 •

edited

Loading

paul-nechifor Feb 19, 2026 •

edited

Loading

paul-nechifor Feb 19, 2026 •

edited

Loading

leshy Feb 19, 2026 •

edited

Loading