test: add fast unit tests for LFS pointer detection#1945
test: add fast unit tests for LFS pointer detection#1945zhangyinxina-ui wants to merge 4 commits intodimensionalOS:devfrom
Conversation
…ipeline CI) (#4) Strictly additive scaffolding that lets `/ship <one-liner>` in Cursor chat drive: branch off origin/dev -> implement -> bash scripts/verify.sh -> PR targeting dev -> Codex review -> squash auto-merge. - scripts/verify.sh: fast+mypy single source of truth; auto-skips cuda and unitree-dds extras on macOS arm64 where wheels/build deps are unavailable. Slow tier and self-hosted CI remain authoritative. - AGENTS.md: append-only Agent Pipeline section + P0/P1/P2/P3 Codex review guidelines tailored to dimos (secrets, robot motion bounds, mypy strict, @Skill invariants, transport renames, all_blueprints regen). Existing content untouched. - .cursor/rules/00-workflow.mdc: alwaysApply rules forbidding direct push to dev/main, force push, --no-verify, neutering verify.sh, and encoding dimos-specific guardrails. - .cursor/commands/ship.md: canonical 11-step /ship flow, base=dev, uses gh REST API to dodge GraphQL race after push. - .github/workflows/agent-pipeline.yml: ubuntu-latest smoke job (verify-fast) -> ruff + pytest --collect-only + mypy. Does NOT replace ci.yml (Linux self-hosted) or macos.yml (macOS self-hosted). Local preflight: mypy dimos/ -> Success: no issues found in 650 source files. pytest left for CI (LFS data download too slow on first local run on this machine). Co-authored-by: update <zhang@local>
) The lightweight `verify-fast` job on `ubuntu-latest` failed during `uv sync` because `pyaudio==0.2.14` (transitively pulled in by `unitree-webrtc-connect-leshy` from the `dimos[unitree]` extras) needs `portaudio.h` to compile its native extension. ubuntu-latest does not ship portaudio headers by default. Mirror what `macos.yml` does (`brew install portaudio libjpeg-turbo`) on the apt side: install `portaudio19-dev`, `libjpeg-turbo8-dev`, and `libturbojpeg0-dev` before `uv sync`. This is also the demo PR validating the agent pipeline end-to-end: branch off origin/dev -> fix -> commit -> push -> open PR -> verify-fast green -> squash merge.
…ages (#6) Adds a callout to the "Adding a New Skill" section in AGENTS.md that spells out the existing package convention so contributors do not mistakenly create __init__.py files when adding skills/modules. The convention: every subpackage under dimos/ (including dimos/agents/skills/, dimos/core/, dimos/perception/, etc.) is a PEP 420 namespace package. The only __init__.py in the dimos/ tree is dimos/__init__.py itself, which exists solely as a lazy re-export shim for the top-level Dimos class. Adding __init__.py files would silently change import resolution for pytest collection, mypy, and the all_blueprints.py regen tooling that walk the dimos/ tree. Docs-only change — does not touch Python sources, test fixtures, or CI logic. agent-pipeline.yml's paths-ignore correctly skips verify-fast for **.md changes. Co-authored-by: update <zhang@local>
Cover _is_lfs_pointer_file with tmp_path fixtures: valid pointer, oversized file, plain text, missing path, and invalid UTF-8. No network or LFS pull. Made-with: Cursor
Greptile SummaryThis PR adds five fast pytest unit tests for The test logic itself is correct and covers all meaningful branches of Confidence Score: 4/5Safe to merge; P2 findings only — the tests themselves are correct and the existing All findings are P2 (style/design): the new CI workflow is a smoke check by design and existing
Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[PR opened / synchronized] --> B{agent-pipeline.yml}
A --> C{ci.yml - self-hosted Linux}
B --> D[uv sync]
D --> E[ruff check/format - non-blocking]
E --> F[pytest --collect-only\nvalidates imports only]
F --> G[mypy dimos/]
G --> H[bash -n verify.sh\nsyntax check only]
C --> I[uv sync]
I --> J[pytest - ACTUALLY RUNS TESTS\nnot tool or mujoco markers]
J --> K[mypy dimos/]
H --> L[agent-pipeline: PASS\nno tests executed]
K --> M[ci.yml: PASS\ntests executed]
style F fill:#f9a,stroke:#c00
style H fill:#f9a,stroke:#c00
style J fill:#9f9,stroke:#090
style L fill:#f9a,stroke:#c00
style M fill:#9f9,stroke:#090
Reviews (1): Last reviewed commit: "test: add fast unit tests for LFS pointe..." | Re-trigger Greptile |
| # Ruff issues are surfaced via annotations but do not block this | ||
| # smoke job; main `ci.yml` is the authoritative formatter gate. | ||
|
|
||
| - name: pytest --collect-only (catch import errors fast) | ||
| run: | | ||
| uv run pytest --collect-only -q -m 'not (tool or mujoco or slow)' dimos/ \ | ||
| > /tmp/pytest-collect.txt 2>&1 | ||
| tail -20 /tmp/pytest-collect.txt | ||
| # If imports break, this fails in seconds rather than 10 min into |
There was a problem hiding this comment.
CI pipeline never actually executes the test suite
The pytest step only runs --collect-only (validates imports and collection, not test logic), and verify.sh is only checked for shell syntax via bash -n. If a test in dimos/utils/test_data_lfs_pointer.py (or any other test) contains a logic error that makes it fail at runtime, this workflow will still pass. The actual test execution happens in ci.yml (self-hosted Linux runner), so tests aren't skipped entirely — but this workflow's value as a "fast second-opinion CI signal" for test correctness is limited since it never runs tests.
Additionally, AGENTS.md and verify.sh both state "Both local developers and CI invoke this exact script", but agent-pipeline.yml never calls verify.sh as an executable.
Consider replacing the collect-only + syntax-check steps with an actual execution of verify.sh (or at minimum uv run pytest -q) to make this workflow meaningful as a test gate:
- name: Run fast test suite
run: |
uv run pytest -q -m 'not (tool or mujoco or slow)' dimos/| echo "" | ||
| echo ">>> [2/3] uv run pytest -q (fast suite: excludes slow/tool/mujoco)" | ||
| uv run pytest -q | ||
| echo "<<< OK" |
There was a problem hiding this comment.
-q conflicts with -v in addopts
pyproject.toml sets addopts = "-s -v -r a ...", which includes -v (verbose). Passing -q on the command line decrements the verbosity level, so the two flags cancel each other out to default verbosity — not the quiet output the -q flag implies. The test suite still runs correctly, but the output format will be neither quiet nor verbose, which can make CI logs confusing.
| echo "<<< OK" | |
| uv run pytest |
Omitting -q lets addopts control verbosity as configured.
| echo "<<< OK" | |
| uv run pytest |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0682ec1fda
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| Darwin) UV_EXTRA_FLAGS+=(--no-extra cuda --no-extra unitree-dds) ;; | ||
| esac | ||
| echo "uv sync ${UV_EXTRA_FLAGS[*]}" | ||
| uv sync "${UV_EXTRA_FLAGS[@]}" |
There was a problem hiding this comment.
Run verify sync in frozen/locked mode
verify.sh is used as the local gate, but uv sync is invoked without --frozen/--locked, so it can silently update uv.lock during verification and pass with a dependency set that isn’t what CI will use. uv sync --help explicitly says --frozen is "Sync without updating the uv.lock file"; without it, stale lockfiles are auto-updated instead of failing fast. In practice this can produce false-green local verification and then CI failures when the lockfile change wasn’t committed.
Useful? React with 👍 / 👎.
|
Thank you for the contribution. But can you please break it up into separate PRs? You're making a lot of agent changes, including ones which imply a certain process. Those need to be discussed with the team and motivated. Also, please respect the PR description template which requires describing the problem each PR is address. There should be at least 2 PRs: one for the LFS changes and one for the agent changes. |
Description
dimos/utils/test_data_lfs_pointer.pywith five fast pytest cases for_is_lfs_pointer_file.git lfs pull; keeps the default fast test suite meaningful for data-layer helpers.How to Test
Optional single file:
Contributor License Agreement
Closes DIM-XXX