Skip to content

CI: run vendored template suites against the SDK checkout - #513

Draft
jdchawla29 wants to merge 1 commit into
jaideep/vendor-environment-templatesfrom
jaideep/templates-ci
Draft

CI: run vendored template suites against the SDK checkout#513
jdchawla29 wants to merge 1 commit into
jaideep/vendor-environment-templatesfrom
jaideep/templates-ci

Conversation

@jdchawla29

@jdchawla29 jdchawla29 commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #512.

Adds a templates CI job: for each of environments/{coding,cua}, uv sync the template's pinned dependencies, install the SDK from the checkout over the hud pin (editable), then run the template's own ruff format --check, ruff check, and pytest.

This makes template breakage an SDK PR failure rather than something discovered after release. The coding template's suite includes a full local rollout (serve → connect → run → grade, golden=1.0/baseline=0.0), so the v6 serving contract is exercised end to end on every PR. Docker-dependent tests self-skip; those move to the nightly/release-gate job in a follow-up.


Note

Low Risk
CI-only change with no runtime or SDK behavior modifications; risk is limited to longer CI runs or flaky template tests blocking merges.

Overview
Adds a templates CI job that runs on every PR alongside the existing SDK test matrix.

For each vendored template (environments/coding and environments/cua), the job syncs the template’s pinned deps, replaces the published hud pin with an editable install of the repo root (uv pip install -e ../..), then runs that template’s ruff format (check), ruff check, and pytest.

This turns template breakage into an SDK PR failure instead of a post-release discovery. The coding template’s pytest suite includes hermetic local rollout coverage (serve → connect → run → grade), so the v6 serving contract is exercised end-to-end on every PR; Docker-only integration tests are expected to skip in this job.

Reviewed by Cursor Bugbot for commit 5b253c2. Bugbot is set up for automated code reviews on this repo. Configure here.

HUD-2276

@jdchawla29
jdchawla29 marked this pull request as draft July 27, 2026 03:21
@jdchawla29
jdchawla29 force-pushed the jaideep/vendor-environment-templates branch from ea60e77 to 76ffbed Compare July 31, 2026 02:49
@jdchawla29
jdchawla29 marked this pull request as ready for review July 31, 2026 02:50
Each environments/ template installs its pinned deps, overrides hud with
the checkout via an editable install, then runs its own ruff and pytest.
Catches SDK changes that break the canonical templates at the PR boundary.
@jdchawla29
jdchawla29 force-pushed the jaideep/templates-ci branch from 8c98a9c to 5b253c2 Compare July 31, 2026 02:54

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5b253c2437

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/ci.yml
- name: Install template with the SDK from this checkout
working-directory: environments/${{ matrix.template }}
run: |
uv sync --all-extras

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Enforce the checked-in template locks

For both matrix entries, if a PR changes environments/<template>/pyproject.toml without updating its uv.lock, this command silently regenerates the lock in the runner and tests that uncommitted state, allowing CI to pass while the checked-in reproducible environment remains stale. uv sync --help documents --locked as “Assert that the uv.lock will remain unchanged”; use that option here so dependency and lock changes must be committed together.

Useful? React with 👍 / 👎.

Comment thread .github/workflows/ci.yml
working-directory: environments/${{ matrix.template }}
run: |
uv sync --all-extras
uv pip install -e ../..

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Resolve the checkout in a clean template dependency graph

When the checkout removes a dependency previously supplied by the released hud package, the preceding sync installs that old dependency from the template lock and this additive editable install leaves it behind. Template tests can therefore keep importing an undeclared dependency and pass even though a clean installation against the checkout would fail; uv pip install --help confirms that removing extraneous packages requires --exact. Resolve hud from ../.. as part of the template environment rather than layering it over the released package's dependency set.

Useful? React with 👍 / 👎.

@jdchawla29
jdchawla29 marked this pull request as draft July 31, 2026 03:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant