From 2b0ae038d782238b439dc9e335a229526409d1d4 Mon Sep 17 00:00:00 2001 From: James Devine Date: Sun, 3 May 2026 09:07:46 +0100 Subject: [PATCH 1/7] feat(compile): replace Python gate evaluator with bundled TypeScript gate.js MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Migrates the trigger-filter gate evaluator from `scripts/gate-eval.py` to a bundled TypeScript artifact `scripts/gate.js` produced by a new `scripts/ado-script/` workspace. The compiler now emits a `NodeTool@0` install step and `node /tmp/ado-aw-scripts/gate.js` instead of `python3`. Architecture (variant A2 in the design walkthrough): - TypeScript workspace at scripts/ado-script/ built with @vercel/ncc. - shared/ modules (auth, ado-client, env-facts, policy state machine, vso-logger) reusable across future bundles. - gate/ entry implementing bypass, fact acquisition, 11 predicate evaluators, and self-cancel — full parity with the deleted Python evaluator (45/45 tests ported, +6 parity guards). Drift-proof codegen: - New hidden CLI subcommand `ado-aw export-gate-schema` emits a schemars-derived JSON Schema from the Rust IR types. - `npm run codegen` chains it through json-schema-to-typescript to produce src/shared/types.gen.ts. - New CI workflow .github/workflows/ado-script.yml runs codegen + `git diff --exit-code` to fail on IR/TS schema drift. Pipeline integration: - TriggerFiltersExtension prepends a NodeTool@0 step pinned to 20.x. - compile_gate_step_external invokes `node` instead of `python3`. - release.yml builds the bundle (npm ci && npm run build) before zipping scripts/, and excludes node_modules/dist/schema from the zip. - New tests/gate_e2e.rs (#[ignore]'d) compiles a real agent, extracts GATE_SPEC, runs gate.js end-to-end, and asserts SHOULD_RUN. - compiler_tests.rs assertions tightened: now check for `node '/tmp/ado-aw-scripts/gate.js'` instead of the loose `python3` match (which falsely passed via base.yml's mcpg-config validation). Cleanup: - Deleted scripts/gate-eval.py, scripts/gate-spec.schema.json, tests/gate_eval_tests.py. Documentation: - New docs/ado-script.md records the A2 decision, codegen pipeline, bundle-size budget (5 MB; gate.js is ~1.1 MB), and how to add new internal bundles (e.g. poll.js). - docs/filter-ir.md rewritten: Node evaluator, NodeTool@0 step, scripts.zip distribution. - AGENTS.md tree + tech stack updated; new entry in docs index. - ado-script-design.md added at repo root as the design walkthrough that produced the A2 decision. Validation: - 173/173 vitest tests pass (45 ports + 6 parity guards + smoke + shared-module units). - Full cargo test suite green. - cargo clippy --all-targets --all-features clean. - E2E: `cd scripts/ado-script && npm run build && cd ../.. && cargo test --test gate_e2e -- --ignored` passes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/workflows/ado-script.yml | 66 + .github/workflows/release.yml | 19 +- .gitignore | 1 + AGENTS.md | 8 +- ado-script-design.md | 235 ++ docs/ado-script.md | 168 ++ docs/filter-ir.md | 67 +- scripts/ado-script/.gitignore | 4 + scripts/ado-script/README.md | 28 + scripts/ado-script/package-lock.json | 1994 +++++++++++++++++ scripts/ado-script/package.json | 29 + scripts/ado-script/src/.gitkeep | 0 .../src/gate/__tests__/ports/INVENTORY.md | 127 ++ .../src/gate/__tests__/ports/equals.test.ts | 23 + .../gate/__tests__/ports/file-glob.test.ts | 68 + .../src/gate/__tests__/ports/glob.test.ts | 48 + .../src/gate/__tests__/ports/helpers.ts | 21 + .../gate/__tests__/ports/integration.test.ts | 34 + .../gate/__tests__/ports/label-set.test.ts | 82 + .../src/gate/__tests__/ports/logical.test.ts | 51 + .../__tests__/ports/numeric-range.test.ts | 36 + .../__tests__/ports/predicate-facts.test.ts | 29 + .../gate/__tests__/ports/ref-prefix.test.ts | 33 + .../gate/__tests__/ports/time-window.test.ts | 30 + .../gate/__tests__/ports/value-set.test.ts | 63 + scripts/ado-script/src/gate/bypass.test.ts | 50 + scripts/ado-script/src/gate/bypass.ts | 22 + scripts/ado-script/src/gate/facts.test.ts | 247 ++ scripts/ado-script/src/gate/facts.ts | 120 + scripts/ado-script/src/gate/index.ts | 60 + .../ado-script/src/gate/predicates.test.ts | 420 ++++ scripts/ado-script/src/gate/predicates.ts | 163 ++ .../ado-script/src/gate/selfcancel.test.ts | 86 + scripts/ado-script/src/gate/selfcancel.ts | 26 + .../ado-script/src/shared/ado-client.test.ts | 115 + scripts/ado-script/src/shared/ado-client.ts | 84 + scripts/ado-script/src/shared/auth.test.ts | 38 + scripts/ado-script/src/shared/auth.ts | 44 + .../ado-script/src/shared/env-facts.test.ts | 77 + scripts/ado-script/src/shared/env-facts.ts | 60 + scripts/ado-script/src/shared/index.ts | 5 + scripts/ado-script/src/shared/policy.test.ts | 114 + scripts/ado-script/src/shared/policy.ts | 136 ++ scripts/ado-script/src/shared/types.gen.ts | 113 + .../ado-script/src/shared/vso-logger.test.ts | 63 + scripts/ado-script/src/shared/vso-logger.ts | 54 + .../fixtures/gate-spec-pr-title-match.json | 22 + scripts/ado-script/test/smoke.test.ts | 67 + scripts/ado-script/tsconfig.json | 17 + scripts/gate-eval.py | 388 ---- scripts/gate-spec.schema.json | 366 --- src/compile/extensions/trigger_filters.rs | 59 +- src/compile/filter_ir.rs | 255 ++- src/main.rs | 65 +- tests/compiler_tests.rs | 477 ++-- tests/export_gate_schema.rs | 30 + tests/gate_e2e.rs | 147 ++ tests/gate_eval_tests.py | 359 --- 58 files changed, 6206 insertions(+), 1407 deletions(-) create mode 100644 .github/workflows/ado-script.yml create mode 100644 ado-script-design.md create mode 100644 docs/ado-script.md create mode 100644 scripts/ado-script/.gitignore create mode 100644 scripts/ado-script/README.md create mode 100644 scripts/ado-script/package-lock.json create mode 100644 scripts/ado-script/package.json create mode 100644 scripts/ado-script/src/.gitkeep create mode 100644 scripts/ado-script/src/gate/__tests__/ports/INVENTORY.md create mode 100644 scripts/ado-script/src/gate/__tests__/ports/equals.test.ts create mode 100644 scripts/ado-script/src/gate/__tests__/ports/file-glob.test.ts create mode 100644 scripts/ado-script/src/gate/__tests__/ports/glob.test.ts create mode 100644 scripts/ado-script/src/gate/__tests__/ports/helpers.ts create mode 100644 scripts/ado-script/src/gate/__tests__/ports/integration.test.ts create mode 100644 scripts/ado-script/src/gate/__tests__/ports/label-set.test.ts create mode 100644 scripts/ado-script/src/gate/__tests__/ports/logical.test.ts create mode 100644 scripts/ado-script/src/gate/__tests__/ports/numeric-range.test.ts create mode 100644 scripts/ado-script/src/gate/__tests__/ports/predicate-facts.test.ts create mode 100644 scripts/ado-script/src/gate/__tests__/ports/ref-prefix.test.ts create mode 100644 scripts/ado-script/src/gate/__tests__/ports/time-window.test.ts create mode 100644 scripts/ado-script/src/gate/__tests__/ports/value-set.test.ts create mode 100644 scripts/ado-script/src/gate/bypass.test.ts create mode 100644 scripts/ado-script/src/gate/bypass.ts create mode 100644 scripts/ado-script/src/gate/facts.test.ts create mode 100644 scripts/ado-script/src/gate/facts.ts create mode 100644 scripts/ado-script/src/gate/index.ts create mode 100644 scripts/ado-script/src/gate/predicates.test.ts create mode 100644 scripts/ado-script/src/gate/predicates.ts create mode 100644 scripts/ado-script/src/gate/selfcancel.test.ts create mode 100644 scripts/ado-script/src/gate/selfcancel.ts create mode 100644 scripts/ado-script/src/shared/ado-client.test.ts create mode 100644 scripts/ado-script/src/shared/ado-client.ts create mode 100644 scripts/ado-script/src/shared/auth.test.ts create mode 100644 scripts/ado-script/src/shared/auth.ts create mode 100644 scripts/ado-script/src/shared/env-facts.test.ts create mode 100644 scripts/ado-script/src/shared/env-facts.ts create mode 100644 scripts/ado-script/src/shared/index.ts create mode 100644 scripts/ado-script/src/shared/policy.test.ts create mode 100644 scripts/ado-script/src/shared/policy.ts create mode 100644 scripts/ado-script/src/shared/types.gen.ts create mode 100644 scripts/ado-script/src/shared/vso-logger.test.ts create mode 100644 scripts/ado-script/src/shared/vso-logger.ts create mode 100644 scripts/ado-script/test/fixtures/gate-spec-pr-title-match.json create mode 100644 scripts/ado-script/test/smoke.test.ts create mode 100644 scripts/ado-script/tsconfig.json delete mode 100644 scripts/gate-eval.py delete mode 100644 scripts/gate-spec.schema.json create mode 100644 tests/export_gate_schema.rs create mode 100644 tests/gate_e2e.rs delete mode 100644 tests/gate_eval_tests.py diff --git a/.github/workflows/ado-script.yml b/.github/workflows/ado-script.yml new file mode 100644 index 00000000..0b0647a8 --- /dev/null +++ b/.github/workflows/ado-script.yml @@ -0,0 +1,66 @@ +name: ado-script Workspace + +on: + pull_request: + paths: + - "scripts/ado-script/**" + - "src/compile/filter_ir.rs" + - "Cargo.toml" + - "Cargo.lock" + - ".github/workflows/ado-script.yml" + +env: + CARGO_TERM_COLOR: always + +jobs: + ado-script: + name: Build, Test & Drift-Check + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - uses: dtolnay/rust-toolchain@stable + + - uses: Swatinem/rust-cache@v2 + + - uses: actions/setup-node@v4 + with: + node-version: "20" + cache: "npm" + cache-dependency-path: scripts/ado-script/package-lock.json + + - name: Install workspace dependencies + working-directory: scripts/ado-script + run: npm ci + + - name: Regenerate types from Rust IR (codegen) + working-directory: scripts/ado-script + run: npm run codegen + + - name: Verify generated TypeScript is up to date + run: | + if ! git diff --exit-code -- scripts/ado-script/src/shared/types.gen.ts; then + echo "" + echo "::error::types.gen.ts is out of date with the Rust IR." + echo "Run 'cd scripts/ado-script && npm run codegen' and commit the result." + exit 1 + fi + + - name: Run TypeScript tests + working-directory: scripts/ado-script + run: npm test + + - name: Type-check + working-directory: scripts/ado-script + run: npm run typecheck + + - name: Build bundle (gate.js) + working-directory: scripts/ado-script + run: npm run build + + - name: Smoke-test bundle + working-directory: scripts/ado-script + run: npx vitest run test/smoke.test.ts + + - name: E2E gate test + run: cargo test --test gate_e2e -- --ignored --nocapture diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index b2090746..1912b6ac 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -56,11 +56,28 @@ jobs: cd target/release cp ado-aw ado-aw-linux-x64 + - name: Set up Node.js for ado-script bundle + uses: actions/setup-node@v4 + with: + node-version: "20" + + - name: Build ado-script TypeScript bundle (gate.js) + working-directory: scripts/ado-script + run: | + npm ci + npm run build + # `npm run build` runs codegen + ncc + copies dist/gate/index.js + # to ../gate.js (i.e. scripts/gate.js), which is then included in + # scripts.zip by the next step. + - name: Package scripts bundle run: | set -euo pipefail cd scripts - zip -r ../scripts.zip . + zip -r ../scripts.zip . \ + -x "ado-script/node_modules/*" \ + -x "ado-script/dist/*" \ + -x "ado-script/schema/*" - name: Upload release assets env: diff --git a/.gitignore b/.gitignore index 05451379..78d855eb 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,5 @@ target examples/sample-agent.yml +scripts/gate.js *.pyc __pycache__/ diff --git a/AGENTS.md b/AGENTS.md index 84a93381..8b8743e9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -120,8 +120,8 @@ Every compiled pipeline runs as three sequential jobs: ├── ado-aw-derive/ # Proc-macro crate: #[derive(SanitizeConfig)], #[derive(SanitizeContent)] ├── examples/ # Example agent definitions ├── scripts/ # Supporting scripts shipped as release artifacts -│ ├── gate-eval.py # Python gate evaluator (data-driven filter evaluation) -│ └── gate-spec.schema.json # JSON Schema for gate spec (generated from Rust types) +│ ├── ado-script/ # TypeScript workspace for bundled gate.js (and future bundles) +│ └── gate.js # Bundled gate evaluator (built from scripts/ado-script/, see docs/ado-script.md) ├── tests/ # Integration tests and fixtures ├── docs/ # Per-concept reference documentation (see index below) ├── Cargo.toml # Rust dependencies @@ -133,6 +133,7 @@ Every compiled pipeline runs as three sequential jobs: - **Language**: Rust (2024 edition) - Note: Rust 2024 edition exists and is the edition used by this project - **CLI Framework**: clap v4 with derive macros - **Error Handling**: anyhow for ergonomic error propagation +- **Bundled scripts**: TypeScript + ncc (`scripts/ado-script/`) — compiled gate evaluator and future internal helpers; see [`docs/ado-script.md`](docs/ado-script.md). - **Async Runtime**: tokio with full features - **YAML Parsing**: serde_yaml - **MCP Server**: rmcp with server and transport-io features @@ -183,6 +184,9 @@ index to jump to the right page. - [`docs/filter-ir.md`](docs/filter-ir.md) — filter expression IR specification: `Fact`/`Predicate` types, three-pass compilation (lower → validate → codegen), gate step generation, adding new filter types. +- [`docs/ado-script.md`](docs/ado-script.md) — `ado-script` workspace + (`scripts/ado-script/`): the bundled TypeScript runtime helpers (today: + `gate.js`), schemars-driven type codegen, and the A2 design decision. - [`docs/local-development.md`](docs/local-development.md) — local development setup notes. diff --git a/ado-script-design.md b/ado-script-design.md new file mode 100644 index 00000000..835e2b20 --- /dev/null +++ b/ado-script-design.md @@ -0,0 +1,235 @@ +# Design exploration: an `actions/github-script` analog for ADO + +> **Mode**: thought experiment. No scope committed. Goal is to map the design +> space, surface trade-offs, and identify the highest-leverage entry point if +> we ever pull the trigger. + +## 1. The concern that motivates this + +`scripts/gate-eval.py` is already 388 lines. It conflates several +responsibilities: + +1. **Spec deserialization** (base64 → JSON → dict) +2. **Fact acquisition** (env vars, REST API for PR metadata, REST API for + iteration changes, datetime arithmetic) — including auth, URL building, + retry/timeout semantics +3. **Predicate evaluation** (10 predicate types, recursive, with overnight + time-window arithmetic and `_strip_ref_prefix` quirks) +4. **Failure-policy state machine** (`fail_closed` / `fail_open` / + `skip_dependents`, with transitive propagation through fact dependencies) +5. **ADO logging-command emission** (`##vso[...]`) +6. **Self-cancel** (PATCH to builds API) + +Every new filter type forces a coordinated change across: +`filter_ir.rs` (Rust IR) → JSON schema → `gate-eval.py` evaluator → +fixtures → docs. The Python file has no static typing, no test harness in CI +(only Rust-side spec-serialization tests), and grows with the IR. + +There are at least two more places where a similar Python/bash blob is on the +roadmap or already exists: + +- The Stage-3 safe-output executor — currently a typed Rust binary + (`src/execute.rs` + `src/safeoutputs/*.rs`). Strong story today, but every + ADO interaction is hand-rolled HTTP via `reqwest`. +- The agent shim & the prepare/setup steps — currently bash interleaved with + ADO macro expansion. + +The user's instinct: rather than letting `gate-eval.py` grow into a +monstrosity (and rather than reinventing it for each new use case), give +ado-aw a single, well-tested primitive — the way `actions/github-script` +gives gh-aw its "drop in JS, get a pre-authed Octokit + context" lever. + +## 2. What `actions/github-script` actually is + +For grounding, the github-script contract: + +```yaml +- uses: actions/github-script@v7 + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + const { data } = await github.rest.issues.createComment({ + owner: context.repo.owner, repo: context.repo.repo, + issue_number: context.issue.number, body: 'hi', + }); + core.setOutput('comment-id', data.id); +``` + +Mechanics worth copying: + +| Property | Detail | +|---|---| +| Language | Node.js (single ecosystem, ncc-bundled, no `npm install` at runtime) | +| Auth | Pre-injected `github` Octokit, token from input | +| Context | Pre-injected `context` (event payload + repo/issue/PR shortcuts) | +| Helpers | `core` (output/secrets/log), `glob`, `io`, `exec`, `fetch` | +| Wrapper | `(async () => {