Export the agent e2e suite as a release bundle for downstream repos

[ling-senpeng13]

What

Packages the agent e2e suite (Conductor.AI.E2eTests/) into a self-contained, version-stamped tarball — conductor-ai-e2e-csharp-<version>.tar.gz — attached to every GitHub release, so downstream repos (e.g. orkes-io/orkes-conductor) can pin the e2e suite to the exact csharp-sdk release they run against. Mirrors conductor-oss/javascript-sdk#134.

• scripts/package-e2e-bundle.sh — stages the 27 e2e sources verbatim + a standalone csproj that swaps the ProjectReference for PackageReference conductor-ai@<version>, so a run exercises the published package. Two .NET-specific wrinkles the JS bundle didn't have: Settings (LlmModel/ServerUrl) is vendored into the bundle, since in-repo it arrives via <Compile Include> from the examples project; and AssemblyName must stay Conductor.AI.E2eTests because Conductor.AI grants it InternalsVisibleTo and a few suites read internal types.
• Bundles ship check-results.py + run.sh. Every test is a [SkippableFact] that skips when the server is unreachable, so a dead server yields an all-skipped run that reads as green — run.sh fails on 0 executed or server-unreachable skips, the same guard agent-e2e.yml applies in-repo.
• scripts/test-package-e2e-bundle.sh — static validator (source parity, package pin, no ProjectReference/<Compile> escapes, assembly name preserved, no leftover @VERSION@; no network). --build additionally packs the local SDK into a temp feed and compiles the bundle against it — the compile-time proof it needs nothing but the package.
• .github/workflows/release-agent-e2e-bundle.yml — packages, validates, generates sha256 sidecars, uploads to the release; same release event as release.yml, plus workflow_dispatch to attach bundles to existing releases. Release tags in this repo are bare versions (3.0.0-rc2), so the tag is used verbatim and v is stripped only for the NuGet version.

Changes outside the bundle-export scope

Flagging these explicitly, since they touch existing files rather than adding new ones. All three are consequences of review findings rather than the original goal — happy to split any of them out if reviewers would rather see them separately.

File	Change	Why it's here
Conductor/conductor-csharp.csproj	README path /package/Conductor/README.md → ../README.md	The absolute path is Dockerfile-only, so dotnet pack failed anywhere else — which the validator's --build mode needs. See below.
.github/workflows/agent-e2e.yml	−25/+3: inline guard heredoc replaced by python3 scripts/check-results.py	The guard existed twice (inline here, generated into the bundle) keyed to a magic string from E2eFixture.cs, with nothing tying the copies together. Deduplicating it necessarily edits this workflow. No behavior change — same logic, same marker.
.github/workflows/pull_request.yml	+21: new e2e_bundle job	--build is the only check that proves the bundle compiles, and it ran nowhere. It needs no published package, so a bundle that can't compile should fail the PR rather than ship attached to a release. Note this adds a job to every PR.

On the csproj fix

Conductor/conductor-csharp.csproj pinned its README at /package/Conductor/README.md — a Dockerfile-only path, so dotnet pack failed anywhere else. Changed to ../README.md, which resolves to the same file inside the Docker build (Dockerfile copies /README.md to both /package/Conductor/README.md and /package/README.md; workdir is /package/Conductor) and matches what Conductor.AI.csproj already does. release.yml packs via docker build --target=publish_release, so the release path is unaffected.

Review hardening (second commit)

A review of the first commit turned up eight issues, mostly variants of the validator couldn't catch the bugs it existed for, and the guard failed open:

1. The release workflow validated a stand-in, not the shipped artifact. It packaged at the real version, then the validator re-packaged at 9.9.9-test in a temp dir and checked that — so every version-dependent assertion never saw the uploaded tarball. Validator now takes --tarball/--version; the workflow points it at what it just built. $VERSION is also shape-checked in the packager, since it's an unescaped sed replacement (a / or & silently corrupted every stamped file).
2. Script injection. TAG="${{ inputs.tag }}" inside run: is substituted before the shell parses the line. Tags now arrive via env: and are shape-checked before use.
3. The vacuous-run guard failed open. No python3 → warn and continue; since dotnet test exits 0 on an all-skipped run, a dead server reported a pass — exactly what the guard prevents. run.sh now refuses to start without python3, checked up front rather than after a 20-minute run. python3 added to the bundle README prerequisites.
4. The guard was duplicated and untested. Extracted to scripts/check-results.py, copied verbatim into the bundle and invoked by agent-e2e.yml. The validator asserts both copies are identical, that UNREACHABLE_MARKER still appears in E2eFixture.cs, and that agent-e2e.yml hasn't re-inlined its own.
5. No drift detection on the generated csproj. It writes the TFM and four test-package versions longhand, so an in-repo bump left the bundle on stale xunit/Test.Sdk. Validator now diffs both csprojs.
6. Packager and validator shared one blind spot. Both used find -maxdepth 1 -name '*.cs', so the parity count agreed whether or not a subdirectory was dropped. Packager copies recursively (excluding obj/, bin/); validator compares full relative path sets, requiring the only extra to be the vendored Shared/Settings.cs.
7. --build ran nowhere. Added the e2e_bundle CI job (see the scope table above).
8. Bundle validity depended on an uncoordinated publish. Both workflows fire on release: published independently, so a failed NuGet publish still attached a bundle pinning a nonexistent version. Packaging still doesn't wait (it's static), but upload now blocks on conductor-ai appearing in the nuget.org flat container, up to 20 minutes. This delays when the asset shows up on a release, and fails the job outright if the publish fails.

Also: the validator errored on unknown args only when they were $1, so a typo silently ran static-only and printed ALL CHECKS PASSED. Proper arg loop now.

Validation

• validator green in all four modes — self-packaged and --tarball, each with and without --build; compiles 0 warnings / 0 errors against a locally packed conductor-ai 9.9.9-test with no project references, confirming internals access survives the package boundary
• dotnet test --list-tests on the extracted bundle discovers 157 tests — identical to the in-repo project
• guard exercised against synthetic TRX: healthy → 0; 0-executed → 1; server-unreachable skip → 1; no TRX → 1
• negative-tested that the new checks actually fail, since finding 6 was precisely a check that couldn't: hostile version 1.0/x&y → rejected (exit 1); tarball with one source removed → FAIL: bundle is missing e2e sources; in-repo xunit bumped → test package drift ... xunit: repo '2.9.9' vs bundle '2.9.3'
• docker build --target=pack_release succeeds with the relative README path, and the resulting conductor-csharp nupkg contains README.md

Not run here: the suite end-to-end, which needs a live server + mcp-testkit + real LLM calls — that's agent-e2e.yml on this PR, over the same sources.

Follow-up

The first release cut after this merges is the first one to carry a bundle; the consuming lane in orkes-io/orkes-conductor pins that version.

Known and deliberately left alone: the tarball isn't reproducible (no --sort/--mtime/--owner), and gh release upload --clobber replaces a bundle and its sha256 sidecar together, so downstream verification can't detect a pinned bundle changing underneath it.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag	Coverage Δ
unittests	3.31% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[kowser-orkes]

Javascript sdk should use 3.0.0-rc4