feat: add sealed probes for private repositories by lpcox · Pull Request #6673 · github/gh-aw-firewall

lpcox · 2026-07-28T02:45:19Z

Summary

add top-level sealedProbes configuration with schema, stdin-config, defaults, validation, and fail-closed preflight support
stage credential-isolated private repository seeds before any runtime container starts
run an offline trusted broker and fresh writable one-repository probe sandbox for every invocation
expose a fixed agent-local sealed-probe CLI and generated skill with exactly three caller-declared outcomes plus reserved ERROR
publish and version the new sealed-probe container image through the release workflow

Security model

The broker and probe have no network access. Private repository credentials are used only by trusted host staging and are excluded from the primary agent. Each probe receives only a fresh ephemeral repository copy and a read-only script, with a read-only root filesystem, non-root identity, dropped capabilities, seccomp, resource bounds, and no Docker socket.

The broker validates the request before launch, enforces the invocation budget, rejects non-zero exits and invalid output, and emits only canonical {"result":"..."} JSON. All failure paths collapse to {"result":"ERROR"} with the same exit status and no diagnostics exposed to the agent.

Closes #6661

Add trusted offline repository staging and a no-network probe broker. Run Python in hardened per-invocation sandboxes and expose a narrow CLI with four-symbol result closure. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 21364940-ae16-48b5-a577-321ef7f3fc74

github-actions · 2026-07-28T02:45:48Z

Documentation Preview

Documentation build failed for this PR. View logs.

Built from commit 3861cc9

github-actions · 2026-07-28T02:48:21Z

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric	Base	PR	Delta
Lines	99.11%	98.82%	📉 -0.29%
Statements	99.03%	98.48%	📉 -0.55%
Functions	99.53%	99.24%	📉 -0.29%
Branches	95.67%	95.15%	📉 -0.52%

📁 Per-file Coverage Changes (4 files)

File	Lines (Before → After)	Statements (Before → After)
`src/services/optional-services.ts`	100.0% → 100.0% (+0.00%)	100.0% → 99.1% (-0.94%)
`src/commands/main-action.ts`	99.3% → 99.3% (+0.01%)	98.7% → 98.7% (+0.02%)
`src/commands/build-config.ts`	97.1% → 97.2% (+0.08%)	97.1% → 97.2% (+0.08%)
`src/log-directory-setup.ts`	96.2% → 100.0% (+3.78%)	96.3% → 100.0% (+3.71%)

✨ New Files (10 files)

src/parsers/sealed-probe-parser.ts: 100.0% lines
src/sealed-probe/manager.ts: 91.5% lines
src/sealed-probe/paths.ts: 100.0% lines
src/sealed-probe/preflight.ts: 88.7% lines
src/sealed-probe/protocol.ts: 93.0% lines
src/sealed-probe/skill.ts: 100.0% lines
src/sealed-probe/staging.ts: 95.7% lines
src/sealed-probe/types.ts: 100.0% lines
src/services/sealed-probe-service.ts: 100.0% lines
src/types/sealed-probe-options.ts: 100.0% lines

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot

Pull request overview

Adds sealed probes for bounded computation over staged private repositories.

Changes:

Adds configuration, validation, staging, protocol, broker, and lifecycle integration.
Adds hardened probe containers, agent wrapper/skill, and extensive tests.
Publishes the sealed-probe image through the release workflow.

Show a summary per file

File	Description
`src/types/wrapper-config.ts`	Adds sealed-probe options.
`src/types/sealed-probe-options.ts`	Defines configuration and defaults.
`src/types/index.ts`	Exports sealed-probe types.
`src/services/sealed-probe-service.ts`	Builds broker and agent wiring.
`src/services/sealed-probe-service.test.ts`	Tests service assembly.
`src/services/sealed-probe-compose.test.ts`	Tests Compose integration.
`src/services/optional-services.ts`	Registers the optional broker.
`src/services/agent-volumes/docker-socket.ts`	Shares Docker socket resolution.
`src/services/agent-environment/excluded-vars.ts`	Excludes GitHub tokens.
`src/services/agent-environment/excluded-vars.test.ts`	Tests credential exclusion.
`src/sealed-probe/wrapper.test.ts`	Tests agent wrapper behavior.
`src/sealed-probe/workflow-integration.test.ts`	Tests staging order.
`src/sealed-probe/types.ts`	Defines staging data types.
`src/sealed-probe/skill.ts`	Generates agent guidance.
`src/sealed-probe/skill.test.ts`	Tests generated guidance.
`src/sealed-probe/protocol.ts`	Implements protocol validation.
`src/sealed-probe/protocol.test.ts`	Tests protocol behavior.
`src/sealed-probe/protocol-parity.test.ts`	Checks TS/JS parity.
`src/sealed-probe/probe-seccomp.test.ts`	Checks seccomp invariants.
`src/sealed-probe/preflight.ts`	Adds fail-closed preflight.
`src/sealed-probe/preflight.test.ts`	Tests preflight checks.
`src/sealed-probe/paths.ts`	Defines artifact paths.
`src/sealed-probe/paths.test.ts`	Tests path derivation.
`src/sealed-probe/staging.ts`	Stages private repository seeds.
`src/sealed-probe/staging.test.ts`	Tests secure staging.
`src/sealed-probe/manager.ts`	Orchestrates setup and teardown.
`src/sealed-probe/manager.test.ts`	Tests lifecycle management.
`src/sealed-probe/broker.test.ts`	Tests broker and launcher behavior.
`src/sealed-probe/end-to-end.test.ts`	Tests wrapper-to-broker flow.
`src/schema.test.ts`	Registers the schema section.
`src/parsers/sealed-probe-parser.ts`	Normalizes configuration.
`src/parsers/sealed-probe-parser.test.ts`	Tests normalization.
`src/image-tag.ts`	Adds sealed-probe digests.
`src/image-tag.test.ts`	Tests digest handling.
`src/container-start.test.ts`	Updates cleanup expectations.
`src/container-lifecycle.ts`	Cleans conflicting brokers.
`src/constants.ts`	Defines the broker name.
`src/config-mapper.ts`	Maps file configuration.
`src/config-file.ts`	Adds raw config typing.
`src/config-file-sealed-probes-validation.test.ts`	Tests schema validation.
`src/config-file-mapping.test.ts`	Tests config mapping.
`src/config-file-loading.test.ts`	Tests stdin loading.
`src/commands/main-action.ts`	Integrates lifecycle cleanup.
`src/commands/build-config.ts`	Builds normalized configuration.
`src/commands/build-config.test.ts`	Tests config construction.
`src/cli-workflow.ts`	Stages seeds before startup.
`src/awf-config-schema.json`	Adds runtime schema.
`docs/awf-config.schema.json`	Publishes configuration schema.
`docs/awf-config-spec.md`	Documents the security model.
`containers/sealed-probe/probe-seccomp.json`	Defines probe syscall policy.
`containers/sealed-probe/Dockerfile`	Builds broker/probe image.
`containers/sealed-probe/broker/workspace.js`	Manages invocation copies.
`containers/sealed-probe/broker/server.js`	Serves the Unix-socket API.
`containers/sealed-probe/broker/protocol.js`	Implements broker protocol.
`containers/sealed-probe/broker/probe-runner.js`	Launches probe containers.
`containers/sealed-probe/broker/healthcheck.js`	Checks broker health.
`containers/sealed-probe/broker/framing.js`	Parses bounded requests.
`containers/sealed-probe/broker/config.js`	Loads broker configuration.
`containers/sealed-probe/broker/broker.js`	Coordinates probe execution.
`containers/sealed-probe/broker/audit.js`	Writes protected diagnostics.
`containers/agent/sealed-probe-wrapper.sh`	Adds the agent CLI.
`containers/agent/entrypoint.sh`	Activates the CLI.
`containers/agent/Dockerfile`	Packages the wrapper.
`CLAUDE.md`	Documents the architecture.
`.github/workflows/release.yml`	Publishes and signs the image.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 65/65 changed files
Comments generated: 10
Review effort level: Medium

lpcox · 2026-07-28T03:06:14Z

@copilot Fix the code for all comments in this review thread.

When a review comment includes a suggested change, apply the suggestion exactly.

Do not make changes beyond what is described in the linked review thread.

Copilot · 2026-07-28T03:43:13Z

@copilot Fix the code for all comments in this review thread.

When a review comment includes a suggested change, app...

All ten review comments addressed in commit fix: sealed probe security and isolation review fixes:

sbx runtime removed — dropped from SealedProbeRuntime, both JSON schemas, config-file.ts, and all tests; every sbx configuration now fails closed at preflight with an unsupported-runtime error.
CodeQL TOCTOU races — staging.ts:278 uses try/catch around readFileSync instead of existsSync + read; test files use openSync(O_RDONLY | O_NOFOLLOW) + fstatSync + readFileSync(fd).
Timeout covers workspace creation — deadline captured before createInvocationWorkspace (which calls the synchronous cpSync); if the copy already exhausted the budget the broker returns ERROR immediately; remaining budget passed as timeoutMs to runProbeContainer.
/health removed from agent-visible socket — the route is gone; broker writes a broker.ready file after listenOnSocket; healthcheck.js checks that file with fs.accessSync.
Staging token via file, not env — AWF_SEALED_PROBE_STAGING_TOKEN_FILE carries the path to a 0600 file; the askpass helper does cat "${file}"; the token value never appears in any child process's environment (and thus not in /proc/<pid>/environ); token file cleaned up in finally.
Work-directory symlink check — prepareSealedProbes calls lstatSync on config.workDir before prepareDirectories; if it is a symlink the function throws immediately.
Masking mounts — an empty sealed-probes-mask/ directory (sibling of sealed-probes/) is mounted over the sealed-probe root for both bare and /host-prefixed paths; isSealedProbeAgentMount recognises the new masking targets.
Aggregate storage bound — /probe is now a size-limited tmpfs (256 MiB); the seed repo is a separate read-only bind mount at /probe/repo; the output is a pre-created bind-mounted file at /probe/out; createInvocationWorkspace makes the broker-side repo copy owner-writable so cleanup succeeds.
Separate probe image — Dockerfile is now a two-stage build: probe stage (python:3.12-alpine, no Node/docker-cli/apk) tagged as sealed-probe; broker stage (node:22-alpine + docker-cli) tagged as sealed-probe-broker; resolveSealedProbeImages returns both refs; IMAGE_DIGEST_KEYS includes sealed-probe-broker.
SKILL.md at standard discovery path — entrypoint.sh copies the file from $AWF_SEALED_PROBE_SKILL into ~/.github/skills/sealed-probe/SKILL.md (both chroot and non-chroot paths), the path scanned by GitHub Copilot skill discovery.

Build and publish both container targets, keep readiness broker-private, and materialize writable repository copies inside bounded tmpfs. Add a real Docker isolation integration test for the probe runtime. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 21364940-ae16-48b5-a577-321ef7f3fc74

github-actions · 2026-07-28T04:24:21Z

✅ Copilot review passed with no inline comments.

@lpcox Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

Validate opened descriptors directly before reading credential-scan fixtures. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 21364940-ae16-48b5-a577-321ef7f3fc74

github-actions · 2026-07-28T04:27:08Z

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric	Base	PR	Delta
Lines	99.11%	98.76%	📉 -0.35%
Statements	99.03%	98.41%	📉 -0.62%
Functions	99.53%	99.13%	📉 -0.40%
Branches	95.67%	95.03%	📉 -0.64%

📁 Per-file Coverage Changes (4 files)

File	Lines (Before → After)	Statements (Before → After)
`src/services/optional-services.ts`	100.0% → 100.0% (+0.00%)	100.0% → 99.1% (-0.93%)
`src/commands/main-action.ts`	99.3% → 99.3% (+0.01%)	98.7% → 98.7% (+0.02%)
`src/commands/build-config.ts`	97.1% → 97.2% (+0.08%)	97.1% → 97.2% (+0.08%)
`src/log-directory-setup.ts`	96.2% → 100.0% (+3.78%)	96.3% → 100.0% (+3.71%)

✨ New Files (10 files)

src/parsers/sealed-probe-parser.ts: 100.0% lines
src/sealed-probe/manager.ts: 88.5% lines
src/sealed-probe/paths.ts: 100.0% lines
src/sealed-probe/preflight.ts: 88.7% lines
src/sealed-probe/protocol.ts: 93.0% lines
src/sealed-probe/skill.ts: 100.0% lines
src/sealed-probe/staging.ts: 95.9% lines
src/sealed-probe/types.ts: 100.0% lines
src/services/sealed-probe-service.ts: 95.3% lines
src/types/sealed-probe-options.ts: 100.0% lines

Coverage comparison generated by scripts/ci/compare-coverage.ts

github-actions · 2026-07-28T04:29:58Z

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric	Base	PR	Delta
Lines	99.11%	98.76%	📉 -0.35%
Statements	99.03%	98.41%	📉 -0.62%
Functions	99.53%	99.13%	📉 -0.40%
Branches	95.67%	95.03%	📉 -0.64%

📁 Per-file Coverage Changes (4 files)

File	Lines (Before → After)	Statements (Before → After)
`src/services/optional-services.ts`	100.0% → 100.0% (+0.00%)	100.0% → 99.1% (-0.93%)
`src/commands/main-action.ts`	99.3% → 99.3% (+0.01%)	98.7% → 98.7% (+0.02%)
`src/commands/build-config.ts`	97.1% → 97.2% (+0.08%)	97.1% → 97.2% (+0.08%)
`src/log-directory-setup.ts`	96.2% → 100.0% (+3.78%)	96.3% → 100.0% (+3.71%)

✨ New Files (10 files)

src/parsers/sealed-probe-parser.ts: 100.0% lines
src/sealed-probe/manager.ts: 88.5% lines
src/sealed-probe/paths.ts: 100.0% lines
src/sealed-probe/preflight.ts: 88.7% lines
src/sealed-probe/protocol.ts: 93.0% lines
src/sealed-probe/skill.ts: 100.0% lines
src/sealed-probe/staging.ts: 95.9% lines
src/sealed-probe/types.ts: 100.0% lines
src/services/sealed-probe-service.ts: 95.3% lines
src/types/sealed-probe-options.ts: 100.0% lines

Coverage comparison generated by scripts/ci/compare-coverage.ts

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 21364940-ae16-48b5-a577-321ef7f3fc74

github-actions · 2026-07-28T05:10:43Z

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric	Base	PR	Delta
Lines	99.11%	99.20%	📈 +0.09%
Statements	99.03%	99.10%	📈 +0.07%
Functions	99.53%	99.45%	📉 -0.08%
Branches	95.67%	95.75%	📈 +0.08%

📁 Per-file Coverage Changes (4 files)

File	Lines (Before → After)	Statements (Before → After)
`src/services/optional-services.ts`	100.0% → 100.0% (+0.00%)	100.0% → 99.1% (-0.93%)
`src/commands/main-action.ts`	99.3% → 99.3% (+0.01%)	98.7% → 98.7% (+0.02%)
`src/commands/build-config.ts`	97.1% → 97.2% (+0.08%)	97.1% → 97.2% (+0.08%)
`src/log-directory-setup.ts`	96.2% → 100.0% (+3.78%)	96.3% → 100.0% (+3.71%)

✨ New Files (10 files)

src/parsers/sealed-probe-parser.ts: 100.0% lines
src/sealed-probe/manager.ts: 100.0% lines
src/sealed-probe/paths.ts: 100.0% lines
src/sealed-probe/preflight.ts: 100.0% lines
src/sealed-probe/protocol.ts: 100.0% lines
src/sealed-probe/skill.ts: 100.0% lines
src/sealed-probe/staging.ts: 100.0% lines
src/sealed-probe/types.ts: 100.0% lines
src/services/sealed-probe-service.ts: 100.0% lines
src/types/sealed-probe-options.ts: 100.0% lines

Coverage comparison generated by scripts/ci/compare-coverage.ts

lpcox · 2026-07-28T05:21:59Z

@copilot address the failed codeql ci checks

Copilot · 2026-07-28T05:27:13Z

@copilot address the failed codeql ci checks

Fixed in the latest commit. Both writeSeedMap (manager.ts:81) and writeSealedProbeSkill (skill.ts:136) now open the target file with O_WRONLY | O_CREAT | O_EXCL | O_NOFOLLOW before writing, ensuring the file is created atomically without following symlinks and without racing an existing file. CodeQL reports 0 alerts after this change.

github-actions · 2026-07-28T05:41:09Z

📰 VERDICT: Smoke Docker Sbx has concluded. All systems operational. This is a developing story. 🎤

github-actions · 2026-07-28T05:41:09Z

🔌 Smoke Services — All services reachable! ✅

github-actions · 2026-07-28T05:41:10Z

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

github-actions · 2026-07-28T05:41:29Z

❌ Security Guard failed. Please review the logs for details.

github-actions · 2026-07-28T05:41:35Z

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

+  // O_EXCL | O_NOFOLLOW: atomically create; fail if a symlink or existing file
+  // is already at this path (insecure-temp-file guard).
+  const fd = fs.openSync(
+    paths.seedMapPath,


+  // O_EXCL | O_NOFOLLOW: atomically create; fail if a symlink or existing file
+  // is already at this path (insecure-temp-file guard).
+  const fd = fs.openSync(
+    paths.skillPath,


github-actions · 2026-07-28T05:41:48Z

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

github-actions · 2026-07-28T05:41:57Z

✅ Build Test Suite completed successfully!

github-actions · 2026-07-28T05:42:44Z

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric	Base	PR	Delta
Lines	99.11%	99.20%	📈 +0.09%
Statements	99.03%	99.10%	📈 +0.07%
Functions	99.53%	99.45%	📉 -0.08%
Branches	95.67%	95.75%	📈 +0.08%

📁 Per-file Coverage Changes (4 files)

File	Lines (Before → After)	Statements (Before → After)
`src/services/optional-services.ts`	100.0% → 100.0% (+0.00%)	100.0% → 99.1% (-0.93%)
`src/commands/main-action.ts`	99.3% → 99.3% (+0.01%)	98.7% → 98.7% (+0.02%)
`src/commands/build-config.ts`	97.1% → 97.2% (+0.08%)	97.1% → 97.2% (+0.08%)
`src/log-directory-setup.ts`	96.2% → 100.0% (+3.78%)	96.3% → 100.0% (+3.71%)

✨ New Files (10 files)

src/parsers/sealed-probe-parser.ts: 100.0% lines
src/sealed-probe/manager.ts: 100.0% lines
src/sealed-probe/paths.ts: 100.0% lines
src/sealed-probe/preflight.ts: 100.0% lines
src/sealed-probe/protocol.ts: 100.0% lines
src/sealed-probe/skill.ts: 100.0% lines
src/sealed-probe/staging.ts: 100.0% lines
src/sealed-probe/types.ts: 100.0% lines
src/services/sealed-probe-service.ts: 100.0% lines
src/types/sealed-probe-options.ts: 100.0% lines

Coverage comparison generated by scripts/ci/compare-coverage.ts

github-actions · 2026-07-28T05:44:11Z

Smoke Test: Claude Engine Validation

Check	Result
API status	✅ PASS
gh check	✅ PASS
File status	✅ PASS

Overall result: PASS

Generated by Smoke Claude for #6673 · haiku45 · 54.5 AIC · ⊞ 3.3K · ◷
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T05:44:26Z

@lpcox — smoke test results for network isolation egress enforcement:

EGRESS_RESULT allow=pass deny=pass

✅ Allowed domain (api.github.com): HTTP 200
✅ Blocked domain (example.com): connection denied (403 from proxy)

Overall: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

example.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "example.com"

See Network Configuration for more information.

🛡️ Egress verdict from Smoke Copilot Network Isolation
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T05:44:28Z

Smoke Test: Copilot BYOK (Direct Mode)

✅ GitHub MCP: Available (read access confirmed)
✅ GitHub.com: HTTP 200
✅ File I/O: Smoke test workflow readable
✅ BYOK Inference: Running in direct mode via api-proxy → api.githubcopilot.com

Overall: PASS — Direct BYOK mode (COPILOT_PROVIDER_API_KEY) is operational.

🔑 BYOK report filed by Smoke Copilot BYOK
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T05:44:38Z

Smoke Test Results

Redis PING: ❌ host.docker.internal name resolution failed
PostgreSQL pg_isready: ❌ no response
PostgreSQL SELECT 1: ❌ name resolution failed

Overall: FAIL — host.docker.internal is not resolvable from this runner environment. Service containers may not be configured or the hostname is not available.

🔌 Service connectivity validated by Smoke Services
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T05:44:44Z

🔥 Smoke Test Results

Test	Status
GitHub MCP	✅ Connected (secrecy policy filtered — expected for private repo)
GitHub.com HTTP	✅ 200 OK
File Write/Read	✅ Pass

Overall: PASS

@lpcox

📰 BREAKING: Report filed by Smoke Copilot
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T05:45:31Z

feat: add sealed probes for private repositories
docs: Runner Doctor 2026-07-27 scan — B13 update + B15/B16 new failure modes
fix: add topologyPeers to policy manifest for accurate audit attribution
GitHub MCP Testing: ✅
Safe Inputs GH CLI Testing: ✅
Playwright Testing: ✅
File Writing Testing: ✅
Bash Tool Testing: ✅
Discussion Interaction Testing: ✅
Build AWF: ✅
Overall Status: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T05:45:54Z

Chroot Version Comparison Results

Runtime	Host Version	Chroot Version	Match?
Python	`Python 3.12.13`	`Python 3.12.13`	✅ YES
Node.js	`v24.18.0`	`v22.23.1`	❌ NO
Go	`go1.22.12`	`go1.22.12`	✅ YES

Overall: ❌ FAILED — Node.js version mismatch (host v24.18.0 vs chroot v22.23.1).

Tested by Smoke Chroot
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T05:45:58Z

Smoke Test Results

GitHub MCP Testing: ❌
GitHub.com Connectivity: ❌
File Writing Testing: ✅
Bash Tool Testing: ✅

Overall status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T05:46:07Z

Smoke Test: API Proxy OpenTelemetry Tracing

Scenario	Result	Details
1. Module Loading	✅ Pass	`otel.js` loads successfully; exports: `startRequestSpan`, `setTokenAttributes`, `setBudgetAttributes`, `endSpan`, `endSpanError`, `shutdown`, `isEnabled` and internal helpers
2. Test Suite	✅ Pass	59 tests passed across `otel.test.js` and `otel-fanout.test.js` (2 suites)
3. Env Var Forwarding	✅ Pass	`src/services/api-proxy-env-config.ts` forwards `GH_AW_OTLP_ENDPOINTS`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`, `GITHUB_AW_OTEL_TRACE_ID`, `GITHUB_AW_OTEL_PARENT_SPAN_ID` to the api-proxy container
4. Token Tracker Integration	✅ Pass	`onUsage` callback exists in `token-tracker-http.js` (line 348/406) as the OTEL hook point
5. OTEL Diagnostics	✅ Pass	Module initializes with graceful degradation (falls back to local file `otel.jsonl` when no OTLP endpoint configured)

Summary: All 5 scenarios pass. OTEL tracing integration is fully functional — spans are created per request with GenAI semantic conventions, parent context propagation via GITHUB_AW_OTEL_TRACE_ID/GITHUB_AW_OTEL_PARENT_SPAN_ID is wired, and fan-out export to multiple OTLP endpoints works correctly.

📡 OTel tracing validated by Smoke OTel Tracing
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T05:47:00Z

@lpcox

GitHub MCP Test: ✅ (pre-fetched data validated)
GitHub.com Connectivity: ✅
File Write/Read Test: ✅
BYOK Inference Test: ✅

Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) authenticated via Microsoft Entra

Overall: PASS

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T05:47:37Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	passed	✅ PASS
Node.js	execa	✅	passed	✅ PASS
Node.js	p-limit	✅	passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for #6673 · sonnet46 · 48.8 AIC · ⊞ 8.4K · ◷
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T05:48:00Z

Smoke Test Results: PASS

GitHub MCP connectivity: ✅
GitHub.com connectivity: ✅
File write/read: ✅
BYOK inference: ✅
Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)
cc @lpcox

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)
_{Add label ready-for-aw to run again}

github-actions · 2026-07-28T05:48:21Z

🔍 Smoke Test: Docker Sbx Validation

Test	Result
GitHub MCP Connectivity	⚠️ Filtered by secrecy policy
GitHub.com HTTP	⚠️ Pre-step data not expanded
File Write/Read	⚠️ Pre-step data not expanded

Overall: ⚠️ INCONCLUSIVE — workflow template variables were not expanded before agent execution; pre-computed test data unavailable.

📰 BREAKING: Report filed by Smoke Docker Sbx
_{Add label ready-for-aw to run again}

Copilot AI review requested due to automatic review settings July 28, 2026 02:45

Copilot started reviewing on behalf of lpcox July 28, 2026 02:45 View session

github-advanced-security AI found potential problems Jul 28, 2026

View reviewed changes

Comment thread src/sealed-probe/skill.test.ts Fixed

Comment thread src/sealed-probe/staging.test.ts Fixed

Comment thread src/sealed-probe/staging.ts Fixed

Copilot AI reviewed Jul 28, 2026

View reviewed changes

Copilot started work on behalf of lpcox July 28, 2026 03:07 View session

fix: sealed probe security and isolation review fixes

5b6ca5b

Copilot finished work on behalf of lpcox July 28, 2026 03:43

github-advanced-security AI found potential problems Jul 28, 2026

View reviewed changes

Comment thread src/sealed-probe/staging.test.ts Fixed

test: remove staging credential scan race

4595cb8

Validate opened descriptors directly before reading credential-scan fixtures. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 21364940-ae16-48b5-a577-321ef7f3fc74

test: restore sealed probe coverage

a45daf5

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 21364940-ae16-48b5-a577-321ef7f3fc74

github-advanced-security AI found potential problems Jul 28, 2026

View reviewed changes

Comment thread src/sealed-probe/manager.ts Fixed

Comment thread src/sealed-probe/skill.ts Fixed

Copilot started work on behalf of lpcox July 28, 2026 05:22 View session

fix: use O_EXCL|O_NOFOLLOW for secure file creation in sealed probe

e1d9d8d

Copilot finished work on behalf of lpcox July 28, 2026 05:27

lpcox added the ready-for-aw label Jul 28, 2026

lpcox temporarily deployed to aoai-model July 28, 2026 05:41 — with GitHub Actions Inactive

github-advanced-security AI found potential problems Jul 28, 2026

View reviewed changes

github-actions Bot added the smoke-claude label Jul 28, 2026

github-actions Bot added smoke-copilot-byok smoke-copilot-network-isolation Copilot network-isolation egress smoke test labels Jul 28, 2026

github-actions Bot added the smoke-copilot label Jul 28, 2026

github-actions Bot added the smoke-codex label Jul 28, 2026

lpcox temporarily deployed to aoai-model July 28, 2026 05:46 — with GitHub Actions Inactive

github-actions Bot added the smoke-copilot-byok-aoai-entra label Jul 28, 2026

lpcox deployed to aoai-model July 28, 2026 05:47 — with GitHub Actions Active

github-actions Bot added the build-test label Jul 28, 2026

github-actions Bot added the smoke-copilot-byok-aoai-apikey label Jul 28, 2026

Uh oh!

Conversation

lpcox commented Jul 28, 2026

Summary

Security model

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation Preview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026

⚠️ Coverage Regression Detected

Overall Coverage

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lpcox commented Jul 28, 2026

Uh oh!

Copilot AI commented Jul 28, 2026

Uh oh!

github-actions Bot commented Jul 28, 2026

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026

⚠️ Coverage Regression Detected

Overall Coverage

Uh oh!

github-actions Bot commented Jul 28, 2026

⚠️ Coverage Regression Detected

Overall Coverage

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026

⚠️ Coverage Regression Detected

Overall Coverage

Uh oh!

lpcox commented Jul 28, 2026

Uh oh!

Copilot AI commented Jul 28, 2026

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 28, 2026

⚠️ Coverage Regression Detected

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading

github-actions Bot commented Jul 28, 2026 •

edited

Loading