Skip to content

Add experiment results to GitHub job summaries#29

Open
Copilot wants to merge 1 commit into
mainfrom
copilot/add-experiment-results-to-job-summary
Open

Add experiment results to GitHub job summaries#29
Copilot wants to merge 1 commit into
mainfrom
copilot/add-experiment-results-to-job-summary

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 26, 2026

Experiment runs uploaded results.json as an artifact but did not surface the formatted results in the Actions job summary. This makes run outcomes visible directly from the workflow page.

  • Job summary output
    • Appends the existing formatted experiment results table to $GITHUB_STEP_SUMMARY when available.
    • Preserves existing console output and results.json artifact generation.
await fs.appendFile(GITHUB_STEP_SUMMARY, `## Experiment results\n\n\`\`\`\n${resultSummaries}\n\`\`\`\n`)

@joshblack joshblack marked this pull request as ready for review May 26, 2026 21:53
Copilot AI review requested due to automatic review settings May 26, 2026 21:53
@joshblack joshblack enabled auto-merge (squash) May 26, 2026 21:53
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR surfaces experiment result summaries directly in GitHub Actions job summaries while preserving the existing console output and results artifact flow.

Changes:

  • Reads GITHUB_STEP_SUMMARY from the environment.
  • Adds a helper to append formatted experiment results to the Actions job summary when available.
  • Reuses the formatted result summary for both console output and summary output.
Show a summary per file
File Description
packages/agent-eval/src/cli.ts Adds GitHub job summary output for formatted experiment results.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 1/1 changed files
  • Comments generated: 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants