Skip to content

Comments

fix: sync test assertions with capitalized "GitHub toolsets" message#17935

Merged
pelikhan merged 2 commits intomainfrom
copilot/fix-permission-validation-tests
Feb 23, 2026
Merged

fix: sync test assertions with capitalized "GitHub toolsets" message#17935
pelikhan merged 2 commits intomainfrom
copilot/fix-permission-validation-tests

Conversation

Copy link
Contributor

Copilot AI commented Feb 23, 2026

After permissions_validation.go was updated to print Missing required permissions for GitHub toolsets: (capital G), six test assertions still matched against the old lowercase github toolsets string, causing strings.Contains to return false and panicking the affected subtests.

Changes

  • permissions_validator_test.go — updated 3 expectContains/expectMsg literals
  • permissions_warning_test.go — updated 2 warningMessage/expectedPhrases literals
  • permissions_no_github_tool_test.go — updated 1 warningMessage literal

All six occurrences now match the production string exactly.

Original prompt

This section details on the original issue you should resolve

<issue_title>[CI Failure Doctor] CI Failure Investigation - Run 37498</issue_title>
<issue_description>### CI Failure Investigation - Run github/gh-aw#37498

Summary

The test failure job for run 37498 (head sha ef2fa31d80b793b624a645f83aff2ebc7bf7cca6) failed because the permission validation tests were still looking for the old lowercase message even though the validator now prints Missing required permissions for GitHub toolsets:.

Failure Details

Root Cause Analysis

After the recent fix to capitalize GitHub inside permissions_validation.go, the human-facing text changed but the validator/warning tests continue to assert a lowercase github toolsets string. Each strings.Contains call now returns false, the assertions panic, and TestFormatValidationMessage / TestValidatePermissions_ComplexScenarios abort before reporting a named failure.

Failed Jobs and Errors

  • test failure: the Go test runner reported ❌ FAILURES FOUND in: test-result-unit.json followed by TestFormatValidationMessage/Missing_permissions_message and the related TestValidatePermissions_ComplexScenarios subtests, plus the warning that "No individual test marked as failed" because the assertions panic during initialization.
Investigation Findings
  • permissions_validation.go now builds lines starting with "Missing required permissions for GitHub toolsets:" and writes the full suggestion text for Options 1 and 2.
  • The validator, warning, and "no github tool" tests still hard-code the lowercase text, so their strings.Contains checks fail and the tests assert before a failure record can be emitted, killing the job early.
  • Running go test ./pkg/workflow locally in this environment tries to download Go 1.25 and fails with golang.org/toolchain@v0.0.1-go1.25.0...: Forbidden, so we could not re-run the suite here.

Recommended Actions

  • Keep the tests locked to the literal message in sync with the production helper (e.g. share a constant or compare case-insensitively) so future user-message tweaks do not break string assertions.
  • Re-run go test ./pkg/workflow (or the full make agent-finish) once the Go 1.25 toolchain is available to verify that these fixes cover the entire suite.

Prevention Strategies

Avoid hard-coding case-sensitive user-facing strings in tests; either reference the message via a shared exported helper or assert with strings.Contains(strings.ToLower(...), strings.ToLower(expected)) so cosmetic text tweaks do not cause regressions.

AI Team Self-Improvement

When writing validation tests for CLI output, either derive the expected message from the same helper that produces it or use case-insensitive comparisons so capitalization-only fixes stay safe.

Historical Context

No similar investigations are recorded in the cached history for this repository.

🩺 Diagnosis provided by CI Failure Doctor

To install this workflow, run gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d. View source at https://github.com/githubnext/agentics/tree/ea350161ad5dcc9624cf510f134c6a9e39a6f94d/workflows/ci-doctor.md.

  • expires on Feb 24, 2026, 12:28 PM UTC

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

…ns validation message

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix permission validation tests after text changes fix: sync test assertions with capitalized "GitHub toolsets" message Feb 23, 2026
@pelikhan pelikhan marked this pull request as ready for review February 23, 2026 15:59
Copilot AI review requested due to automatic review settings February 23, 2026 15:59
@pelikhan pelikhan merged commit 06dfe6c into main Feb 23, 2026
@pelikhan pelikhan deleted the copilot/fix-permission-validation-tests branch February 23, 2026 16:00
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates workflow permission-related tests to match the capitalized production message Missing required permissions for GitHub toolsets: so string-contains assertions no longer fail.

Changes:

  • Updated expected warning/validation message literals from github toolsetsGitHub toolsets across six assertions.
  • Kept test expectations aligned with pkg/workflow/permissions_validation.go’s current output.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
pkg/workflow/permissions_warning_test.go Updates warning message/phrases to match the capitalized “GitHub toolsets” header.
pkg/workflow/permissions_validator_test.go Updates expected validation message header string to match production output.
pkg/workflow/permissions_no_github_tool_test.go Updates warning message header string used by the “github tool configured” test case.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI Failure Doctor] CI Failure Investigation - Run 37498

2 participants