Skip to content
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .github/FLAKY_CI_FAILURE_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
title: '[Flaky CI]: {{ env.JOB_NAME }}'
labels: Tests
---

### Flakiness Type

Other / Unknown

### Name of Job

{{ env.JOB_NAME }}

### Name of Test

_Not available - check the run link for details_

### Link to Test Run

{{ env.RUN_LINK }}

---

_This issue was automatically created._
80 changes: 80 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1168,7 +1168,87 @@ jobs:
# Always run this, even if a dependent job failed
if: always()
runs-on: ubuntu-24.04
permissions:
issues: write
steps:
- name: Check out current commit
if: github.ref == 'refs/heads/develop' && (contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled'))
uses: actions/checkout@v6
with:
sparse-checkout: .github

- name: Create issues for failed jobs
if: github.ref == 'refs/heads/develop' && (contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled'))
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');

// Fetch actual job details from the API to get descriptive names
const jobs = await github.rest.actions.listJobsForWorkflowRun({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: context.runId,
per_page: 100
});
Comment thread
nicohrubec marked this conversation as resolved.

const failedJobs = jobs.data.jobs.filter(job =>
job.conclusion === 'failure' || job.conclusion === 'cancelled'
);

if (failedJobs.length === 0) {
console.log('No failed jobs found');
return;
}

// Read and parse template
const template = fs.readFileSync('.github/FLAKY_CI_FAILURE_TEMPLATE.md', 'utf8');
const [, frontmatter, bodyTemplate] = template.match(/^---\n([\s\S]*?)\n---\n([\s\S]*)$/);
Comment thread
nicohrubec marked this conversation as resolved.

// Get existing open issues with Tests label
const existing = await github.rest.issues.listForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
state: 'open',
labels: 'Tests',
Comment thread
nicohrubec marked this conversation as resolved.
per_page: 100
});
Comment thread
nicohrubec marked this conversation as resolved.

for (const job of failedJobs) {
const jobName = job.name;
const jobUrl = job.html_url;

// Replace template variables
const vars = {
'JOB_NAME': jobName,
'RUN_LINK': jobUrl
};

let title = frontmatter.match(/title:\s*'(.*)'/)[1];
let issueBody = bodyTemplate;
for (const [key, value] of Object.entries(vars)) {
const pattern = new RegExp(`\\{\\{\\s*env\\.${key}\\s*\\}\\}`, 'g');
title = title.replace(pattern, value);
issueBody = issueBody.replace(pattern, value);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

String replace special patterns corrupt job name substitution

The replace() calls interpret special dollar-sign sequences in the replacement string ($&, $', $$, $1, etc.) rather than treating them literally. If a job name happens to contain patterns like $& or $1, the issue title and body would contain the matched template placeholder or captured groups instead of the literal job name. While job names rarely contain such patterns, this could cause confusing issue titles when they do.

Fix in Cursor Fix in Web

}

const existingIssue = existing.data.find(i => i.title === title);
Comment thread
nicohrubec marked this conversation as resolved.
Outdated
Comment thread
nicohrubec marked this conversation as resolved.
Outdated

if (existingIssue) {
console.log(`Issue already exists for ${jobName}: #${existingIssue.number}`);
continue;
}

const newIssue = await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title: title,
body: issueBody.trim(),
labels: ['Tests']
});
console.log(`Created issue #${newIssue.data.number} for ${jobName}`);
}

- name: Check for failures
if: cancelled() || contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled')
run: |
Expand Down
Loading