Skip to content

ci(chromatic): skip Chromatic on Dependabot PRs#7425

Merged
Zaimwa9 merged 1 commit intomainfrom
ci/skip-chromatic-on-dependabot
May 6, 2026
Merged

ci(chromatic): skip Chromatic on Dependabot PRs#7425
Zaimwa9 merged 1 commit intomainfrom
ci/skip-chromatic-on-dependabot

Conversation

@talissoncosta
Copy link
Copy Markdown
Contributor

@talissoncosta talissoncosta commented May 5, 2026

Thanks for submitting a PR! Please check the boxes below:

  • I have read the Contributing Guide.
  • I have added information to docs/ if required so people know about the feature.
  • I have filled in the "Changes" section below.
  • I have filled in the "How did you test this code" section below.

Changes

Follow-up to a question raised on #7375 by @Zaimwa9 — Chromatic CI is currently failing on every Dependabot npm PR with ✖ Missing project token.

TL;DR. Dependabot PRs can't read CHROMATIC_PROJECT_TOKEN (separate secrets store), so Chromatic fails with "Missing project token". This skips Chromatic on Dependabot PRs only. main's post-merge push still runs Chromatic with full secrets, so coverage isn't lost.

Cause

GitHub runs Dependabot PRs with a separate secrets store (Settings → Secrets and variables → Dependabot) that doesn't include the CHROMATIC_PROJECT_TOKEN we have in the regular Actions secrets. The Chromatic action then aborts with Missing project token.

This isn't a misconfiguration — it's the platform protecting us. If Actions secrets were exposed to Dependabot PRs, a malicious package-lock.json poisoned dep could exfiltrate them via postinstall scripts (see e.g. the Codecov 2021 incident for the canonical version of this attack).

Confirmed via the workflow logs of #7375's last run:

Secret source: Dependabot
✖ Missing project token

Fix

Skip the Chromatic job when github.actor == 'dependabot[bot]':

if: |
  github.actor != 'dependabot[bot]'
  && (github.event_name == 'push' || github.event.pull_request.draft == false)

When the Dependabot PR eventually merges into main, the push event runs Chromatic with the parent's secrets — so any visual regression a bump introduces is still caught, one merge later instead of pre-merge. Trade-off accepted given dep bumps rarely change visual output.

Alternatives considered

  • Add CHROMATIC_PROJECT_TOKEN to Dependabot secrets store. Would make Chromatic run on Dependabot PRs the same as any other. Rejected: widens the secret-access surface for the bot whose PRs are most likely to bring in arbitrary transitive deps.
  • Switch to pull_request_target event. Foot-gun. Gives PR code access to the base repo's secrets — exactly the attack vector the platform-level isolation is preventing. Hard pass.

How did you test this code?

@vercel
Copy link
Copy Markdown

vercel Bot commented May 5, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated (UTC)
docs Ignored Ignored Preview May 5, 2026 4:49pm
flagsmith-frontend-preview Ignored Ignored Preview May 5, 2026 4:49pm
flagsmith-frontend-staging Ignored Ignored Preview May 5, 2026 4:49pm

Request Review

@github-actions github-actions Bot added ci-cd Build, test and deployment related and removed ci-cd Build, test and deployment related labels May 5, 2026
Dependabot PRs run with a separate secrets store (Settings → Secrets →
Dependabot) that doesn't include CHROMATIC_PROJECT_TOKEN, so the action
aborts with "Missing project token". Skip the job for Dependabot —
main's post-merge push runs Chromatic with full secrets, so any visual
regression a bump introduces is still caught one merge later.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added ci-cd Build, test and deployment related and removed ci-cd Build, test and deployment related labels May 5, 2026
@talissoncosta talissoncosta force-pushed the ci/skip-chromatic-on-dependabot branch from 4b84781 to 349efa7 Compare May 5, 2026 16:49
@github-actions github-actions Bot added ci-cd Build, test and deployment related and removed ci-cd Build, test and deployment related labels May 5, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 5, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.44%. Comparing base (c771931) to head (349efa7).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #7425   +/-   ##
=======================================
  Coverage   98.44%   98.44%           
=======================================
  Files        1398     1398           
  Lines       52646    52646           
=======================================
  Hits        51826    51826           
  Misses        820      820           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Zaimwa9 Zaimwa9 marked this pull request as ready for review May 6, 2026 08:54
@Zaimwa9 Zaimwa9 requested a review from a team as a code owner May 6, 2026 08:54
@Zaimwa9 Zaimwa9 requested review from germangarces and removed request for a team May 6, 2026 08:54
Copy link
Copy Markdown

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Code review skipped — your organization's overage spend limit has been reached.

Code review is billed via overage credits. To resume reviews, an organization admin can raise the monthly limit at claude.ai/admin-settings/claude-code.

Once credits are available, reopen this pull request to trigger a review.

@github-actions github-actions Bot added ci-cd Build, test and deployment related and removed ci-cd Build, test and deployment related labels May 6, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 6, 2026

Docker builds report

Image Build Status Security report
ghcr.io/flagsmith/flagsmith-api-test:pr-7425 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-e2e:pr-7425 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-api:pr-7425 Finished ✅ Results
ghcr.io/flagsmith/flagsmith:pr-7425 Finished ✅ Results
ghcr.io/flagsmith/flagsmith-private-cloud:pr-7425 Finished ✅ Results
ghcr.io/flagsmith/flagsmith-frontend:pr-7425 Finished ✅ Results

@chromatic-com
Copy link
Copy Markdown

chromatic-com Bot commented May 6, 2026

Important

Testing in progress…

🟢 UI Tests: 284 tests unchanged
UI Review: Comparing 142 stories…
Storybook icon Storybook Publish: 142 stories published

@chromatic-com
Copy link
Copy Markdown

chromatic-com Bot commented May 6, 2026

Tip

All tests passed and all changes approved!

🟢 UI Tests: 284 tests unchanged
🟢 UI Review: 142 stories published -- no changes
Storybook icon Storybook Publish: 142 stories published

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 6, 2026

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  11 passed

Details

stats  11 tests across 8 suites
duration  42.9 seconds
commit  349efa7
info  🔄 Run: #16534 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  11 passed

Details

stats  11 tests across 8 suites
duration  10.9 seconds
commit  349efa7
info  🔄 Run: #16534 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  2 passed

Details

stats  2 tests across 2 suites
duration  50 seconds
commit  349efa7
info  🔄 Run: #16534 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  17 passed

Details

stats  17 tests across 14 suites
duration  1 minute, 16 seconds
commit  349efa7
info  🔄 Run: #16534 (attempt 1)

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 6, 2026

Visual Regression

16 screenshots compared. See report for details.
View full report

@Zaimwa9 Zaimwa9 merged commit c8023f0 into main May 6, 2026
58 checks passed
@Zaimwa9 Zaimwa9 deleted the ci/skip-chromatic-on-dependabot branch May 6, 2026 09:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cd Build, test and deployment related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants