Skip to content

test: ci #7900

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tmilewski wants to merge 5 commits into from
Closed

test: ci #7900

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
568c9ff
test: ci
tmilewski Feb 20, 2026
d07400e
chore: Test
tmilewski Feb 20, 2026
34a1f13
test: ci
tmilewski Feb 20, 2026
e6b699a
test: ci cache
tmilewski Feb 20, 2026
e69a0d9
test: ci
tmilewski Feb 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/ninety-laws-sort.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@clerk/backend': patch
---

TEST
213 changes: 144 additions & 69 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ jobs:

steps:
- name: Checkout Repo
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
fetch-depth: 100 # Enough for changeset status comparison, much faster than full history
fetch-tags: false
Expand All @@ -69,10 +69,10 @@ jobs:
- name: Setup
id: config
uses: ./.github/actions/init-blacksmith
# with:
# turbo-signature: ${{ secrets.TURBO_REMOTE_CACHE_SIGNATURE_KEY }}
# turbo-team: ${{ vars.TURBO_TEAM }}
# turbo-token: ${{ secrets.TURBO_TOKEN }}
with:
turbo-signature: ${{ secrets.TURBO_REMOTE_CACHE_SIGNATURE_KEY }}
turbo-team: ${{ vars.TURBO_TEAM }}
turbo-token: ${{ secrets.TURBO_TOKEN }}

- name: Verify lockfile is deduped
run: pnpm dedupe --check
Expand Down Expand Up @@ -107,7 +107,7 @@ jobs:

steps:
- name: Checkout Repo
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
fetch-depth: 1
fetch-tags: false
Expand All @@ -117,17 +117,23 @@ jobs:
- name: Setup
id: config
uses: ./.github/actions/init-blacksmith
# with:
# turbo-signature: ${{ secrets.TURBO_REMOTE_CACHE_SIGNATURE_KEY }}
# turbo-summarize: ${{ env.TURBO_SUMMARIZE }}
# turbo-team: ${{ vars.TURBO_TEAM }}
# turbo-token: ${{ secrets.TURBO_TOKEN }}
with:
turbo-signature: ${{ secrets.TURBO_REMOTE_CACHE_SIGNATURE_KEY }}
turbo-summarize: ${{ env.TURBO_SUMMARIZE }}
turbo-team: ${{ vars.TURBO_TEAM }}
turbo-token: ${{ secrets.TURBO_TOKEN }}

- name: Turbo Build
run: pnpm turbo build $TURBO_ARGS --only

- name: Save Turbo Cache
uses: useblacksmith/cache/save@v5
with:
path: .turbo-cache
key: turbo-build-${{ github.sha }}

- name: Upload Turbo Summary
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v6
if: ${{ env.TURBO_SUMMARIZE == 'true' }}
continue-on-error: true
with:
Expand All @@ -153,7 +159,7 @@ jobs:

steps:
- name: Checkout Repo
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
fetch-depth: 1
fetch-tags: false
Expand All @@ -163,11 +169,17 @@ jobs:
- name: Setup
id: config
uses: ./.github/actions/init-blacksmith
# with:
# turbo-signature: ${{ secrets.TURBO_REMOTE_CACHE_SIGNATURE_KEY }}
# turbo-summarize: ${{ env.TURBO_SUMMARIZE }}
# turbo-team: ${{ vars.TURBO_TEAM }}
# turbo-token: ${{ secrets.TURBO_TOKEN }}
with:
turbo-signature: ${{ secrets.TURBO_REMOTE_CACHE_SIGNATURE_KEY }}
turbo-summarize: ${{ env.TURBO_SUMMARIZE }}
turbo-team: ${{ vars.TURBO_TEAM }}
turbo-token: ${{ secrets.TURBO_TOKEN }}

- name: Restore Turbo Cache
uses: useblacksmith/cache/restore@v5
with:
path: .turbo-cache
key: turbo-build-${{ github.sha }}

- name: Check size using bundlewatch
continue-on-error: true
Expand All @@ -178,19 +190,13 @@ jobs:
CI_COMMIT_SHA: ${{ github.event.pull_request.head.sha }}
CI_REPO_NAME: ${{ vars.REPO_NAME }}
CI_REPO_OWNER: ${{ vars.REPO_OWNER }}
run: pnpm turbo bundlewatch $TURBO_ARGS

- name: Lint packages using publint
run: pnpm turbo lint:publint $TURBO_ARGS

- name: Lint types using attw
run: pnpm turbo lint:attw $TURBO_ARGS
run: pnpm turbo bundlewatch --affected $TURBO_ARGS

- name: Run lint
run: pnpm turbo lint $TURBO_ARGS
- name: Run linting (publint, attw, eslint)
run: pnpm turbo lint:publint lint:attw lint --affected $TURBO_ARGS

- name: Upload Turbo Summary
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v6
if: ${{ env.TURBO_SUMMARIZE == 'true' }}
continue-on-error: true
with:
Expand Down Expand Up @@ -219,12 +225,22 @@ jobs:
matrix:
include:
- node-version: 22
test-filter: "**"
filter-label: "**"
test-filter: "--filter=@clerk/backend"
filter-label: "@clerk/backend"
- node-version: 22
test-filter: "--filter=@clerk/clerk-js"
filter-label: "@clerk/clerk-js"
- node-version: 22
test-filter: "--filter=@clerk/shared --filter=@clerk/nextjs --filter=@clerk/react"
filter-label: "@clerk/shared, @clerk/nextjs, @clerk/react"
- node-version: 22
test-filter: "--filter=!@clerk/backend --filter=!@clerk/clerk-js --filter=!@clerk/shared --filter=!@clerk/nextjs --filter=!@clerk/react"
filter-label: "remaining packages"
run-typedoc: true

steps:
- name: Checkout Repo
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
fetch-depth: 1
fetch-tags: false
Expand All @@ -235,47 +251,89 @@ jobs:
id: config
uses: ./.github/actions/init-blacksmith
with:
# Ensures that all builds are cached appropriately with a consistent run name `Unit Tests (20)`.
node-version: ${{ matrix.node-version }}
# turbo-signature: ${{ secrets.TURBO_REMOTE_CACHE_SIGNATURE_KEY }}
# turbo-summarize: ${{ env.TURBO_SUMMARIZE }}
# turbo-team: ${{ vars.TURBO_TEAM }}
# turbo-token: ${{ secrets.TURBO_TOKEN }}
turbo-signature: ${{ secrets.TURBO_REMOTE_CACHE_SIGNATURE_KEY }}
turbo-summarize: ${{ env.TURBO_SUMMARIZE }}
turbo-team: ${{ vars.TURBO_TEAM }}
turbo-token: ${{ secrets.TURBO_TOKEN }}

- name: Restore Turbo Cache
uses: useblacksmith/cache/restore@v5
with:
path: .turbo-cache
key: turbo-build-${{ github.sha }}

- name: Run tests in packages
run: |
if [ "${{ matrix.test-filter }}" = "**" ]; then
echo "Running full test suite on Node ${{ matrix.node-version }}"
pnpm turbo test $TURBO_ARGS
else
echo "Running tests: ${{ matrix.filter-label }}"
pnpm turbo test $TURBO_ARGS ${{ matrix.test-filter }}
fi
echo "Running tests: ${{ matrix.filter-label }}"
pnpm turbo test $TURBO_ARGS ${{ matrix.test-filter }}
env:
NODE_VERSION: ${{ matrix.node-version }}

- name: Run Typedoc tests
run: |
# Only run Typedoc tests for one matrix version and main test run
if [ "${{ matrix.node-version }}" == "22" ] && [ "${{ matrix.test-filter }}" = "**" ]; then
pnpm turbo run //#test:typedoc
fi
if: ${{ matrix.run-typedoc }}
run: pnpm turbo run //#test:typedoc
env:
NODE_VERSION: ${{ matrix.node-version }}

- name: Upload Turbo Summary
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v6
if: ${{ env.TURBO_SUMMARIZE == 'true' }}
continue-on-error: true
with:
name: turbo-summary-report-unit-${{ github.run_id }}-${{ github.run_attempt }}-node-${{ matrix.node-version }}
name: turbo-summary-report-unit-${{ github.run_id }}-${{ github.run_attempt }}-${{ strategy.job-index }}
path: .turbo/runs
retention-days: 5

integration-tests:
# needs: [check-permissions, build-packages]
integration-setup:
needs: [check-permissions]
if: ${{ github.event_name != 'pull_request' || github.event.pull_request.draft == false }}
name: Integration Test Setup
runs-on: "blacksmith-8vcpu-ubuntu-2204"
permissions:
contents: read
actions: write
defaults:
run:
shell: bash
timeout-minutes: ${{ vars.TIMEOUT_MINUTES_NORMAL && fromJSON(vars.TIMEOUT_MINUTES_NORMAL) || 10 }}
Comment on lines +290 to +301
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Actionlint will flag the custom runner label.
Line 292 uses blacksmith-8vcpu-ubuntu-2204, which actionlint doesn’t recognize by default. If this is a self-hosted label, add it to actionlint.yaml (or include self-hosted in runs-on) to avoid lint failures.

🧰 Tools
🪛 actionlint (1.7.11)

[error] 292-292: label "blacksmith-8vcpu-ubuntu-2204" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2025-vs2026", "windows-2022", "windows-11-arm", "ubuntu-slim", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "macos-latest", "macos-latest-xlarge", "macos-latest-large", "macos-26-xlarge", "macos-26-large", "macos-26", "macos-15-intel", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xlarge", "macos-14-large", "macos-14", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file

(runner-label)

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/ci.yml around lines 288 - 299, The CI job
"integration-setup" uses a custom runner label "blacksmith-8vcpu-ubuntu-2204"
which actionlint flags; fix this by either adding "self-hosted" to the runs-on
array for the integration-setup job (e.g., runs-on: [self-hosted,
'blacksmith-8vcpu-ubuntu-2204']) or by adding the custom label
"blacksmith-8vcpu-ubuntu-2204" to your actionlint configuration
(actionlint.yaml) so actionlint recognizes it; update the runs-on or
actionlint.yaml and rerun the linter.


steps:
- name: Checkout Repo
uses: actions/checkout@v6
with:
fetch-depth: 1
fetch-tags: false
filter: "blob:none"
show-progress: false

- name: Setup
id: config
uses: ./.github/actions/init-blacksmith
with:
turbo-signature: ${{ secrets.TURBO_REMOTE_CACHE_SIGNATURE_KEY }}
turbo-team: ${{ vars.TURBO_TEAM }}
turbo-token: ${{ secrets.TURBO_TOKEN }}

- name: Version packages for snapshot
run: npm run version-packages:snapshot ci

- name: Verdaccio
uses: ./.github/actions/verdaccio
with:
publish-cmd: |
if [ "$(pnpm config get registry)" = "https://registry.npmjs.org/" ]; then echo 'Error: Using default registry' && exit 1; else pnpm turbo build $TURBO_ARGS --only && pnpm changeset publish --no-git-tag --tag latest; fi

- name: Upload Verdaccio Storage
uses: actions/upload-artifact@v6
with:
name: verdaccio-storage-${{ github.run_id }}-${{ github.run_attempt }}
path: .verdaccio/storage
Comment on lines +329 to +333
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Avoid run_attempt in Verdaccio storage artifact name.
On “Re-run failed jobs,” integration-setup may not rerun, so the download step (Line 440+) will look for a new run_attempt artifact that doesn’t exist. Use a stable name (e.g., run_id only) or force integration-setup to rerun.

🛠️ Proposed fix (stable artifact name across reruns) 
-          name: verdaccio-storage-${{ github.run_id }}-${{ github.run_attempt }}
+          name: verdaccio-storage-${{ github.run_id }}
-          name: verdaccio-storage-${{ github.run_id }}-${{ github.run_attempt }}
+          name: verdaccio-storage-${{ github.run_id }}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- name: Upload Verdaccio Storage
uses: actions/upload-artifact@v6
with:
name: verdaccio-storage-${{ github.run_id }}-${{ github.run_attempt }}
path: .verdaccio/storage
- name: Upload Verdaccio Storage
uses: actions/upload-artifact@v6
with:
name: verdaccio-storage-${{ github.run_id }}
path: .verdaccio/storage
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/ci.yml around lines 327 - 331, The artifact name in the
"Upload Verdaccio Storage" step uses `${{ github.run_id }}-${{
github.run_attempt }}` which changes on reruns and breaks the downloader; change
the artifact naming to a stable value (e.g., use only `${{ github.run_id }}`) so
the upload step ("Upload Verdaccio Storage") produces a consistent artifact name
that the download step can reliably reference across reruns.

retention-days: 1

integration-tests:
needs: [check-permissions, integration-setup]
if: ${{ github.event_name != 'pull_request' || github.event.pull_request.draft == false }}
name: Integration Tests (${{ matrix.test-name }}, ${{ matrix.test-project }}${{ matrix.next-version && format(', {0}', matrix.next-version) || '' }})
permissions:
contents: read
Expand Down Expand Up @@ -330,7 +388,7 @@ jobs:

steps:
- name: Checkout Repo
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
fetch-depth: 1
fetch-tags: false
Expand All @@ -341,9 +399,9 @@ jobs:
id: config
uses: ./.github/actions/init-blacksmith
with:
# turbo-signature: ${{ secrets.TURBO_REMOTE_CACHE_SIGNATURE_KEY }}
# turbo-team: ${{ vars.TURBO_TEAM }}
# turbo-token: ${{ secrets.TURBO_TOKEN }}
turbo-signature: ${{ secrets.TURBO_REMOTE_CACHE_SIGNATURE_KEY }}
turbo-team: ${{ vars.TURBO_TEAM }}
turbo-token: ${{ secrets.TURBO_TOKEN }}
playwright-enabled: true

- name: Verify jq is installed
Expand Down Expand Up @@ -379,16 +437,27 @@ jobs:
echo "affected=${AFFECTED}"
echo "affected=${AFFECTED}" >> $GITHUB_OUTPUT

- name: Version packages for snapshot
- name: Download Verdaccio Storage
if: ${{ steps.task-status.outputs.affected == '1' }}
run: npm run version-packages:snapshot ci
uses: actions/download-artifact@v6
with:
name: verdaccio-storage-${{ github.run_id }}-${{ github.run_attempt }}
path: .verdaccio/storage

- name: Verdaccio
- name: Start Verdaccio
if: ${{ steps.task-status.outputs.affected == '1' }}
uses: ./.github/actions/verdaccio
with:
publish-cmd: |
if [ "$(pnpm config get registry)" = "https://registry.npmjs.org/" ]; then echo 'Error: Using default registry' && exit 1; else pnpm turbo build $TURBO_ARGS --only && pnpm changeset publish --no-git-tag --tag latest; fi
run: |
pnpm set registry http://localhost:4873
nohup ./node_modules/.bin/verdaccio --config ./verdaccio.install.yaml &
for i in {1..10}; do
if curl -f http://localhost:4873/ > /dev/null 2>&1; then
echo "Verdaccio is up and running"
break
fi
echo "Waiting for Verdaccio to start (attempt $i)..."
sleep 2
done
pnpm config set //localhost:4873/:_authToken secretToken

Comment on lines +453 to 467
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Fail fast if Verdaccio never starts.
If the loop times out, the job currently continues and later steps fail with less actionable errors. Add a guard to exit when the registry never becomes healthy.

🛠️ Proposed fix (explicit failure on startup timeout) 
       - name: Start Verdaccio
         if: ${{ steps.task-status.outputs.affected == '1' }}
         run: |
           pnpm set registry http://localhost:4873
           nohup ./node_modules/.bin/verdaccio --config ./verdaccio.install.yaml &
+          started=false
           for i in {1..10}; do
             if curl -f http://localhost:4873/ > /dev/null 2>&1; then
               echo "Verdaccio is up and running"
+              started=true
               break
             fi
             echo "Waiting for Verdaccio to start (attempt $i)..."
             sleep 2
           done
+          if [ "$started" != "true" ]; then
+            echo "Verdaccio failed to start" >&2
+            exit 1
+          fi
           pnpm config set //localhost:4873/:_authToken secretToken
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/ci.yml around lines 447 - 461, The "Start Verdaccio"
step's health-check loop may time out silently causing later steps to fail;
modify the step so after the for-loop that checks curl -f http://localhost:4873/
you explicitly fail the job if Verdaccio never becomes healthy (e.g., detect
that the loop completed without break and run an exit 1 with a clear error
message). Update the same step that runs nohup ./node_modules/.bin/verdaccio and
the for i in {1..10} loop to add a guard (post-loop conditional) that echoes an
explanatory error and exits non-zero when the registry is unreachable, ensuring
subsequent steps don't run against a missing registry.

- name: Edit .npmrc [link-workspace-packages=false]
run: sed -i -E 's/link-workspace-packages=(deep|true)/link-workspace-packages=false/' .npmrc
Expand Down Expand Up @@ -468,7 +537,7 @@ jobs:

- name: Upload test-results
if: ${{ cancelled() || failure() }}
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v6
with:
name: playwright-traces-${{ github.run_id }}-${{ github.run_attempt }}-${{ matrix.test-name }}${{ matrix.next-version && format('-next{0}', matrix.next-version) || '' }}
path: integration/test-results
Expand All @@ -488,7 +557,7 @@ jobs:

steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
fetch-depth: 1
fetch-tags: false
Expand All @@ -500,10 +569,16 @@ jobs:
with:
turbo-enabled: true
node-version: 22
# turbo-signature: ${{ secrets.TURBO_REMOTE_CACHE_SIGNATURE_KEY }}
# turbo-summarize: ${{ env.TURBO_SUMMARIZE }}
# turbo-team: ${{ vars.TURBO_TEAM }}
# turbo-token: ${{ secrets.TURBO_TOKEN }}
turbo-signature: ${{ secrets.TURBO_REMOTE_CACHE_SIGNATURE_KEY }}
turbo-summarize: ${{ env.TURBO_SUMMARIZE }}
turbo-team: ${{ vars.TURBO_TEAM }}
turbo-token: ${{ secrets.TURBO_TOKEN }}

- name: Restore Turbo Cache
uses: useblacksmith/cache/restore@v5
with:
path: .turbo-cache
key: turbo-build-${{ github.sha }}

- name: Publish with pkg-pr-new
run: pnpm run build && pnpx pkg-pr-new@${{ vars.PKG_PR_NEW_VERSION || '0.0.49' }} publish --compact --pnpm './packages/*'
8 changes: 8 additions & 0 deletions packages/backend/src/api/endpoints/BetaFeaturesApi.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,12 @@ export class BetaFeaturesAPI extends AbstractAPI {
bodyParams: params,
});
}

public async get() {
// TODO: Remove - Strictly a test endpoint
return this.request<void>({
method: 'GET',
path: joinPaths(basePath),
});
}
Comment on lines +39 to +45
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Test endpoint should not be merged to production code.

This method is explicitly marked as a test endpoint via the TODO comment. Adding test-only code to production API classes is a merge-blocking concern—this could accidentally ship and expose an unintended endpoint.

If this is purely for CI testing purposes, consider:

  • Moving it to a test fixture or mock
  • Using a separate test-only module that isn't published
  • Removing before merge if the PR objective is only CI workflow changes
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@packages/backend/src/api/endpoints/BetaFeaturesApi.ts` around lines 39 - 45,
The get method in BetaFeaturesApi is a test-only endpoint (see the TODO) and
must not be merged into production; remove this test method from the
BetaFeaturesApi class or relocate it into a non-published test fixture/module
(e.g., a dedicated test file or mock API) so it isn't part of the published
package, and update any callers or tests to import the new test-only module
instead of BetaFeaturesApi.get (ensure you stop using basePath/request from the
production API in tests or mark the module as test-only).

}
Loading