chore(ci): run karma on local headless browsers instead of BrowserStack

[Essk]

Description

CI has been red for months, including on the 3.17.5 release push. The unit, playback and playback-min matrix jobs hang in run npm test until the 6 hour timeout.

From the logs, the BrowserStack sessions never start, and karma waits for them forever. Why they never start hasn't been diagnosed — whether the account or credentials behind the BROWSER_STACK_USERNAME / BROWSER_STACK_ACCESS_KEY secrets are still valid isn't visible from the repo side. But the hang is avoidable entirely by skipping BrowserStack: with those env vars removed from the job, videojs-generate-karma-config falls back to local browser detection, which is exactly the configuration the coverage test type already runs (it empties browserstackLaunchers) and passes in about two minutes on the same runner.

In our own player stack we've found that testing in real (cloud) browsers really only adds value for Safari-exclusive playback behaviour — native HLS, AirPlay integration and the like — and we've reworked our test stack to use headless browsers everywhere else, which made CI both faster and more stable. In this repo, the BrowserStack setup provides no Safari coverage anyway: scripts/karma.conf.js filters /(Safari|Chromium)/ out of the browser list, so the effective BrowserStack set was only bsChrome plus a bsFirefox pinned to version 64 (a 2018 build). Restoring genuine Safari coverage would be worthwhile, but it's a separate piece of work from repairing CI. If the secrets are confirmed working again later, restoring the two env vars restores the old behavior; the BrowserStack launcher plumbing is untouched.

Running the suite on current local browsers surfaced two further issues that this PR also fixes:

1. preferHeadless: false (boilerplate from the 2019 generator migration) made local detection launch headful Chrome and Firefox on the runner. Removed, so the standard headless variants run.
2. Firefox froze the unit suite at exactly test Update rollup to the latest version 🚀 #782. The suite creates a dedicated web worker per segment loader per test, and Firefox caps dedicated workers at 512 per origin (dom.workers.maxPerDomain) — silently queueing creation past the cap rather than erroring. Once crossed, a fresh transmuxer worker never starts, its probeTs message is never processed, and the awaited appended event never fires; QUnit has no default test timeout, so karma's no-activity watchdog produces a deterministic Executed 781 of 1410 … DISCONNECTED retry loop for ~21 minutes. (The instrumented coverage build passes on Firefox only because it runs slowly enough for GC to reclaim orphaned workers; Chrome has no equivalent cap.) Fixed with a FirefoxHeadlessHighWorkerLimit launcher that raises the pref. The long-term fix is terminating leaked workers in test teardown, at which point the pref becomes belt-and-braces — proposed as a follow-up rather than blocking the CI repair.

Specific Changes proposed

• Remove the two BROWSER_STACK_* env vars from the ci job so karma detects and uses the runner's local browsers.
• Remove preferHeadless: false from scripts/karma.conf.js.
• Add a FirefoxHeadlessHighWorkerLimit custom launcher (Firefox with dom.workers.maxPerDomain: 10000) and map it in place of detected FirefoxHeadless.

No changes to the test-type matrix or the Firefox version pin (which is inert without BrowserStack).

Verification

• Locally (macOS), CI_TEST_TYPE=unit with the final config passes fully on Firefox 153: TOTAL: 1403 SUCCESS; with default prefs it reproduces the CI freeze at exactly test Update rollup to the latest version 🚀 #782. Chrome passes throughout (TOTAL: 2806 SUCCESS across two instances).
• This PR's own checks demonstrate the fix, since pull_request workflow runs use the merge ref: all four matrix jobs now pass in 1.5–3 minutes on both Chrome and Firefox.
• Unblocks fix: forward parse708captions and captionServices to the mp4 caption parser #1611, which cannot currently get a green CI run.

Requirements Checklist

• Feature implemented / Bug fixed
• If necessary, more likely in a feature request than a bug fix
  • Unit Tests updated or fixed
  • Docs/guides updated
  • Example created (starter template on JSBin)
• Reviewed by Two Core Contributors

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.00%. Comparing base (a9f9d7a) to head (58cb5ca).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1612   +/-   ##
=======================================
  Coverage   84.00%   84.00%           
=======================================
  Files          44       44           
  Lines       11713    11713           
  Branches     2625     2625           
=======================================
  Hits         9840     9840           
  Misses       1873     1873           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Essk]

CI results on this PR: the BrowserStack hang is fixed, but it has uncovered a second, unrelated problem in the unit job that this PR does not fix.

Green (previously hung to the 6h timeout, now finish in ~2.5 min):

• ci (ubuntu-latest, coverage) — TOTAL: 2806 SUCCESS
• ci (ubuntu-latest, playback) — TOTAL: 41 SUCCESS
• ci (ubuntu-latest, playback-min) — TOTAL: 41 SUCCESS

All three log Launching browsers Chrome, Firefox, i.e. the runner's local Chrome 151 and Firefox 153 — which is also a coverage improvement over the BrowserStack set, where Firefox was pinned to 64.

Still red: ci (ubuntu-latest, unit)

Chrome passes (Executed 1403 of 1410 (skipped 7) SUCCESS), but Firefox stalls partway and never recovers:

Firefox 153.0 (Linux x86_64): Executed 781 of 1410 (skipped 3) DISCONNECTED (5 mins 28.313 secs / 18.896 secs)
  Disconnected reconnect failed before timeout of 300000ms (ping timeout)

It then loops through browserDisconnectTolerance: 3 reconnects at 5 minutes each and fails the job at ~21 minutes.

This looks deterministic rather than flaky: it stalled at exactly 781 of 1410 in both the push and pull_request runs, on Firefox 152 and 153 respectively. Notably coverage runs the same test set (see scripts/rollup.config.js — unit and coverage build the same bundle) and Firefox completes all 1403 in about 10 seconds there. The only difference is istanbul instrumentation, so the timing change appears to be what hides the hang — which suggests a race in a test rather than an infrastructure problem.

I have deliberately not touched the tests or karma config to force this green. Worth deciding whether to merge this as-is (it takes CI from one green job to three, and turns a 6-hour hang into a 21-minute failure) and track the Firefox unit stall separately, or hold this until that stall is diagnosed too.

[Essk]

Root cause of the unit failure found and fixed.

The unit suite creates a dedicated web worker per segment loader per test. Firefox caps dedicated workers at 512 per origin (dom.workers.maxPerDomain) and silently queues worker creation past the cap rather than erroring. Around test #782 the cap is crossed, so that test's fresh transmuxer worker never starts, its probeTs message is never processed, and the awaited appended event never fires. QUnit has no default test timeout, so the suite stalls until karma's no-activity watchdog kills the browser — producing the deterministic Executed 781 of 1410 … DISCONNECTED retry loop at ~21 minutes.

This also explains the apparent paradox that coverage (same tests) passes on Firefox: the istanbul-instrumented build runs slowly enough that GC reclaims orphaned blob workers before the cap bites. Chrome has no equivalent cap. Reproduced locally on Firefox 153 (macOS): default prefs hang at exactly 781; with dom.workers.maxPerDomain: 10000 the full suite passes (TOTAL: 1403 SUCCESS).

Fix here: a FirefoxHeadlessHighWorkerLimit launcher raising the pref, mapped in place of detected FirefoxHeadless. The diagnostic logging commit has been reverted. Long-term follow-up worth filing: terminate leaked workers in test teardown (many tests never dispose their loaders/decrypters), at which point the pref becomes belt-and-braces.

[Essk]

Looks like the agent I asked to debug this left some "helpful" comments. I'll leave them, as while not the easiest to read they are accurate and leave context for future investigations. However, apologies for the 🤖 spam.