Gate E2E tests behind APPLESCRIPT_E2E flag; auto-skip in CI#10
Gate E2E tests behind APPLESCRIPT_E2E flag; auto-skip in CI#10frouaix merged 2 commits intoissue-1-e2e-testingfrom
Conversation
Co-authored-by: frouaix <876178+frouaix@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Updates the macOS AppleScript E2E integration suite to avoid running on CI runners where Automation permissions and real apps are unavailable, preventing CI hangs/failures while keeping local opt-in execution.
Changes:
- Adds a module-level
skipReasonthat skips the whole suite whenCIis set orAPPLESCRIPT_E2Eis not set. - Uses
node:test’s{ skip: reason }option ondescribe()to provide a clear skip message.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const skipReason: string | undefined = process.env.CI | ||
| ? "Skipped in CI environment (set APPLESCRIPT_E2E=1 and unset CI to run)" | ||
| : !process.env.APPLESCRIPT_E2E | ||
| ? "Set APPLESCRIPT_E2E=1 to run E2E tests locally" | ||
| : undefined; |
There was a problem hiding this comment.
skipReason currently doesn't enforce the documented "macOS required" constraint. If someone sets APPLESCRIPT_E2E=1 on Linux/Windows (or runs on a non-darwin CI runner), the suite will still execute and likely fail/hang. Consider adding a process.platform !== "darwin" skip reason (with a clear message) so the suite can never run off-macOS.
| const skipReason: string | undefined = process.env.CI | |
| ? "Skipped in CI environment (set APPLESCRIPT_E2E=1 and unset CI to run)" | |
| : !process.env.APPLESCRIPT_E2E | |
| ? "Set APPLESCRIPT_E2E=1 to run E2E tests locally" | |
| : undefined; | |
| const skipReason: string | undefined = | |
| process.platform !== "darwin" | |
| ? "AppleScript E2E tests require macOS (process.platform !== 'darwin')" | |
| : process.env.CI | |
| ? "Skipped in CI environment (set APPLESCRIPT_E2E=1 and unset CI to run)" | |
| : !process.env.APPLESCRIPT_E2E | |
| ? "Set APPLESCRIPT_E2E=1 to run E2E tests locally" | |
| : undefined; |
| describe("E2E app tests", () => { | ||
| const skipReason: string | undefined = process.env.CI | ||
| ? "Skipped in CI environment (set APPLESCRIPT_E2E=1 and unset CI to run)" | ||
| : !process.env.APPLESCRIPT_E2E |
There was a problem hiding this comment.
The gate is documented as APPLESCRIPT_E2E=1, but the check uses truthiness (!process.env.APPLESCRIPT_E2E). Values like APPLESCRIPT_E2E=0 are truthy and would incorrectly enable these tests. Consider checking explicitly for the expected value (e.g., process.env.APPLESCRIPT_E2E === "1") so the flag behaves as documented.
| : !process.env.APPLESCRIPT_E2E | |
| : process.env.APPLESCRIPT_E2E !== "1" |
E2E tests in
test/integration/e2e-apps.test.tsspawn real macOS apps and require Automation permissions — running them unconditionally in CI (whereCI=true) causes failures or hangs on GitHub Actions runners.Changes
skipReasonevaluated at module load time; the entire suite skips whenprocess.env.CIis set orprocess.env.APPLESCRIPT_E2Eis unset{ skip: reason }option ondescribefor clean, informative output with zero test infrastructure overheadTo run locally:
APPLESCRIPT_E2E=1 pnpm test:integration🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.