Skip to content

Gate E2E tests behind APPLESCRIPT_E2E flag; auto-skip in CI#10

Merged
frouaix merged 2 commits intoissue-1-e2e-testingfrom
copilot/sub-pr-7-another-one
Feb 22, 2026
Merged

Gate E2E tests behind APPLESCRIPT_E2E flag; auto-skip in CI#10
frouaix merged 2 commits intoissue-1-e2e-testingfrom
copilot/sub-pr-7-another-one

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Feb 22, 2026

E2E tests in test/integration/e2e-apps.test.ts spawn real macOS apps and require Automation permissions — running them unconditionally in CI (where CI=true) causes failures or hangs on GitHub Actions runners.

Changes

  • Skip condition: Added a skipReason evaluated at module load time; the entire suite skips when process.env.CI is set or process.env.APPLESCRIPT_E2E is unset
  • Skip messaging: Uses Node's native { skip: reason } option on describe for clean, informative output with zero test infrastructure overhead
const skipReason: string | undefined = process.env.CI
  ? "Skipped in CI environment (set APPLESCRIPT_E2E=1 and unset CI to run)"
  : !process.env.APPLESCRIPT_E2E
    ? "Set APPLESCRIPT_E2E=1 to run E2E tests locally"
    : undefined;

describe("E2E app tests", { skip: skipReason }, () => {  });

To run locally: APPLESCRIPT_E2E=1 pnpm test:integration


🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

Co-authored-by: frouaix <876178+frouaix@users.noreply.github.com>
Copilot AI changed the title [WIP] Update E2E tests based on feedback from #7 Gate E2E tests behind APPLESCRIPT_E2E flag; auto-skip in CI Feb 22, 2026
Copilot AI requested a review from frouaix February 22, 2026 01:44
@frouaix frouaix marked this pull request as ready for review February 22, 2026 02:22
Copilot AI review requested due to automatic review settings February 22, 2026 02:22
@frouaix frouaix merged commit 792bc3b into issue-1-e2e-testing Feb 22, 2026
1 check passed
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the macOS AppleScript E2E integration suite to avoid running on CI runners where Automation permissions and real apps are unavailable, preventing CI hangs/failures while keeping local opt-in execution.

Changes:

  • Adds a module-level skipReason that skips the whole suite when CI is set or APPLESCRIPT_E2E is not set.
  • Uses node:test’s { skip: reason } option on describe() to provide a clear skip message.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +115 to +119
const skipReason: string | undefined = process.env.CI
? "Skipped in CI environment (set APPLESCRIPT_E2E=1 and unset CI to run)"
: !process.env.APPLESCRIPT_E2E
? "Set APPLESCRIPT_E2E=1 to run E2E tests locally"
: undefined;
Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

skipReason currently doesn't enforce the documented "macOS required" constraint. If someone sets APPLESCRIPT_E2E=1 on Linux/Windows (or runs on a non-darwin CI runner), the suite will still execute and likely fail/hang. Consider adding a process.platform !== "darwin" skip reason (with a clear message) so the suite can never run off-macOS.

Suggested change
const skipReason: string | undefined = process.env.CI
? "Skipped in CI environment (set APPLESCRIPT_E2E=1 and unset CI to run)"
: !process.env.APPLESCRIPT_E2E
? "Set APPLESCRIPT_E2E=1 to run E2E tests locally"
: undefined;
const skipReason: string | undefined =
process.platform !== "darwin"
? "AppleScript E2E tests require macOS (process.platform !== 'darwin')"
: process.env.CI
? "Skipped in CI environment (set APPLESCRIPT_E2E=1 and unset CI to run)"
: !process.env.APPLESCRIPT_E2E
? "Set APPLESCRIPT_E2E=1 to run E2E tests locally"
: undefined;

Copilot uses AI. Check for mistakes.
describe("E2E app tests", () => {
const skipReason: string | undefined = process.env.CI
? "Skipped in CI environment (set APPLESCRIPT_E2E=1 and unset CI to run)"
: !process.env.APPLESCRIPT_E2E
Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The gate is documented as APPLESCRIPT_E2E=1, but the check uses truthiness (!process.env.APPLESCRIPT_E2E). Values like APPLESCRIPT_E2E=0 are truthy and would incorrectly enable these tests. Consider checking explicitly for the expected value (e.g., process.env.APPLESCRIPT_E2E === "1") so the flag behaves as documented.

Suggested change
: !process.env.APPLESCRIPT_E2E
: process.env.APPLESCRIPT_E2E !== "1"

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants