Skip to content

fix(ci): re-enable Android E2E tests with BrowserStack Detox#44

Open
tsdamas wants to merge 8 commits into
mainfrom
SDKS-5053
Open

fix(ci): re-enable Android E2E tests with BrowserStack Detox#44
tsdamas wants to merge 8 commits into
mainfrom
SDKS-5053

Conversation

@tsdamas
Copy link
Copy Markdown
Contributor

@tsdamas tsdamas commented May 28, 2026

Summary by CodeRabbit

  • Chores
    • Pinned Xcode for iOS CI and improved simulator selection logic.
    • Added manual workflow trigger and re-enabled Android end-to-end jobs in CI.
    • Expanded Android BrowserStack E2E to run codegen for additional React Native modules.
    • Updated Detox/BrowserStack test dependency for Android runs.
    • Registered additional Android React Native modules and adjusted packaging to exclude specific META-INF entries.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 28, 2026

Warning

Review limit reached

@tsdamas, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 1 minute and 55 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9d01cf73-5d4e-4f7b-85b4-f143966cbd2e

📥 Commits

Reviewing files that changed from the base of the PR and between d892fd1 and be2385d.

📒 Files selected for processing (2)
  • .github/workflows/browserstack-prep-ios-artifacts.yml
  • .github/workflows/build-and-test-ios.yml
📝 Walkthrough

Walkthrough

Enables Android E2E CI jobs, expands BrowserStack Android codegen tasks, updates PingTestRunner scripts/dependencies and Android packaging/includes, and pins/simplifies Xcode selection and simulator resolution in the iOS CI workflow.

Changes

Android E2E Testing Enablement

Layer / File(s) Summary
Enable Android E2E jobs in CI
.github/workflows/ci.yml, .github/workflows/e2e-tests.yml
Adds workflow_dispatch and removes if: false guards to enable browserstack-android and e2e-android jobs.
BrowserStack workflow and Gradle codegen expansion
.github/workflows/browserstack-e2e-android.yml
Removed top TODO comment; expanded generateCodegenArtifactsFromSchema task list to include :ping-identity_rn-fido and :ping-identity_rn-oath.
PingTestRunner scripts, modules, and packaging
PingTestRunner/package.json, PingTestRunner/android/settings.gradle, PingTestRunner/android/app/build.gradle
Prefixed Gradle assemble tasks in build:bs:android script; bumped detox to @browserstack/detox@20.38.0-cloud.3; added new Android subproject includes; added packaging.resources exclusions for selected META-INF files.

iOS Build Tooling Update

Layer / File(s) Summary
Xcode version pinning and simulator selection
.github/workflows/build-and-test-ios.yml
Pinned XCODE_VERSION to 26.2; replaced dynamic Xcode/runtime probing with direct xcode-select to the pinned app and xcodebuild -version; replaced xcrun simctl + jq enumeration with parsing of xcodebuild -showdestinations to pick a simulator (prefer iPhone 17) and export SIMULATOR_UDID, SIMULATOR_NAME, and OS_VERSION.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 I hopped through YAML and Gradle lines,
Unhid the jobs and pinned the Xcode signs,
Codegen grew with FIDO, OATH in tow,
Detox bumped — the runners now can go,
APKs slimmed down as resources go.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the primary objective: re-enabling Android E2E tests with BrowserStack Detox, which is reflected in multiple file changes across ci.yml, e2e-tests.yml, and related test configuration files.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch SDKS-5053

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@tsdamas tsdamas marked this pull request as draft May 28, 2026 21:33
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 28, 2026

PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://ForgeRock.github.io/ping-react-native-sdk/docs-preview/pr-44/

Built to branch gh-pages at 2026-05-29 23:21 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/build-and-test-ios.yml:
- Around line 116-124: The current extraction of AVAILABLE using awk '/Available
destinations/,/Ineligible destinations/' | head -n -1 is non-portable on macOS;
update the pipeline that sets AVAILABLE (which reads DESTINATIONS) to use a
single awk invocation that prints lines after the "Available destinations"
header and stops at "Ineligible destinations" if present (i.e., use awk state
flags to skip the two header/footer lines and tolerate a missing "Ineligible
destinations"), so DEST_LINE (which greps DESTINATIONS/AVAILABLE for
"platform:iOS Simulator" and "name:iPhone") receives the correct block on BSD
head.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d896e12c-3403-4dc2-b2f2-a1ff5b7b6e84

📥 Commits

Reviewing files that changed from the base of the PR and between 100c49b and 14ad5e7.

⛔ Files ignored due to path filters (2)
  • .yarn/install-state.gz is excluded by !**/.yarn/**, !**/*.gz
  • yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (5)
  • .github/workflows/browserstack-e2e-android.yml
  • .github/workflows/build-and-test-ios.yml
  • .github/workflows/ci.yml
  • .github/workflows/e2e-tests.yml
  • PingTestRunner/package.json
💤 Files with no reviewable changes (2)
  • .github/workflows/e2e-tests.yml
  • .github/workflows/ci.yml

Comment thread .github/workflows/build-and-test-ios.yml
@tsdamas tsdamas marked this pull request as ready for review May 29, 2026 19:06
@tsdamas tsdamas closed this May 29, 2026
@tsdamas tsdamas reopened this May 29, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/build-and-test-ios.yml:
- Around line 87-88: Replace the hard-coded xcode-select step ("Select Xcode
version") with the runtime-aware "Resolve Xcode version" logic used in the
e2e-tests workflow (or add an equivalent preflight) so the job verifies a
provisioned iOS simulator runtime before switching Xcode; specifically, reuse
the resolver that determines an Xcode installation with matching simulator
runtimes (the same step that precedes "Resolve simulator destination") and run
that check before calling xcode-select, and then remove or narrow the use of
job-level continue-on-error so simulator/setup failures aren’t silently masked.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3da92390-5376-4a9f-a296-3acc5021cb6c

📥 Commits

Reviewing files that changed from the base of the PR and between 8bb854f and d892fd1.

⛔ Files ignored due to path filters (2)
  • .yarn/install-state.gz is excluded by !**/.yarn/**, !**/*.gz
  • yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (7)
  • .github/workflows/browserstack-e2e-android.yml
  • .github/workflows/build-and-test-ios.yml
  • .github/workflows/ci.yml
  • .github/workflows/e2e-tests.yml
  • PingTestRunner/android/app/build.gradle
  • PingTestRunner/android/settings.gradle
  • PingTestRunner/package.json
💤 Files with no reviewable changes (1)
  • .github/workflows/e2e-tests.yml
🚧 Files skipped from review as they are similar to previous changes (3)
  • .github/workflows/ci.yml
  • .github/workflows/browserstack-e2e-android.yml
  • PingTestRunner/android/settings.gradle

Comment on lines 87 to +88
- name: Select Xcode version
run: sudo xcode-select -s "${{ env.XCODE_RESOLVED_PATH }}" && /usr/bin/xcodebuild -version
run: sudo xcode-select -s /Applications/Xcode_${{ env.XCODE_VERSION }}.app && /usr/bin/xcodebuild -version
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Locate and print the relevant sections for the iOS workflow and the shared resolver
FILE1=".github/workflows/build-and-test-ios.yml"
FILE2=".github/workflows/e2e-tests.yml"

echo "=== $FILE1 (around select Xcode version + simulator destination) ==="
# Print around the referenced line numbers with some buffer
nl -ba "$FILE1" | sed -n '60,140p'

echo
echo "=== $FILE2 (around Xcode/simulator resolver) ==="
nl -ba "$FILE2" | sed -n '35,120p'

echo
echo "=== $FILE1 (full job blocks with continue-on-error) ==="
# Show lines containing continue-on-error and any "Resolve simulator destination" step name
rg -n "continue-on-error|Resolve simulator destination|xcode-select|Xcode_|xcodebuild -showdestinations" "$FILE1" || true

Repository: ForgeRock/ping-react-native-sdk

Length of output: 240


🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE1=".github/workflows/build-and-test-ios.yml"
FILE2=".github/workflows/e2e-tests.yml"

echo "=== $FILE1 (around select Xcode version + simulator destination) ==="
awk 'NR>=60 && NR<=140 {printf "%d:%s\n", NR, $0}' "$FILE1" | sed -n '1,200p'

echo
echo "=== $FILE2 (around Xcode/simulator resolver) ==="
awk 'NR>=35 && NR<=120 {printf "%d:%s\n", NR, $0}' "$FILE2" | sed -n '1,200p'

echo
echo "=== $FILE1 (locations of key steps/flags) ==="
rg -n "continue-on-error|Resolve simulator destination|xcode-select|Xcode_|xcodebuild -showdestinations" "$FILE1" || true

Repository: ForgeRock/ping-react-native-sdk

Length of output: 9971


🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE1=".github/workflows/build-and-test-ios.yml"
awk 'NR>=1 && NR<=260 {printf "%d:%s\n", NR, $0}' "$FILE1" | sed -n '1,260p'

Repository: ForgeRock/ping-react-native-sdk

Length of output: 9302


🌐 Web query:

GitHub Actions job-level continue-on-error support semantics

💡 Result:

GitHub Actions supports continue-on-error at the job level via jobs.<job_id>.continue-on-error (boolean / expression). When set to true, failures in that job don’t fail the overall workflow run, but they do still mark the job as failed (so you’ll typically see a red X in the UI) and any steps inside the job still behave as usual unless you also use step-level continue-on-error. Semantics to know 1) What changes at the job level - jobs.<job_id>.continue-on-error: true prevents the job from failing the overall workflow run; i.e., downstream workflows/jobs that depend on the run won’t be blocked the same way as with a hard failure. [1] - Even with job-level continue-on-error enabled, the job itself is still considered failed in the job’s own conclusion/visual status. [2] 2) How step expressions like failure behave - With job-level continue-on-error, a failed step can still trigger conditions like if: ${{ failure }} in subsequent steps because the step actually failed (even though the job is allowed to continue). This is consistent with reported behavior: the workflow runs subsequent steps, but the job is still marked as an overall failure. [3] 3) Job status vs workflow status - There’s a distinction between “job did not fail the workflow” and “job is green.” An observed outcome is: subsequent steps can run, but the job will be an overall failure (red X), even though the workflow may be allowed to proceed. [2] 4) Nested/indirect continue-on-error limitations - continue-on-error at a higher level does not necessarily suppress failures from deeply nested actions (e.g., action calling another action calling a node action). This means job-level continue-on-error may not “mask” all underlying tool failures across all nesting layers. [4] Practical guidance - Use job-level continue-on-error when you want the workflow to keep going even if that entire job encounters an error, but you still want the job to be visible as failed for diagnostics. [1][2] - If you need specific steps to not trigger conditional logic (or you want to control exactly which later steps run), consider step-level continue-on-error and/or explicit conditions based on step outcomes rather than relying on failure alone. The step-triggered behavior is documented/observed under job-level continue-on-error. [3] Example (job-level) jobs: test: runs-on: ubuntu-latest continue-on-error: true steps: - run: exit 1 - run: echo "This runs, but the job is still failed"

Citations:


Keep the runtime-compatible Xcode resolver (avoid pinned hard-switch) in iOS workflow

  • .github/workflows/build-and-test-ios.yml hard-switches to /Applications/Xcode_${{ env.XCODE_VERSION }}.app (lines 87-88) but doesn’t use the runtime-aware “Resolve Xcode version” logic from .github/workflows/e2e-tests.yml.
  • If that pinned Xcode lacks a provisioned iOS simulator runtime, “Resolve simulator destination” (lines 114-127) can fail with “No eligible iPhone simulator found”, preventing the later iOS test steps from running; with job-level continue-on-error: true (line 29), the overall workflow run can still succeed even when iOS tests never execute.
  • Reuse the existing resolver (or equivalent preflight verifying simulator runtime availability) before running xcode-select, and avoid relying on continue-on-error to mask simulator/setup failures when tests are expected to run.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/build-and-test-ios.yml around lines 87 - 88, Replace the
hard-coded xcode-select step ("Select Xcode version") with the runtime-aware
"Resolve Xcode version" logic used in the e2e-tests workflow (or add an
equivalent preflight) so the job verifies a provisioned iOS simulator runtime
before switching Xcode; specifically, reuse the resolver that determines an
Xcode installation with matching simulator runtimes (the same step that precedes
"Resolve simulator destination") and run that check before calling xcode-select,
and then remove or narrow the use of job-level continue-on-error so
simulator/setup failures aren’t silently masked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant