From d419ee7a11e0be59d098c9071ee4564466cc2e67 Mon Sep 17 00:00:00 2001 From: Chaitanya Sharma Date: Wed, 22 Jul 2026 23:24:41 +0530 Subject: [PATCH 1/3] docs(smartui): document the app capture half of the Figma-App CLI workflow The Figma-App CLI page documented only the Figma upload side of the workflow. Following it end to end produced a build containing Figma frames with nothing to compare them against, because the steps that capture and upload the app screenshots were never described. Adds the missing half and fixes several blocking and structural issues. Blocking fix: - Add LT_USERNAME and LT_ACCESS_KEY to the environment variable step. upload-figma-app signs its request with these and exits with "Missing LT_USERNAME in Environment Variables" when they are unset. Both sibling Figma pages already document them, this page did not. New content: - Step 6, upload the app to the real device cloud - Step 7, Appium capabilities including visual: true and smartUI.project, with a note that the app side is keyed by project name and not by PROJECT_TOKEN - Step 8, the screenshot hook shown in context rather than only the .png naming rule in isolation - A "How the comparison is paired" section covering the three conditions that must line up: same project, Figma build marked as baseline, and exact screenshot name match Reference gaps: - Add a Configuration Options table for depth, screenshot_names length and uniqueness rules, orientation, smartIgnore, exact device name matching, and the schema's rejection of unknown keys - Document the --fetch-results flag - Add Windows CMD and PowerShell env var tabs to match sibling pages Structural fixes: - Replace the two duplicate "Device Names" tabs, which held identical screenshot naming content and said nothing about device names - Regroup Troubleshooting by symptom, removing the orphaned Symptoms and Solutions fragments and the duplicate Project Type tabs - Move the support links footer out of the last tab so it always renders - Add the breadcrumb JSON-LD used by sibling SmartUI pages - Update the Figma URL example to the current /design/ form - Add trailing slashes to internal links Removes the "ensure both Figma and SDK uploads use the same --buildName" guidance. The app side has no such flag, it uses the smartUI.build capability, and the figma-web page's working sample deliberately uses different build names on each half. Replaced with the pairing rules that are verifiable in the CLI source. Flagged for reviewer confirmation against the backend comparison logic. Verified with a production build: no broken links, and the page renders clean in local preview. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_0183Ta7Aq3y1eqoxfi4mBBnC --- docs/smartui-cli-figma-app.md | 454 +++++++++++++++++++++++----------- 1 file changed, 312 insertions(+), 142 deletions(-) diff --git a/docs/smartui-cli-figma-app.md b/docs/smartui-cli-figma-app.md index 17158309a..0e4fbdf6b 100644 --- a/docs/smartui-cli-figma-app.md +++ b/docs/smartui-cli-figma-app.md @@ -1,4 +1,4 @@ ---- +--- id: smartui-cli-figma-app title: Getting Started with TestMu AI's SmartUI Figma-App CLI sidebar_label: Figma-App CLI @@ -24,31 +24,72 @@ import BrandName, { BRAND_URL } from '@site/src/component/BrandName'; --- + + SmartUI Figma-App CLI lets you compare **mobile app screenshots captured on real devices** with your **Figma design frames** to detect visual mismatches and ensure accurate implementation of mobile UI. +The workflow has two halves. First you upload your Figma frames to SmartUI as the baseline using the CLI. Then you run your Appium test suite on the real device cloud so your app screenshots land in the same project and get compared against those frames. This guide covers both halves end to end. + --- ## Prerequisites - Node.js and npm installed - SmartUI account with App Automation plan -- Real device screenshots captured via Appium, SDK, or SmartUI platform +- Your Username and Access Key from the [Account Settings](https://accounts.lambdatest.com/security/username-accesskey) page - Figma Personal Access Token ([how to get one](https://help.figma.com/hc/en-us/articles/8085703771159-Manage-personal-access-tokens)) +- An Appium test suite for your app +- Your app uploaded to the real device cloud so you have an `app_url`. See [Upload your app](/support/docs/upload-apps-on-real-device-cloud/) --- -## Understanding Figma Tokens +## Understanding the Tokens and Credentials -| Token | Where It’s Used | Description | +The Figma-App workflow needs four values. Three of them are environment variables and one goes inside your config file. + +| Token | Where It Is Used | Description | |----------------------|------------------|-----------------------------------------------------------------------------| -| `FIGMA_TOKEN` | Env Variable | Your Figma **Personal Access Token** to authenticate with the Figma API | -| `figma_file_token` | `designs.json` | Figma **file ID**, extracted from the Figma file URL | -| `figma_ids` | `designs.json` | List of **frame or node IDs** you want to compare visually | +| `PROJECT_TOKEN` | Env Variable | Your SmartUI project token, copied from the project on the SmartUI dashboard | +| `FIGMA_TOKEN` | Env Variable | Your Figma **Personal Access Token** to authenticate with the Figma API | +| `LT_USERNAME` | Env Variable | Your username, used to authenticate the upload | +| `LT_ACCESS_KEY` | Env Variable | Your access key, used to authenticate the upload | +| `figma_file_token` | `designs.json` | Figma **file ID**, extracted from the Figma file URL | +| `figma_ids` | `designs.json` | List of **frame or node IDs** you want to compare visually | + +:::warning + +`LT_USERNAME` and `LT_ACCESS_KEY` are mandatory. The `upload-figma-app` command signs its request with them, and it exits with `Missing LT_USERNAME in Environment Variables` or `Missing LT_ACCESS_KEY in Environment Variables` if either one is unset. + +::: > Example Figma URL: -> `https://www.figma.com/file/abc12345/file-name?node-id=2417-58969` +> `https://www.figma.com/design/abc12345/file-name?node-id=2417-58969` > - `figma_file_token`: `abc12345` > - `figma_ids`: `2417-58969` +> +> Older Figma links use `/file/` instead of `/design/`. Both forms carry the file ID in the same position. --- @@ -65,13 +106,15 @@ SmartUI Figma-App CLI lets you compare **mobile app screenshots captured on real - Tags (optional) 5. Click **Submit** +Note down both the **project name** and the **project token**. You need the token for the CLI upload and the name for your Appium capabilities. + --- ### 2. Install SmartUI CLI ```bash -npm install @lambdatest/smartui-cli -```` +npm install -g @lambdatest/smartui-cli +``` --- @@ -83,6 +126,8 @@ Run the following to create your initial design file: npx smartui config:create-figma-app designs.json ``` +The file must have a `.json` extension, and the command refuses to overwrite a file that already exists. + #### Sample `designs.json` ```json title="designs.json" @@ -90,12 +135,12 @@ npx smartui config:create-figma-app designs.json "mobile": [ { "name": "Pixel 8", - "platform": ["Android 14"], + "platform": ["android 14"], "orientation": "portrait" } ], "figma": { - "depth": 2, + "depth": 1, "configs": [ { "figma_file_token": "abc12345", @@ -106,91 +151,173 @@ npx smartui config:create-figma-app designs.json } } ``` + +#### Configuration Options + +| Config Key | Description | Usage | +| ---------- | ----------- | ----- | +| `mobile[].name` | Device name. This must be an exact match for a supported device, and the same device you run your Appium test on. An unsupported value fails validation with `unsupported mobile device name`. The generated config seeds valid examples you can start from. | Mandatory | +| `mobile[].platform` | Operating system and version for the device, for example `["android 14"]` or `["ios 17"]`. | Optional | +| `mobile[].orientation` | Either `portrait` or `landscape`. No other value is accepted. | Optional | +| `figma.depth` | Positive integer representing how deep into the Figma document tree to traverse. Setting it to `2` returns pages and all top level objects on each page. Leaving it out returns all nodes. | Optional | +| `figma.configs[].figma_file_token` | File token for your Figma file. You can list multiple files in the same configuration. | Mandatory | +| `figma.configs[].figma_ids` | List of node or frame IDs you want to compare. Values must be unique. | Mandatory | +| `figma.configs[].screenshot_names` | Names given to the uploaded frames. If you supply this array it must have exactly the same number of entries as `figma_ids`, in the same order. Names must be unique across the whole file. | Optional | +| `smartIgnore` | Top level boolean that enables SmartUI's automatic ignore behavior for dynamic content. | Optional | + +:::note + +The config schema rejects unknown keys. If you add a property that is not listed above, the CLI logs `Additional property "" is not allowed` and the upload fails validation. + +::: + --- ### 4. Set Environment Variables +Set all four values before running the upload. + + + + +```bash +export PROJECT_TOKEN="123456#1234abcd-****-****-****-************" +export FIGMA_TOKEN="figd_****************************************" +export LT_USERNAME="YOUR_USERNAME" +export LT_ACCESS_KEY="YOUR_ACCESS_KEY" +``` + + + + ```bash -export PROJECT_TOKEN="your_smartui_project_token" -export FIGMA_TOKEN="your_figma_personal_token" +set PROJECT_TOKEN="123456#1234abcd-****-****-****-************" +set FIGMA_TOKEN="figd_****************************************" +set LT_USERNAME="YOUR_USERNAME" +set LT_ACCESS_KEY="YOUR_ACCESS_KEY" +``` + + + + +```powershell +$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************" +$env:FIGMA_TOKEN="figd_****************************************" +$env:LT_USERNAME="YOUR_USERNAME" +$env:LT_ACCESS_KEY="YOUR_ACCESS_KEY" ``` + + + --- -### 5. Run the Comparison +### 5. Upload your Figma designs as the baseline ```bash -npx smartui upload-figma-app designs.json +npx smartui upload-figma-app designs.json --buildName "figma-baseline" --markBaseline ``` +Uploaded frames are stored with a `.png` suffix. A frame named `homepage` in `screenshot_names` becomes `homepage.png` in the build. Remember this, because your app screenshot names have to match it in Step 8. + #### Optional Flags | Flag | Description | | ---------------- | ------------------------------------------------- | | `--markBaseline` | Mark this build as a new baseline for future runs | | `--buildName` | Assign a custom name to this comparison build | +| `--fetch-results [filename]` | Poll for build results and print them. Pass a file name such as `results.json` to also write them to disk, which is useful in CI | #### Example ```bash -npx smartui upload-figma-app designs.json --buildName "v1.0.0" --markBaseline +npx smartui upload-figma-app designs.json --buildName "v1.0.0" --markBaseline --fetch-results results.json ``` --- -### View SmartUI Results +### 6. Upload your app -You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. +Your Appium test needs an app that lives on the real device cloud. Upload your `.apk` or `.ipa` and note the `app_url` that is returned. -cmd +```bash +curl -u "$LT_USERNAME:$LT_ACCESS_KEY" \ +-X POST "https://manual-api.lambdatest.com/app/upload/realDevice" \ +-F "appFile=@/path/to/your/app.apk" \ +-F "name=YourAppName" +``` -## Best Practices +The response contains an `app_id` that you use as `lt://APP_ID`. For other upload options see [Upload your app](/support/docs/upload-apps-on-real-device-cloud/). - - +--- -**Build Names** +### 7. Configure your Appium capabilities + +This is the half that produces the app screenshots. The device you set here must match the device in `designs.json`, otherwise the frames and the screenshots are captured at different viewports and every comparison reports a mismatch. + +```javascript title="NodeJS example" +let capabilities = { + deviceName: "Pixel 8", // must match mobile[].name in designs.json + platformName: "android", + platformVersion: "14", // must match mobile[].platform + isRealMobile: true, // Mandatory + app: "lt://APP_ID", // Mandatory + //highlight-next-line + visual: true, // Mandatory + name: "Figma app comparison", + build: "Real Device App Build", + //highlight-start + "smartUI.project": "", // Mandatory, the project NAME not the project token + "smartUI.build": "", // Optional + "smartUI.baseline": false, // Leave false, your Figma build is the baseline + //highlight-end +}; + +let gridUrl = + "https://" + + "" + + ":" + + "" + + `@mobile-hub.lambdatest.com/wd/hub`; + +let driver = await new webdriver.Builder() + .usingServer(gridUrl) + .withCapabilities(capabilities) + .build(); +``` -```bash - npx smartui upload-figma-app designs.json --buildName "v1.0.0" - ``` +:::warning - - +The app side is identified by `smartUI.project`, which takes the project **name**. The `PROJECT_TOKEN` you exported in Step 4 authenticates the CLI upload only. It is not used by the Appium capabilities. -**Screenshot Names** +::: -- Good: `homepage-screen`, `login-form`, `dashboard-tab` - - Avoid: `test1`, `screenshot`, `design-1` - - Ensure `screenshot_names` in your config match the order of `figma_ids` +:::warning - - +`visual: true` is mandatory. Without it no screenshots are sent to SmartUI and the build is reported with an `Error` status. -**Device Names** +::: -**Screenshot Naming for SDK Comparisons** +--- -**Critical**: When comparing Figma designs with app screenshots captured via SDKs, add `.png` extension to your SDK screenshot names. +### 8. Capture screenshots with matching names -Figma-uploaded screenshots automatically have `.png` appended (e.g., `homepage.png`), so your SDK screenshots must match: +Add the screenshot hook after the point in your script where the screen you care about is rendered. + +**Critical**: Figma frames are stored with `.png` appended, so your app screenshot names must include the extension to line up with them. -**In your Appium/SDK code:** ```javascript -// ❌ Wrong - will not match Figma screenshot -driver.execute("smartui.takeScreenshot", {name: "homepage"}); +// ❌ Wrong, will not match the Figma frame +await driver.execute("smartui.takeScreenshot", {name: "homepage"}); -// ✅ Correct - matches Figma screenshot name -driver.execute("smartui.takeScreenshot", {name: "homepage.png"}); +// ✅ Correct, matches the Figma frame homepage.png +await driver.execute("smartui.takeScreenshot", {name: "homepage.png"}); ``` -**Example for different frameworks:** - - + ```javascript -// JavaScript await driver.execute("smartui.takeScreenshot", {name: "homepage.png"}); ``` @@ -198,167 +325,210 @@ await driver.execute("smartui.takeScreenshot", {name: "homepage.png"}); ```java -// Java driver.execute("smartui.takeScreenshot", Map.of("name", "homepage.png")); ``` - + -This ensures that Figma screenshots (e.g., `homepage.png`) match app screenshots (e.g., `homepage.png`) in the same build. +```python +driver.execute_script("smartui.takeScreenshot", {"name": "homepage.png"}) +``` - - -**Device Names** + -**Screenshot Naming for SDK Comparisons** +:::note -**Critical**: When comparing Figma designs with app screenshots captured via SDKs, add `.png` extension to your SDK screenshot names. +Appium with SmartUI supports viewport based screenshot comparisons only. Crop your Figma frames to a single viewport so they line up with what the device captures. -Figma-uploaded screenshots automatically have `.png` appended (e.g., `homepage.png`), so your SDK screenshots must match: +::: -**In your Appium/SDK code:** -```javascript -// ❌ Wrong - will not match Figma screenshot -driver.execute("smartui.takeScreenshot", {name: "homepage"}); +Run your test suite as you normally would. -// ✅ Correct - matches Figma screenshot name -driver.execute("smartui.takeScreenshot", {name: "homepage.png"}); +```bash +npm i && node your_test_script.js ``` -**Example for different frameworks:** +--- - - +### 9. View SmartUI Results -```javascript -// JavaScript -await driver.execute("smartui.takeScreenshot", {name: "homepage.png"}); +You can see the SmartUI dashboard to view the results. This will help you identify the Mismatches from the existing `Baseline` build and do the required visual testing. + +cmd + +--- + +## How the comparison is paired + +Three things have to line up for a Figma frame and an app screenshot to be compared: + +1. **Same project.** The CLI upload targets it by `PROJECT_TOKEN` and the Appium run targets it by `smartUI.project`. Both must resolve to the same Real Devices project. +2. **Figma build is the baseline.** Upload the Figma designs with `--markBaseline`, or approve that build on the dashboard. Your app run then becomes the comparison build. +3. **Screenshot names match exactly.** A frame stored as `homepage.png` is only compared against an app screenshot named `homepage.png`. + +The device in `designs.json` and the device in your capabilities should also match, so both sides are captured at the same viewport. + +--- + +## Best Practices + + + + +**Build Names** + +Give each run a build name you can trace back to a release or a commit. + +```bash +npx smartui upload-figma-app designs.json --buildName "v1.0.0" ``` - + -```java -// Java -driver.execute("smartui.takeScreenshot", Map.of("name", "homepage.png")); -``` +**Screenshot Names** + +- Good: `homepage-screen`, `login-form`, `dashboard-tab` +- Avoid: `test1`, `screenshot`, `design-1` +- Ensure `screenshot_names` in your config match the order of `figma_ids`, and that the two arrays are the same length +- Keep names unique across the whole config file. Duplicates fail validation with `Found duplicate screenshot names in figma config` +- Remember the `.png` suffix on the app side - + + +**Device Names** -This ensures that Figma screenshots (e.g., `homepage.png`) match app screenshots (e.g., `homepage.png`) in the same build. +- Use the exact device name string. `Pixel 8` is valid, `pixel8` and `Pixel-8` are not +- Use the same device in `designs.json` and in your Appium capabilities. A Figma frame captured for one viewport will not line up with a screenshot taken on a device of a different size +- Keep `orientation` consistent between your Figma frames and the device +- Start from the file produced by `npx smartui config:create-figma-app`, which seeds valid device names + + + + +**CI Runs** + +- Set all four environment variables as secrets in your pipeline +- Use `--fetch-results results.json` so the pipeline can read the outcome instead of relying on console output +- Refresh the Figma baseline as a separate job, not on every commit, so design changes are a deliberate step ## Troubleshooting - - +### Uploads fail with an authentication error + + + -Verify Figma Token +Check that all four values are set in the shell you are running from. ```bash - echo $FIGMA_TOKEN - ``` - Ensure the token is valid and has not expired. Generate a new token from [Figma Settings](https://www.figma.com/settings). +echo $PROJECT_TOKEN +echo $FIGMA_TOKEN +echo $LT_USERNAME +echo $LT_ACCESS_KEY +``` - - +A missing `LT_USERNAME` or `LT_ACCESS_KEY` is the most common cause. The CLI reports the missing variable by name. -Check File Token + + -- Verify the `figma_file_token` in your `designs.json` matches the file ID from the Figma URL - - Ensure you have access to the Figma file +Ensure the Figma token is valid and has not expired. Generate a new token from [Figma Settings](https://www.figma.com/settings). - - -Validate Node IDs + -- Confirm `figma_ids` in your configuration are valid - - Check that the nodes exist in the Figma file -**Symptoms**: -- Figma screenshots don't match app screenshots -- Comparison shows mismatches even when designs are identical -**Solutions**: +- Ensure the project is created as **Real Devices** type, not CLI +- Verify the project exists on the SmartUI dashboard +- Verify `PROJECT_TOKEN` belongs to that same project - + -Check Screenshot Names +### The config file is rejected -- Ensure SDK screenshots include `.png` extension (e.g., `homepage.png`) - - Verify screenshot names match exactly between Figma config and SDK code - - Ensure `screenshot_names` array matches the order of `figma_ids` + + - - +- Verify the `figma_file_token` in your `designs.json` matches the file ID from the Figma URL +- Ensure the account behind your Figma token has access to that file -Verify Device Sizes + + -- Ensure device dimensions match Figma frame sizes - - Check that device names in config match actual device capabilities - - Verify orientation (portrait/landscape) matches between Figma and device +- Confirm the `figma_ids` in your configuration are valid and unique +- Check that the nodes still exist in the Figma file +- Increase `figma.depth` if nested frames are not being picked up - + + +If you supply `screenshot_names`, it must have the same number of entries as `figma_ids`. A mismatch fails with `Mismatch in Figma Ids and Screenshot Names in figma config`. -Check Build Names + + -- Ensure both Figma and SDK uploads use the same `--buildName` - - Verify builds are in the same project +The schema rejects keys it does not recognize. Check the Configuration Options table above and remove anything that is not listed. - + -Project Type +### Figma frames and app screenshots are not being compared -- Ensure project is created as **Real Devices** type (not CLI) - - Verify project exists in SmartUI dashboard -**Symptoms**: -- "Invalid project token" error -- Uploads fail with authentication errors -**Solutions**: + + - - +- Ensure app screenshots include the `.png` extension, for example `homepage.png` +- Verify names match exactly between the Figma config and your Appium code, including case +- Ensure `screenshot_names` matches the order of `figma_ids` -Verify Project Token + + -```bash - echo $PROJECT_TOKEN - ``` - Ensure the token is set correctly and matches your SmartUI project. +- Verify `smartUI.project` in your capabilities is the project **name** for the same project whose token you used for the CLI upload +- Confirm the Figma build was marked as the baseline, either with `--markBaseline` or by approving it on the dashboard - + -Check Project Type +- Ensure the device in `designs.json` matches the device in your Appium capabilities +- Verify orientation matches on both sides +- Check that your Figma frame dimensions match the device viewport -- Ensure project is created as **Real Devices** type - - Verify project exists in SmartUI dashboard -If you encounter issues not covered here: -- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) for detailed solutions -- Check [Figma CLI Documentation](/support/docs/smartui-cli-figma) for basic Figma workflows -- Check [Figma-Web CLI Documentation](/support/docs/smartui-cli-figma-web) for web comparison workflows -- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources -- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) + + + +- Confirm `visual: true` is set in your capabilities. Without it the build is reported as `Error` and no screenshots reach SmartUI +- Confirm `isRealMobile: true` is set +- Confirm the screenshot hook runs after the screen has finished rendering +If you encounter issues not covered here: + +- Review the [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide/) for detailed solutions +- Check [Figma CLI Documentation](/support/docs/smartui-cli-figma/) for basic Figma workflows +- Check [Figma-Web CLI Documentation](/support/docs/smartui-cli-figma-web/) for web comparison workflows +- Visit [TestMu AI Support](https://www.testmuai.com/support) for additional resources +- Contact support at support@testmuai.com or use [24/7 Chat Support](https://www.testmuai.com/support) ## Additional Resources -- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide) -- [Figma CLI Documentation](/support/docs/smartui-cli-figma) -- [Figma-Web CLI Documentation](/support/docs/smartui-cli-figma-web) -- [Appium Hooks Documentation](/support/docs/smartui-appium-hooks) -- [Baseline Management](/support/docs/smartui-baseline-management) -- [Running Your First Project](/support/docs/smartui-running-your-first-project) +- [Comprehensive Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide/) +- [Figma CLI Documentation](/support/docs/smartui-cli-figma/) +- [Figma-Web CLI Documentation](/support/docs/smartui-cli-figma-web/) +- [Appium Hooks Documentation](/support/docs/smartui-appium-hooks/) +- [SmartUI Appium SDK](/support/docs/smartui-appium-sdk/) +- [Upload your app to the real device cloud](/support/docs/upload-apps-on-real-device-cloud/) +- [Baseline Management](/support/docs/smartui-baseline-management/) +- [Running Your First Project](/support/docs/smartui-running-your-first-project/) - [SmartUI API Documentation](https://www.testmuai.com/support/api-doc/) - [How to generate a Figma token](https://help.figma.com/hc/en-us/articles/8085703771159-Manage-personal-access-tokens) - [SmartUI CLI Docs](/support/docs/smartui-cli/) From 6a1537e8a3d9a5a379f72979fa9a8e6602d81713 Mon Sep 17 00:00:00 2001 From: Chaitanya Sharma Date: Thu, 23 Jul 2026 13:47:31 +0530 Subject: [PATCH 2/3] docs(smartui): correct Figma-App page against live verification Ran the full workflow live against a new Real Devices project, a new Figma file, and a real Pixel 8 / Android 14 device. Four builds were produced and compared. Several statements on the page, including some this branch had introduced, did not survive the run. Fixes a broken code sample: - `driver.execute("smartui.takeScreenshot", {name: "..."})` throws `Error response status: 1`. The config-object key is `screenshotName`, not `name`. This snippet was wrong on the published page and this branch had reproduced it unchanged. Documented both forms that were proven to work on device: the `smartui.takeScreenshot=.png` string form and the `{screenshotName: ".png"}` config form. Corrects the credentials section: - LT_USERNAME and LT_ACCESS_KEY are NOT required by upload-figma-app. Running with only PROJECT_TOKEN and FIGMA_TOKEN set, in a clean directory with no cached state, uploads successfully. PROJECT_TOKEN authenticates that endpoint. The earlier "mandatory, exits with Missing LT_USERNAME" warning on this branch was wrong. - The two variables are still needed, but for the Appium grid URL on the app side, so the table now says where each value is actually consumed. - Replaced the warning with the real trap: an empty-string value fails while an unset value passes, because the guard compares against "". Corrects two long-standing errors in Step 1: - Approver(s) is mandatory and pre-filled, not optional. - The button is labelled Continue, not Submit. Softens the unknown-key claim: - An unrecognised property logs a warning and the upload continues. It does not fail validation. Reworded in the options note and in the troubleshooting tab. Also notes that the app upload response returns `app_url` directly in lt:// form rather than an app_id the reader has to assemble. The pairing rules this branch introduced are now confirmed empirically. The Figma build and the Appium build carried different build names and still compared, Baseline being the Figma frame and Captured the device screenshot, so removing the old same-buildName guidance was correct. Verified with a production build: no broken links. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_0183Ta7Aq3y1eqoxfi4mBBnC --- docs/smartui-cli-figma-app.md | 52 +++++++++++++++++++++++++---------- 1 file changed, 37 insertions(+), 15 deletions(-) diff --git a/docs/smartui-cli-figma-app.md b/docs/smartui-cli-figma-app.md index 0e4fbdf6b..d5d142779 100644 --- a/docs/smartui-cli-figma-app.md +++ b/docs/smartui-cli-figma-app.md @@ -67,20 +67,23 @@ The workflow has two halves. First you upload your Figma frames to SmartUI as th ## Understanding the Tokens and Credentials -The Figma-App workflow needs four values. Three of them are environment variables and one goes inside your config file. +The Figma-App workflow needs four values across its two halves. Two are used by the CLI upload, two by the Appium run. | Token | Where It Is Used | Description | |----------------------|------------------|-----------------------------------------------------------------------------| -| `PROJECT_TOKEN` | Env Variable | Your SmartUI project token, copied from the project on the SmartUI dashboard | +| `PROJECT_TOKEN` | Env Variable | Your SmartUI project token. Authenticates the **CLI upload** | | `FIGMA_TOKEN` | Env Variable | Your Figma **Personal Access Token** to authenticate with the Figma API | -| `LT_USERNAME` | Env Variable | Your username, used to authenticate the upload | -| `LT_ACCESS_KEY` | Env Variable | Your access key, used to authenticate the upload | +| `LT_USERNAME` | Env Variable | Your username. Used in the **Appium grid URL** | +| `LT_ACCESS_KEY` | Env Variable | Your access key. Used in the **Appium grid URL** | | `figma_file_token` | `designs.json` | Figma **file ID**, extracted from the Figma file URL | | `figma_ids` | `designs.json` | List of **frame or node IDs** you want to compare visually | :::warning -`LT_USERNAME` and `LT_ACCESS_KEY` are mandatory. The `upload-figma-app` command signs its request with them, and it exits with `Missing LT_USERNAME in Environment Variables` or `Missing LT_ACCESS_KEY in Environment Variables` if either one is unset. +Do not set any of these to an empty string. The CLI treats an empty value as an error and exits with +`Missing FIGMA_TOKEN in Environment Variables` (or the matching `LT_USERNAME` / `LT_ACCESS_KEY` message), +while leaving the same variable unset is accepted. If you hit that message, check for a stray +`export FIGMA_TOKEN=` in your shell. ::: @@ -101,10 +104,10 @@ The Figma-App workflow needs four values. Three of them are environment variable 2. Click **New Project** 3. Select **Real Devices** as the platform 4. Enter: - - Project Name - - Approvers (optional) + - Project Name (required) + - Approver(s) (required, pre-filled with your own user) - Tags (optional) -5. Click **Submit** +5. Click **Continue** Note down both the **project name** and the **project token**. You need the token for the CLI upload and the name for your Appium capabilities. @@ -167,7 +170,7 @@ The file must have a `.json` extension, and the command refuses to overwrite a f :::note -The config schema rejects unknown keys. If you add a property that is not listed above, the CLI logs `Additional property "" is not allowed` and the upload fails validation. +If you add a property that is not listed above, the CLI logs `Additional property "" is not allowed` as a warning and then continues with the upload. The extra key is ignored rather than applied, so check for this warning if a setting you added appears to have no effect. ::: @@ -247,7 +250,7 @@ curl -u "$LT_USERNAME:$LT_ACCESS_KEY" \ -F "name=YourAppName" ``` -The response contains an `app_id` that you use as `lt://APP_ID`. For other upload options see [Upload your app](/support/docs/upload-apps-on-real-device-cloud/). +The response contains an `app_url` field, already in `lt://APP...` form, which is the value you pass as the `app` capability in the next step. For other upload options see [Upload your app](/support/docs/upload-apps-on-real-device-cloud/). --- @@ -308,31 +311,50 @@ Add the screenshot hook after the point in your script where the screen you care ```javascript // ❌ Wrong, will not match the Figma frame -await driver.execute("smartui.takeScreenshot", {name: "homepage"}); +await driver.execute("smartui.takeScreenshot=homepage"); // ✅ Correct, matches the Figma frame homepage.png +await driver.execute("smartui.takeScreenshot=homepage.png"); +``` + +:::warning + +When you pass a config object, the screenshot name key is `screenshotName`. Passing `name` throws +`Error response status: 1` and the test fails. + +```javascript +// ❌ Wrong, throws await driver.execute("smartui.takeScreenshot", {name: "homepage.png"}); + +// ✅ Correct +await driver.execute("smartui.takeScreenshot", {screenshotName: "homepage.png"}); ``` +::: + ```javascript -await driver.execute("smartui.takeScreenshot", {name: "homepage.png"}); +// simple form +await driver.execute("smartui.takeScreenshot=homepage.png"); + +// config form +await driver.execute("smartui.takeScreenshot", {screenshotName: "homepage.png"}); ``` ```java -driver.execute("smartui.takeScreenshot", Map.of("name", "homepage.png")); +((JavaScriptExecutor) driver).executeScript("smartui.takeScreenshot=homepage.png"); ``` ```python -driver.execute_script("smartui.takeScreenshot", {"name": "homepage.png"}) +driver.execute_script("smartui.takeScreenshot=homepage.png") ``` @@ -474,7 +496,7 @@ If you supply `screenshot_names`, it must have the same number of entries as `fi -The schema rejects keys it does not recognize. Check the Configuration Options table above and remove anything that is not listed. +An unrecognised key does not stop the upload, it only logs `Additional property "" is not allowed` and is then ignored. If a setting seems to have no effect, look for that warning and check it against the Configuration Options table above. From 1762e5f620426f95b60d21c1de67fe4cf88a1d1b Mon Sep 17 00:00:00 2001 From: Chaitanya Sharma Date: Thu, 23 Jul 2026 17:37:01 +0530 Subject: [PATCH 3/3] docs(smartui): fix broken Java snippet, add Figma rate limits, drop unverified claims Second verification pass. Every code sample on the page has now been executed, and any statement that could not be executed was removed or narrowed rather than left standing. Fixes a second broken code sample: - The Java tab published `((JavaScriptExecutor) driver)`. That type does not exist. Selenium's interface is `JavascriptExecutor`, with a lower case s in script. Compiling the published spelling against selenium-api fails with `cannot find symbol`; the corrected spelling compiles clean. The corrected snippet was then run against a real Pixel 8 and the resulting screenshot was confirmed in the build. Note the same wrong spelling appears on the Appium hooks page. Verified and kept: - Python tab run on a real Pixel 8, screenshot confirmed in the build. - Duplicate `figma_ids` rejected with `figma_ids must contain unique values`. - Several `figma_file_token` entries in one config pass validation. Adds a troubleshooting group for Figma API limits: - Uploads fail during Processing App Figma when the Figma token sits in the low rate-limit bucket, which is where Viewer and Collab seats on free Starter workspaces land. Records the symptom, that a direct REST call can still succeed while the upload fails, and the practical ways out. - Adds the `Invalid token` case and the `file_content:read` scope the upload needs. Removed or narrowed because they could not be verified: - Deleted the claim that Appium with SmartUI supports viewport based comparisons only. Inherited from another page, never exercised. - Narrowed the device mismatch guidance from a causal claim about every comparison failing to a plain instruction to use the same device. - Narrowed `figma.depth` to what the generated config shows. The documented traversal semantics were never exercised. - Narrowed `smartIgnore` to what is observable, that the schema accepts it and it is forwarded with the upload. - Narrowed `--fetch-results` to what `--help` states. The flag exists, but writing results to disk remains unproven because the Figma rate limit blocked every attempt. Verified with a production build: no broken links. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_0183Ta7Aq3y1eqoxfi4mBBnC --- docs/smartui-cli-figma-app.md | 49 ++++++++++++++++++++++++++--------- 1 file changed, 37 insertions(+), 12 deletions(-) diff --git a/docs/smartui-cli-figma-app.md b/docs/smartui-cli-figma-app.md index d5d142779..2c39d42e7 100644 --- a/docs/smartui-cli-figma-app.md +++ b/docs/smartui-cli-figma-app.md @@ -162,11 +162,11 @@ The file must have a `.json` extension, and the command refuses to overwrite a f | `mobile[].name` | Device name. This must be an exact match for a supported device, and the same device you run your Appium test on. An unsupported value fails validation with `unsupported mobile device name`. The generated config seeds valid examples you can start from. | Mandatory | | `mobile[].platform` | Operating system and version for the device, for example `["android 14"]` or `["ios 17"]`. | Optional | | `mobile[].orientation` | Either `portrait` or `landscape`. No other value is accepted. | Optional | -| `figma.depth` | Positive integer representing how deep into the Figma document tree to traverse. Setting it to `2` returns pages and all top level objects on each page. Leaving it out returns all nodes. | Optional | +| `figma.depth` | Positive integer controlling how deep into the Figma document tree the fetch traverses. The generated config uses `1`. | Optional | | `figma.configs[].figma_file_token` | File token for your Figma file. You can list multiple files in the same configuration. | Mandatory | | `figma.configs[].figma_ids` | List of node or frame IDs you want to compare. Values must be unique. | Mandatory | | `figma.configs[].screenshot_names` | Names given to the uploaded frames. If you supply this array it must have exactly the same number of entries as `figma_ids`, in the same order. Names must be unique across the whole file. | Optional | -| `smartIgnore` | Top level boolean that enables SmartUI's automatic ignore behavior for dynamic content. | Optional | +| `smartIgnore` | Top level boolean accepted by the config schema and forwarded with the upload. | Optional | :::note @@ -229,7 +229,7 @@ Uploaded frames are stored with a `.png` suffix. A frame named `homepage` in `sc | ---------------- | ------------------------------------------------- | | `--markBaseline` | Mark this build as a new baseline for future runs | | `--buildName` | Assign a custom name to this comparison build | -| `--fetch-results [filename]` | Poll for build results and print them. Pass a file name such as `results.json` to also write them to disk, which is useful in CI | +| `--fetch-results [filename]` | Poll for build results after the upload. Accepts an optional output file name, for example `results.json` | #### Example @@ -256,7 +256,7 @@ The response contains an `app_url` field, already in `lt://APP...` form, which i ### 7. Configure your Appium capabilities -This is the half that produces the app screenshots. The device you set here must match the device in `designs.json`, otherwise the frames and the screenshots are captured at different viewports and every comparison reports a mismatch. +This is the half that produces the app screenshots. Use the same device here as in `designs.json` so both sides are captured at the same viewport. ```javascript title="NodeJS example" let capabilities = { @@ -347,7 +347,8 @@ await driver.execute("smartui.takeScreenshot", {screenshotName: "homepage.png"}) ```java -((JavaScriptExecutor) driver).executeScript("smartui.takeScreenshot=homepage.png"); +// the Selenium interface is JavascriptExecutor, with a lower case s in script +((JavascriptExecutor) driver).executeScript("smartui.takeScreenshot=homepage.png"); ``` @@ -360,12 +361,6 @@ driver.execute_script("smartui.takeScreenshot=homepage.png") -:::note - -Appium with SmartUI supports viewport based screenshot comparisons only. Crop your Figma frames to a single viewport so they line up with what the device captures. - -::: - Run your test suite as you normally would. ```bash @@ -434,7 +429,7 @@ npx smartui upload-figma-app designs.json --buildName "v1.0.0" **CI Runs** - Set all four environment variables as secrets in your pipeline -- Use `--fetch-results results.json` so the pipeline can read the outcome instead of relying on console output +- Use `--fetch-results` so the pipeline polls for the build outcome after the upload returns - Refresh the Figma baseline as a separate job, not on every commit, so design changes are a deliberate step @@ -501,6 +496,36 @@ An unrecognised key does not stop the upload, it only logs `Additional property +### The upload fails while fetching from Figma + + + + +The upload authenticates and then fails during **Processing App Figma** with a message like: + +``` +Failed to retrieve figma files, Figma API rate limit reached for your token. +Your file is on the 'starter' plan tier, and your token's rate-limit bucket is 'low'. +``` + +Figma applies the lowest rate-limit bucket to tokens that act as a Viewer or Collab seat on a file, which includes free Starter workspaces. A handful of uploads in quick succession is enough to exhaust it. + +What helps: + +- Wait before retrying. Short backoff often is not enough, so leave a longer gap between attempts +- Reduce how many `figma_ids` you fetch per run, and avoid re-running the upload while iterating on unrelated config +- Use a token belonging to an Editor seat on a paid Figma tier, which is placed in a higher bucket + +A direct call to the Figma REST API can still succeed while the upload fails, because the upload makes several calls per run. + + + + +If the message is `Invalid token` rather than a rate limit, the token itself is being rejected. Regenerate it from [Figma Settings](https://www.figma.com/settings) and make sure it carries the `file_content:read` scope, which is what allows reading file contents and rendering images. + + + + ### Figma frames and app screenshots are not being compared