diff --git a/CLAUDE.md b/CLAUDE.md index b8c0c405..94f9957c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -12,11 +12,38 @@ WorkOS CLI for installing AuthKit integrations and managing WorkOS resources (or ## Non-TTY Behavior - **Output**: Auto-switches to JSON when piped or `--json` flag. `WORKOS_FORCE_TTY=1` overrides. -- **Auth**: Exits code 4 instead of opening browser. Requires prior `workos auth login` or `WORKOS_API_KEY` env var. +- **Auth**: Exits code 4 instead of opening browser. Resource commands (organization, user, role, permission, membership, invitation, session, event, feature-flag, org-domain, portal, webhook, config) use the dashboard session from a prior `workos auth login`; expired access tokens refresh silently while the stored refresh token is valid, so only a truly dead session exits 4. `WORKOS_API_KEY` applies only to `workos api` and the still-REST commands (`connection`, `directory`, `audit-log`, `api-key`, `vault`, plus the workflow/debug commands `seed`, `setup-org`, `onboard-user`, `verify-login`, `debug-sso`, `debug-sync`, `migrations`). - **Errors**: Structured JSON to stderr: `{ "error": { "code": "...", "message": "..." } }` - **Exit codes**: 0=success, 1=error, 2=cancelled, 4=auth required (follows `gh` CLI convention) - **Headless flags**: `--no-branch`, `--no-commit`, `--create-pr`, `--no-git-check`. CI mode (`WORKOS_MODE=ci`) auto-continues past a dirty tree without `--no-git-check`; agent mode requires the flag. +## JSON Output Conventions + +`--json` output is a public API: users script against it with jq and in CI. The +backend's vocabulary is an implementation detail and must never leak through +untranslated, or the next backend migration becomes another user-visible break. +Route every enum and metadata field through `src/utils/output-conventions.ts` +rather than hand-normalizing per command. + +- **Keys are camelCase.** +- **Enum values are lowercase.** Backends emit assorted casings (`Verified`, + `PENDING`, `Active`); the CLI emits one convention. Use `enumOut()`. +- **Enum input is case-insensitive.** Whatever the CLI prints for a field it + accepts for that field ("forgiving in, canonical out"). Use `enumIn()`. +- **`state` is the lifecycle-state key** on every resource, not `status`. +- **`metadata` is an object map**, not GraphQL's array of pairs, so + `.metadata.foo` resolves in jq. Use `metadataToMap()`. +- **Internal/backend-only fields are dropped** from curated shapes. + +When a spec mocks a backend response, feed it the backend's real casing +(`'Verified'`) and assert the lowercase output. A mock that feeds already-correct +values never exercises the normalization, which is exactly how a casing bug +shipped once already. + +`scripts/parity-smoke.ts` compares this branch against `../main` and fails on any +unexpected field divergence. Its `ACCEPTED` map lists deliberate curations only; +a casing-only difference appearing there is a bug, not an accepted divergence. + ## Tech Constraints - **Bun** only; the shipped CLI is a Bun-compiled standalone binary diff --git a/MIGRATION.md b/MIGRATION.md new file mode 100644 index 00000000..d696b931 --- /dev/null +++ b/MIGRATION.md @@ -0,0 +1,577 @@ +# Migration guide: GraphQL resource commands (`--json` output) + +## What changed + +About a dozen `workos` resource commands moved from the REST backend to the +dashboard GraphQL account plane. This ships as one breaking release. + +Command names, flags, and positionals are unchanged. `workos org list`, +`workos user get `, `workos webhook create --url ...` all take the same +arguments they always did. The only thing that changed is the **`--json` output +shape**. If you script against `--json` with `jq` or in CI, read this guide. If +you only read the human-readable tables, nothing changes for you. + +Why the change: the backend's vocabulary (enum casing, metadata encoding, field +names) was leaking into `--json`, which made it fragile. The CLI now applies one +consistent contract (`src/utils/output-conventions.ts`) so the next backend +migration is not another user-visible break. Two rules of that contract: + +- Enum values are always lowercase (snake_case if multiword). +- Enum input is case-insensitive, so any value the CLI prints is accepted back. + +> Source note: the old REST shapes below are derived from the release's +> `feat!` commit bodies, the `scripts/parity-smoke.ts` `ACCEPTED` map, and the +> README correction commit. The sibling `../main` checkout was not reachable +> from this worktree, so a few old REST values could not be independently +> confirmed; those are flagged inline and listed under +> [Unconfirmed facts](#unconfirmed-facts). + +## Breaking changes at a glance + +| Change | Old | New | Affected commands | +| ------------------------- | -------------------------------------------------------------------- | --------------------------------------------------- | --------------------------------------------------------------- | +| List envelope | `{ "data": [...], "list_metadata": {...} }` | `{ "": [...], "pagination": {...} }` | all list views | +| `metadata` encoding | object map `{"team":"blue"}` (array of pairs during prerelease) | object map `{"team":"blue"}` | organization, user | +| Lifecycle key | `status` | `state` | user identities, membership, session | +| Enum casing | mixed (`Pending`, `Active`, `Verified`, `SOME`) | lowercase (`pending`, `active`, `verified`, `some`) | invitation, webhook, org-domain, membership, role, feature-flag | +| `role.type` | `EnvironmentRole` / `OrganizationRole` | `environment` / `organization` | role | +| `webhook.state` | `enabled` | `active` | webhook | +| `invitation.organization` | flat `organizationId` string | nested `{ "id", "name" }` object | invitation | +| `feature-flag.enabled` | flat flag value | derived from the active environment's state | feature-flag | +| Internal fields | present (`stripeCustomerId`, `resourceTypeId`, request context, ...) | dropped | all | + +## metadata (silent failure - read this first) + +`metadata` is an **object map** in the final release: + +```json +{ "metadata": { "team": "blue" } } +``` + +`.metadata.team` resolves in `jq`. Affects `organization` and `user`. + +Why this section is called out: during the GraphQL prerelease, `metadata` was +briefly emitted as GraphQL's array-of-pairs transport form: + +```json +{ "metadata": [{ "key": "team", "value": "blue" }] } +``` + +That form **fails silently**. `jq '.metadata.team'` returns empty rather than +erroring, so a broken pipeline looks like it is returning "no value" instead of +crashing. The final release folds it back to a map. + +Action: + +- If you script against a stable REST release, `.metadata.team` worked before + and still works. No change. +- If you already migrated a script to the array form during the prerelease + (`.metadata[] | select(.key=="team") | .value`), **revert it** to + `.metadata.team`. + +```bash +# WRONG (prerelease array form) - remove this +jq -r '.metadata[] | select(.key=="team") | .value' + +# RIGHT +jq -r '.metadata.team' +``` + +## Enum casing + +All enum values in `--json` are now lowercase, snake_case if multiword. Every +value the migrated commands emit today is single word (`active`, `verified`, +`pending`, `environment`, `organization`, `standard`, `dns`, `manual`, `some`), +so the snake_case rule is not yet visible in output, but a future multiword enum +will render as e.g. `user_registration`, not `userregistration`. + +Enum **input is case-insensitive**, so values round-trip: whatever the CLI +prints for a field, it accepts back for that field. + +```bash +# Both accepted; the CLI prints `verified`. +workos organization create foo.com:verified +workos organization create foo.com:Verified +``` + +Old -> new values (from the `feat!` commit body): + +| Field | Old | New | +| -------------------------------------- | --------------------- | --------------------- | +| `invitation.state` | `Pending` | `pending` | +| `webhook.state` | `Active`* | `active` | +| `org-domain.state` | `Verified` | `verified` | +| `org-domain.verificationStrategy` | `Dns` / `Manual` | `dns` / `manual` | +| `membership.state` / `membership.type` | `Active` / `Standard` | `active` / `standard` | +| `role.type` | `Environment`* | `environment` | +| `feature-flag.accessType` | `SOME` | `some` | + +\* See [Deliberate differences from REST](#deliberate-differences-from-the-old-rest-output): +the REST plane used different vocabulary for `webhook.state` (`enabled`) and +`role.type` (`EnvironmentRole`). The casings above are the GraphQL backend +passthrough values that the release normalizes. + +Fix pattern (do not hardcode casing; downcase before comparing): + +```bash +# Fragile +jq 'select(.state == "Pending")' + +# Robust +jq 'select((.state | ascii_downcase) == "pending")' +``` + +## Lifecycle key rename: `status` -> `state` + +The lifecycle key is `state` on every resource, never `status`. This affects +three places that previously used `status`: + +| Command | Old | New | +| --------------- | ---------------------- | --------------------- | +| user identities | `.identities[].status` | `.identities[].state` | +| membership | `.status` | `.state` | +| session | `.status` | `.state` | + +```bash +# Old +jq -r '.identities[].status' +# New +jq -r '.identities[].state' +``` + +## Per-command reference + +Each section shows the new payload (from the command's `*.spec.ts`) and the +`jq` change. List views are wrapped in `{ "": [...], "pagination": {...} }`; +the examples show a single row unless the envelope is the point. + +### organization + +New: + +```json +{ + "id": "org_1", + "name": "FooCorp", + "createdAt": "2026-01-01T00:00:00.000Z", + "usersCount": 3, + "allowProfilesOutsideOrganization": false, + "externalId": null, + "domains": [{ "id": "dom_1", "domain": "foo.com", "state": "verified" }], + "metadata": { "team": "blue" } +} +``` + +List envelope changed from `{ data, list_metadata }` to `{ organizations, pagination }`. +Internal fields like `stripeCustomerId` are dropped. `domains[].state` is lowercase. + +```bash +# List row ids: old envelope -> new envelope +jq -r '.data[].id' # old +jq -r '.organizations[].id' # new + +# Filter verified domains (casing changed) +jq '.organizations[].domains[] | select(.state == "verified")' + +# Read a metadata key +jq -r '.organization.metadata.team' +``` + +### user + +New: + +```json +{ + "id": "user_1", + "email": "jane@example.com", + "firstName": "Janet", + "lastName": "Doe", + "metadata": { "team": "blue" }, + "identities": [ + { + "id": "ident_1", + "state": "active", + "organization": { "id": "org_1", "name": "FooCorp" }, + "roles": [{ "id": "role_1", "name": "member" }] + } + ] +} +``` + +`identities[].status` became `identities[].state` and is lowercase. `metadata` +is a map. Internal fields (`googleOauthProfile`, identity `customAttributes`) are +dropped. List envelope is `{ users, pagination }`. + +```bash +# Identity state: renamed status -> state, lowercased +jq -r '.user.identities[] | select(.state == "active") | .id' + +# Metadata key +jq -r '.user.metadata.team' +``` + +### role + +New: + +```json +{ + "id": "role_env", + "slug": "admin", + "name": "Admin", + "description": "Administrator", + "type": "environment", + "permissions": ["users:read"], + "createdAt": "2026-01-01T00:00:00.000Z", + "updatedAt": "2026-02-01T00:00:00.000Z" +} +``` + +`type` dropped the redundant `Role` suffix and is lowercase: +`EnvironmentRole` -> `environment`, `OrganizationRole` -> `organization`. +`permissions` is an array of permission slugs. + +```bash +# Old: matched the REST vocabulary +jq 'select(.type == "EnvironmentRole")' +# New +jq 'select(.type == "environment")' +``` + +### permission + +New: + +```json +{ + "id": "perm_1", + "slug": "users:read", + "name": "Read users", + "description": "Read user records", + "system": false, + "createdAt": "2026-01-01T00:00:00.000Z", + "updatedAt": "2026-02-01T00:00:00.000Z" +} +``` + +No enum or metadata changes. Internal fields (`environmentId`, +`isEnabledForApiKeys`) are dropped. List envelope is `{ permissions, pagination }`. + +```bash +jq -r '.permissions[] | select(.system == false) | .slug' +``` + +### membership + +New: + +```json +{ + "id": "om_1", + "userId": "user_1", + "organizationId": "org_1", + "state": "active", + "type": "standard", + "role": "member", + "roles": ["member"], + "directoryUserId": null, + "createdAt": "2026-01-01T00:00:00.000Z", + "updatedAt": "2026-01-01T00:00:00.000Z" +} +``` + +`status` -> `state`; `state` and `type` are lowercase (`Active`/`Standard` -> +`active`/`standard`). List envelope is `{ memberships, pagination }`. + +```bash +# Old +jq -r '.data[] | select(.status == "Active") | .id' +# New +jq -r '.memberships[] | select(.state == "active") | .id' +``` + +### invitation + +New: + +```json +{ + "id": "invite_1", + "email": "jane@example.com", + "state": "pending", + "createdAt": "2026-01-01T00:00:00.000Z", + "organization": { "id": "org_1", "name": "FooCorp" } +} +``` + +Two changes: `state` is lowercase (`Pending` -> `pending`), and the flat +`organizationId` string is now a nested `organization` object. + +```bash +# Organization id: flat string -> nested object +jq -r '.organizationId' # old +jq -r '.organization.id' # new + +# Filter pending (casing changed) +jq '.invitations[] | select(.state == "pending")' +``` + +### session + +New: + +```json +{ + "id": "session_1", + "state": "active", + "createdAt": "...", + "updatedAt": "...", + "expiresAt": "2026-03-01T00:00:00.000Z", + "endedAt": null, + "ipAddress": "...", + "userAgent": "Mozilla/5.0", + "provider": "Password", + "organization": { "id": "org_1", "name": "FooCorp" }, + "impersonator": null +} +``` + +`status` -> `state`; values are lowercase (`active`, `revoked`, `expired`). + +```bash +# Old +jq -r '.data[] | select(.status == "Active") | .id' +# New +jq -r '.sessions[] | select(.state == "active") | .id' +``` + +### event + +New: + +```json +{ + "id": "event_1", + "event": "dsync.user.created", + "data": { "directory_id": "dir_1" }, + "createdAt": "2026-01-01T00:00:00.000Z", + "updatedAt": "2026-01-01T00:00:00.000Z" +} +``` + +The event type is under `event`. Internal request `context` and `metadata` that +GraphQL carries are dropped. List envelope is `{ events, pagination }`. + +```bash +jq -r '.events[] | select(.event == "dsync.user.created") | .id' +``` + +**Sort order reversed. This one changes which rows you get, not just their shape.** + +| | old (REST) | new (GraphQL) | +| ------------------------------- | -------------------- | ------------------------- | +| `event list` order | oldest first | newest first | +| `event list --limit 10` returns | the 10 oldest events | the 10 most recent events | + +If you took the first element to mean "the latest event", it now means the +opposite of what it used to, and no error is raised: + +```bash +# Old: this was the OLDEST event. Taking [0] to mean "latest" was already a bug, +# it just happened to be a different bug. +workos event list --events user.created --limit 1 | jq -r '.data[0].id' + +# New: [0] is genuinely the most recent event. +workos event list --events user.created --limit 1 | jq -r '.events[0].id' +``` + +`--after` still requests the next page, but "next" now moves toward older +events because the feed starts with the newest. Treat cursors as opaque and do +not reuse a cursor saved by the REST version after upgrading. + +If you were paginating to the end of the REST feed just to reach recent events, +you can now drop that work and read from the first page. + +### feature-flag + +New (`get` detail): + +```json +{ + "id": "flag_1", + "slug": "beta", + "name": "...", + "description": "...", + "enabled": false, + "defaultEnabled": true, + "accessType": "some", + "organizationTargets": [{ "id": "org_1", "name": "FooCorp" }], + "userTargets": [{ "id": "user_1", "email": "a@example.com" }], + "tags": [] +} +``` + +`accessType` is lowercase (`SOME` -> `some`). `enabled` now reflects the +**active environment's** flag state, not a flat global flag. List rows carry the +list subset (through `enabled`); `get` adds targeting fields. + +```bash +# Old +jq 'select(.accessType == "SOME")' +# New +jq 'select(.accessType == "some")' + +jq -r '.flags[] | select(.enabled) | .slug' +``` + +### org-domain + +New: + +```json +{ + "id": "org_domain_1", + "domain": "example.com", + "state": "verified", + "organizationId": "org_1", + "subdomain": null, + "verificationStrategy": "dns", + "verificationContent": "workos-verify=abc123", + "domainCaptureEnabled": false +} +``` + +`state` (`Verified` -> `verified`) and `verificationStrategy` +(`Dns`/`Manual` -> `dns`/`manual`) are lowercase. + +```bash +# Old +jq 'select(.state == "Verified")' +# New +jq 'select(.state == "verified")' + +jq -r 'select(.verificationStrategy == "dns") | .verificationContent' +``` + +### portal + +New: + +```json +{ + "id": "portal_setup_link_1", + "link": "https://setup.workos.com/abc", + "intents": ["sso"], + "state": "active", + "expiresAt": "2026-08-01T00:00:00Z" +} +``` + +The URL is under `link`. `intents` are echoed in the CLI's own lowercase +vocabulary (`Sso` -> `sso`). `state` is lowercase. + +```bash +jq -r '.portalSetupLink.link' +jq -r '.portalSetupLink.intents[]' +``` + +### webhook + +New: + +```json +{ + "id": "we_123", + "url": "https://example.com/hook", + "events": ["dsync.user.created"], + "state": "active", + "createdAt": "2024-01-01T00:00:00Z" +} +``` + +Two changes: the endpoint URL is under `url` (REST used `endpoint_url`), and +`state` is `active` where REST said `enabled`. List envelope is +`{ webhookEndpoints, pagination }`. + +```bash +# URL: renamed field +jq -r '.endpoint_url' # old (REST) +jq -r '.url' # new + +# State: vocabulary + presence changed +jq 'select(.status == "enabled")' # old (REST) +jq 'select(.state == "active")' # new +``` + +### config + +`config redirect add`, `config cors add`, and `config homepage-url set` emit +small confirmation objects, not resource shapes: + +```json +{ "uri": "https://app.example.com/callback", "alreadyExists": false } +{ "origin": "https://app.example.com", "alreadyExists": true } +{ "homepageUrl": "https://app.example.com", "applicationId": "app_1" } +``` + +These are new commands on the dashboard plane; no field renames from a prior +shape apply. + +## Deliberate differences from the old REST output + +These are intentional curations, declared in the `ACCEPTED` map of +`scripts/parity-smoke.ts`. They are not bugs, and the parity smoke reports them +as INFO, not FAIL. + +| Field | REST | Now | Reason | +| ------------------------- | -------------------------------------- | ------------------------------------------- | ---------------------------------------------------------------------------------- | +| `role.type` | `EnvironmentRole` / `OrganizationRole` | `environment` / `organization` | Drops the redundant `Role` suffix. | +| `webhook.state` | `enabled` | `active` | Aligns webhook lifecycle with session's `active`/`expired`/`revoked` vocabulary. | +| `invitation.organization` | flat `organizationId` | nested `{ id, name }` | Structural: consistent with other nested org references. | +| `user.identities` | different shape, `status` key | curated identities, `status` -> `state` | Structural + lifecycle-key rename. | +| `feature-flag.enabled` | flat flag | derived from the active environment's state | Flag state is per-environment server-side; the CLI reports the active environment. | + +Enum casing is deliberately absent from `ACCEPTED`: the CLI lowercases all enum +values, so a casing-only difference is treated as a bug that must fail the +parity run, not an accepted divergence. + +## How to verify your scripts + +`scripts/parity-smoke.ts` compares this release against a `../main` (REST) +checkout against the same WorkOS environment. It matches rows by id, compares +every shared field, and fails on any divergence not listed in `ACCEPTED`. Use it +to confirm your understanding of a field before and after. + +```bash +bun run scripts/parity-smoke.ts # read-only +bun run scripts/parity-smoke.ts --seed # seed fixtures so lists are non-empty +``` + +Prereqs are documented at the top of the script: a dashboard `workos auth login` +on this branch, a `WORKOS_API_KEY` for the REST plane pinned to the same +environment, and `../main` checked out on `main`. + +## Unconfirmed facts + +The sibling `../main` checkout was not reachable from this worktree, so the +following old REST values could not be independently verified against source: + +1. Exact REST casing for `invitation.state`, `org-domain.state`, + `org-domain.verificationStrategy`, `membership.state`/`membership.type`, and + `feature-flag.accessType`. The old values shown come from the `feat!` commit + body, which describes them as GraphQL backend passthrough casing normalized + in this release. Whether the last stable REST release emitted that exact + casing to users is not confirmed here. +2. The precise old REST envelope key names beyond `data` / `list_metadata` + (confirmed via the README correction commit e67867d for `org list`); other + commands are assumed to have used the same REST envelope. +3. The full old REST field set for `user.identities`, `session`, and `portal`. + The `ACCEPTED` map confirms the direction of the structural changes but not + the complete prior shapes. + +Confirmed directly from source: all new shapes (from `*.spec.ts`), the +conventions (`src/utils/output-conventions.ts`), the deliberate divergences +(`ACCEPTED` in `scripts/parity-smoke.ts`), the metadata array-vs-map history and +the lifecycle key rename (`feat!` commit 22d441d), and the snake_case enum rule +(commit 461df62). diff --git a/README.md b/README.md index 572d0394..963b2b48 100644 --- a/README.md +++ b/README.md @@ -125,6 +125,8 @@ Resource Management: feature-flag Manage feature flags webhook Manage webhooks config Manage redirect URIs, CORS, homepage URL + authkit Manage AuthKit app config (redirect URIs, CORS, logout URIs) + branding Manage branding images (logo, icon, favicon) portal Generate Admin Portal links vault Manage encrypted secrets api-key Manage per-org API keys @@ -144,7 +146,7 @@ Workflows: debug-sync Diagnose directory sync issues ``` -All management commands support `--json` for structured output (auto-enabled in non-TTY) and `--api-key` to override the active environment's key. +All management commands support `--json` for structured output (auto-enabled in non-TTY). Still-REST commands (`connection`, `directory`, `audit-log`, `api-key`, `vault`, `workos api`, and the workflow/debug commands) also accept `--api-key` to override the active environment's key; the other resource commands authenticate with your `workos auth login` session (see [Resource Management](#resource-management)). ### Unclaimed Environments @@ -303,7 +305,11 @@ API keys are stored in the system keychain via `@napi-rs/keyring`, with a JSON f ### Resource Management -All resource commands follow the same pattern: `workos [args] [--options]`. API keys resolve via: `--api-key` flag → `WORKOS_API_KEY` env var → active environment's stored key. +All resource commands follow the same pattern: `workos [args] [--options]`. + +Most resource commands — organization, user, role, permission, membership, invitation, session, event, feature-flag, org-domain, portal, webhook, config, authkit, branding — authenticate with your WorkOS dashboard session from `workos auth login` and target the active environment (see `workos env`). Pass `--environment-id ` on any of them to target a different environment for a single invocation. Access tokens refresh automatically while the session is valid, so a logged-in machine keeps working headlessly; a dead session exits with code 4. + +The remaining commands (`connection`, `directory`, `audit-log`, `api-key`, `vault`) still use the REST plane, as does the raw escape hatch `workos api`. Those resolve an API key via: `--api-key` flag → `WORKOS_API_KEY` env var → active environment's stored key. #### organization @@ -312,68 +318,72 @@ workos organization create [domain:state ...] workos organization update [domain] [state] workos organization get workos organization list [--domain] [--limit] [--before] [--after] [--order] -workos organization delete +workos organization delete [--yes] ``` #### user ```bash workos user get -workos user list [--email] [--organization] [--limit] -workos user update [--first-name] [--last-name] [--email-verified] [--password] [--external-id] -workos user delete +workos user list [--email] [--limit] [--before] [--after] [--order] +workos user update [--first-name] [--last-name] [--email] [--locale] [--external-id] +workos user delete [--yes] ``` #### role +Role mutations change the privilege surface; non-interactive callers must pass `--yes`. + ```bash workos role list [--org ] workos role get [--org ] -workos role create --slug --name [--org ] -workos role update [--name] [--description] [--org ] -workos role delete --org -workos role set-permissions --permissions [--org ] -workos role add-permission [--org ] -workos role remove-permission --org +workos role create --slug --name [--description] [--org ] [--yes] +workos role update [--name] [--description] [--org ] [--yes] +workos role delete --org [--yes] +workos role set-permissions --permissions [--org ] [--yes] +workos role add-permission [--org ] [--yes] +workos role remove-permission --org [--yes] ``` #### permission ```bash -workos permission list [--limit] +workos permission list workos permission get -workos permission create --slug --name [--description] -workos permission update [--name] [--description] -workos permission delete +workos permission create --slug --name [--description] [--yes] +workos permission update [--name] [--description] [--yes] +workos permission delete [--yes] ``` #### membership +`--org` and `--user` are mutually exclusive on `list`; pagination flags apply to `--org` listings only. + ```bash -workos membership list [--org] [--user] [--limit] +workos membership list (--org | --user ) [--limit] [--before] [--after] [--order] workos membership get workos membership create --org --user [--role] -workos membership update [--role] -workos membership delete -workos membership deactivate +workos membership update [--role] [--yes] +workos membership delete [--yes] +workos membership deactivate [--yes] workos membership reactivate ``` #### invitation ```bash -workos invitation list [--org] [--email] [--limit] +workos invitation list [--org] [--email] [--limit] [--before] [--after] workos invitation get workos invitation send --email [--org] [--role] [--expires-in-days] -workos invitation revoke +workos invitation revoke [--yes] workos invitation resend ``` #### session ```bash -workos session list [--limit] -workos session revoke +workos session list [--limit] [--before] [--after] +workos session revoke [--yes] ``` #### connection @@ -397,7 +407,7 @@ workos directory list-groups --directory [--limit] #### event ```bash -workos event list --events [--org] [--range-start] [--range-end] [--limit] +workos event list --events [--range-start] [--range-end] [--after] [--limit] ``` #### audit-log @@ -414,7 +424,7 @@ workos audit-log get-retention #### feature-flag ```bash -workos feature-flag list [--limit] +workos feature-flag list [--limit] [--before] [--after] [--order] workos feature-flag get workos feature-flag enable workos feature-flag disable @@ -427,7 +437,7 @@ workos feature-flag remove-target ```bash workos webhook list workos webhook create --url --events -workos webhook delete +workos webhook delete [--yes] ``` #### config @@ -438,12 +448,53 @@ workos config cors add workos config homepage-url set ``` +`config` adds a single entry to a list. To replace a whole list, use `authkit` below. + +#### authkit + +Per-environment AuthKit app configuration. Unlike `config`, these setters replace the entire list, and each accepts `--dry-run` to validate without saving. + +```bash +workos authkit redirect-uris list [--limit] +workos authkit redirect-uris set --uri [--uri ...] [--default ] [--dry-run] +workos authkit cors get +workos authkit cors set --origin [--origin ...] [--dry-run] +workos authkit logout-uris list [--limit] +workos authkit logout-uris set --uri [--uri ...] [--default ] [--dry-run] +``` + +Wildcard web origins are rejected — an accepted `*` would allow every browser origin, which is equivalent to disabling CORS. + +#### branding + +The logo, icon, and favicon an environment renders, each with a light and dark variant. Branding is not AuthKit-only: the same record drives hosted AuthKit pages and transactional emails. + +```bash +workos branding get +workos branding set +workos branding set [--logo] [--logo-dark] [--icon] [--icon-dark] [--favicon] [--favicon-dark] +``` + +Slots are `logo`, `logo-dark`, `icon`, `icon-dark`, `favicon`, and `favicon-dark`. Set one image positionally, or several at once with the matching flags — the two forms cannot be combined in a single invocation. + +```bash +# One image +workos branding set icon ./icon.png + +# Several at once +workos branding set --logo ./logo.png --logo-dark ./logo-dark.png --favicon ./favicon.ico +``` + +Images must be under 400 KB each and one of `.png`, `.jpg`, `.jpeg`, `.gif`, `.svg`, `.webp`, `.avif`, or `.ico`. Only the images you name are changed; the rest are left as they are. There is no way to clear an image from the CLI — upload a replacement instead. + #### portal ```bash -workos portal generate-link --intent --org [--return-url] [--success-url] +workos portal generate-link --intent --org ``` +Supported intents: `sso`, `dsync`, `log_streams`, `domain_verification`, `certificate_renewal`. Generating a link expires prior links of the same intent. + #### vault ```bash @@ -473,7 +524,7 @@ workos api-key delete workos org-domain get workos org-domain create --org workos org-domain verify -workos org-domain delete +workos org-domain delete [--yes] ``` ### Installer Options @@ -512,11 +563,11 @@ mkdir my-app && cd my-app && workos install # With visual dashboard (experimental) workos dashboard -# JSON output (explicit) -workos org list --json --api-key sk_test_xxx +# JSON output (explicit; requires a prior `workos auth login`) +workos org list --json # Pipe-friendly (auto-detects non-TTY) -workos org list --api-key sk_test_xxx | jq '.data[].name' +workos org list | jq '.organizations[].name' # Machine-readable command discovery workos --help --json | jq '.commands[].name' @@ -539,11 +590,15 @@ The CLI also auto-detects non-TTY environments (piped output, CI, coding agents) ### JSON Output +> **Upgrading?** The resource commands moved to the dashboard GraphQL API and +> their `--json` shapes changed in one breaking release. See +> [MIGRATION.md](MIGRATION.md) for the old-to-new field table and `jq` snippets. + All commands produce structured JSON when piped or with `--json`: ```bash -workos org list --api-key sk_test_xxx | jq . -# → { "data": [...], "list_metadata": { "before": null, "after": "..." } } +workos org list | jq . +# → { "organizations": [...], "pagination": { "before": null, "after": "..." } } workos env list --json # → { "data": [{ "name": "prod", "type": "production", "active": true, ... }] } @@ -552,8 +607,8 @@ workos env list --json Errors go to stderr as structured JSON: ```bash -workos org list 2>&1 -# → { "error": { "code": "no_api_key", "message": "No API key configured..." } } +workos org list 2>&1 # not logged in → exit code 4 +# → { "error": { "code": "auth_required", "message": "Not logged in..." } } ``` ### Agent Mode @@ -603,13 +658,16 @@ workos install --api-key sk_test_xxx --client-id client_xxx --no-commit 2>/dev/n ### Environment Variables -| Variable | Effect | -| ------------------------ | --------------------------------------------------------- | -| `WORKOS_API_KEY` | API key for management commands (bypasses stored config) | -| `WORKOS_API_BASE_URL` | Override API base URL (set automatically by `workos dev`) | -| `WORKOS_MODE` | Interaction mode: `human`, `agent`, or `ci` | -| `WORKOS_FORCE_TTY=1` | Force human (non-JSON) **output** mode even when piped | -| `WORKOS_TELEMETRY=false` | Disable telemetry | +| Variable | Effect | +| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `WORKOS_API_KEY` | API key for `workos api`, the still-REST commands (connection, directory, audit-log, api-key, vault), and workflow/debug commands. Resource commands use the `workos auth login` session instead | +| `WORKOS_API_URL` | Override the API base URL for all CLI commands (e.g. a local API) | +| `WORKOS_API_BASE_URL` | Accepted alias for `WORKOS_API_URL` (what `workos dev` sets) | +| `WORKOS_MODE` | Interaction mode: `human`, `agent`, or `ci` | +| `WORKOS_FORCE_TTY=1` | Force human (non-JSON) **output** mode even when piped | +| `WORKOS_TELEMETRY=false` | Disable telemetry | + +> `WORKOS_API_URL` controls where the **CLI's own** commands (`workos user`, `workos api`, etc.) send requests — set it to point the CLI at a locally-running API. This is distinct from `workos dev`, which sets API vars for your **app's** dev process so it talks to the in-process emulator. ### Command Discovery diff --git a/package.json b/package.json index cd0a4651..428c6134 100644 --- a/package.json +++ b/package.json @@ -89,7 +89,9 @@ "eval:diff": "bun run ./tests/evals/index.ts diff", "eval:prune": "bun run ./tests/evals/index.ts prune", "eval:logs": "bun run ./tests/evals/index.ts logs", - "eval:show": "bun run ./tests/evals/index.ts show" + "eval:show": "bun run ./tests/evals/index.ts show", + "catalog:vendor": "bun run ./scripts/vendor-catalog.ts", + "justification:check": "bun run ./scripts/check-justification.ts" }, "author": "WorkOS", "license": "MIT" diff --git a/scripts/check-justification.ts b/scripts/check-justification.ts new file mode 100644 index 00000000..33c172ef --- /dev/null +++ b/scripts/check-justification.ts @@ -0,0 +1,39 @@ +#!/usr/bin/env tsx +/** + * CI gate for the command justification manifest. + * + * Loads the curated manifest and the vendored Management catalog, runs + * `validateManifest`, and exits non-zero if any entry is incomplete or maps to + * an operation the catalog does not contain. Wired to `pnpm justification:check` + * and run in CI so an unjustified or drifted command cannot ship. + * + * Usage: + * pnpm justification:check + */ + +import { getManifest } from '../src/catalog/manifest.js'; +import { loadManagementCatalog } from '../src/catalog/loader.js'; +import { validateManifest } from '../src/catalog/justification.js'; + +function main(): void { + const manifest = getManifest(); + // Validate against the full catalog (including feature-flag-gated ops): a + // manifest entry may legitimately target a flagged operation, and we only + // want to fail on genuinely-unknown `mapsTo` values, not on visibility. + const catalog = loadManagementCatalog(undefined, { includeFeatureFlagged: true }); + + const { ok, errors } = validateManifest(manifest, catalog); + + if (ok) { + console.log(`justification:check — OK (${manifest.length} command(s) validated)`); + process.exit(0); + } + + console.error(`justification:check — FAILED (${errors.length} error(s)):`); + for (const error of errors) { + console.error(` - ${error}`); + } + process.exit(1); +} + +main(); diff --git a/scripts/command-smoke.sh b/scripts/command-smoke.sh index 0f5cb640..0cd644f9 100644 --- a/scripts/command-smoke.sh +++ b/scripts/command-smoke.sh @@ -11,10 +11,12 @@ # POSIX sh only: this runs in debian-slim and Alpine containers (no bash) # and under Git Bash on the Windows runners. # -# When WORKOS_API_KEY is provided, an authenticated section also runs: list -# plus a create → get → delete round-trip against that environment. CI passes -# a dedicated staging-environment key; fork PRs receive no secrets and skip -# it. The offline contract checks always run with the key withheld, so the +# When WORKOS_API_KEY is provided, an authenticated section also runs: a list +# against a still-REST resource command (migrated resource commands take the +# dashboard session and refuse API keys) plus a create → get → delete +# organization round-trip through `workos api`, the raw-REST escape hatch. +# CI passes a dedicated staging-environment key; fork PRs receive no secrets +# and skip it. The offline contract checks always run with the key withheld, so the # exit-4 assertion stays deterministic. # # Usage: [WORKOS_API_KEY=sk_...] sh command-smoke.sh /path/to/workos @@ -52,7 +54,7 @@ cleanup() { # Never orphan the round-trip organization in the shared staging # environment, even when a check between create and delete fails. if [ -n "$ORG_ID" ] && [ "$org_deleted" -eq 0 ]; then - WORKOS_API_KEY="$SMOKE_API_KEY" "$BIN" organization delete "$ORG_ID" --insecure-storage >/dev/null 2>&1 || true + WORKOS_API_KEY="$SMOKE_API_KEY" "$BIN" api "/organizations/$ORG_ID" -X DELETE -y --insecure-storage >/dev/null 2>&1 || true fi rm -rf "$SANDBOX" } @@ -113,43 +115,47 @@ if [ -n "$SMOKE_API_KEY" ]; then # On failure, surface the CLI's structured stderr error — it never # contains key material (keys are masked in all output). err_file="$SANDBOX/stderr" - out=$(WORKOS_API_KEY="$SMOKE_API_KEY" "$BIN" organization list --json --insecure-storage 2>"$err_file") + out=$(WORKOS_API_KEY="$SMOKE_API_KEY" "$BIN" connection list --json --insecure-storage 2>"$err_file") code=$? case "$out" in *'"data"'*) json_ok=1 ;; *) json_ok=0 ;; esac - if [ "$code" -eq 0 ] && [ "$json_ok" -eq 1 ]; then pass "authenticated organization list exits 0 with data"; else fail "authenticated organization list (exit $code): $(cat "$err_file")"; fi + if [ "$code" -eq 0 ] && [ "$json_ok" -eq 1 ]; then pass "authenticated connection list exits 0 with data"; else fail "authenticated connection list (exit $code): $(cat "$err_file")"; fi + # The write round-trip goes through `workos api`: organization is on the + # dashboard plane now, and every other still-REST create needs an org id + # this script can no longer mint. `api` pretty-prints, hence the space- + # tolerant id parse. -y because mutating api calls refuse otherwise. ORG_NAME="cli-smoke-$$-$(date +%s)" - out=$(WORKOS_API_KEY="$SMOKE_API_KEY" "$BIN" organization create "$ORG_NAME" --json --insecure-storage 2>"$err_file") + out=$(WORKOS_API_KEY="$SMOKE_API_KEY" "$BIN" api /organizations -d "{\"name\":\"$ORG_NAME\"}" -y --insecure-storage 2>"$err_file") code=$? # Org ids are org_; the closing-quote anchor keeps nested # org_domain_* ids from matching. - ORG_ID=$(printf '%s' "$out" | sed -n 's/.*"id":"\(org_[A-Za-z0-9]*\)".*/\1/p') + ORG_ID=$(printf '%s' "$out" | sed -n 's/.*"id": *"\(org_[A-Za-z0-9]*\)".*/\1/p') if [ "$code" -eq 0 ] && [ -n "$ORG_ID" ]; then org_deleted=0 - pass "organization create returns an id ($ORG_ID)" + pass "api organization create returns an id ($ORG_ID)" else - fail "organization create (exit $code): $out $(cat "$err_file")" + fail "api organization create (exit $code): $out $(cat "$err_file")" fi if [ -n "$ORG_ID" ]; then - out=$(WORKOS_API_KEY="$SMOKE_API_KEY" "$BIN" organization get "$ORG_ID" --json --insecure-storage 2>"$err_file") + out=$(WORKOS_API_KEY="$SMOKE_API_KEY" "$BIN" api "/organizations/$ORG_ID" --insecure-storage 2>"$err_file") code=$? case "$out" in *"$ORG_NAME"*) json_ok=1 ;; *) json_ok=0 ;; esac - if [ "$code" -eq 0 ] && [ "$json_ok" -eq 1 ]; then pass "organization get returns the created organization"; else fail "organization get (exit $code)"; fi + if [ "$code" -eq 0 ] && [ "$json_ok" -eq 1 ]; then pass "api organization get returns the created organization"; else fail "api organization get (exit $code)"; fi - WORKOS_API_KEY="$SMOKE_API_KEY" "$BIN" organization delete "$ORG_ID" --json --insecure-storage >/dev/null 2>&1 + WORKOS_API_KEY="$SMOKE_API_KEY" "$BIN" api "/organizations/$ORG_ID" -X DELETE -y --insecure-storage >/dev/null 2>&1 code=$? if [ "$code" -eq 0 ]; then org_deleted=1 - pass "organization delete cleans up" + pass "api organization delete cleans up" else - fail "organization delete (exit $code) — cleanup trap will retry" + fail "api organization delete (exit $code) — cleanup trap will retry" fi fi else diff --git a/scripts/parity-smoke.ts b/scripts/parity-smoke.ts new file mode 100644 index 00000000..7f85bab6 --- /dev/null +++ b/scripts/parity-smoke.ts @@ -0,0 +1,1079 @@ +/** + * Parity smoke: compare command output between this branch (dashboard GraphQL + * plane) and ../main (REST plane), against the same WorkOS environment. + * + * Buckets: + * + * CONTROL — commands that stayed REST on both branches (connection, + * directory, audit-log). Same code both sides → strict shape+data + * parity. A failure here means env/auth misalignment, so every + * result below is untrustworthy. Check these first. + * LIST — the migrated commands' list views. Shapes intentionally differ + * (approved feat!), so this matches rows by id and compares EVERY + * field present on both sides. Known-intentional differences are + * declared in ACCEPTED and reported as INFO, not FAIL; anything + * else is a FAIL. Branch-only keys (curated additions) and + * main-only keys (intentionally dropped internals) are reported + * as INFO. + * GET — detail views. Sources an id/slug from the branch list, calls + * `get` on both binaries, compares the same way. + * WRITE — opt-in (--mutate). Cross-plane read-after-write: create an + * organization on one plane, read it back on the OTHER, then + * delete it. Running `create` on both and diffing would compare + * two DIFFERENT entities and prove nothing; reading across planes + * is the actual claim the migration makes. + * NEW — branch-only commands. Exit 0 + valid JSON on the branch binary. + * + * Live parity gaps: feature flags cannot be seeded outside the dashboard; + * session requires a real login; webhook/event have no `get`; org-domain has + * `get` but no list from which to source an id; portal/config are mutation-only. + * Their complete JSON contracts are pinned by json-contract.spec.ts. + * + * Prereqs: + * 1. From this branch: `workos auth login` (dashboard OAuth session). + * 2. export WORKOS_API_KEY=sk_... for main's REST plane. Its environment + * MUST match the dashboard session's active env, or every row diverges. + * 3. ../main = a checkout of this repo on main. + * + * Usage: + * bun run scripts/parity-smoke.ts # read-only + * bun run scripts/parity-smoke.ts --seed # + seed fixtures so lists are non-empty + * bun run scripts/parity-smoke.ts --seed --mutate # + cross-plane write round-trip + * bun run scripts/parity-smoke.ts --seed --invite --mutate --strict + * # release gate + * + * Env: PARITY_BRANCH_BIN, PARITY_MAIN_BIN (default: bun /src/bin.ts), + * PARITY_ENV_ID (pins --environment-id on branch commands). + * + * Exit: 0 if every executed check passed, 1 otherwise. + */ +import { spawn } from 'node:child_process'; +import { existsSync, mkdtempSync, readFileSync } from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; + +const BRANCH_DIR = path.resolve(import.meta.dirname, '..'); +const MAIN_DIR = path.resolve(BRANCH_DIR, '../main'); +const ENV_ID = process.env.PARITY_ENV_ID; +const MUTATE = process.argv.includes('--mutate'); +const SEED = process.argv.includes('--seed'); +const STRICT = process.argv.includes('--strict'); +// Invitations are seeded separately because sending one attempts real email +// delivery. The address used is @example.com (RFC 2606 reserved, black-holed), +// and the invitation is revoked during cleanup, but it stays opt-in. +const INVITE = process.argv.includes('--invite'); + +if (STRICT) { + const missing = [!SEED && '--seed', !INVITE && '--invite', !MUTATE && '--mutate'].filter(Boolean); + if (missing.length) { + console.error(`--strict requires ${missing.join(', ')}`); + process.exit(2); + } +} + +if (!existsSync(path.join(MAIN_DIR, 'src/bin.ts'))) { + console.error(`main checkout not found: ${MAIN_DIR}/src/bin.ts`); + process.exit(2); +} +for (const dir of [BRANCH_DIR, MAIN_DIR]) { + const pkg = JSON.parse(readFileSync(path.join(dir, 'package.json'), 'utf8')); + if (pkg.name !== 'workos') { + console.error(`${dir} is not the workos CLI (package.json name="${pkg.name}")`); + process.exit(2); + } +} + +// --- credential + cwd isolation --------------------------------------------------- +// Bun auto-loads `.env.local` from the CWD. Each worktree ships its own, with a +// DIFFERENT WORKOS_API_KEY, so running each binary from its own directory made +// the two planes authenticate against two different environments and compare +// unrelated data. Run everything from an empty scratch dir (nothing to load) and +// inject ONE key into both children. +const NEUTRAL_CWD = mkdtempSync(path.join(os.tmpdir(), 'parity-smoke-')); + +function envLocalKey(dir: string): string | undefined { + const f = path.join(dir, '.env.local'); + if (!existsSync(f)) return undefined; + const m = readFileSync(f, 'utf8').match(/^WORKOS_API_KEY=(.*)$/m); + return m?.[1]?.trim().replace(/^["']|["']$/g, '') || undefined; +} + +// Precedence: explicit env var > this branch's .env.local. Never main's, so the +// REST plane is pinned to the same environment the branch targets. +const API_KEY = process.env.WORKOS_API_KEY ?? envLocalKey(BRANCH_DIR); +const API_KEY_SOURCE = process.env.WORKOS_API_KEY ? 'env' : envLocalKey(BRANCH_DIR) ? 'branch .env.local' : 'NONE'; + +function binFor(dir: string): { cmd: string; args: string[] } { + const override = dir === BRANCH_DIR ? process.env.PARITY_BRANCH_BIN : process.env.PARITY_MAIN_BIN; + if (override) { + const [cmd, ...args] = override.trim().split(/\s+/); + return { cmd, args }; + } + return { cmd: 'bun', args: [path.join(dir, 'src/bin.ts')] }; +} + +interface RunResult { + rc: number; + stdout: string; + stderr: string; +} +const COMMAND_TIMEOUT_MS = 60_000; +function run(dir: string, cliArgs: string[]): Promise { + const { cmd, args } = binFor(dir); + return new Promise((resolve) => { + const childEnv: NodeJS.ProcessEnv = { ...process.env }; + if (API_KEY) childEnv.WORKOS_API_KEY = API_KEY; + const p = spawn(cmd, [...args, ...cliArgs], { cwd: NEUTRAL_CWD, env: childEnv, stdio: ['ignore', 'pipe', 'pipe'] }); + let stdout = ''; + let stderr = ''; + let settled = false; + const finish = (result: RunResult) => { + if (settled) return; + settled = true; + clearTimeout(timer); + resolve(result); + }; + const timer = setTimeout(() => { + p.kill('SIGKILL'); + finish({ rc: -1, stdout, stderr: `${stderr}\ncommand timed out after ${COMMAND_TIMEOUT_MS / 1000}s` }); + }, COMMAND_TIMEOUT_MS); + p.stdout.on('data', (d) => { + stdout += d; + }); + p.stderr.on('data', (d) => { + stderr += d; + }); + p.on('error', (err) => finish({ rc: -1, stdout, stderr: String(err) })); + p.on('close', (rc, signal) => + finish({ rc: rc ?? -1, stdout, stderr: signal ? `${stderr}\nterminated by ${signal}` : stderr }), + ); + }); +} + +function parseJson(stdout: string): any { + try { + return JSON.parse(stdout); + } catch { + return undefined; + } +} + +const envArgs = (): string[] => (ENV_ID ? ['--environment-id', ENV_ID] : []); +const shortErr = (s: string): string => s.split('\n').filter(Boolean).slice(0, 2).join(' | '); + +// --- value normalization --------------------------------------------------------- +// Volatile or timing-dependent keys are never compared. +const VOLATILE = new Set([ + 'createdAt', + 'created_at', + 'updatedAt', + 'updated_at', + 'lastSignedInAt', + 'last_signed_in_at', + 'emailVerifiedAt', + 'email_verified_at', + 'occurredAt', + 'occurred_at', + 'expiresAt', + 'expires_at', + 'timestamp', + 'before', + 'after', + 'listMetadata', + 'list_metadata', + 'pagination', +]); + +function normalize(v: unknown): unknown { + if (v === null || v === undefined) return null; + if (Array.isArray(v)) return v.map(normalize).sort(cmp); + if (typeof v === 'object') { + const o: Record = {}; + for (const k of Object.keys(v as Record).sort()) { + if (!VOLATILE.has(k)) o[k] = normalize((v as Record)[k]); + } + return o; + } + return v; +} +function cmp(a: unknown, b: unknown): number { + const ka = JSON.stringify(a); + const kb = JSON.stringify(b); + return ka < kb ? -1 : ka > kb ? 1 : 0; +} +const canon = (v: unknown): string => JSON.stringify(normalize(v)); + +function itemsOf(out: any, key?: string): any[] { + if (out && typeof out === 'object') { + if (key) return Array.isArray(out[key]) ? out[key] : []; + if (Array.isArray(out.data)) return out.data; + for (const k of Object.keys(out)) if (Array.isArray(out[k])) return out[k]; + } + return []; +} + +/** Unwrap a single-entity payload: branch `{organization:{...}}`, main raw or `{data:{...}}`. */ +function entityOf(out: any): any { + if (!out || typeof out !== 'object') return undefined; + if (out.id) return out; + if (out.data && typeof out.data === 'object' && !Array.isArray(out.data)) return out.data; + for (const k of Object.keys(out)) { + const v = out[k]; + if (v && typeof v === 'object' && !Array.isArray(v) && v.id) return v; + } + return undefined; +} + +/** Recursively find the first string value matching `_...` — envelope-agnostic. */ +function findId(out: any, prefix: string): string | undefined { + if (typeof out === 'string') return out.startsWith(`${prefix}_`) ? out : undefined; + if (Array.isArray(out)) { + for (const v of out) { + const r = findId(v, prefix); + if (r) return r; + } + return undefined; + } + if (out && typeof out === 'object') { + for (const k of Object.keys(out)) { + const r = findId(out[k], prefix); + if (r) return r; + } + } + return undefined; +} + +// --- accepted divergences -------------------------------------------------------- +// Field-level differences that are intentional per the branch specs. Reported as +// INFO so they stay visible, never as FAIL. Anything NOT listed here that differs +// is a real regression. +// Every entry is a DELIBERATE vocabulary or structural curation, documented in +// the migration guide. Enum CASING is deliberately absent: the CLI normalizes +// all enum values to lowercase via utils/output-conventions, so a casing-only +// difference is a bug, not an accepted divergence, and must fail the run. +const ACCEPTED: Record = { + 'role.type': + 'vocabulary: branch emits environment/organization; REST emitted EnvironmentRole/OrganizationRole. The redundant Role suffix is dropped.', + 'webhook.state': + "vocabulary: branch emits active; REST emitted enabled. Aligned with session's active/expired/revoked.", + 'invitation.organization': 'structural: branch nests {id,name}; REST emitted a flat organizationId', + 'user.identities': 'structural: branch nests curated identities and renames status->state; REST shape differs', + 'feature-flag.enabled': 'semantic: branch derives from the active environment state; REST exposed a flat flag', +}; + +/** + * Branch key -> main key, for fields the curated shapes RENAMED. Without this a + * renamed field is "branch-only" on one side and "dropped" on the other, so it + * is never compared and the row passes vacuously. + */ +const ALIAS: Record> = { + webhook: { url: 'endpoint_url', state: 'status' }, + invitation: { email: 'email' }, +}; + +// Top-level curation that is already documented and snapshot-pinned. In strict +// mode, any NEW branch-only or main-only key fails instead of being waved +// through as an informational shape difference. +const EXPECTED_BRANCH_ONLY: Record = { + organization: ['usersCount'], + user: ['authenticationFactors', 'hasPassword', 'identities', 'sessionCount'], + invitation: ['organization'], +}; +const EXPECTED_MAIN_ONLY: Record = { + organization: ['object'], + user: ['emailVerified', 'lastSignInAt', 'object'], + role: ['object', 'resourceTypeSlug'], + permission: ['object', 'resourceTypeSlug'], + invitation: [ + 'acceptInvitationUrl', + 'acceptedAt', + 'acceptedUserId', + 'inviterUserId', + 'object', + 'organizationId', + 'revokedAt', + 'token', + ], + webhook: ['object', 'secret'], + event: ['context'], +}; + +function unexpectedCuration(cmd: string, side: 'branch' | 'main', keys: Iterable): string[] { + const expected = new Set((side === 'branch' ? EXPECTED_BRANCH_ONLY : EXPECTED_MAIN_ONLY)[cmd] ?? []); + return [...keys].filter((key) => !expected.has(key)); +} + +interface FieldDiff { + key: string; + branch: unknown; + main: unknown; +} +interface RowCompare { + diffs: FieldDiff[]; + accepted: FieldDiff[]; + branchOnly: string[]; + mainOnly: string[]; +} + +const isPlainObject = (v: unknown): v is Record => + typeof v === 'object' && v !== null && !Array.isArray(v); + +/** + * Reduce both sides to the structure they SHARE, recursively. The curated + * shapes drop internal fields at every depth (an organization domain keeps + * {id,domain,state} and drops object/organizationId/verificationStrategy), so a + * wholesale compare of a nested object reports intentional curation as a diff. + * Projecting to shared keys first means only genuine value divergences survive. + * A type mismatch (array vs object) is preserved so it still reports. + */ +function projectShared(b: any, m: any): [any, any] { + if (Array.isArray(b) && Array.isArray(m)) { + if (b.length !== m.length) return [b, m]; + // Pair elements by id when both sides carry one; otherwise pair by sorted order. + const byId = b.every((x) => isPlainObject(x) && x.id) && m.every((x) => isPlainObject(x) && x.id); + const bs = byId ? [...b].sort((x, y) => cmp(x.id, y.id)) : [...b].sort(cmp); + const ms = byId ? [...m].sort((x, y) => cmp(x.id, y.id)) : [...m].sort(cmp); + const pb: any[] = []; + const pm: any[] = []; + for (let i = 0; i < bs.length; i++) { + const [x, y] = projectShared(bs[i], ms[i]); + pb.push(x); + pm.push(y); + } + return [pb, pm]; + } + if (isPlainObject(b) && isPlainObject(m)) { + const pb: Record = {}; + const pm: Record = {}; + for (const k of Object.keys(b)) { + if (VOLATILE.has(k) || !(k in m)) continue; + const [x, y] = projectShared(b[k], m[k]); + pb[k] = x; + pm[k] = y; + } + return [pb, pm]; + } + return [b, m]; +} + +function compareEntity(cmd: string, b: any, m: any): RowCompare { + const diffs: FieldDiff[] = []; + const accepted: FieldDiff[] = []; + const branchOnly: string[] = []; + const mainOnly: string[] = []; + const aliased = new Set(); + for (const k of Object.keys(b)) { + if (VOLATILE.has(k)) continue; + const mk = k in m ? k : ALIAS[cmd]?.[k]; + if (!mk || !(mk in m)) { + branchOnly.push(k); + continue; + } + aliased.add(mk); + const [pb, pm] = projectShared(b[k], m[mk]); + if (canon(pb) !== canon(pm)) { + const label = mk === k ? k : `${k}↔${mk}`; + const d = { key: label, branch: b[k], main: m[mk] }; + (ACCEPTED[`${cmd}.${k}`] ? accepted : diffs).push(d); + } + } + for (const k of Object.keys(m)) { + if (VOLATILE.has(k)) continue; + if (!(k in b) && !aliased.has(k)) mainOnly.push(k); + } + return { diffs, accepted, branchOnly, mainOnly }; +} + +// --- registry -------------------------------------------------------------------- +const CONTROL = [ + { label: 'connection list', args: ['connection', 'list', '--json'] }, + { label: 'directory list', args: ['directory', 'list', '--json'] }, + { label: 'audit-log list-actions', args: ['audit-log', 'list-actions', '--json'] }, +]; + +interface ListCheck { + cmd: string; + label: string; + args: string[]; + /** Branch envelope key; main's REST envelope always uses `data`. */ + listKey: string; + /** id/slug key used to match rows across sides and to feed the GET check. */ + idKey: string; + /** `get` subcommand, if the command has one. */ + get?: { sub: string; argFrom: string }; + /** + * Set when the two planes page in different sort orders, so a bounded `--limit` + * window legitimately returns different rows. Exact-set matching would be + * comparing the newest N against the oldest N. Instead: require a non-empty + * overlap and compare every field on the rows that DO appear on both. + */ + orderDiverges?: { note: string }; +} + +const LISTS: ListCheck[] = [ + { + cmd: 'organization', + label: 'organization', + args: ['organization', 'list', '--json'], + listKey: 'organizations', + idKey: 'id', + get: { sub: 'get', argFrom: 'id' }, + }, + { + cmd: 'user', + label: 'user', + args: ['user', 'list', '--json'], + listKey: 'users', + idKey: 'id', + get: { sub: 'get', argFrom: 'id' }, + }, + { + cmd: 'role', + label: 'role', + args: ['role', 'list', '--json'], + listKey: 'roles', + idKey: 'slug', + get: { sub: 'get', argFrom: 'slug' }, + }, + { + cmd: 'permission', + label: 'permission', + args: ['permission', 'list', '--json'], + listKey: 'permissions', + idKey: 'slug', + get: { sub: 'get', argFrom: 'slug' }, + }, + { + cmd: 'invitation', + label: 'invitation', + args: ['invitation', 'list', '--json'], + listKey: 'invitations', + idKey: 'id', + get: { sub: 'get', argFrom: 'id' }, + }, + { + cmd: 'feature-flag', + label: 'feature-flag', + args: ['feature-flag', 'list', '--json'], + listKey: 'flags', + idKey: 'slug', + get: { sub: 'get', argFrom: 'slug' }, + }, + { + cmd: 'webhook', + label: 'webhook', + args: ['webhook', 'list', '--json'], + listKey: 'webhookEndpoints', + idKey: 'id', + }, + // organization.created and user.created both fire during --seed/--mutate, so + // this has real rows to compare rather than empty-vs-empty. + { + cmd: 'event', + label: 'event', + args: ['event', 'list', '--events', 'organization.created,user.created', '--limit', '100', '--json'], + listKey: 'events', + idKey: 'id', + orderDiverges: { + note: 'GraphQL returns events newest-first; REST returned oldest-first. A bounded --limit window therefore covers opposite ends of the feed.', + }, + }, +]; + +const NEW_ONLY = [ + { label: 'whoami', args: ['whoami', '--json'] }, + { label: 'project list', args: ['project', 'list', '--json'] }, + { label: 'team members', args: ['team', 'members', '--json'] }, + { label: 'authkit redirect-uris list', args: ['authkit', 'redirect-uris', 'list', '--json', ...envArgs()] }, + { label: 'branding get', args: ['branding', 'get', '--json', ...envArgs()] }, +]; + +// --- reporting ------------------------------------------------------------------- +type Status = 'PASS' | 'FAIL' | 'AUTH' | 'SKIP' | 'INFO'; +interface Row { + kind: string; + label: string; + status: Status; + detail: string; +} +const rows: Row[] = []; +const notes: string[] = []; +const C = { + green: '\x1b[32m', + red: '\x1b[31m', + yellow: '\x1b[33m', + cyan: '\x1b[36m', + dim: '\x1b[2m', + reset: '\x1b[0m', +}; +const push = (kind: string, label: string, status: Status, detail: string) => + rows.push({ kind, label, status, detail }); + +/** Poll eventual cross-plane state for up to 15 seconds. */ +async function eventually(check: () => Promise): Promise { + for (let attempt = 0; attempt < 15; attempt++) { + if (await check()) return true; + await new Promise((resolve) => setTimeout(resolve, 1000)); + } + return false; +} + +// --- checks ---------------------------------------------------------------------- +async function checkControl(c: { label: string; args: string[] }): Promise { + const [b, m] = await Promise.all([run(BRANCH_DIR, c.args), run(MAIN_DIR, c.args)]); + if (b.rc === 4 || m.rc === 4) return push('CONTROL', c.label, 'AUTH', `branch rc=${b.rc}, main rc=${m.rc}`); + if (b.rc !== 0 || m.rc !== 0) { + return push( + 'CONTROL', + c.label, + 'FAIL', + `exit branch=${b.rc} main=${m.rc} ${shortErr(b.stderr)}${shortErr(m.stderr)}`, + ); + } + const jb = parseJson(b.stdout); + const jm = parseJson(m.stdout); + if (jb === undefined || jm === undefined) return push('CONTROL', c.label, 'FAIL', 'invalid JSON on one side'); + push( + 'CONTROL', + c.label, + canon(jb) === canon(jm) ? 'PASS' : 'FAIL', + canon(jb) === canon(jm) ? 'shape+data equal' : 'normalized output differs', + ); +} + +/** Returns the branch list items so the GET phase can source identifiers. */ +async function checkList(c: ListCheck): Promise { + const args = [...c.args, ...envArgs()]; + const mainArgs = c.args; // main takes no --environment-id + const [b, m] = await Promise.all([run(BRANCH_DIR, args), run(MAIN_DIR, mainArgs)]); + if (b.rc === 4 || m.rc === 4) { + push('LIST', c.label, 'AUTH', `branch rc=${b.rc}, main rc=${m.rc}`); + return undefined; + } + if (b.rc !== 0 || m.rc !== 0) { + push('LIST', c.label, 'FAIL', `exit branch=${b.rc} main=${m.rc} ${shortErr(b.stderr)}${shortErr(m.stderr)}`); + return undefined; + } + const jb = parseJson(b.stdout); + const jm = parseJson(m.stdout); + if (jb === undefined || jm === undefined) { + push('LIST', c.label, 'FAIL', 'invalid JSON on one side'); + return undefined; + } + + const bi = itemsOf(jb, c.listKey); + const mi = itemsOf(jm); + if (bi.length === 0 && mi.length === 0) { + push('LIST', c.label, 'SKIP', 'both sides empty — proves the call works, proves nothing about data'); + return bi; + } + const bIds = new Set(bi.map((i) => i?.[c.idKey])); + const mIds = new Set(mi.map((i) => i?.[c.idKey])); + const onlyB = [...bIds].filter((x) => !mIds.has(x)); + const onlyM = [...mIds].filter((x) => !bIds.has(x)); + if (c.orderDiverges) { + const shared = [...bIds].filter((x) => mIds.has(x)); + notes.push(`${C.cyan}INFO${C.reset} ${c.cmd} ordering: ${c.orderDiverges.note}`); + if (shared.length === 0) { + push( + 'LIST', + c.label, + 'SKIP', + `no overlap between the two windows (branch ${bi.length}, main ${mi.length}); widen --limit to compare rows`, + ); + return bi; + } + // Fall through and compare only the shared rows. + } else if (onlyB.length || onlyM.length) { + push( + 'LIST', + c.label, + 'FAIL', + `entity sets differ (branch ${bi.length}, main ${mi.length}; only-branch=${JSON.stringify(onlyB).slice(0, 60)} only-main=${JSON.stringify(onlyM).slice(0, 60)})`, + ); + return bi; + } + // Same entity set: compare every shared field, row by row. + let unexpected = 0; + let acceptedN = 0; + const seenBranchOnly = new Set(); + const seenMainOnly = new Set(); + let comparedRows = 0; + for (const bRow of bi) { + const mRow = mi.find((x) => x?.[c.idKey] === bRow?.[c.idKey]); + if (!mRow) continue; + comparedRows++; + const r = compareEntity(c.cmd, bRow, mRow); + unexpected += r.diffs.length; + acceptedN += r.accepted.length; + r.branchOnly.forEach((k) => seenBranchOnly.add(k)); + r.mainOnly.forEach((k) => seenMainOnly.add(k)); + for (const d of r.diffs) { + notes.push( + `${C.red}FAIL${C.reset} ${c.cmd}.${d.key}: branch=${JSON.stringify(d.branch)} main=${JSON.stringify(d.main)}`, + ); + } + for (const d of r.accepted) { + notes.push( + `${C.cyan}INFO${C.reset} ${c.cmd}.${d.key} (accepted): branch=${JSON.stringify(d.branch)} main=${JSON.stringify(d.main)}`, + ); + } + } + if (STRICT) { + const unexpectedBranchOnly = unexpectedCuration(c.cmd, 'branch', seenBranchOnly); + const unexpectedMainOnly = unexpectedCuration(c.cmd, 'main', seenMainOnly); + for (const key of unexpectedBranchOnly) + notes.push(`${C.red}FAIL${C.reset} ${c.cmd}: unexpected branch-only key ${key}`); + for (const key of unexpectedMainOnly) + notes.push(`${C.red}FAIL${C.reset} ${c.cmd}: unexpected main-only key ${key}`); + unexpected += unexpectedBranchOnly.length + unexpectedMainOnly.length; + } + const extras = [ + seenBranchOnly.size ? `branch-only keys: ${[...seenBranchOnly].join(',')}` : '', + seenMainOnly.size ? `dropped: ${[...seenMainOnly].join(',')}` : '', + acceptedN ? `${acceptedN} accepted diff(s)` : '', + ] + .filter(Boolean) + .join('; '); + const scope = c.orderDiverges + ? `${comparedRows} overlapping row(s) compared (windows differ by sort order)` + : `${comparedRows} row(s) matched, every shared field compared`; + push( + 'LIST', + c.label, + unexpected === 0 ? 'PASS' : 'FAIL', + `${scope}${extras ? ` — ${extras}` : ''}${unexpected ? ` — ${unexpected} UNEXPECTED diff(s)` : ''}`, + ); + return bi; +} + +async function checkGet(c: ListCheck, branchItems: any[] | undefined): Promise { + if (!c.get) return push('GET', c.label, 'SKIP', 'no get subcommand on either side'); + if (!branchItems || branchItems.length === 0) + return push('GET', c.label, 'SKIP', 'no row available to source an identifier'); + const ident = branchItems[0]?.[c.get.argFrom]; + if (!ident) return push('GET', c.label, 'SKIP', `list row has no ${c.get.argFrom}`); + + const [b, m] = await Promise.all([ + run(BRANCH_DIR, [c.cmd, c.get.sub, String(ident), '--json', ...envArgs()]), + run(MAIN_DIR, [c.cmd, c.get.sub, String(ident), '--json']), + ]); + if (b.rc === 4 || m.rc === 4) return push('GET', c.label, 'AUTH', `branch rc=${b.rc}, main rc=${m.rc}`); + if (b.rc !== 0 || m.rc !== 0) { + return push('GET', c.label, 'FAIL', `exit branch=${b.rc} main=${m.rc} ${shortErr(b.stderr)}${shortErr(m.stderr)}`); + } + const be = entityOf(parseJson(b.stdout)); + const me = entityOf(parseJson(m.stdout)); + if (!be || !me) return push('GET', c.label, 'FAIL', 'could not unwrap entity from one side'); + const r = compareEntity(c.cmd, be, me); + const unexpectedShape = STRICT + ? [...unexpectedCuration(c.cmd, 'branch', r.branchOnly), ...unexpectedCuration(c.cmd, 'main', r.mainOnly)] + : []; + for (const key of unexpectedShape) notes.push(`${C.red}FAIL${C.reset} ${c.cmd} get: unexpected one-sided key ${key}`); + for (const d of r.diffs) { + notes.push( + `${C.red}FAIL${C.reset} ${c.cmd} get .${d.key}: branch=${JSON.stringify(d.branch)} main=${JSON.stringify(d.main)}`, + ); + } + push( + 'GET', + c.label, + r.diffs.length === 0 && unexpectedShape.length === 0 ? 'PASS' : 'FAIL', + `${c.get.argFrom}=${String(ident).slice(0, 28)} — every shared field compared${r.accepted.length ? `; ${r.accepted.length} accepted` : ''}${r.diffs.length + unexpectedShape.length ? `; ${r.diffs.length + unexpectedShape.length} UNEXPECTED` : ''}`, + ); +} + +/** + * Cross-plane read-after-write. Create on one plane, read on the other, delete. + * Grammar differs by side: branch `delete --yes`, main `delete` (no flag). + */ +async function checkWriteRoundTrip(from: 'branch' | 'main'): Promise { + const label = from === 'branch' ? 'create@graphql → read@rest' : 'create@rest → read@graphql'; + const name = `parity-smoke-${Date.now()}`; + const creator = from === 'branch' ? BRANCH_DIR : MAIN_DIR; + const reader = from === 'branch' ? MAIN_DIR : BRANCH_DIR; + + const createArgs = + from === 'branch' + ? ['organization', 'create', name, '--json', ...envArgs()] + : ['organization', 'create', name, '--json']; + const c = await run(creator, createArgs); + if (c.rc !== 0) return push('WRITE', label, 'FAIL', `create failed rc=${c.rc} ${shortErr(c.stderr)}`); + const orgId = findId(parseJson(c.stdout), 'org'); + if (!orgId) return push('WRITE', label, 'FAIL', 'could not extract org id from create output'); + + try { + const readArgs = + reader === BRANCH_DIR + ? ['organization', 'get', orgId, '--json', ...envArgs()] + : ['organization', 'get', orgId, '--json']; + const r = await run(reader, readArgs); + if (r.rc !== 0) + return push('WRITE', label, 'FAIL', `read-back failed rc=${r.rc} ${shortErr(r.stderr)} (org ${orgId})`); + const ent = entityOf(parseJson(r.stdout)); + if (!ent) return push('WRITE', label, 'FAIL', `read-back returned no entity (org ${orgId})`); + if (ent.id !== orgId) return push('WRITE', label, 'FAIL', `read-back id mismatch: ${ent.id} != ${orgId}`); + if (ent.name !== name) return push('WRITE', label, 'FAIL', `read-back name mismatch: ${ent.name} != ${name}`); + push('WRITE', label, 'PASS', `${orgId} written on one plane, read identically on the other`); + } finally { + // Branch requires --yes; main takes no confirmation flag. + const delArgs = + creator === BRANCH_DIR + ? ['organization', 'delete', orgId, '--yes', '--json', ...envArgs()] + : ['organization', 'delete', orgId, '--json']; + const d = await run(creator, delArgs); + if (d.rc !== 0) { + push('CLEANUP', label, 'FAIL', `delete failed for ${orgId} (rc=${d.rc})`); + } else { + // Deletes replicate between planes asynchronously. Poll rather than + // treating a normal delay as a leak. + const absent = await eventually(async () => { + const [branchList, mainList] = await Promise.all([ + run(BRANCH_DIR, ['organization', 'list', '--json', ...envArgs()]), + run(MAIN_DIR, ['organization', 'list', '--json']), + ]); + return [branchList, mainList].every( + (result) => result.rc === 0 && !itemsOf(parseJson(result.stdout)).some((org) => org.id === orgId), + ); + }); + push( + 'CLEANUP', + label, + absent ? 'PASS' : 'FAIL', + absent ? `${orgId} absent on both planes` : `${orgId} still exists or could not be verified after 15s`, + ); + } + } +} + +/** + * Create disposable fixtures so the list/get checks run against real rows + * instead of comparing empty-vs-empty. Seeds via the BRANCH (GraphQL) binary, + * which doubles as a write-path exercise: both planes then read what GraphQL + * wrote. Returns a cleanup thunk. + * + * `feature-flag` is the one command that cannot be seeded at all: there is no + * create subcommand on either side AND no POST /feature-flags REST endpoint, so + * a flag can only come into existence through the dashboard UI. + * + * `user` has no CLI create either, but POST /user_management/users exists, so it + * is seeded through `workos api`. Events need no seeding of their own: creating + * an organization and a user emits organization.created and user.created. + */ +async function seed(): Promise<() => Promise> { + const ts = Date.now(); + const cleanups: Array<() => Promise> = []; + + const pslug = `parity-seed-${ts}`; + const p = await run(BRANCH_DIR, [ + 'permission', + 'create', + '--slug', + pslug, + '--name', + `Parity Seed ${ts}`, + '--yes', + '--json', + ...envArgs(), + ]); + if (p.rc === 0) { + push('SEED', 'permission', 'PASS', pslug); + notes.push(`${C.dim}seeded permission ${pslug}${C.reset}`); + cleanups.push(async () => { + await run(BRANCH_DIR, ['permission', 'delete', pslug, '--yes', '--json', ...envArgs()]); + }); + } else { + push('SEED', 'permission', 'FAIL', `create failed rc=${p.rc} ${shortErr(p.stderr)}`); + } + + const w = await run(BRANCH_DIR, [ + 'webhook', + 'create', + '--url', + `https://example.com/parity-${ts}`, + '--events', + 'dsync.user.created', + '--json', + ...envArgs(), + ]); + const wid = findId(parseJson(w.stdout), 'we'); + if (w.rc === 0 && wid) { + push('SEED', 'webhook', 'PASS', wid); + notes.push(`${C.dim}seeded webhook ${wid}${C.reset}`); + cleanups.push(async () => { + await run(BRANCH_DIR, ['webhook', 'delete', wid, '--yes', '--json', ...envArgs()]); + }); + } else { + push('SEED', 'webhook', 'FAIL', `create failed rc=${w.rc} ${shortErr(w.stderr)}`); + } + + // No CLI `user create` on either plane, so seed over raw REST. This also + // emits a user.created event for the event check. + const email = `parity-seed-${ts}@example.com`; + const u = await run(BRANCH_DIR, [ + 'api', + '/user_management/users', + '--method', + 'POST', + '--data', + JSON.stringify({ email, password: `Parity-Seed-${ts}!aB9`, email_verified: true }), + '--yes', + ]); + const uid = findId(parseJson(u.stdout), 'user'); + if (u.rc === 0 && uid) { + push('SEED', 'user', 'PASS', uid); + notes.push(`${C.dim}seeded user ${uid}${C.reset}`); + cleanups.push(async () => { + await run(BRANCH_DIR, ['api', `/user_management/users/${uid}`, '--method', 'DELETE', '--yes']); + }); + } else { + push('SEED', 'user', 'FAIL', `create failed rc=${u.rc} ${shortErr(u.stderr)}`); + } + + if (INVITE) { + const inviteEmail = `parity-invite-${ts}@example.com`; + const inv = await run(BRANCH_DIR, [ + 'api', + '/user_management/invitations', + '--method', + 'POST', + '--data', + JSON.stringify({ email: inviteEmail }), + '--yes', + ]); + const invId = findId(parseJson(inv.stdout), 'invitation'); + if (inv.rc === 0 && invId) { + push('SEED', 'invitation', 'PASS', invId); + notes.push(`${C.dim}seeded invitation ${invId}${C.reset}`); + cleanups.push(async () => { + await run(BRANCH_DIR, ['api', `/user_management/invitations/${invId}/revoke`, '--method', 'POST', '--yes']); + // Sending an invitation also creates a pending USER record, and revoking + // the invitation does not remove it. Without this the run leaks one user + // per invocation, which then pollutes the next run's user comparison. + const found = await run(BRANCH_DIR, ['api', `/user_management/users?email=${encodeURIComponent(inviteEmail)}`]); + const orphan = findId(parseJson(found.stdout), 'user'); + if (orphan) await run(BRANCH_DIR, ['api', `/user_management/users/${orphan}`, '--method', 'DELETE', '--yes']); + }); + } else { + push('SEED', 'invitation', 'FAIL', `create failed rc=${inv.rc} ${shortErr(inv.stderr)}`); + } + } else { + notes.push(`${C.dim}invitation not seeded (pass --invite; it attempts real email delivery)${C.reset}`); + } + + await new Promise((r) => setTimeout(r, 2500)); + return async () => { + for (const c of [...cleanups].reverse()) await c(); + // A successful delete command is not enough: read the resources back and + // prove no active fixture remains. Revoked invitations intentionally stay + // as audit records, so their final state is checked rather than requiring + // the row to disappear. + const clean = await eventually(async () => { + const [permission, webhook, user, invitation] = await Promise.all([ + run(BRANCH_DIR, ['permission', 'list', '--json', ...envArgs()]), + run(BRANCH_DIR, ['webhook', 'list', '--json', ...envArgs()]), + run(BRANCH_DIR, ['user', 'list', '--json', ...envArgs()]), + run(BRANCH_DIR, ['invitation', 'list', '--json', ...envArgs()]), + ]); + const results = [permission, webhook, user, invitation]; + if (results.some((result) => result.rc !== 0 || parseJson(result.stdout) === undefined)) return false; + if ([permission, webhook, user].some((result) => result.stdout.includes(String(ts)))) return false; + const seededInvitation = itemsOf(parseJson(invitation.stdout)).find((row) => + JSON.stringify(row).includes(String(ts)), + ); + return !seededInvitation || seededInvitation.state === 'revoked'; + }); + push( + 'CLEANUP', + 'seed fixtures', + clean ? 'PASS' : 'FAIL', + clean ? `no active resource contains marker ${ts}` : `active resource containing marker ${ts} remains after 15s`, + ); + }; +} + +async function checkEventPagination(): Promise { + const base = ['event', 'list', '--events', 'organization.created,user.created', '--limit', '3', '--json']; + let firstRun: RunResult | undefined; + let first: any; + let firstRows: any[] = []; + let cursor: string | undefined; + const ready = await eventually(async () => { + firstRun = await run(BRANCH_DIR, [...base, ...envArgs()]); + if (firstRun.rc !== 0) return true; + first = parseJson(firstRun.stdout); + firstRows = itemsOf(first); + cursor = first?.pagination?.after; + return firstRows.length > 0 && Boolean(cursor); + }); + if (firstRun?.rc === 4) return push('PAGINATION', 'event', 'AUTH', 'auth required'); + if (firstRun?.rc !== 0) return push('PAGINATION', 'event', 'FAIL', `page 1 exit ${firstRun?.rc ?? -1}`); + if (!ready || !cursor) { + return push( + 'PAGINATION', + 'event', + STRICT ? 'FAIL' : 'SKIP', + `page 1 has ${firstRows.length} row(s) and ${cursor ? 'a' : 'no'} next cursor after 15s`, + ); + } + + const secondRun = await run(BRANCH_DIR, [...base, '--after', cursor, ...envArgs()]); + if (secondRun.rc !== 0) return push('PAGINATION', 'event', 'FAIL', `page 2 exit ${secondRun.rc}`); + const second = parseJson(secondRun.stdout); + const secondRows = itemsOf(second); + if (secondRows.length === 0) return push('PAGINATION', 'event', 'FAIL', 'next cursor returned an empty page'); + + const firstIds = new Set(firstRows.map((row) => row.id)); + const overlap = secondRows.filter((row) => firstIds.has(row.id)); + const oldestFirstPage = firstRows + .map((row) => row.createdAt) + .filter(Boolean) + .sort()[0]; + const newestSecondPage = secondRows + .map((row) => row.createdAt) + .filter(Boolean) + .sort() + .at(-1); + if (overlap.length) return push('PAGINATION', 'event', 'FAIL', `${overlap.length} row(s) repeated on page 2`); + if (!oldestFirstPage || !newestSecondPage || newestSecondPage > oldestFirstPage) { + return push('PAGINATION', 'event', 'FAIL', 'page 2 moves forward in time'); + } + push('PAGINATION', 'event', 'PASS', `${firstRows.length} + ${secondRows.length} distinct rows; page 2 is not newer`); +} + +async function checkNew(c: { label: string; args: string[] }): Promise { + const r = await run(BRANCH_DIR, c.args); + if (r.rc === 4) return push('NEW', c.label, 'AUTH', 'auth required'); + if (r.rc !== 0) return push('NEW', c.label, 'FAIL', `exit ${r.rc} ${shortErr(r.stderr)}`); + if (parseJson(r.stdout) === undefined) return push('NEW', c.label, 'FAIL', 'invalid JSON'); + push('NEW', c.label, 'PASS', 'valid JSON'); +} + +// --- run ------------------------------------------------------------------------- +console.log(`parity-smoke branch=${BRANCH_DIR}`); +console.log( + ` main=${MAIN_DIR}${ENV_ID ? ` env=${ENV_ID}` : ''}${MUTATE ? ' [--mutate]' : ''}${SEED ? ' [--seed]' : ''}${INVITE ? ' [--invite]' : ''}${STRICT ? ' [--strict]' : ''}`, +); +console.log(` cwd=${NEUTRAL_CWD} (isolated so no worktree .env.local loads)`); +console.log(` api key source: ${API_KEY_SOURCE}\n`); +if (API_KEY_SOURCE === 'NONE') { + console.log( + `${C.yellow}no WORKOS_API_KEY resolved; REST-plane commands will fail or fall back to stored config${C.reset}\n`, + ); + if (STRICT) process.exit(2); +} + +await Promise.all(CONTROL.map(checkControl)); + +// Preflight: prove both planes are looking at the SAME environment before any +// comparison is believed. Disjoint non-empty organization sets mean different +// environments, and every downstream row would be a meaningless comparison. +{ + const [b, m] = await Promise.all([ + run(BRANCH_DIR, ['organization', 'list', '--json', ...envArgs()]), + run(MAIN_DIR, ['organization', 'list', '--json']), + ]); + if (b.rc === 4 || m.rc === 4) { + push('PREFLIGHT', 'same-environment', 'AUTH', `branch rc=${b.rc}, main rc=${m.rc}`); + } else if (b.rc !== 0 || m.rc !== 0) { + push('PREFLIGHT', 'same-environment', 'FAIL', `branch rc=${b.rc}, main rc=${m.rc}`); + } else { + const parsedBranch = parseJson(b.stdout); + const parsedMain = parseJson(m.stdout); + if (parsedBranch === undefined || parsedMain === undefined) { + push('PREFLIGHT', 'same-environment', 'FAIL', 'invalid organization JSON on one side'); + } else { + const bi = itemsOf(parsedBranch); + const mi = itemsOf(parsedMain); + const bIds = new Set(bi.map((i: any) => i?.id)); + const mIds = new Set(mi.map((i: any) => i?.id)); + const sameIds = bIds.size === mIds.size && [...bIds].every((id) => mIds.has(id)); + if (bi.length === 0 || mi.length === 0) { + push( + 'PREFLIGHT', + 'same-environment', + 'SKIP', + `cannot confirm alignment: branch has ${bi.length} org(s), main has ${mi.length}.`, + ); + } else if (!sameIds) { + push( + 'PREFLIGHT', + 'same-environment', + 'FAIL', + `organization sets differ (branch ${bi.length}, main ${mi.length}); comparisons would be meaningless`, + ); + } else { + push('PREFLIGHT', 'same-environment', 'PASS', `${bIds.size} identical organization id(s) on both planes`); + } + } + } + const preflight = rows.find((row) => row.kind === 'PREFLIGHT'); + if (preflight?.status !== 'PASS') { + console.log(`${C.red}aborting: could not prove both planes use the same environment${C.reset}`); + for (const r of rows) console.log(` ${r.status} ${r.kind} ${r.label} ${r.detail}`); + process.exit(1); + } +} + +const unseed = SEED ? await seed() : undefined; +const listItems = new Map(); +await Promise.all( + LISTS.map(async (c) => { + listItems.set(c.cmd, await checkList(c)); + }), +); +await Promise.all(LISTS.map((c) => checkGet(c, listItems.get(c.cmd)))); +await Promise.all(NEW_ONLY.map(checkNew)); +if (MUTATE) { + await checkWriteRoundTrip('branch'); + await checkWriteRoundTrip('main'); +} else { + push('WRITE', 'cross-plane round-trip', 'SKIP', 're-run with --mutate to exercise the write path'); +} +await checkEventPagination(); +if (unseed) await unseed(); + +// --- report ---------------------------------------------------------------------- +const w = Math.max(...rows.map((r) => r.label.length), 20); +const order = ['PREFLIGHT', 'SEED', 'CONTROL', 'LIST', 'GET', 'PAGINATION', 'WRITE', 'NEW', 'CLEANUP']; +for (const kind of order) { + const group = rows.filter((r) => r.kind === kind); + if (!group.length) continue; + console.log(`${C.dim}${kind}${C.reset}`); + for (const r of group) { + const color = + r.status === 'PASS' + ? C.green + : r.status === 'FAIL' + ? C.red + : r.status === 'AUTH' + ? C.yellow + : r.status === 'INFO' + ? C.cyan + : C.dim; + console.log(` ${color}${r.status.padEnd(4)}${C.reset} ${r.label.padEnd(w)} ${C.dim}${r.detail}${C.reset}`); + } +} +if (notes.length) { + console.log(`\n${C.dim}field-level detail${C.reset}`); + for (const n of notes) console.log(` ${n}`); +} + +const n = (s: Status) => rows.filter((r) => r.status === s).length; +const nfail = n('FAIL'); +const allowedStrictSkip = (row: Row) => + (row.kind === 'GET' && + ['webhook', 'event'].includes(row.label) && + row.detail === 'no get subcommand on either side') || + (row.kind === 'LIST' && row.label === 'feature-flag' && row.detail.startsWith('both sides empty')) || + (row.kind === 'GET' && row.label === 'feature-flag' && row.detail === 'no row available to source an identifier'); +const unexpectedSkips = STRICT ? rows.filter((row) => row.status === 'SKIP' && !allowedStrictSkip(row)) : []; +const blocked = nfail > 0 || (STRICT && (n('AUTH') > 0 || unexpectedSkips.length > 0)); +if (unexpectedSkips.length) { + console.log(`${C.red}strict: ${unexpectedSkips.length} unexpected skip(s):${C.reset}`); + for (const row of unexpectedSkips) console.log(` ${row.kind} ${row.label}: ${row.detail}`); +} +console.log( + `\n${blocked ? C.red : C.green}${n('PASS')} pass, ${nfail} fail, ${n('AUTH')} auth, ${n('SKIP')} skip${STRICT ? ` (${unexpectedSkips.length} unexpected)` : ''}${C.reset}`, +); +process.exit(blocked ? 1 : 0); diff --git a/scripts/smoke-dashboard-plane.sh b/scripts/smoke-dashboard-plane.sh new file mode 100755 index 00000000..6ce8f2f0 --- /dev/null +++ b/scripts/smoke-dashboard-plane.sh @@ -0,0 +1,464 @@ +#!/usr/bin/env bash +# Live smoke test for the dashboard-plane resource commands (organization, user, +# role, permission, membership, invitation, session, event, feature-flag, +# org-domain, portal, webhook, config) against the real WorkOS API. +# +# Prereq: `workos auth login` (device flow) with the environment you want to +# target set as the active env. Then: +# +# ./scripts/smoke-dashboard-plane.sh # read-only (safe) +# ./scripts/smoke-dashboard-plane.sh --mutate # + CRUD round-trips in a disposable org +# ./scripts/smoke-dashboard-plane.sh --config-writes # + config redirect/cors add (snapshot & restore) +# ./scripts/smoke-dashboard-plane.sh --branding-writes # + branding image upload (multipart; see below) +# ./scripts/smoke-dashboard-plane.sh --keep # don't clean up created resources +# +# --branding-writes is the only tier that uploads files. It exercises the +# GraphQL multipart transport, which is what an MCP client cannot do. It +# REPLACES the environment's real logo/icon/favicon with generated test images +# and CANNOT restore them: the API returns asset paths, not the original bytes, +# and there is no re-upload-from-URL operation. Run it on a sandbox environment +# you do not mind re-branding. +# +# Env overrides: +# WORKOS_BIN command to invoke the CLI (default: node /dist/bin.js) +# SMOKE_EVENT_TYPES comma-separated event types for `event list` (default: user.created) +# SMOKE_BRANDING_ENV_ID environment for --branding-writes (default: the active one). +# Point this at a throwaway environment so the tier +# never rebrands anything you care about. Creating one: +# `workos project create scratch --yes` gives a fresh +# project whose environments start with no branding. +# +# Exit code: 0 if every executed test passed, 1 otherwise. +set -u + +REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +MUTATE=0 CONFIG_WRITES=0 BRANDING_WRITES=0 KEEP=0 +for a in "$@"; do + case "$a" in + --mutate) MUTATE=1 ;; + --config-writes) CONFIG_WRITES=1 ;; + --branding-writes) BRANDING_WRITES=1 ;; + --keep) KEEP=1 ;; + -h|--help) sed -n '2,26p' "${BASH_SOURCE[0]}"; exit 0 ;; + *) echo "unknown flag: $a (see --help)"; exit 2 ;; + esac +done + +if [ -n "${WORKOS_BIN:-}" ]; then + # shellcheck disable=SC2206 + BIN=($WORKOS_BIN) +else + [ -f "$REPO/dist/bin.js" ] || { echo "dist/bin.js not found — run: pnpm build"; exit 1; } + BIN=(node "$REPO/dist/bin.js") +fi + +# The migrated commands must run on the OAuth token alone; surface any REST +# leftovers by removing the API-key plane from the environment entirely. +if [ -n "${WORKOS_API_KEY:-}" ]; then + echo "note: WORKOS_API_KEY is set — unsetting it for this run (migrated commands must not need it)" + unset WORKOS_API_KEY +fi +unset WORKOS_FORCE_TTY 2>/dev/null || true + +WORK="$(mktemp -d "${TMPDIR:-/tmp}/workos-smoke.XXXXXX")" +c_g=$'\033[32m' c_r=$'\033[31m' c_y=$'\033[33m' c_d=$'\033[2m' c_0=$'\033[0m' +N=0 NPASS=0 NFAIL=0 NSKIP=0 +FAILURES=() +LAST="" + +say() { printf '\n%s\n' "$1"; } + +# t