refactor(onboarding): Encapsulate the quickstart experiment in one module by khvn26 · Pull Request #8263 · Flagsmith/flagsmith

khvn26 · 2026-08-11T12:05:23Z

Thanks for submitting a PR! Please check the boxes below:

I have read the Contributing Guide.
I have added information to docs/ if required so people know about the feature.
I have filled in the "Changes" section below.
I have filled in the "How did you test this code" section below.

Changes

Follow-up to #8242, per the review suggestion to encapsulate experiments for easy cleanup.

All frontend code the onboarding quickstart experiment touches now lives in common/experiments/onboardingQuickstart.ts. The hook-proof consumers import plain functions from the same module.

Cleaning up the experiment should be much easier.

How did you test this code?

Moved and extended the unit tests.

The onboarding flow decision is now made in the backend, per organisation. Serving a multivariate split assignment records the experiment exposure; the first SDK evaluation of any of the organisation's environments records the conversion, both against the organisation targeting key. - flagsmith 6.2.0 and openfeature-provider-flagsmith from its experimentation-tracking PR branch - events enabled on the Flagsmith-on-Flagsmith client via FLAGSMITH_ON_FLAGSMITH_SERVER_EVENTS_API_URL (set in staging and production ECS task definitions; never enabled in offline mode) - get_openfeature_client() attaches the provider's exposure hook, so any multivariate split evaluation records an exposure - onboarding_variant field on the organisation serialiser beep boop

The gate no longer evaluates the Flagsmith-on-Flagsmith flag client-side; the backend decides per organisation and the exposure is recorded server-side. beep boop

Also refreshes a stale line reference in the generated events catalogue. beep boop

The frontend decides the onboarding variant before the organisation exists, evaluating with an anonymous identity. Storing that identity's identifier as the organisation's targeting key keeps bucketing and experiment events on one identifier from the first anonymous evaluation onward. Write-only on create and immutable after; org.<id> when unset. beep boop

The entry decision is made under a server-assigned anonymous identity before the organisation exists: the frontend identifies with an empty identifier, reads the flag under the returned one (recording the exposure), and stores identifier and variant. The identifier becomes the organisation's targeting key at creation in both arms, and the stored variant drives routing, the chromeless layout and the gate. The backend no longer serves onboarding_variant: with one Flagsmith-on-Flagsmith project across frontend and API, the flag decision needs no API surface. beep boop

…xperiment-events

When the routing timeout won the race, the late decision still wrote its variant and targeting key — read from whatever identity the SDK held by then, possibly the logged-in user. Decisions are now persisted only when they win the race. beep boop

The render check already treats a missing decision as the legacy page, so the defaulted copy was redundant. beep boop

…ation The stored targeting key survived invite acceptance and keyless creation paths, so a user's first organisation created long after the entry decision could consume it. Any organisation membership now clears the key. beep boop

…dule All frontend code the experiment touches now lives in common/experiments/onboardingQuickstart.ts, including a useOnboardingQuickstart hook that owns the gate's variant read and analytics tagging. Cleaning up the experiment is deleting the module and unpicking its importers. beep boop

vercel · 2026-08-11T12:05:25Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
flagsmith-frontend-preview	Ready	Preview	Aug 11, 2026 12:06pm
flagsmith-frontend-staging	Ready	Preview	Aug 11, 2026 12:06pm

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Ignored	Preview	Aug 11, 2026 12:06pm

coderabbitai · 2026-08-11T12:05:35Z

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 2bd597c4-2ae6-4a68-a538-978696684e2c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Zaimwa9 · 2026-08-11T12:48:04Z

@themis-blindfold review

themis-blindfold · 2026-08-11T12:56:43Z

⚖️ Themis review: ✅ Ship it

Clean encapsulation of the onboarding quickstart experiment into a single frontend module (onboardingQuickstart.ts), with backend plumbing to pin experiment bucketing via Organisation.targeting_key and track environment.first_evaluated conversion events through OpenFeature. The race/timeout pattern in App.js is well-designed: the anonymous-identity decision is never persisted unless it wins the race, and a missing identifier correctly downgrades to control. The serializer enforces immutability on update, and the field is write-only. The dependency bumps (flagsmith 5→6, openfeature-provider-flagsmith 0.2→1.0) are major version jumps, but CI passed cleanly including unit tests and linting.

Area	Score
🎯 Correctness	5/5
🧪 Test coverage	5/5
📐 Code quality	5/5
🚀 Product impact	3/5

📝 Walkthrough

Backend targeting key — Organisation.targeting_key (nullable, immutable after create) pins experiment bucketing to the anonymous identifier assigned during the frontend entry decision. Migration is schema-only.
Conversion tracking — record_environment_first_evaluation now fires an OpenFeature track() call with the organisation's targeting key. Events flow only when FLAGSMITH_ON_FLAGSMITH_SERVER_EVENTS_API_URL is configured; offline mode and unconfigured deployments are unaffected.
Flagsmith SDK upgrade — flagsmith 5.3→6.2 and openfeature-provider-flagsmith 0.2→1.0 bring EventProcessorConfig and FlagsmithExposureHook, both used here.
Frontend encapsulation — All experiment logic moved from getOnboardingVariant.ts (deleted) and scattered call sites into common/experiments/onboardingQuickstart.ts. The entry decision, persistence, stored-value access, cleanup, and the React hook all live in one module.
Entry decision redesign — App.js now calls decideOnboardingEntry() under a server-assigned anonymous identity (rather than the logged-in user), and stores the identifier as the org's targeting_key at creation, so bucketing is stable.
Infrastructure — Events API URL added to all six ECS task definitions (staging + production × admin-api, sdk-api, task-processor).

🧪 How to verify

On staging, sign up as a new user without providing an org name; confirm the entry decision routes to /getting-started or /create within 2 seconds and the onboarding_variant localStorage key is set.
Create an organisation from the single-page flow; verify the POST to /api/v1/organisations/ includes targeting_key and the response does not echo it back.
Attempt a PATCH/PUT to the organisation with a different targeting_key; confirm it is silently ignored and the stored value is unchanged.
Trigger a first SDK evaluation against the new org's environment; confirm the environment.first_evaluated event appears in the events API with the correct targeting key.
Run pytest api/tests/unit/environments/onboarding/ api/tests/unit/integrations/flagsmith/ api/tests/unit/organisations/test_unit_organisations_models.py api/tests/unit/organisations/test_unit_organisations_serializers.py and yarn test --testPathPattern onboardingQuickstart locally.

Automate: steps 4–5 are already covered by CI; step 2 could be a lightweight integration test asserting the response body omits targeting_key.

Product take: This is plumbing for experiment observability, not a user-facing change. It makes the quickstart experiment measurable (conversion events tied to bucketing) and makes future cleanup straightforward (delete one module). Solid infrastructure work.

🧭 Assumptions & unverified claims

The FlagsmithExposureHook from openfeature-provider-flagsmith 1.0.0 is assumed to be non-throwing on hook execution; the provider package is not installed in the review environment, so this could not be verified against source. The AbstractProvider.track() base implementation is a no-op, confirmed in the installed OpenFeature SDK.
The server-assigned anonymous identifier from flagsmith.identify('') is assumed to fit within the 64-character max_length of Organisation.targeting_key. If the Flagsmith API ever returns a longer identifier, the Django model validation would reject it at org creation time — no silent truncation, but an unexpected 400.

An experiment module that's born ready for its own funeral — respect. · reviewed at 9b16eb5

khvn26 and others added 14 commits August 7, 2026 17:06

feat(onboarding): Read onboarding variant from the organisation API

1b4d68c

The gate no longer evaluates the Flagsmith-on-Flagsmith flag client-side; the backend decides per organisation and the exposure is recorded server-side. beep boop

chore: Update documentation artefacts

015d7fd

chore: Update documentation artefacts

1935afa

build(deps): openfeature-provider-flagsmith 1.0.0 from PyPI

c3944b1

Also refreshes a stale line reference in the generated events catalogue. beep boop

improve test_initialise_provider tests

9d61fbe

Merge remote-tracking branch 'origin/main' into feat/onboarding-org-e…

745dda5

…xperiment-events

chore: Update documentation artefacts

4169bca

refactor(onboarding): Merge the gate's variant variables

a295c74

The render check already treats a missing decision as the legacy page, so the defaulted copy was redundant. beep boop

github-actions Bot added the front-end Issue related to the React Front End Dashboard label Aug 11, 2026

vercel Bot deployed to Preview – flagsmith-frontend-staging August 11, 2026 12:06 View deployment

vercel Bot deployed to Preview – flagsmith-frontend-preview August 11, 2026 12:06 View deployment

github-actions Bot added the refactor label Aug 11, 2026

Base automatically changed from feat/onboarding-org-experiment-events to main August 11, 2026 12:08

Navigation Menu

Uh oh!

refactor(onboarding): Encapsulate the quickstart experiment in one module - #8263

refactor(onboarding): Encapsulate the quickstart experiment in one module#8263
khvn26 wants to merge 14 commits into
mainfrom
refactor/onboarding-experiment-hook

khvn26 commented Aug 11, 2026

Uh oh!

vercel Bot commented Aug 11, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Aug 11, 2026

Review skipped

Uh oh!

Zaimwa9 commented Aug 11, 2026

Uh oh!

themis-blindfold Bot commented Aug 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

khvn26 commented Aug 11, 2026

Changes

How did you test this code?

Uh oh!

vercel Bot commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Aug 11, 2026

Review skipped

Uh oh!

Zaimwa9 commented Aug 11, 2026

Uh oh!

themis-blindfold Bot commented Aug 11, 2026

⚖️ Themis review: ✅ Ship it

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vercel Bot commented Aug 11, 2026 •

edited

Loading