Skip to content

perf: Denormalise feature names on Code Reference scans and add GIN index#6844

Open
matthewelwell wants to merge 6 commits intomainfrom
perf/further-feature-list-improvements
Open

perf: Denormalise feature names on Code Reference scans and add GIN index#6844
matthewelwell wants to merge 6 commits intomainfrom
perf/further-feature-list-improvements

Conversation

@matthewelwell
Copy link
Contributor

Changes

Closes #6836

This PR denormalises the feature names directly into an ArrayField on the FeatureFlagCodeReferencesScan. The feature names were previously nested inside the JSONField which caused performance issues with the database queries when listing features for the dashboard.

How did you test this code?

Added tests.

@matthewelwell matthewelwell requested a review from a team as a code owner March 4, 2026 17:47
@matthewelwell matthewelwell requested review from gagantrivedi and removed request for a team March 4, 2026 17:47
@vercel
Copy link

vercel bot commented Mar 4, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated (UTC)
docs Ignored Ignored Preview Mar 4, 2026 6:09pm
flagsmith-frontend-preview Ignored Ignored Preview Mar 4, 2026 6:09pm
flagsmith-frontend-staging Ignored Ignored Preview Mar 4, 2026 6:09pm

Request Review

@github-actions github-actions bot added api Issue related to the REST API performance labels Mar 4, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Mar 4, 2026

Docker builds report

Image Build Status Security report
ghcr.io/flagsmith/flagsmith-e2e:pr-6844 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-api-test:pr-6844 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-frontend:pr-6844 Finished ✅ Results
ghcr.io/flagsmith/flagsmith-api:pr-6844 Finished ✅ Results
ghcr.io/flagsmith/flagsmith:pr-6844 Finished ✅ Results
ghcr.io/flagsmith/flagsmith-private-cloud:pr-6844 Finished ✅ Results

@matthewelwell matthewelwell marked this pull request as draft March 4, 2026 17:49
@matthewelwell matthewelwell marked this pull request as ready for review March 4, 2026 18:12
@codecov
Copy link

codecov bot commented Mar 4, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.34%. Comparing base (800ac8d) to head (2b03644).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6844      +/-   ##
==========================================
+ Coverage   98.32%   98.34%   +0.01%     
==========================================
  Files        1332     1342      +10     
  Lines       49401    49723     +322     
==========================================
+ Hits        48575    48898     +323     
+ Misses        826      825       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

operations = [
migrations.AddField(
model_name="featureflagcodereferencesscan",
field=ArrayField(models.TextField(), default=list),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly sure there are other fields in this app that don't support oracle, but we probably need to think on how to exclude this from EE.

@gagantrivedi
Copy link
Member

The PR migrates the last_feature_found_at subquery and the get_code_references_for_feature_flag function to use the new feature_names array, but the count calculation in
  counts_by_repository (the heavier operation) still does a full jsonb_path_query_array + jsonb_array_length against the raw code_references JSON blob:

  count=Func(
      Func(
          F("code_references"),
          Value("$[*] ? (@.feature_name == $feature_name)"),
          JSONObject(feature_name=F("feature_name")),
          function="jsonb_path_query_array",
      ),
      function="jsonb_array_length",
  ),

  This generates SQL like:

  jsonb_array_length(
    jsonb_path_query_array(
      V0."code_references",
      '$[*] ? (@.feature_name == $feature_name)',
      JSONB_BUILD_OBJECT('feature_name', "features_feature"."name")
    )
  )

The only solution I see here is to accept that the data model doesn't work for how we query it, and change it to flatten everything and get rid of the JSON column. Otherwise, we'd keep going in circles making the data access layer more complicated than it needs to be.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Issue related to the REST API performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve query performance when listing features that have code references

2 participants