Skip to content

Conversation

@adrianviquez
Copy link
Contributor

@adrianviquez adrianviquez commented Jan 9, 2026

The data export is going to need some SQL at the end of the day, as we can't just do a sql dump since we're doing specific models + data. You have to manually create that dump file by specifying your requirements, including which models you want to export data of, which fields within those models to exclude, etc.

https://linear.app/getsentry/issue/CCMRG-1994/create-sql-generator-file

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.


Note

Introduces a new sql_generator.py to produce additive, batched PostgreSQL INSERT ... ON CONFLICT statements for owner-scoped data exports.

  • ExportContext prefetches hierarchy IDs and applies ownership/date filtering (with full exports for select models and hierarchy-based filtering for others)
  • generate_upsert_sql builds batched UPSERTs per model using BATCH_SIZE, honoring composite conflict keys (e.g., core.Branch, timeseries.Measurement)
  • Robust serialize_value handles enums, JSON, UUIDs, timestamps, bytea, and float/Decimal edge cases (NaN/Infinity)
  • generate_full_export writes a complete SQL script with header, per-model batches, and BEGIN/COMMIT
  • generate_sequence_resets appends sequence reset statements for auto-increment PKs; generate_timescale_export targets TIMESCALE_MODELS

Written by Cursor Bugbot for commit bdd8028. This will update automatically on new commits. Configure here.

@adrianviquez adrianviquez requested a review from a team January 9, 2026 00:54
@sentry
Copy link

sentry bot commented Jan 9, 2026

Codecov Report

❌ Patch coverage is 0% with 211 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.36%. Comparing base (270978f) to head (bdd8028).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...s/shared/shared/owner_data_export/sql_generator.py 0.00% 211 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #641      +/-   ##
==========================================
- Coverage   93.79%   93.36%   -0.44%     
==========================================
  Files        1291     1292       +1     
  Lines       46878    47119     +241     
  Branches     1517     1567      +50     
==========================================
+ Hits        43971    43992      +21     
- Misses       2598     2818     +220     
  Partials      309      309              
Flag Coverage Δ
apiunit 96.56% <ø> (+<0.01%) ⬆️
sharedintegration 38.08% <0.00%> (-0.59%) ⬇️
sharedunit 87.12% <0.00%> (-1.34%) ⬇️
workerintegration 59.14% <ø> (-0.03%) ⬇️
workerunit 91.26% <ø> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@codecov-notifications
Copy link

codecov-notifications bot commented Jan 9, 2026

Codecov Report

❌ Patch coverage is 0% with 211 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...s/shared/shared/owner_data_export/sql_generator.py 0.00% 211 Missing ⚠️

📢 Thoughts on this report? Let us know!

@codspeed-hq
Copy link

codspeed-hq bot commented Jan 9, 2026

Merging this PR will not alter performance

✅ 9 untouched benchmarks


Comparing adrian/export-sql-generator (bdd8028) with main (270978f)1

Open in CodSpeed

Footnotes

  1. No successful run was found on main (1aa11d2) during the generation of this report, so 270978f was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants