Skip to content

Stream large database exports#203

Open
Gimyoonsoo wants to merge 2 commits into
outerbase:mainfrom
Gimyoonsoo:algora-59-stream-exports
Open

Stream large database exports#203
Gimyoonsoo wants to merge 2 commits into
outerbase:mainfrom
Gimyoonsoo:algora-59-stream-exports

Conversation

@Gimyoonsoo
Copy link
Copy Markdown

Summary

Changes

  • Add shared streaming export helpers and batched table reads.
  • Update database dump export to stream table schemas and row inserts.
  • Update CSV and JSON table exports to stream batches while preserving file download headers.
  • Add focused tests for streamed dump, CSV, JSON, batched reads, and stream responses.

Testing

  • .\node_modules.bin\vitest.exe run src/export
  • Result: passed, 31 passed / 0 failed
  • .\node_modules.bin\prettier.exe --check src/export/index.ts src/export/dump.ts src/export/csv.ts src/export/json.ts src/export/index.test.ts src/export/dump.test.ts src/export/csv.test.ts src/export/json.test.ts
  • Result: passed
  • git diff --check
  • Result: passed, with line-ending warnings only
  • .\node_modules.bin\vite.exe build --mode client; if ($LASTEXITCODE -eq 0) { .\node_modules.bin\vite.exe build }
  • Result: passed; build printed existing non-fatal Failed to find Response internal state key warnings
  • .\node_modules.bin\tsc.exe --noEmit --pretty false
  • Result: failed because of existing unrelated errors in plugins/cdc, src/cache, src/do, and src/operation; no errors pointed to the modified export files.

Notes

  • No authentication, payment, database, deployment, or security-sensitive configuration changes were made.
  • No secrets, credentials, production data, or security-sensitive information were accessed.

Algora

/claim #59

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant