Commit cacca8f
committed
core/seed: stream dialects from JSONL, move the standard libraries into it
A dialect is now a directory of JSONL files rather than one JSON document:
dialect.jsonl names it and carries its rules, and types, operators, casts
and functions each get a file of one record per line. Records are applied
as they are read, so a function list running to thousands of entries is
never held in memory as a whole, and a dialect only writes the files it
needs.
With a record-per-line format the engines' standard libraries fit in it,
so they move out of Go:
- MySQL and SQLite: stdlib.go held the function list as Go literals. It is
now functions.jsonl, and defaultSchema reads it. 6.6k lines of Go gone.
- PostgreSQL: pg_catalog.go carried ~2700 functions ahead of the system
tables. Those move to the dialect directory too, leaving the generated
file to the tables alone — 30k lines of Go gone — and sqlc-pg-gen learns
to write the JSONL, so a regeneration produces the same layout.
Each list is read once per process and feeds both the analysis core and the
catalog the legacy compiler builds, so there is one copy of the data rather
than one per consumer. The binary drops about 3 MB.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011MnoUabwBWW9gaEn2Nj7eG1 parent 34b38c8 commit cacca8f
35 files changed
Lines changed: 3961 additions & 37366 deletions
File tree
- internal
- core/seed
- engine
- clickhouse
- dialect
- dolphin
- dialect
- googlesql
- dialect
- postgresql
- dialect
- sqlite
- dialect
- tools/sqlc-pg-gen
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
196 | 196 | | |
197 | 197 | | |
198 | 198 | | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
199 | 202 | | |
200 | 203 | | |
201 | 204 | | |
| |||
0 commit comments