Skip to content

migrate-dataform: make the migration report class-first, and grade findings by severity #56

Description

@ihistand

Correction to the original framing. migration-report.json and migration-report.md already exist and are already machine-readable. This is not a new deliverable; it is a restructuring of one. What follows is the delta.

What exists

Per-file findings with line numbers, counts of clean / rewritten / flagged, warnings, and a "finish the migration" section. Every flagged location is also marked inline with a SQLANVIL-MIGRATE: comment.

What is missing, and why it matters

Grouping is by file; triage happens by class. Porting acuantia, the question that drove every decision was never "what is wrong with this file" — it was "how many sites does this construct have, and is that worth doing or descoping". A file-first report cannot answer that without the reader aggregating it by hand. Class-first, sorted by count, with the file list nested underneath, is the same data arranged for the decision actually being made.

That framing is what led to deleting four features outright rather than porting them, which was the single largest reduction in remaining work. Nobody makes that call from a per-file list.

No severity. These are three different problems and the report presents them identically:

  • fails at compile — cheap, found immediately
  • fails at run time — expensive, found late, and only if that action is reached; in acuantia the real tail was repeatedly ~5× what the failure list showed, because failures hide behind other failures
  • succeeds but means something different — the dangerous one. at time zone applied to a bare timestamp converts the wrong way and never errors. SPLIT(x, '') translated literally returns one element instead of characters. Neither raises.

The third class deserves to be loudest, and today it is indistinguishable from the first.

No recommended treatment per class. Each flag carries a note, but the notes are hints ("consider jsonb or a composite type") rather than the worked answer. Once a class has a known-good port — and #53, #54, #57 and #58 are exactly that — the report should say so and point at it.

No audit of what was rewritten. The report counts rewrites but does not group them by rule, so there is no way to review the automatic changes without diffing the tree. Several rewrites in the acuantia port were subtly wrong on the first attempt and were caught only by reading them. A reviewer needs to see "SAFE_DIVIDE: 15 sites, here they are".

Shape of the change

Group findings by rule/class; per class emit count, severity, the BigQuery construct, the recommended PostgreSQL treatment, whether it was rewritten or flagged, and the file/line list. Sort by count. Keep the per-file view as a secondary section — it is the right shape once you are actually editing.

Both outputs stay: JSON for an agent working class-by-class largest-first and re-running to measure progress, Markdown for a human doing the same with less ceremony.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions