Skip to content

migrate-dataform: the SAFE_CAST rewrite silently removes safety #55

Description

@ihistand

Filed separately from #53 because this is a bug producing wrong output for anyone migrating today, and it should be fixable and releasable without waiting on the wider rules work.

{ pattern: /\bSAFE_CAST\s*\(/g, replacement: "CAST(", kind: "rewrite",
  note: "SAFE_CAST → CAST — Postgres CAST raises on bad input; verify the data or guard it" }

SAFE_CAST returns NULL where a value will not convert. Rewriting it to CAST does not preserve that — it converts a silent NULL into a hard failure, and the note asking the user to "verify the data" is asking them to audit every row of every source.

What it looks like in practice: the run fails hours later, in a different file from the one being read, with

invalid input syntax for type bigint: "0750069967 "

and the SQL that failed is code the converter itself wrote. In acuantia the NULL-on-bad-input behaviour was deliberate and load-bearing — the sources carry Excel error strings (#DIV/0!, #N/A), SAP identifiers with stray characters, and non-numeric HubSpot team ids.

Fix

PostgreSQL 16 added pg_input_is_valid(text, type), which asks exactly the question SAFE_CAST asks:

SAFE_CAST(x AS t)  ->  case when pg_input_is_valid(x::text, 't') then x::t end

Guard on the text form, cast x itself so an already-numeric value converts normally. This is exact rather than approximate, so it belongs as a rewrite, not a flag.

Two things worth carrying into the implementation:

  • Test it against a column, not a literal. PostgreSQL constant-folds 'abc'::bigint at plan time, so the pattern looks broken when tested on constants and works correctly on real data. That will waste someone's afternoon otherwise.
  • One divergence to document: for a fractional numeric cast to an integer type, BigQuery's SAFE_CAST rounds where this yields NULL. Text-sourced values — the reason SAFE_CAST is used — are unaffected.

Requires PostgreSQL 16+. For older targets the fallback is a per-type guard function, or keeping the flag.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions