fix(snowflake): stop FK collision across tables sharing a constraint name#30473
fix(snowflake): stop FK collision across tables sharing a constraint name#30473ulixius9 wants to merge 2 commits into
Conversation
… constraint names Snowflake constraint names are unique per table, not per schema. Cloned tables (CREATE TABLE ... CLONE / LIKE) copy the constraint name, so SHOW IMPORTED KEYS IN SCHEMA can return the same fk_name for several tables. get_schema_foreign_keys grouped rows by fk_name alone, merging distinct tables' columns into one constraint filed under whichever table appeared first. That produced foreign-key constraints referencing columns from another table (e.g. CUSTOMER acquiring SUPPLIER's S_NATIONKEY), which the server rejected with 400 "Invalid column name", skipping the table patch entirely. Key the grouping on (fk_name, table_name) so same-named constraints on different tables stay separate. Genuine composite FKs on a single table still group. Validated against live Snowflake: reproduced the leak with the old code and confirmed correct per-table reflection with the fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
❌ PR checklist incompleteThis PR cannot be merged until the following are addressed on its linked issue:
The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically. Maintainers can bypass this check by adding the |
| from unittest import TestCase | ||
| from unittest.mock import Mock | ||
|
|
||
| from snowflake.sqlalchemy.snowdialect import SnowflakeDialect | ||
|
|
||
| from metadata.ingestion.source.database.snowflake.utils import ( | ||
| get_schema_foreign_keys, | ||
| ) | ||
|
|
||
|
|
||
| def _fk_row( | ||
| fk_name, | ||
| fk_table_name, | ||
| fk_column_name, | ||
| pk_table_name, |
There was a problem hiding this comment.
💡 Quality: New test inherits from unittest.TestCase
The new test module test_snowflake_foreign_key_collision.py defines SnowflakeForeignKeyCollisionTest(TestCase) and uses setUp, which violates the project's testing convention to use pytest with plain assert, fixtures, and unittest.mock (no TestCase inheritance). It already uses plain assert, so convert it to a module-level pytest style: replace the class/setUp with a @pytest.fixture that builds the dialect and mock connection, and make the tests plain functions accepting that fixture.
Was this helpful? React with 👍 / 👎
|
|
||
| class SnowflakeForeignKeyCollisionTest(TestCase): |
There was a problem hiding this comment.
The new test class inherits from unittest.TestCase and uses setUp, contrary to the repository's pytest fixture convention. This introduces a second lifecycle style and makes fixture composition and parametrization harder when extending these regression cases.
Context Used: CLAUDE.md (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
🔴 Playwright Results — workflow failedValidated commit ✅ 104 passed · ❌ 2 failed · 🟡 1 flaky · ⏭️ 3 skipped · 🧰 0 lifecycle flaky PerformanceBlocking targets: ❌ unmet · Optimization targets: 🟡 in progress Shard-job maxima below are not the full workflow wall time; the linked run includes build, fixture, planning, and reporting. 🕒 Full workflow signal wall (to summary) 52m 47s ⏱️ Max setup 3m 3s · max shard execution 14m 28s · max shard-job elapsed before upload 19m 42s · reporting 7s 🌐 204.72 requests/attempt · 1.79 app boots/UI scenario · 0.00% common-shard skew Optimization targets still in progress:
Genuine Failures (failed on all attempts)❌
|
Code Review 👍 Approved with suggestions 0 resolved / 1 findingsUpdates the Snowflake SQLAlchemy dialect to group foreign keys by table name and constraint name, preventing cross-table collisions for cloned tables with shared names. Consider modernizing the new test module to avoid inheriting directly from unittest.TestCase. 💡 Quality: New test inherits from unittest.TestCase📄 ingestion/tests/unit/topology/database/test_snowflake_foreign_key_collision.py:22-36 The new test module 🤖 Prompt for agentsOptionsDisplay: compact → Showing less information. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Gitar |
Describe your changes:
Fixes #
I fixed Snowflake foreign-key reflection so constraints no longer collide across tables that share a constraint name. Snowflake constraint names are unique per table (not per schema), and cloned tables (
CREATE TABLE ... CLONE / LIKE) copy the name, soSHOW IMPORTED KEYS IN SCHEMAcan return the samefk_namefor several tables.get_schema_foreign_keysgrouped rows byfk_namealone, merging distinct tables' columns into one constraint filed under whichever table appeared first — producing FK constraints that reference another table's columns (e.g.CUSTOMERacquiringSUPPLIER'sS_NATIONKEY), which the server rejects with400 "Invalid column name"and skips the table patch. The grouping is now keyed on(fk_name, table_name)so same-named constraints on different tables stay separate, while genuine composite FKs on a single table still group together.Type of change:
High-level design:
N/A — small, self-contained change in the Snowflake SQLAlchemy dialect override.
Tests:
Use cases covered
400 Invalid column nameand the clone lost its FK entirely).Unit tests
ingestion/tests/unit/topology/database/test_snowflake_foreign_key_collision.py(4 cases: no cross-table merge, no column leak, composite FK still groups, distinct FKs on one table stay separate).Ingestion integration tests
Manual testing performed
Validated against a live Snowflake account: reproduced the exact leak (
CUSTOMER→['C_NATIONKEY','S_NATIONKEY'],SUPPLIER→[]) with the current release code, then confirmed correct per-table reflection (CUSTOMER→['C_NATIONKEY'],SUPPLIER→['S_NATIONKEY']) with the fix, order-independently.UI screen recording / screenshots:
Not applicable.
Checklist:
Greptile Summary
Fixes Snowflake foreign-key reflection collisions by:
Confidence Score: 5/5
The PR appears safe to merge.
No blocking failures remain.
Important Files Changed
Reviews (2): Last reviewed commit: "Merge branch 'main' into port-of-spain" | Re-trigger Greptile