Skip to content

feat: auto-apply Snowflake ownership object tags on publish#26

Open
Vinay Shende (vinay79n) wants to merge 4 commits into
mainfrom
feat/tagging-tables
Open

feat: auto-apply Snowflake ownership object tags on publish#26
Vinay Shende (vinay79n) wants to merge 4 commits into
mainfrom
feat/tagging-tables

Conversation

@vinay79n

@vinay79n Vinay Shende (vinay79n) commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Implements the table-ownership RFC (RFC: Snowflake table ownership via object tag) by tagging tables through the existing
publish chokepoint instead of hundreds of hand-written ALTER statements.

  • Add _snowflake/object_tags.py: build_table_tags (derive OWNER/TEAM/DOMAIN/
    PROJECT
    from Metaflow context, default STATUS=active, validate STATUS/SLA),
    build_set_tag_sql, and apply_table_tags (warn-don't-fail on missing tag defs)
  • Wire prod-only tagging into publish() and publish_pandas() via a new optional
    tags override; non-prod (DATA_SCIENCE_STAGE) runs are left untouched
  • Add metaflow/registry.py: create_ownership_registry_view() + TABLE_OWNERSHIP_
    REGISTRY
    view (plain view, adoption-based, no refresh needed)
  • Export create_ownership_registry_view from metaflow package
  • Add unit tests (100% coverage of object_tags.py)
  • Docs: publish/publish_pandas ownership-tags sections, registry view page
  • Bump version 0.4.2 -> 0.5.0

ClickUp card - DS-2491

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class Snowflake table-ownership tagging to the Metaflow publish “chokepoints” and introduces an admin helper view that surfaces those tags via a central registry view.

Changes:

  • Introduces _snowflake/object_tags.py to derive/validate ownership tags, generate ALTER TABLE … SET TAG SQL, and apply tags (warn-only on tagging failures).
  • Wires optional tags overrides into publish() and publish_pandas() and applies tags for production tables.
  • Adds create_ownership_registry_view() + docs to create/query the TABLE_OWNERSHIP_REGISTRY view; bumps version to 0.5.0.

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
uv.lock Bumps locked project version to 0.5.0.
pyproject.toml Bumps package version to 0.5.0 and updates authors.
src/ds_platform_utils/_snowflake/object_tags.py New tag derivation/SQL builder/apply helpers for Snowflake object tags.
src/ds_platform_utils/metaflow/write_audit_publish.py Adds tags parameter and applies tags post-SWAP in production.
src/ds_platform_utils/metaflow/pandas.py Adds tags parameter and applies tags in production for both direct and S3-stage paths.
src/ds_platform_utils/metaflow/registry.py New helper to create the ownership registry view in Snowflake.
src/ds_platform_utils/metaflow/init.py Exports create_ownership_registry_view.
tests/unit_tests/snowflake/test__object_tags.py New unit tests for tag derivation, SQL construction, and apply behavior.
README.md Links to the new create_ownership_registry_view docs page.
docs/metaflow/publish.md Documents the new tags param and ownership-tag behavior.
docs/metaflow/publish_pandas.md Documents the new tags param and ownership-tag behavior.
docs/metaflow/create_ownership_registry_view.md New docs for creating/querying the registry view.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/ds_platform_utils/metaflow/pandas.py Outdated
Comment thread src/ds_platform_utils/_snowflake/object_tags.py
Comment thread docs/metaflow/publish.md Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants