Add Feature Engineering reference files to databricks-ml-training-serving#570
Closed
Hehehe421 wants to merge 1 commit into
Closed
Add Feature Engineering reference files to databricks-ml-training-serving#570Hehehe421 wants to merge 1 commit into
Hehehe421 wants to merge 1 commit into
Conversation
…ving Extends the skill with two new reference files covering Feature Engineering in Unity Catalog: - 3-feature-store.md: Standard API (GA). Covers FeatureEngineeringClient, create_table/write_table, FeatureLookup with point-in-time, fe.log_model with lineage, score_batch, and online store publishing. - 4-declarative-features.md: Declarative API (Public Preview, >=0.15.0). Covers DeltaTableSource, create_feature with SlidingWindow/TumblingWindow/ RollingWindow, materialize_features, and Feature Serving Endpoints. Also updates SKILL.md description with feature store trigger terms so the skill activates on Feature Engineering prompts. Evaluated with SkillForge L1+L3: composite 0.96, no regressions.
Author
|
Migrating contribution to databricks-eng/universe per the latest agent-skills pipeline guidance. Branch: sixuan-he_data/feature/feature-store-skill on databricks-eng/universe-dev. PR will open against universe:master once Tian's declarative-features.md lands. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extends the
databricks-ml-training-servingskill with two new reference files covering Feature Engineering in Unity Catalog.3-feature-store.md(287 lines) — Standard API (GA). CoversFeatureEngineeringClient,create_table/write_table,FeatureLookupwith point-in-time,fe.log_modelwith lineage,score_batch, and online store publishing via Lakebase.4-declarative-features.md(296 lines) — Declarative API (Public Preview,>=0.15.0). CoversDeltaTableSource,create_featurewithSlidingWindow/TumblingWindow/RollingWindow,materialize_features, and Feature Serving Endpoints. Streaming Declarative Features (Private Preview) referenced as a callout only — not covered in code.SKILL.md— description updated with 14 feature store trigger terms so the skill activates on Feature Engineering prompts (feature store,FeatureLookup,score_batch,publish_table,create_feature, etc.).Why
The existing skill covered classical ML and MLflow training but did not address Feature Engineering — a core ML pattern for Databricks customers needing training-serving consistency, feature reuse, point-in-time correctness, and real-time lookups.
The existing
SKILL.mdalso referenced `mlflow.pyfunc.spark_udf` as the batch scoring path. For feature-store-backed models this is the wrong API — `fe.score_batch` is required to auto-join feature tables via lineage. The new files document the correct path; the trigger terms inSKILL.mddescription ensure the agent uses the right one.Source material
Evaluation
Evaluated with SkillForge L1 (unit) + L3 (static document quality):
Test plan
summer_catalog.dbdemos_fs_travelin a vm-summer workspace (T1-T4 manual prompts via Databricks Assistant)