Skip to content

fix: UDF/ODFV source rehydrate (+ Postgres / online cache) - #6655

Open
aniketpalu wants to merge 6 commits into
feast-dev:masterfrom
aniketpalu:fix/udf-source-rehydrate-postgres-online-cache
Open

fix: UDF/ODFV source rehydrate (+ Postgres / online cache)#6655
aniketpalu wants to merge 6 commits into
feast-dev:masterfrom
aniketpalu:fix/udf-source-rehydrate-postgres-online-cache

Conversation

@aniketpalu

@aniketpalu aniketpalu commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes three issues found while testing BatchFeatureView UDFs and OnDemandFeatureViews with Spark transformations (SparkApplication / SparkComputeEngine path) against a remote feature server.

All three blocked a clean UDF/ODFV materialize → online path. This PR is scoped to that story. Shared helper feast.transformation.udf_rehydrate is introduced so the same contract can later be wired into RayTransformation.from_proto (intentionally not changed here).

1. Spark BFV UDF — prefer source over dill (exit 139)

Problem: Materializing a BatchFeatureView with a Python UDF on Spark could segfault the driver (exit 139). Registry stores both dill (body) and source (body_text / udf_string); the Spark path executed the dill-restored callable, which is unsafe with DataFrame ops on our Spark stack.

Change: SparkTransformationNode resolves the UDF via shared resolve_udf: exec trusted udf_string when present; fall back to the existing callable / dill. SparkFeatureBuilder passes udf_string through.

Files: spark/nodes.py, spark/feature_builder.py, transformation/udf_rehydrate.py

2. ODFV / local transformations — source-first from_proto

Problem: ODFV serve (and feast-apply rehydrate from SQL registry) used dill.loads(body) only. That fails across Python minors (e.g. apply on Spark driver 3.10, serve on feature-server 3.12) and when body_text includes @on_demand_feature_view(...) (NameError / CrashLoop if source is exec'd naively).

Change: PandasTransformation.from_proto and PythonTransformation.from_proto use the same resolve_udf (strip leading decorators, then exec; dill fallback).

Out of scope (follow-up): RayTransformation.from_proto remains dill.loads for now.

Files: pandas_transformation.py, python_transformation.py, udf_rehydrate.py

3. Postgres empty feature_name_columns → SELECT *

Problem: BatchFeatureView python/pandas mode signals "all columns" with empty feature_name_columns. Postgres offline store still projected only join keys + timestamps, starving the UDF of input features.

Change: When feature_name_columns is empty, pull_latest / pull_all use a.* / alias.* (aligned with Ray offline-store semantics).

Files: postgres_offline_store/postgres.py + unit tests

4. Fresh online registry reads

Problem: After materialize completes, online serving could briefly (or stickily) gate on cached MATERIALIZING / stale FV metadata.

Change: Online request path uses allow_cache=False when resolving features / feature services (utils._get_online_request_context, feature server _get_features).

Files: utils.py, feature_server.py

Out of scope

  • RayTransformation.from_proto / Ray compute-engine dill paths (follow-up using udf_rehydrate)
  • Remote materialization async failure → GENERATED wipe (separate remote-mat PR)
  • Redis same-timestamp idempotency

Which issue(s) this PR fixes

Found during BYOS Spark UDF/ODFV E2E; no single upstream issue required. Related discussions: Spark UDF materialize reliability, ODFV cross-image Python skew.

Checks

  • I've made sure the tests are passing.
  • My commits are signed off (git commit -s)
  • My PR title follows conventional commits format

Testing Strategy

  • Unit tests
  • Integration tests
  • Manual tests
  • Testing is not required for this change

Misc

…istry reads

Rehydrate BatchFeatureView/ODFV callables from body_text (strip leading
decorators) before dill.loads to avoid Spark driver exit 139 and
cross-Python serve failures. Treat empty Postgres feature_name_columns as
SELECT *. Disable registry cache on the online request path so FeatureView
state gates see AVAILABLE_ONLINE immediately after materialize.
RayTransformation.from_proto left unchanged (follow-up).

Signed-off-by: Aniket Paluskar <apaluska@redhat.com>
@aniketpalu
aniketpalu requested a review from a team as a code owner July 29, 2026 09:35
Comment thread sdk/python/feast/transformation/udf_rehydrate.py Fixed
@codecov-commenter

codecov-commenter commented Jul 29, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 66.43357% with 48 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.42%. Comparing base (6019bd6) to head (fd2178d).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
sdk/python/feast/transformation/udf_rehydrate.py 64.65% 27 Missing and 14 partials ⚠️
.../python/feast/infra/compute_engines/spark/nodes.py 50.00% 3 Missing ⚠️
...ast/infra/compute_engines/spark/feature_builder.py 0.00% 2 Missing ⚠️
..._stores/contrib/postgres_offline_store/postgres.py 84.61% 1 Missing and 1 partial ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6655      +/-   ##
==========================================
+ Coverage   46.36%   46.42%   +0.05%     
==========================================
  Files         414      415       +1     
  Lines       50072    50206     +134     
  Branches     7154     7190      +36     
==========================================
+ Hits        23218    23307      +89     
- Misses      25231    25262      +31     
- Partials     1623     1637      +14     
Flag Coverage Δ
go-feature-server 30.58% <ø> (ø)
python-unit 47.72% <66.43%> (+0.05%) ⬆️
Files with missing lines Coverage Δ
...thon/feast/transformation/pandas_transformation.py 73.68% <100.00%> (+0.95%) ⬆️
...thon/feast/transformation/python_transformation.py 87.50% <100.00%> (+0.46%) ⬆️
...ast/infra/compute_engines/spark/feature_builder.py 0.00% <0.00%> (ø)
..._stores/contrib/postgres_offline_store/postgres.py 46.13% <84.61%> (+1.05%) ⬆️
.../python/feast/infra/compute_engines/spark/nodes.py 26.36% <50.00%> (+0.29%) ⬆️
sdk/python/feast/transformation/udf_rehydrate.py 64.65% <64.65%> (ø)

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6019bd6...fd2178d. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@aniketpalu aniketpalu changed the title fix: Fixes three issues found while testing BatchFeatureView UDFs and OnDemandFeatureViews with Spark transformations (SparkApplication / SparkComputeEngine path) against a remote feature server fix: UDF/ODFV source rehydrate (+ Postgres / online cache) Jul 29, 2026
Signed-off-by: Aniket Paluskar <apaluska@redhat.com>

@franciscojavierarceo franciscojavierarceo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two blocking concerns from the remote diff:

  1. feature_server.py and _get_online_request_context() now pass allow_cache=False for feature-service and feature-view resolution on every online request. For SQL-backed registries this moves backing-registry I/O into the serving hot path, which can materially regress latency and registry load. Please refresh/invalidate the affected cached definitions when UDF state changes (or isolate and justify this serving behavior separately) instead of globally bypassing the cache.
  2. CodeQL's open finding in transformation/udf_rehydrate.py is valid: the decorator-stripping regex has nested unbounded repetition and processes registry-provided source text. Please replace it with a linear parser/line-based approach and add an adversarial regression case.

@ntkathole

Copy link
Copy Markdown
Member

@aniketpalu Please fix CI

aniketpalu and others added 3 commits July 30, 2026 14:10
Post-materialize MATERIALIZING gate lag is handled by client retry /
registry TTL refresh, not by bypassing the registry cache on every
online request. Addresses review feedback on feast-dev#6655.

Signed-off-by: Aniket Paluskar <apaluska@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Avoid CodeQL ReDoS finding on nested @/newline regex when rehydrating
body_text. Add multiline and adversarial @ spam unit tests.

Signed-off-by: Aniket Paluskar <apaluska@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants