Skip to content

feat: Add opt-in filter_by_created_timestamp cutoff to get_historical_features - #6617

Open
addenergyx wants to merge 9 commits into
feast-dev:masterfrom
addenergyx:at-event-time
Open

feat: Add opt-in filter_by_created_timestamp cutoff to get_historical_features#6617
addenergyx wants to merge 9 commits into
feast-dev:masterfrom
addenergyx:at-event-time

Conversation

@addenergyx

@addenergyx addenergyx commented Jul 19, 2026

Copy link
Copy Markdown

What this PR does / why we need it

Adds an opt-in filter_by_created_timestamp: bool = False parameter to get_historical_features that enforces created_timestamp <= entity_timestamp in point-in-time joins.

Today created_timestamp_column is used only as a dedup tiebreaker, never as a filter, so a training row at entity timestamp T can be served a value that did not exist yet at T (backfills, late corrections). With filter_by_created_timestamp=True, retrieval reflects what was actually known as of each entity row's timestamp, i.e. what the online store would have served at that moment.

store.get_historical_features(
    entity_df=entity_df,
    features=features,
    filter_by_created_timestamp=True,  # default False: existing behavior
)

Design

  • Threaded through FeatureStore.get_historical_features using the same optional-kwargs pattern as start_date/end_date: only passed to the offline store when True, so the default path is untouched and third-party stores are unaffected unless a user explicitly opts in.
  • For SQL-template stores, the shared build_point_in_time_query gains a filter_by_created_timestamp template variable, and each template's __base join gains one guarded predicate:
    {% if filter_by_created_timestamp and featureview.created_timestamp_column %}
    AND subquery.created_timestamp <= entity_dataframe.entity_timestamp
    {% endif %}
    Everything else (event-timestamp window, TTL, dedup, ROW_NUMBER) is unchanged.
  • Feature views without a created_timestamp_column are unaffected.

Store coverage

Implemented: BigQuery, Redshift, Spark, Trino, Athena, Postgres, ClickHouse, Couchbase (SQL templates); DuckDB, MSSQL, Oracle (via the shared ibis point_in_time_join); Dask/file (pandas filter mirroring _filter_ttl); Hybrid (delegates, re-checking the resolved child store).

Stores declare support via a supports_filter_by_created_timestamp class attribute on OfflineStore (default False). A single guard in the passthrough provider raises NotImplementedError when the flag is set for a store that does not declare support, so no store can silently ignore it and new stores are covered automatically. Unsupported today: Snowflake (its ASOF JOIN dedups before the join and MATCH_CONDITION takes a single predicate, so the cutoff cannot be expressed without restructuring the query), Remote (needs protocol support to forward the flag), Ray, and MongoDB.

Notes

  • Rows with a NULL created_timestamp are excluded by the predicate in all implementations; the docstring notes the column should be non-null when the mode is used. (In the dask store, unmatched entity rows from the left join are still preserved.)
  • In the dask store, an entity row whose matching feature rows are all filtered out disappears from the result instead of surviving with null features. This matches the store's existing TTL filtering behavior and is a pre-existing trait of its incremental join, not something this PR changes.
  • Orthogonal to ttl: TTL bounds by event time, this bounds by known time.

Which issue(s) this PR fixes

Fixes #6615

Misc

Tests added in sdk/python/tests/unit/infra/offline_stores/test_filter_by_created_timestamp.py:

  • Template rendering: predicate present iff filter_by_created_timestamp=True and a created_timestamp_column is set; default rendering unchanged.
  • Dask and ibis (DuckDB) end-to-end: same event timestamp with two versions created before/after the entity timestamp; default serves the later-created value, filter_by_created_timestamp=True serves the version known at the entity timestamp.

ruff check/ruff format clean; mypy feast reports the same 17 pre-existing errors as master (none introduced).

addenergyx and others added 3 commits July 19, 2026 18:38
…ical_features

When a feature view has a created_timestamp_column, it is currently used
only as a dedup tiebreaker in point-in-time joins, so retrieval can serve
feature values whose created_timestamp is after the entity row's event
timestamp (backfills, late corrections). This leaks future information
into training data and makes training sets non-reproducible.

Add an opt-in at_event_time flag (default False) to get_historical_features
that adds a created_timestamp <= entity_timestamp predicate to the
point-in-time join, so retrieval reflects what was known as of each entity
row's timestamp. Implemented for the SQL template stores (BigQuery,
Redshift, Spark, Trino, Athena, Postgres, ClickHouse, Couchbase), the
ibis-based stores (DuckDB, MSSQL, Oracle) and the dask store. Snowflake,
Remote and Ray raise NotImplementedError when the flag is set.

Fixes feast-dev#6615

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: David <david-adeniji@hotmail.co.uk>
Adds a universal offline store integration test covering at_event_time
(default returns backfilled values, at_event_time=True only returns values
created at or before the entity timestamp, stores without support skip via
NotImplementedError) and documents the flag on the point-in-time joins
concept page.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: David <david-adeniji@hotmail.co.uk>
…ress review feedback

Rename the flag to filter_by_created_timestamp to match the repo's
mechanical flag naming and sit alongside created_timestamp_column.

Review fixes:
- HybridOfflineStore now forwards optional kwargs to the delegated store
  instead of raising TypeError for supported underlying stores.
- The dask filter now excludes feature rows with a null created timestamp
  (consistent with the SQL predicate) while still keeping unmatched entity
  rows from the left join.
- The integration test only treats NotImplementedError from
  get_historical_features itself as an unsupported-store skip.
- Add template-render tests for all SQL dialects and a dask null
  created-timestamp test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: David <david-adeniji@hotmail.co.uk>
@addenergyx addenergyx changed the title feat: Add opt-in at_event_time created_timestamp cutoff to get_historical_features feat: Add opt-in filter_by_created_timestamp cutoff to get_historical_features Jul 19, 2026
addenergyx and others added 2 commits July 19, 2026 19:30
…re capability flag

Simplification pass over the feature, reviewed again by Codex:

- Declare support via OfflineStore.supports_filter_by_created_timestamp
  (default False) and check it once in the passthrough provider via
  ensure_filter_by_created_timestamp_supported, replacing the three
  per-store NotImplementedError guards. Unsupported stores can no longer
  silently ignore the flag.
- The hybrid store re-checks the resolved child store before delegating.
- Standardize every supporting store on an explicit
  filter_by_created_timestamp parameter instead of kwargs.get lookups.
- Document the flag in the OfflineStore.get_historical_features docstring
  alongside start_date/end_date.
- Return the dask created-timestamp filter lazily so the mask fuses into
  the following _drop_duplicates persist instead of forcing an extra
  materialization.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: David <david-adeniji@hotmail.co.uk>
@addenergyx
addenergyx marked this pull request as ready for review July 21, 2026 17:07
@addenergyx
addenergyx requested review from a team and sudohainguyen as code owners July 21, 2026 17:07
@addenergyx
addenergyx requested review from ejscribner, robhowley and tokoko and removed request for a team July 21, 2026 17:07
addenergyx and others added 3 commits July 21, 2026 18:10
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: David <david-adeniji@hotmail.co.uk>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: David <david-adeniji@hotmail.co.uk>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: David <david-adeniji@hotmail.co.uk>

@franciscojavierarceo franciscojavierarceo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Dask implementation can drop entity rows. It performs the left merge first and then _filter_created_timestamp() removes rows whose matched feature has created_timestamp > entity_timestamp. If every candidate for an entity is too new, the merge produced no unmatched/null row to fall back to, so filtering removes that entity entirely. get_historical_features() should preserve entity-dataframe cardinality and return null features in this case. Please apply the cutoff in/before the join or reconstruct unmatched rows, and add a regression where all matching candidates are future-created.

feature_table[timestamp_field] <= entity_table[event_timestamp_col],
)

if filter_by_created_timestamp and created_timestamp_field:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This predicate casts the created timestamp to UTC but the existing event-timestamp comparison on the same join doesn't cast. Compare with the existing predicate at line 433-434 which does no cast.

@codecov-commenter

codecov-commenter commented Jul 30, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 76.66667% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.22%. Comparing base (104ad10) to head (6abf239).
⚠️ Report is 5 commits behind head on master.

Files with missing lines Patch % Lines
sdk/python/feast/feature_store.py 33.33% 1 Missing and 1 partial ⚠️
...feast/infra/offline_stores/hybrid_offline_store.py 33.33% 2 Missing ⚠️
sdk/python/feast/infra/passthrough_provider.py 0.00% 1 Missing and 1 partial ⚠️
...ffline_stores/contrib/mssql_offline_store/mssql.py 0.00% 1 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6617      +/-   ##
==========================================
+ Coverage   45.93%   46.22%   +0.28%     
==========================================
  Files         414      414              
  Lines       50006    50035      +29     
  Branches     7147     7152       +5     
==========================================
+ Hits        22972    23130     +158     
+ Misses      25423    25294     -129     
  Partials     1611     1611              
Flag Coverage Δ
go-feature-server 30.58% <ø> (ø)
python-unit 47.51% <76.66%> (+0.31%) ⬆️
Files with missing lines Coverage Δ
sdk/python/feast/infra/offline_stores/bigquery.py 49.59% <100.00%> (+0.08%) ⬆️
...line_stores/contrib/athena_offline_store/athena.py 39.15% <100.00%> (+0.32%) ⬆️
...res/contrib/clickhouse_offline_store/clickhouse.py 32.55% <100.00%> (+0.39%) ⬆️
...tores/contrib/couchbase_offline_store/couchbase.py 30.76% <100.00%> (+30.76%) ⬆️
...line_stores/contrib/oracle_offline_store/oracle.py 39.75% <100.00%> (+0.18%) ⬆️
..._stores/contrib/postgres_offline_store/postgres.py 45.23% <100.00%> (+0.14%) ⬆️
...ffline_stores/contrib/spark_offline_store/spark.py 36.25% <100.00%> (+0.09%) ⬆️
...ffline_stores/contrib/trino_offline_store/trino.py 50.67% <100.00%> (+0.22%) ⬆️
sdk/python/feast/infra/offline_stores/dask.py 50.97% <100.00%> (+0.48%) ⬆️
sdk/python/feast/infra/offline_stores/duckdb.py 39.27% <100.00%> (+0.20%) ⬆️
... and 9 more

... and 2 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 104ad10...6abf239. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Opt-in created_timestamp <= entity_timestamp cutoff in point-in-time joins (as-of-known-time retrieval)

4 participants