feat: expose filters and lifecycle options in REST list and search (#89)#92
Open
dcfocus wants to merge 2 commits into
Open
feat: expose filters and lifecycle options in REST list and search (#89)#92dcfocus wants to merge 2 commits into
dcfocus wants to merge 2 commits into
Conversation
) Append raw records first and enrich them with embeddings later. Adds an `embedding` field to the record patch path so a vector can be attached or replaced by id or external_id after the original write. - core: RecordPatch.embedding, applied in update_visible_record - api: RecordPatchDto.embedding; copied in core + server patch_from_dto - python: embedding param on PyO3 update and Context.update() - Rust client inherits the field via the UpdateRecordRequest JSON body A record without an embedding is durably stored but excluded from vector search until enriched, since search skips null-embedding records. Tests: Rust core round-trip (by id + external_id) and Python e2e (raw-first -> enrich, incl. add_many bulk). README documents the pattern. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ance-format#89) Bring the REST `search` and `list` endpoints to parity with the core and Python APIs by reusing the existing RecordFilters JSON semantics and LifecycleQueryOptions, mirroring the existing `retrieve` plumbing. - api: add `filters`/`include_expired`/`include_retired` to SearchRequest (legacy {query,limit} payloads still deserialize); change the ContextStoreApi trait so `search` takes `&SearchRequest` and `list` takes filters + lifecycle flags. - core: route trait `search`/`list` through search_filtered_with_options / list_filtered_with_options. - server: `search` route applies filters + lifecycle; `list_records` accepts `filters` (URL JSON string) plus `include_expired`/`include_retired` query params, matching the `related_records` style. - unified + client: dispatch and RemoteContextStore updated; low-level list_records sends the new query params. Adds serde_json dep to the lance-context crate (trait now exposes serde_json::Value). - tests: 8 server route tests (search & list x metadata/built-in field/ expired/retired/relationships), a list invalid-filters test, and 3 api serde backward-compat tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brings the REST
searchandlistendpoints to parity with the core and Python APIs by addingfilters,include_expired, andinclude_retired, reusing the existingRecordFiltersJSON semantics andLifecycleQueryOptions— mirroring the plumbing already used by RESTretrieve. No new query language, no SQL, and no change to default visibility for expired/retired/superseded records.Closes #89.
What changed (#89)
lance-context-api—SearchRequestgainsfilters/include_expired/include_retired(legacy{query, limit}payloads still deserialize). TheContextStoreApitrait now takessearch(&SearchRequest)andlist(limit, offset, filters, include_expired, include_retired)(breaking change to the trait; acceptable pre-1.0).lance-context-core— traitsearch/listroute through the existingsearch_filtered_with_options/list_filtered_with_options.lance-context-server—searchapplies filters + lifecycle;list_recordsacceptsfilters(URL JSON string) plusinclude_expired/include_retiredquery params, matching the existingrelated_recordsstyle.lance-context(unified) +lance-context-client— dispatch andRemoteContextStoreupdated; the low-level clientlist_recordssends the new query params. Adds aserde_jsondependency to thelance-contextcrate (the trait now exposesserde_json::Value).Tests
searchandlist, each covering: metadata filter, built-in field filter,include_expired,include_retired(superseded), andinclude_relationshipstoggling.listinvalid-filters-JSON rejection test.lance-context-apiserde tests, including backward compatibility for the pre-Expose filters and lifecycle options consistently in REST list and search APIs #89SearchRequestpayload.Verification
cargo fmt --all -- --checkcargo clippy --workspace --all-targets -- -D warningscargo test: core (44), server (14, incl. 8 new), api (3) — all green.🤖 Generated with Claude Code