-
Notifications
You must be signed in to change notification settings - Fork 14
Description
Description
The GET /api/published_components/ endpoint currently supports only basic substring matching (name_substring, published_by_substring). To support the new homepage Components tab (TangleML/tangle-ui#1963) with filtering comparable to the pipeline runs list, the endpoint needs richer filtering capabilities.
Currently there are under 100 components in the system, so client-side filtering works today. However, as the library grows this won't scale — server-side filtering is the right long-term solution.
Current state
| Param | Type | Description |
|---|---|---|
name_substring |
string |
Case-insensitive name match |
published_by_substring |
string |
Case-insensitive publisher match |
digest |
string |
Exact digest match |
include_deprecated |
bool |
Include deprecated components |
No support for combining filters, boolean logic, input/output search, or date filtering.
Proposed changes
Phase 1: Extend simple query params
Add these additional query parameters to the existing endpoint:
| Param | Type | Description |
|---|---|---|
description_substring |
string | null |
Case-insensitive search in component description |
input_name_substring |
string | null |
Search across component input names |
output_name_substring |
string | null |
Search across component output names |
input_type_substring |
string | null |
Search across component input types |
output_type_substring |
string | null |
Search across component output types |
This matches the frontend's existing search filter options (Name, Input Name, Input Type, Output Name, Output Type) which are currently performed client-side only — Phase 1 simply moves that logic to the backend.
Why Phase 1 before filter_query? The filter_query system (Phase 2) is a meaningful effort to extend to components. Phase 1 unblocks the UI now with minimal backend surface area. If Phase 2 lands soon, Phase 1 params can be deprecated then. If the team prefers to skip Phase 1 and use client-side filtering as a stopgap until Phase 2 is ready, that's a reasonable alternative worth discussing.
Phase 2 (future): filter_query support
Adopt the same filter_query JSON predicate system used by GET /api/pipeline_runs/ with component-specific system keys:
system/component.name— ValueContains, ValueEqualssystem/component.description— ValueContainssystem/component.published_by— ValueContains, ValueEqualssystem/component.deprecated— ValueEqualssystem/component.input_names— ValueContainssystem/component.output_names— ValueContains
This is a larger effort and can be deferred. Phase 1 covers the immediate needs.
Implementation notes
- Input/output names and types are stored inside the component spec (YAML/JSON blob in the
componenttable) - Given the current scale (under 100 components), querying the
component.specJSON column directly is likely sufficient for Phase 1 without denormalization. If the DB engine doesn't support JSON column queries well, denormalizing searchable fields to columns onPublishedComponentRowis the fallback. - Denormalization becomes more relevant at Phase 2 scale and can be revisited then.
Relevant files
component_library_api_server.py—PublishedComponentService.list()methodbackend_types_sql.py—PublishedComponentRowandComponentmodels