Skip to content

feat: [BE] Add filter_query support to published components endpoint #180

@Mbeaulne

Description

@Mbeaulne

Description

The GET /api/published_components/ endpoint currently supports only basic substring matching (name_substring, published_by_substring). To support the new homepage Components tab (TangleML/tangle-ui#1963) with filtering comparable to the pipeline runs list, the endpoint needs richer filtering capabilities.

Currently there are under 100 components in the system, so client-side filtering works today. However, as the library grows this won't scale — server-side filtering is the right long-term solution.

Current state

Param Type Description
name_substring string Case-insensitive name match
published_by_substring string Case-insensitive publisher match
digest string Exact digest match
include_deprecated bool Include deprecated components

No support for combining filters, boolean logic, input/output search, or date filtering.

Proposed changes

Phase 1: Extend simple query params

Add these additional query parameters to the existing endpoint:

Param Type Description
description_substring string | null Case-insensitive search in component description
input_name_substring string | null Search across component input names
output_name_substring string | null Search across component output names
input_type_substring string | null Search across component input types
output_type_substring string | null Search across component output types

This matches the frontend's existing search filter options (Name, Input Name, Input Type, Output Name, Output Type) which are currently performed client-side only — Phase 1 simply moves that logic to the backend.

Why Phase 1 before filter_query? The filter_query system (Phase 2) is a meaningful effort to extend to components. Phase 1 unblocks the UI now with minimal backend surface area. If Phase 2 lands soon, Phase 1 params can be deprecated then. If the team prefers to skip Phase 1 and use client-side filtering as a stopgap until Phase 2 is ready, that's a reasonable alternative worth discussing.

Phase 2 (future): filter_query support

Adopt the same filter_query JSON predicate system used by GET /api/pipeline_runs/ with component-specific system keys:

  • system/component.name — ValueContains, ValueEquals
  • system/component.description — ValueContains
  • system/component.published_by — ValueContains, ValueEquals
  • system/component.deprecated — ValueEquals
  • system/component.input_names — ValueContains
  • system/component.output_names — ValueContains

This is a larger effort and can be deferred. Phase 1 covers the immediate needs.

Implementation notes

  • Input/output names and types are stored inside the component spec (YAML/JSON blob in the component table)
  • Given the current scale (under 100 components), querying the component.spec JSON column directly is likely sufficient for Phase 1 without denormalization. If the DB engine doesn't support JSON column queries well, denormalizing searchable fields to columns on PublishedComponentRow is the fallback.
  • Denormalization becomes more relevant at Phase 2 scale and can be revisited then.

Relevant files

  • component_library_api_server.pyPublishedComponentService.list() method
  • backend_types_sql.pyPublishedComponentRow and Component models

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions