Skip to content

[FEATURE] Ordered UIDs filter — return collections in externally resolved order (ORDER BY FIELD) #148

Description

@klamparski

Is your feature request related to a problem?

When a collection endpoint is backed by an external system that decides which records match and in what order — a search engine returning relevance-ranked results (Elasticsearch, Algolia, Solr), a recommendation service, or a manually curated list — there is currently no way to make t3api return the records in that order.

A custom filter can easily constrain the collection to the matching records (uid IN (...)), but the ordering is lost: Extbase's query API can only express ORDER BY <column> ASC|DESC, not "order by the position in this UID list" (ORDER BY FIELD(uid, ...) in MySQL/MariaDB). Projects work around this by sorting in PHP after fetching — which breaks pagination and totalItems, and performs poorly, since the whole result set has to be fetched and hydrated into Extbase objects on every request just to serve a single page — or by hand-crafting raw statements inside custom operation handlers, bypassing the whole filter infrastructure.

Describe the solution you'd like

This issue adds the following:

1. QueryModifierInterface — a second-pass extension point for filters

A filter may additionally implement:

interface QueryModifierInterface
{
    public function modifyQuery(QueryInterface $query, ApiFilter $apiFilter, CollectionOperation $operation): void;
}

After the constraints of all filters have been combined and applied, t3api calls modifyQuery() on every filter implementing the interface, passing the fully constrained query and the collection operation being served (mirroring API Platform's filter contract, where apply() receives the Operation). Modifiers run in the same request-driven order as filterProperty(). This is the seam for adjustments the QOM constraint model cannot express — most notably a custom ORDER BY.

The composability rules will be documented explicitly: QOM-level changes compose across modifiers; a modifier that replaces the query with a Doctrine QueryBuilder statement must check $query->getStatement() first and mutate an already-present builder instead of reconverting, so it does not silently discard an earlier modifier's work.

2. AbstractOrderedUidsFilter — a ready-made base class on that seam

A concrete filter implements a single method:

protected function resolveOrderedUids($values, ApiFilter $apiFilter): ?array;
// null      → no lookup performed, filter inactive
// []        → lookup performed, nothing matched (empty collection)
// [12, 5, 8] → matching UIDs, in the order to return

The base class then:

  • constrains the collection to the resolved UIDs (uid IN (...)),
  • re-applies the order as ORDER BY FIELD(uid, ...) via a Doctrine QueryBuilder statement,
  • keeps its state per @ApiFilter declaration, so one filter class can be declared under several parameter names on the same resource,
  • composes with other ordered-UIDs filters: constraints intersect, the first filter in the request's query string stays the primary ordering and the following ones are appended as tie-breakers.

Example usage:

class RelevanceSearchFilter extends AbstractOrderedUidsFilter
{
    public function __construct(private readonly SearchEngineClient $searchEngineClient)
    {
    }

    protected function resolveOrderedUids($values, ApiFilter $apiFilter): ?array
    {
        $phrase = trim((string)(is_array($values) ? reset($values) : $values));

        return $phrase === '' ? null : $this->searchEngineClient->searchUids($phrase);
    }
}

/api/recipes?search=pasta then returns records in the search engine's relevance order.

3. Correct pagination and totalItems for statement-backed queries

The paginated members query gets its own QueryBuilder clone in AbstractCollectionResponse, so limit/offset never leak into the unpaginated count query — totalItems stays the full match count on every page.

4. Documentation and tests

  • A new documentation page for the ordered UIDs filter and a "Query modifiers" section in the custom filters chapter (including the composability rules).
  • Unit test coverage for the new classes and functional (end-to-end dispatcher) coverage: paginated collections with and without a QueryBuilder-statement query, both pages, plus modifier composition — two ordered-UIDs filter declarations, an ordered-UIDs ranking combined with a statement modifier contributing an extra WHERE condition (in both hand-off directions), and the FIELD() ranking observably breaking the ties of an earlier statement ordering.

Describe alternatives you've considered

  • Sorting in PHP after fetching — breaks pagination (limit/offset must be applied by the database) and forces fetching and hydrating the full result set into Extbase objects on every request, so the cost grows with the collection instead of the page size.
  • Raw statement in a custom operation handler — loses the composability of the filter infrastructure (other filters, security expressions, pagination) and duplicates boilerplate in every project.
  • A global query-extension registry (API Platform's QueryCollectionExtension/QueryItemExtension) — a complementary mechanism for cross-cutting concerns rather than request-parameter-driven filtering; in t3api filters are the collection-scoped, opt-in, self-documenting (hydra:search/OpenAPI) primitive, so the ordered-UIDs use case belongs to the filter contract. A cross-cutting customization hook can be added later as an additive feature without breaking anything.
  • Persisting the external ordering into a sortable column — requires write access and constant re-syncing; unusable for per-request orderings like search relevance.

Additional context

  • ORDER BY FIELD() is MySQL/MariaDB specific, which is documented on the new documentation page.
  • Query modifiers run for collection GET operations only — the same scope in which filters apply at all.
  • Follow-up for the next major (noted as @todo in the code): add the CollectionOperation parameter to FilterInterface::filterProperty() as well — it cannot happen within 5.x without breaking existing custom filter implementations.

Code of Conduct

  • I agree to follow this project's Code of Conduct.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions