Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/instructions/java.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ applyTo: '**/*.java'
- Focus on readability, maintainability, and performance when refactoring identified issues.
- Use IDE and code editor warnings and suggestions to catch common patterns early in development.
- Add JavaDoc comments for all public classes and methods to enhance code understandability.
- Use Markdown syntax in JavaDoc instead of HTML tags.
- Follow the Java Language Specification and standard conventions for code style and formatting.

## Best practices
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,4 +54,4 @@ repos:
language: system
pass_filenames: false
files: \.java$
stages: [commit]
stages: [pre-commit]
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[WARNING] hook id `spotless-check` uses deprecated stage names (commit) which will be removed in a future version.  run: `pre-commit migrate-config` to automatically fix this.

2 changes: 2 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ services:
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-idpcore_password}
ports:
- "5437:5432"
volumes:
- ./scripts/init-extensions.sql:/docker-entrypoint-initdb.d/01-init-extensions.sql:ro
networks:
- postgres
restart: unless-stopped
Expand Down
130 changes: 95 additions & 35 deletions docs/src/concepts/entity-filtering.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,21 @@
---
title: Filtering entities
description: Query and filter template-scoped entities using the q= filter DSL
description: Query and filter entities
---

Entity filtering extends the existing list-entities endpoint with an optional `q` query parameter. It lets you narrow down results by attributes, property values, and relations using a simple DSL with AND logic.
## Filtering options

> [!WARNING]
> This is a first version of entity filtering with known constraints: `<` and `>` operators are not supported for relation filters, each key can appear at most once per query, and OR logic between criteria is not supported. These limitations will be addressed in future versions.
| Feature | Option 1: GET Query DSL (`q=`) | Option 2: POST JSON Search |
| :--------------------- | :------------------------------------- | :-------------------------------------------------------- |
| **Best for** | Quick, simple, template-scoped filters | Complex, nested logic, or global cross-template discovery |
| **Endpoint scope** | Scoped to a single template | Cross-template (Global system search) |
| **Logical connectors** | AND only | AND / OR |
| **Operators** | `=`, `:`, `<`, `>` | `EQ`, `NEQ`, `CONTAINS`, `STARTS_WITH`, `GT`, `LTE`, etc. |
| **Global text search** | No, strictly filtering | Yes, via optional free-text `query` property |

## Endpoint
## Option 1: Simple GET Filter DSL (`q` parameter)

The `q` parameter is an optional addition to the existing list-entities endpoint:
Entity filtering extends the existing list-entities endpoint with an optional `q` query parameter. It lets you narrow down results by attributes, property values, and relations using a simple DSL with AND logic.

```http
GET /api/v1/entities/{templateIdentifier}?q=<filter>
Expand All @@ -28,33 +33,33 @@ curl "http://localhost:8084/api/v1/entities/service?q=name:api;property.status=p

---

## Syntax
### DSL Syntax

Each filter is a semicolon-separated list of criteria:

```text
<key><operator><value>[;<key><operator><value>...]
```

All criteria are combined with **AND** logic, meaning every criterion must match for an entity to appear in the results.
All criteria are combined with **AND** logic, meaning every criterion must match for an entity to appear in the results. It also means users cannot duplicate criteria, each key can appear at most once per filter `q`expression.

### Operators

| Operator | Symbol | Behavior | Example |
| ------------- | ------ | ---------------------------------- | ----------------------- |
| Equals | `=` | Exact match (case-insensitive) | `name=payment-service` |
| Contains | `:` | Partial match (case-insensitive) | `name:payment` |
| Less than | `<` | Less than comparison | `property.port<9000` |
| Greater than | `>` | Greater than comparison | `property.port>1000` |
| Operator | Symbol | Behavior | Example |
| ------------ | ------ | -------------------------------- | ---------------------- |
| Equals | `=` | Exact match (case-insensitive) | `name=payment-service` |
| Contains | `:` | Partial match (case-insensitive) | `name:payment` |
| Less than | `<` | Less than comparison | `property.port<9000` |
| Greater than | `>` | Greater than comparison | `property.port>1000` |

> [!WARNING]
> `<` and `>` are only supported for attribute and property filters. Using them on relation filters returns an HTTP `400 Bad Request`.
> `<` and `>` are only supported for property filters (of type NUMBER). Using them on attributes (identifier, name) or relation filters returns an HTTP `400 Bad Request`.

---

## Key Types
### Key Types

### Attribute Filters
#### Attribute Filters

Filter by a direct entity field. Supported fields are `identifier` and `name`.

Expand All @@ -63,7 +68,7 @@ identifier=checkout-service
name:api
```

### Property Filters
#### Property Filters

Filter by a property value using `property.<name>`, where `<name>` is the property's name as defined in the template.

Expand Down Expand Up @@ -96,9 +101,9 @@ relation.database.identifier=my-postgres-1
relation.database.name:prod
```

### Reverse Relation Filters
#### Reverse Relation Filters

Use `relations_as_target.<name>.<property>` to find entities that *appear as targets* in a relation of type `<name>`. The `<property>` must be `identifier` or `name` and refers to the **source** entity in that relation.
Use `relations_as_target.<name>.<property>` to find entities that _appear as targets_ in a relation of type `<name>`. The `<property>` must be `identifier` or `name` and refers to the **source** entity in that relation.

```text
relations_as_target.owned_by.name:platform-team
Expand All @@ -107,7 +112,7 @@ relations_as_target.uses.identifier=service-1

---

## Combining Criteria
### Combining Criteria

Join multiple criteria with `;` to narrow results further:

Expand All @@ -124,7 +129,7 @@ curl "http://localhost:8084/api/v1/entities/service?q=relation.database=my-postg

---

## Examples
### Examples

```bash
# Find a service by exact identifier
Expand All @@ -145,23 +150,78 @@ curl "http://localhost:8084/api/v1/entities/service?q=relations_as_target.api-li

---

## Known Constraints
## Option 2: POST /api/v1/entities/search (JSON advanced search)

For more complex queries across all templates you can use the JSON search endpoint which accepts a nested filter tree, free-text `query`, pagination and sorting.

Request shape (`EntitySearchRequestDtoIn`):

```json
{
"query": "checkout",
"filter": {
"connector": "AND",
"criteria": [
{ "field": "template", "operation": "EQ", "value": "microservice" },
{
"connector": "OR",
"criteria": [
{ "field": "property.language", "operation": "EQ", "value": "JAVA" },
{ "field": "property.language", "operation": "EQ", "value": "KOTLIN" }
]
}
]
},
"page": 0,
"size": 20,
"sort": "identifier:asc"
}
```

### Request body syntax

This first version of entity filtering has the following constraints:
- `filter` is a tree of group nodes and criterion nodes. A group node has `connector` (one of `AND`,`OR`) and a non-empty `criteria` array. A criterion node must include `field`, `operation` and `value`.
- Supported `operation` values: `EQ`, `NEQ`, `CONTAINS`, `NOT_CONTAINS`, `STARTS_WITH`, `ENDS_WITH`, `GT`, `GTE`, `LT`, `LTE`.
- Free-text `query` (optional) performs a case-insensitive contains search across identifier, name, template identifier and all property values.
- Pagination: `page` is zero-based, `size` defaults to 20. Maximum allowed `size` is 500.

| Constraint | Detail |
| ----------------------------- | -------------------------------------------------------------------------- |
| Operators on relation filters | `<` and `>` are not supported for any relation filter type |
| Logic | AND only—no OR/NOT between criteria |
| Duplicate criteria | Each key can appear at most once per query |
| Max criteria per query | 10 |
| Max key length | 255 characters |
| Max value length | 255 characters |
| Property value type-awareness | All comparisons are string-based regardless of property type |
### What it does

Exceeding the numeric limits returns an HTTP `400 Bad Request` with a descriptive error message.
- Execute cross-template searches with precise logical compositions and full-text assistance.
- Combine `query` and `filter` to first narrow by free-text and then apply structured criteria (or vice-versa).

---
Quick curl example

```bash
curl -sS -X POST "http://localhost:8084/api/v1/entities/search" \
-H 'Content-Type: application/json' \
-d '{"query":"checkout","filter":{"connector":"AND","criteria":[{"field":"template","operation":"EQ","value":"microservice"}]},"page":0,"size":20}'
```

### Field reference

- `query` (optional): free-text string (max 255 chars) searched case-insensitively in `identifier`, `name`, `templateIdentifier`, and property values.
- `filter` (optional): nested `FilterNodeDtoIn` JSON structure. Use a `template` criterion to scope results to a specific template when needed.
- `page` / `size`: pagination (zero-based `page`). `size` defaults to 20 and the server enforces a maximum of 500.
- `sort`: `field:asc|desc`. Allowed sort fields: `identifier`, `name`, `templateIdentifier`.

### Operator semantics

| Operator | Meaning |
| --------------------------- | --------------------------------------------------------------- |
| `EQ` | Exact, case-insensitive equality |
| `NEQ` | Not equal (case-insensitive) |
| `CONTAINS` | Case-insensitive string match |
| `NOT_CONTAINS` | Negated string match |
| `STARTS_WITH` / `ENDS_WITH` | Prefix / suffix match |
| `GT` / `GTE` / `LT` / `LTE` | Ordering comparisons (string or numeric depending on the field) |

### Enforced limits

- Maximum filter nesting depth: 5 levels. Requests exceeding this fail with `400 Bad Request` and a descriptive error message.
- Maximum total criterion count across the tree: 50. Requests exceeding this fail with `400 Bad Request`.
- Maximum `query` length: 255 characters.
- Maximum `size` (page size): 500.

## Next Steps

Expand Down
3 changes: 3 additions & 0 deletions scripts/init-extensions.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
-- Initialize PostgreSQL extensions required by the application.
-- This script runs once on first container startup (docker-entrypoint-initdb.d).
CREATE EXTENSION IF NOT EXISTS pg_trgm;
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
package com.decathlon.idp_core.domain.constant;

import lombok.AccessLevel;
import lombok.NoArgsConstructor;

/// Domain constants for the entity filter query DSL safety limits.
@NoArgsConstructor(access = AccessLevel.PRIVATE)
public final class FilterConstraints {

/// Maximum number of filter criteria per `q` query string (DoS prevention).
public static final int MAX_CRITERIA_COUNT = 10;

/// Maximum length (in characters) of a key or value in a single filter
/// criterion.
public static final int MAX_KEY_VALUE_LENGTH = 255;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
package com.decathlon.idp_core.domain.constant;

import java.util.Set;

import lombok.AccessLevel;
import lombok.NoArgsConstructor;

/// Domain constants for search and filter query safety limits.
@NoArgsConstructor(access = AccessLevel.PRIVATE)
public final class SearchConstraints {

/// Maximum number of entities returned per page in a search request.
public static final int MAX_PAGE_SIZE = 500;

/// Maximum length (in characters) of the free-text `query` parameter.
public static final int MAX_QUERY_LENGTH = 255;

/// Maximum nesting depth of a
/// [com.decathlon.idp_core.domain.model.entity.SearchFilterNode] tree.
public static final int MAX_NESTING_DEPTH = 5;

/// Maximum total number of criterion nodes across a
/// [com.decathlon.idp_core.domain.model.entity.SearchFilterNode] tree.
public static final int MAX_TOTAL_CRITERIA = 50;

/// Fields on which search results may be sorted.
public static final Set<String> ALLOWED_SORT_FIELDS = Set.of("identifier", "name",
"templateIdentifier");
}
Original file line number Diff line number Diff line change
Expand Up @@ -91,4 +91,26 @@ public static String minMaxConstraintViolated(String constraint) {
public static final String FILTER_DUPLICATE_CRITERION = "Multiple filters for the same property are not supported";
public static final String FILTER_TYPE_MISMATCH = "Operation '%s' is not applicable for field '%s'.";
public static final String FILTER_PROPERTY_TYPE_NOT_NUMERIC = "Operation '%s' is not applicable for property '%s': only NUMBER properties support comparison operators.";

// Search filter validation messages
public static final String SEARCH_INVALID_CONNECTOR = "Invalid connector '%s'. Supported values: AND, OR";
public static final String SEARCH_INVALID_OPERATOR = "Invalid operation '%s'. Supported values: EQ, NEQ, CONTAINS, NOT_CONTAINS, STARTS_WITH, ENDS_WITH, GT, GTE, LT, LTE";
public static final String SEARCH_INVALID_FIELD = "Unknown field '%s'. Supported fields: template, identifier, name, relation, property.{name}, relation.{name}, relation.{name}.identifier, relation.{name}.name, relations_as_target, relations_as_target.{name}.identifier, relations_as_target.{name}.name";
public static final String SEARCH_TOO_MANY_CRITERIA = "Search filter exceeds maximum of %d total criteria";
public static final String SEARCH_NESTING_TOO_DEEP = "Search filter exceeds maximum nesting depth of %d";
public static final String SEARCH_CRITERION_MISSING_FIELD = "A criterion node must have a non-blank 'field'";
public static final String SEARCH_CRITERION_MISSING_OPERATION = "A criterion node must have a non-blank 'operation'";
public static final String SEARCH_CRITERION_MISSING_VALUE = "A criterion node must have a non-blank 'value'";
public static final String SEARCH_GROUP_MISSING_CONNECTOR = "A group node must have a non-blank 'connector'";
public static final String SEARCH_GROUP_MISSING_CRITERIA = "A group node must have a non-empty 'criteria' list";
public static final String SEARCH_QUERY_TOO_LONG = "Search query must not exceed %d characters";
public static final String SEARCH_NUMERIC_OPERATOR_REQUIRES_PROPERTY = "Operator '%s' is only valid for property.{name} fields";
public static final String SEARCH_INVALID_SORT_FIELD = "Invalid sort field '%s'. Supported fields: identifier, name, templateIdentifier";
public static final String SEARCH_INVALID_SORT_FORMAT = "Invalid sort expression '%s'. Expected format: field or field:asc|desc";
public static final String SEARCH_PAGE_SIZE_TOO_LARGE = "Page size must not exceed %d";
public static final String SEARCH_PAGE_INVALID = "Page index must be 0 or greater";
public static final String SEARCH_SIZE_INVALID = "Page size must be greater than 0";
public static final String SEARCH_NUMERIC_OPERATOR_INVALID_VALUE = "Value '%s' is not a valid number for operator '%s'";
public static final String SEARCH_NUMERIC_OPERATOR_PROPERTY_TYPE_MISMATCH = "Property '%s' in template '%s' is of type %s; operators GT, GTE, LT, LTE require type NUMBER";

}

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
package com.decathlon.idp_core.domain.exception.filter;

/// Domain exception thrown when the `q` filter query string contains invalid syntax.
///
/// **Business semantics:** Signals that the caller provided a malformed filter query
/// for the `GET /api/v1/entities/{template}?q=` endpoint. This exception should be
/// mapped to HTTP 400 Bad Request by the infrastructure layer.
public class InvalidFilterDslException extends RuntimeException {

public InvalidFilterDslException(String message) {
super(message);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
package com.decathlon.idp_core.domain.exception.search;

/// Domain exception thrown when a search filter tree or free-text query contains invalid syntax.
///
/// **Business semantics:** Signals that the caller provided a malformed search request
/// for the `POST /api/v1/entities/search` endpoint. This exception should be mapped to
/// HTTP 400 Bad Request by the infrastructure layer.
public class InvalidSearchQueryException extends RuntimeException {

public InvalidSearchQueryException(String message) {
super(message);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
package com.decathlon.idp_core.domain.model.entity;
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RawSearchFilterNode représente un arbre de filtre brut utilisé pour le parsing et la validation, plutôt qu'une entité métier.
Je me demande si le package domain.model.entity reflète bien sa responsabilité.
Est-ce qu'un package dédié au search/filter (domain.search, domain.filter etc.) ne serait pas plus explicite ?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Effectivement j'ai déplacé les objets domaines liés au search dans un package dédié domain.search 👍


import java.util.List;

/// A domain-native raw node of the search filter tree, produced by the infrastructure mapper
/// before domain parsing and validation.
///
/// **Responsibility:** Carries the unvalidated, string-only representation of a filter tree
/// from the API adapter into the domain parser. All fields are raw strings — no enums,
/// no framework types. Structural conversion from [com.decathlon.idp_core.infrastructure.adapters.api.dto.in.FilterNodeDtoIn]
/// to this type is handled by the infrastructure mapper; validation and enum resolution
/// are handled by the domain parser.
///
/// **Nodes:**
/// - [Group] — a logical group with a raw connector string and child nodes
/// - [Criterion] — a leaf predicate with raw field, operation, and value strings
public sealed interface RawSearchFilterNode {

/// A logical group combining multiple child [RawSearchFilterNode]s.
///
/// @param connector raw connector string (e.g. "AND", "OR"); may be null or
/// blank until validated
/// @param nodes child nodes; may be null until validated
record Group(String connector, List<RawSearchFilterNode> nodes) implements RawSearchFilterNode {
public Group {
nodes = nodes != null ? List.copyOf(nodes) : List.of();
}
}

/// A leaf predicate in the filter tree.
///
/// @param field raw field name (e.g. "template", "property.language"); may be
/// null until validated
/// @param operation raw operation string (e.g. "EQ", "CONTAINS"); may be null
/// until validated
/// @param value raw value string; may be null until validated
record Criterion(String field, String operation, String value) implements RawSearchFilterNode {
}
}
Loading
Loading