From f4330d773aed89f4decf50ff8d191d69a23a00e0 Mon Sep 17 00:00:00 2001 From: Sky Brewer Date: Mon, 29 Jun 2026 14:13:09 +0200 Subject: [PATCH] docs(api): add API conventions, migrations, and README Document API conventions including RSQL query syntax, bulk operations, jobs/batch definitions, and migration notes under dev-docs/api/. Co-Authored-By: Claude Opus 4.8 --- dev-docs/api/README.md | 53 +++++ dev-docs/api/conventions.md | 397 ++++++++++++++++++++++++++++++++++++ dev-docs/api/migrations.md | 93 +++++++++ 3 files changed, 543 insertions(+) create mode 100644 dev-docs/api/README.md create mode 100644 dev-docs/api/conventions.md create mode 100644 dev-docs/api/migrations.md diff --git a/dev-docs/api/README.md b/dev-docs/api/README.md new file mode 100644 index 00000000..6f07bf77 --- /dev/null +++ b/dev-docs/api/README.md @@ -0,0 +1,53 @@ +# ChannelFinder API design docs + +Internal design notes for the ChannelFinder REST API — not user-facing reference +(that is the live [Swagger UI](http://localhost:8080/swagger-ui/index.html) and +the [ReadTheDocs site](https://channelfinder.readthedocs.io/en/latest/)). + +## Contents + +| File | What it is | +| ----------------- | ------------------------------------------------------------------------------------------ | +| `migrations.md` | Why today's API (v0) is being reshaped into v1, the consolidation targets, and the rationale per decision. | +| `conventions.md` | The REST rules v1 follows — HTTP semantics, status codes, naming, errors, pagination. | + +Read migrations.md first for the direction and reasoning, then conventions.md for +the binding rules. + +## The domain in one paragraph + +ChannelFinder is a directory of **channels**. A channel has a unique `name`, an +`owner`, a set of **properties** (name-value pairs), and a set of **tags** +(names). Properties and tags are defined independently and then attached to +channels. Clients query for channels matching property/tag/name expressions. +Alongside the three domain resources are two operational resources — jobs +(handles on async work) and processors (channel post-processors) — plus the +out-of-band Actuator surface for service info, health, and metrics. See the +top-level [README](../../README.md) for what ChannelFinder is and how to run it. + +## Use cases + +Who uses ChannelFinder and how — what the design is answerable to: + +| Actor | Access | Auth | Cadence / scale | +| -------------------------------------------- | -------------------- | ----------------------------- | ------------------------------------------------ | +| RecCeiver / channel producer | write | `channel` (+`property`/`tag`) | Bursty: thousands of channels per IOC start/stop | +| Phoebus & other read clients | read | none | Interactive and frequent | +| Metadata curator / admin | write | `tag`, `property`, `admin` | Occasional, deliberate | +| Bundled admin web UI (`static/`) | read + write | `channel`, `tag`, `property` | Ad-hoc, browser, same-origin | +| Channel processors (e.g. Archiver Appliance) | internal side effect | n/a (server-side) | Automatic on every create/update | +| Operators / monitoring | read | none | Continuous scrape | + +How those patterns shape the API: + +- **Bulk-first writes.** Producers reconcile thousands of channels per IOC, so a whole reconcile is one `POST /channels:batch` command, and an oversized one can run as an async job. +- **Open, search-centric reads.** Discovery is unauthenticated and query-dominated, so v0's three paging/count surfaces collapse into one `GET /channels`, with richer filtering deferred to RSQL. +- **Definitions gate attachments.** Properties and tags are defined independently and then attached, so they are both top-level resources and channel sub-resources, and attaching an undefined one is a `422`. +- **Metadata drives side effects.** Channel properties (`archive`, `archiver`, `pvStatus`) trigger processors, so a run is an explicit `POST /channels:process` command and processors are inspectable via `/processors`. +- **Some work outlives a request.** A directory-wide processor run or an oversized bulk write returns a job the client polls. + +## Status + +These docs describe a target, not a frozen contract. The codebase is structured +for versioned controllers (`web/v0/...`), leaving room to add `v1` alongside +without breaking existing clients. diff --git a/dev-docs/api/conventions.md b/dev-docs/api/conventions.md new file mode 100644 index 00000000..8011e878 --- /dev/null +++ b/dev-docs/api/conventions.md @@ -0,0 +1,397 @@ +# ChannelFinder REST API conventions + +Design rules for the v1 ChannelFinder HTTP API. They describe the target style, +not the current API, which predates them; migrations.md covers the gap and the +rationale. + +The domain has three resources: + +- **channels** — a `name` (unique identifier), an `owner`, a set of properties, and a set of tags. +- **properties** — a name-value definition (`name`, `owner`) attachable to channels with a per-channel `value`. +- **tags** — a name-only label (`name`, `owner`) attachable to channels. + +Two operational resources sit alongside the domain — addressable resources that +model machinery, not directory data: + +- **jobs** (`/jobs/{job_id}`) — the handle on a long-running async operation. +- **processors** (`/processors`) — the channel post-processors; inspect and configure here, run with `POST /channels:process`. + +Service metadata, health, and metrics are not resources; they live on the Spring +Boot Actuator surface (`/actuator/*`), outside the versioned API. + +On a channel, properties serialize as a name→value map +(`"properties": { "cell": "2" }`) and tags as a name list +(`"tags": ["active"]`): the per-channel datum is just the value, the `owner` is +carried by the definition under `/properties`, and a property name cannot appear +twice. + +## Hard rules + +- **Nouns, not verbs, in resource paths.** `/channels`, not `/channels/process`. A genuinely non-CRUD action uses an explicit colon-verb command — see Commands. +- **Plural resources, consistently** — `/channels`, `/properties`, `/tags`; never `/channel`. +- **HTTP method semantics** — `GET` never mutates state. `POST` creates, `PUT` replaces (full representation), `PATCH` partially updates, `DELETE` removes. +- **Idempotency** — `GET`, `PUT`, `DELETE` must be idempotent. `POST` and `PATCH` need not be. +- **Correct status codes** — 2xx success, 4xx client fault, 5xx server fault. Never `200` with an error body. `DELETE` returns `204 No Content`. + - `400 Bad Request` for syntax errors: malformed JSON, missing required fields, wrong types, invalid parameter formats. + - `422 Unprocessable Entity` for semantic violations: a well-formed request that fails a business rule (name collision, attaching an undefined property/tag, referencing a missing channel). + - `401 Unauthorized` for an unauthenticated write; `403 Forbidden` for an authenticated caller lacking the required role. +- **Error responses use [RFC 9457 Problem Details](https://www.rfc-editor.org/rfc/rfc9457)** (`type`, `title`, `status` required; `detail`, `instance` optional) with content type `application/problem+json`. Use `"about:blank"` as `type` for generic HTTP errors. +- **Creating `POST` returns `201 Created`** with the full created resource and a `Location` header. `PUT`/`PATCH` updates return `200 OK` with the full updated resource. Bare `201`/`200` with no body are not allowed — callers should not need a follow-up `GET` for server-assigned fields. +- **Nullable schema properties must be listed in `required`.** A nullable-and-optional property generates a triple-state type (`prop?: T | null`) in generated clients. If the backend always sends the key (null when absent), add it to `required`; if it omits the key entirely, drop `nullable`. +- **List vs detail representations** — a collection `GET` returns a lean summary schema; an item `GET` and create/update responses return the full schema. Never use `null` to mean "omitted in the list projection" — split the schema instead. Reserve `null` for genuine domain optionality. +- **Request and response are separate schemas.** A write body holds only what the caller may set; the response holds the read projection. Never reuse one schema for both, and never put a field on a write body only to ignore it (e.g. a definition's `channels[]` on a property write). +- **Attachment is written only through the channel.** `/properties` and `/tags` define and read the vocabulary; a channel's membership is written only through `/channels/{channel_name}/properties` and `/channels/{channel_name}/tags`. No second path mutates membership from the definition side. + +## Authentication & authorization + +Access is split by HTTP method, not by path: + +- **Reads (`GET`) are unauthenticated.** +- **Writes (`PUT`/`POST`/`PATCH`/`DELETE`) require authentication** and a role scoped to the resource type — `channel`, `property`, `tag`, or `admin`. Roles derive from group membership. +- `401` for a missing/invalid credential; `403` for a valid credential without the required role. +- **Role is the only authorization gate.** An `owner` is recorded for provenance but not enforced per call — a caller with the right role may write a resource owned by another. +- **Two transports: HTTP Basic and Bearer tokens.** Bearer application/service tokens are recommended for machine producers (e.g. RecCeiver); Basic is kept for compatibility. A request carries exactly one scheme — never both in one `Authorization` header. + +## Naming + +- **Lowercase kebab-case** for literal path segments — `/channels`, `/channels/{channel_name}/properties`. +- **`snake_case`** for path parameters, query parameters, and JSON properties in both directions (`channel_name`, `property_name`, `next_cursor`, `total_count`, `sort_id`). +- A query or filter value that names a field (e.g. a sort key) uses the same spelling as the JSON property. +- Sort is `sort=:` where direction is `asc` or `desc` — never a boolean flag. Ascending is the default. +- Use standard hyphenated HTTP header names; do not invent `snake_case` headers. +- `:camelCase` for command/action segments (colon verbs). + +## Operation IDs + +Every operation must have a unique, stable `operationId` in `camelCase`. +springdoc-openapi maps it directly to the generated controller method name. + +**Pattern:** `{verb}{Resource}` or `{verb}{Resource}{SubResource}` — no +separator, no tag prefix. Use a semantic verb, not the HTTP method name +(`createChannel`, not `postChannel`). + +| Verb | When to use | +| -------- | ---------------------------------------------------------------- | +| `list` | Collection `GET` returning many items | +| `get` | Single-resource `GET` | +| `query` | Collection `GET` that is a search/filter façade, not a plain list | +| `create` | `POST` that persists a new resource | +| `update` | `PATCH`, or `PUT` that replaces a resource | +| `delete` | `DELETE` | +| `add` | `PATCH` that attaches/merges properties/tags without removing the unmentioned | +| `remove` | Detaching a property/tag from a channel | +| `batch` | Mixed-method bulk write command — `POST /channels:batch` | +| `process`| Processor run command — `POST /channels:process` | +| `cancel` | Colon-verb command that stops a resource — `POST /jobs/{job_id}:cancel` | + +Examples: + +- `GET /channels` → `queryChannels` +- `GET /channels/{channel_name}` → `getChannel` +- `POST /channels` → `createChannel` +- `POST /channels:batch` → `batchChannels` +- `PATCH /channels/{channel_name}` → `updateChannel` +- `DELETE /channels/{channel_name}` → `deleteChannel` +- `GET /channels/{channel_name}/properties` → `listChannelProperties` +- `PUT /channels/{channel_name}/properties` → `updateChannelProperties` +- `PATCH /channels/{channel_name}/properties` → `addChannelProperties` +- `GET /properties/{property_name}/values` → `listPropertyValues` +- `PUT /channels/{channel_name}/properties/{property_name}` → `updateChannelProperty` +- `DELETE /channels/{channel_name}/tags/{tag_name}` → `removeChannelTag` +- `POST /channels:process` → `processChannels` +- `GET /processors` → `listProcessors` +- `GET /processors/{processor_name}` → `getProcessor` +- `PATCH /processors/{processor_name}` → `updateProcessor` +- `GET /jobs/{job_id}` → `getJob` +- `POST /jobs/{job_id}:cancel` → `cancelJob` + +## Querying channels + +Channel search uses plain, named, documented query parameters: + +- Name glob, page size, and sort: `GET /channels?name=SR*&size=50&sort=channel_name:asc`. +- Tag presence and property values are named parameters too — `tag=active`, `cell=2`. +- **Multiple parameters are ANDed.** That conjunction of simple equality/glob filters is the entire query algebra of v1's first surface — no magic characters, no `GET`-with-body, no operators smuggled into values. + +This drops v0's implicit boolean DSL (`!`-suffix negation, `|,;` disjunction, +`prop!=*` for absence). Negation, OR across values, and grouped precedence belong +to the RSQL layer below. + +### Pagination + +Two contracts, both stateless at the API boundary (no per-client cursor session): + +- **Offset** (`size` + `from`) — the default for interactive discovery, bounded by the backing store's result window. +- **Opaque `cursor` token** — for full traversal (bulk export, "process all"), returned as `next_cursor` and passed back as `?cursor=…`. + +The cursor is opaque so the storage engine's pagination primitive never reaches +the contract; the backend maps it to whatever the engine currently uses +(`search_after` + a point-in-time id today). Query responses carry +`{ total_count, channels[], next_cursor? }`. + +### Richer queries: RSQL + +When the simple AND-of-filters surface is not enough, the richer query layer is +**[RSQL](https://github.com/jirutka/rsql-parser)** (a readable superset of FIQL) +carried in a single `filter` query parameter: + +``` +GET /channels?filter=(cell==2,cell==3);tag!=disabled +``` + +`;` is AND, `,` is OR, and parentheses group — expressing the cross-field OR and +grouped precedence v0 cannot. It is a strict superset of v0's query capability; +every v0 construct maps: + +| v0 capability | v0 form | RSQL | +| ------------------------ | ---------------- | ---------------------------------- | +| Name glob | `~name=SR*` | `name==SR*` | +| AND across fields | distinct params | `name==SR*;cell==2` | +| OR within a field | `~tag=a,b` | `tag=in=(a,b)` / `(tag==a,tag==b)` | +| Tag presence | `~tag=active` | `tag==active` | +| Tag negation | `~tag!=active` | `tag!=active` / `tag=out=(a,b)` | +| Property equality (glob) | `cell=2` | `cell==2` | +| Property value negation | `cell!=2` | `cell!=2` | +| Property present | `cell=*` | `cell==*` | +| Property absent | `cell!=*` | `cell!=*` | + +Three mapping rules must be written into the spec — they pin down semantics v0 +left implicit: + +1. **Selector namespacing.** A property can be named `name` or `tag`, so reserve those selectors for the channel name and tags and address properties unambiguously (e.g. a `property.` prefix). +2. **Wildcard + case-insensitivity.** `*` and `?` in `==`/`!=` arguments are wildcards; matching is case-insensitive. +3. **Absence vs. value negation.** `cell!=*` is "property absent"; `cell!=2` is "has `cell` and value ≠ 2". + +**Alternatives.** **OData `$filter`** is equally capable and more +widely tooled, but verbose and a large surface to adopt for one slice. +**JSON:API filtering** is a convention, not a full algebra — it needs a custom +profile to pin down OR/grouping. **GraphQL**'s one genuine pull is field +selection, not filtering: it has no native filter algebra (negation/OR/grouping +still need a bespoke `where` input), brings a `200 OK`-with-`errors` convention +that fights correct status codes, and a single POST surface that defeats `GET` +caching. The cheaper REST answer to over-fetching is sparse fieldsets +(`GET /channels?fields=name,properties.temperature`) plus the list-vs-detail +projections already required here. + +### Aggregating property values + +A second, faceting read answers "what values does a property take, and how common +is each?" + +**`GET /properties/{property_name}/values`** returns that facet, scoped by the +same query parameters as `GET /channels` — so +`GET /properties/ioc_id/values?name=SR:*` is "distinct `ioc_id`s among `SR:*` +channels." The response is a values projection, sorted and paginated over the +values (not the channels): + +```jsonc +GET /properties/ioc_id/values +{ + "total_distinct": 42, + "values": [ + { "value": "ioc01", "count": 130 }, + { "value": "ioc02", "count": 118 } + ], + "next_cursor": "…" // optional, same opaque-cursor contract as channels +} +``` + +`sort` orders by `value` or `count` (`sort=count:desc` for the largest groups +first); `size`/`from` and the opaque `cursor` page the values as for channel +queries. The aggregation is opaque on purpose — an Elasticsearch `terms` (+ +`cardinality`) aggregation today, exposing only values and counts, never internal +buckets, `doc_count_error`, or `after_key`. + +## Bulk operations + +A producer reconcile is rarely one kind of change — it creates some channels, +updates others, and removes the rest in a single pass. That heterogeneous write +is not a single CRUD operation, so it is a colon-verb command. The client computes +the change set; the server applies it and does not reconcile — no server-side +declarative sync, no query-scoped delete. A producer that does not track what it +holds builds the operations by diffing its desired set against a lean +`GET /channels?&fields=name`. + +**`POST /channels:batch`** takes one document that groups operations by HTTP +method, each method keeping its single-resource meaning: + +```jsonc +POST /channels:batch +{ + "post": [ { "name": "SR:C01", "properties": { "cell": "1" }, "tags": ["active"] } ], + "put": [ { "name": "SR:C02", "properties": { "cell": "2" }, "tags": [] } ], + "patch": [ { "name": "SR:C03", "properties": { "cell": "3" } } ], + "delete": [ "SR:C99" ] +} +``` + +The keys are HTTP method tokens, lower-cased for the `snake_case` wire rule; each +maps to an array of channel representations (or, for `delete`, channel names). Each +item behaves exactly as the same method on the single-channel endpoints: `put` +replaces the whole channel (an omitted property/tag is dropped); `patch` merges +(unmentioned properties/tags are kept); `post` creates and nothing else. A +reconcile that must drop disappeared metadata uses `put` with the full desired +state. + +- **Per-item, not transactional.** Each item is applied and reported independently; a `200`/`job.succeeded` means the batch ran, not that every item passed. The outcome is one `results[]`, keyed by channel `name` (present on every item, including `delete`, so no correlation token is needed): each entry carries its `status`, a failure its RFC 9457 `error`. A sync batch returns it inline, an async batch in `job.result`. +- **Defined order.** Creates (`post`), then replaces (`put`), then merges (`patch`), then deletes (`delete`); document order within a method. No cross-item referencing, and a name appearing twice is a malformed document (`400`). +- **Sync by default, async on request.** The batch runs inline unless the client sends `Prefer: respond-async` (or it exceeds the server's size threshold), in which case it returns a job. + +```jsonc +{ "results": [ + { "name": "SR:C01", "method": "post", "status": 201 }, + { "name": "SR:C03", "method": "patch", "status": 422, + "error": { "type": "about:blank", "title": "Unprocessable Entity", + "status": 422, "detail": "property 'cell' is not defined" } }, + { "name": "SR:C99", "method": "delete", "status": 204 } +] } +``` + +**Alternatives.** The established mixed-method formats — SCIM 2.0 Bulk, JSON:API +Atomic Operations, OData `$batch` — use one flat `operations[]` list, each item +naming its method. Grouping by method keeps the four arrays homogeneous, so each +generates a clean client type instead of the `oneOf` discriminated union OAS-3.0 +codegen handles worst, and channel `name` already supplies the per-item key those +formats add a `bulkId` for. The per-item, non-transactional model follows +Elasticsearch `_bulk` (the backend), not the all-or-nothing of those changeset +formats. + +Single-resource writes keep the plain methods — `POST /channels`, +`PATCH /channels/{channel_name}`, `DELETE /channels/{channel_name}`. A channel's +property/tag lists stay directly addressable, with the add-vs-replace split: + +- `PUT /channels/{channel_name}/properties` replaces the channel's whole property list (idempotent). +- `PATCH /channels/{channel_name}/properties` adds or updates named properties without removing the unmentioned. + +## Commands + +Two operations are genuinely actions, not single-resource mutations — the one +place a colon-verb command fits: + +- **`POST /channels:process`** runs the channel processors over a set of channels, replacing the v0 `PUT /processors/process/{query,channels,all}` paths. +- **`POST /channels:batch`** applies a mixed create/replace/update/delete — see Bulk operations. + +Colon-verb commands signal a mutation/side effect; never use them for read-only +operations. Both can run over the whole directory and so return a job rather than +blocking. + +## Processors + +Channel processors are server-side post-processors that react to channel metadata +— the Archiver Appliance processor (`AAChannelProcessor`) registers a channel for +archiving when it carries the right `archive`/`archiver`/`pvStatus` properties. +`/processors` is where this machinery is inspected and configured; it holds no +directory data. + +Running the processors over channels is the `POST /channels:process` command +(always async), not a write under `/processors`. `/processors` itself is +define/inspect plus a small config surface: + +- **`GET /processors`** → `listProcessors` — registered processors with their info and config (`name`, `enabled`, processor-specific fields). +- **`GET /processors/{processor_name}`** → `getProcessor` — one processor's detail. +- **`PATCH /processors/{processor_name}`** → `updateProcessor` — partial update of mutable config, principally toggling it on or off: + + ```jsonc + PATCH /processors/aa + { "enabled": false } + ``` + + `enabled` is mutable state on the resource, so it is a `PATCH` on the noun — not a `PUT .../enabled` flag sub-path. + +`GET /processors` is unauthenticated; the `PATCH` config write requires the +`admin` role. Replaces v0's `GET /processors`, `GET /processors/count`, and +`PUT /processor/{name}/enabled`. + +## Asynchronous operations (jobs) + +Some operations cannot finish inside one request/response: a directory-wide +processor sweep, or a bulk write of tens of thousands of channels. Rather than +hold the connection open, v1 models the work as a job resource the client submits +and polls. Its two operations are `getJob` (`GET /jobs/{job_id}`) and `cancelJob` +(`POST /jobs/{job_id}:cancel`). + +**The pattern.** + +1. The client submits the work. The server returns **`202 Accepted`** with a `Location: /jobs/{job_id}` header and a `Retry-After` hint. The body is the initial job representation. +2. The client polls **`GET /jobs/{job_id}`** until the job leaves a non-terminal state. `Retry-After` is echoed while running. +3. The job's `result` carries the outcome once terminal. + +**The job resource.** `GET /jobs/{job_id}` returns: + +```jsonc +{ + "job_id": "…", + "kind": "process_channels" | "batch_channels" | …, + "status": "pending" | "running" | "succeeded" | "failed" | "canceled", + "created_at": "…", + "updated_at": "…", + "progress": { "done": 8200, "total": 12000 }, // optional + "result": { … }, // present when terminal + "error": { … } // RFC 9457, on infra failure +} +``` + +A job is a noun with a lifecycle: async work is server state, so it is an +addressable resource, not a hidden session id. Jobs are retained for a bounded +window, then reaped. + +**Opting into async.** Endpoints declare their mode rather than the server +silently choosing `200`-or-`202`: + +- **`POST /channels:process`** (and the other process commands) are always async — unbounded by directory size. They return `202` + a job. +- **`POST /channels:batch`** is synchronous by default and becomes async when the client sends **`Prefer: respond-async`** ([RFC 7240](https://www.rfc-editor.org/rfc/rfc7240)). The server may answer a synchronous request with `202` if the batch exceeds a configured size, signalling it with `Preference-Applied`. + +**`job.succeeded` ≠ every item succeeded.** A batch's outcome is per-item — some +channels persist, some fail a business rule. That per-item report is one schema, +two delivery modes: a synchronous batch returns the `results[]` array inline; an +async batch puts the identical `results[]` in `job.result`. Each failed item +embeds its own RFC 9457 problem detail. A job reaching `succeeded` means the +operation ran to completion, not that every item passed — failures live in the +results, never collapsed into a `200`-with-error-body or a `207 Multi-Status`. + +**Idempotent submission.** A submission may carry an **`Idempotency-Key`** header; +a repeat with the same key returns the same job instead of starting a second. + +**Cancellation.** A running job is stopped with **`POST /jobs/{job_id}:cancel`**, +moving it to `canceled`; already-applied work in a partially-run batch is not +rolled back, and the `result` records how far it got. + +## Application info & observability + +Service metadata (name, version, build), backend (Elasticsearch) status, health, +and metrics are not domain resources and do not live in the versioned API. They +are served by Spring Boot Actuator, already enabled +(`management.endpoints.web.exposure.include=prometheus, metrics, health, info`): + +- **`GET /actuator/health`** — liveness/readiness and backend reachability. +- **`GET /actuator/info`** — service name, version, and build metadata. +- **`GET /actuator/metrics`**, **`GET /actuator/prometheus`** — metrics scrape. + +These sit outside the `/v1` resource space and follow Actuator's own conventions, +not the rules above — do not restyle them with `snake_case`, `operationId`s, or +Problem Details. v0's custom `GET /ChannelFinder` service-info endpoint is +replaced by `/actuator/info` + `/actuator/health`. + +## Soft rules + +- At most **2 levels of resources** — `/channels/{channel_name}/properties/{property_name}` is the floor; do not nest deeper. +- Prefer `PATCH` over `PUT` for partial edits. +- Query parameters (only) filter lists; they never carry a request body's worth of structure when a body is the honest representation. + +## Codegen constraints + +If a typed client is generated via an OAS-3.0-era toolchain, array-form nullables +(`type: ["string", "null"]`) fall back to `any`. Until a 3.1-aware generator is +in use, write every nullable as `oneOf`: + +```yaml +# ✅ correct +my_field: + oneOf: + - type: string + - type: "null" + +# ❌ wrong — codegen produces `any` +my_field: + type: ["string", "null"] +``` diff --git a/dev-docs/api/migrations.md b/dev-docs/api/migrations.md new file mode 100644 index 00000000..783a34c2 --- /dev/null +++ b/dev-docs/api/migrations.md @@ -0,0 +1,93 @@ +# Migration: v0 → v1 + +Why today's ChannelFinder API (the live `web/v0/controller` classes) is being +reshaped into a more RESTful surface, the resulting targets, and the decisions +behind each so they need not be re-litigated. v1 is the target direction; +conventions.md is the binding rule reference. The codebase already lives under +`web/v0/...`, leaving room for a `v1` package alongside it without breaking +existing clients. + +--- + +## Legacy problems to avoid + +| Issue | Examples / pattern | +| ---------------------------- | -------------------------------------------------------------------------------------------------------------------------- | +| Inverted method semantics | `PUT` creates (`create`, `create_1`…`create_5`), `POST` updates — the reverse of REST norms. | +| RPC-style paths | `PUT /resources/processors/process/{query,channels,all}` — verbs in the path, not resources + methods. | +| Non-descriptive operationIds | Generated, collision-suffixed: `list`, `list_1`, `read`, `read_1`, `query_1`, `create_1`…`create_5`. | +| Untyped schemas | Bodies typed as bare `type: object`; responses served as `*/*` instead of `application/json`. | +| Three ways to page/count | `/scroll` + `/scroll/{scrollId}`, `/channels/count`, and `/channels/combined` all coexist. | +| No error contract | No error schema; failures are unmodelled, so clients cannot rely on a shape. | +| List vs detail entangled | `Tag` embeds a full `channels[]`; a `withChannels` boolean toggles whether it is populated. | +| Request = response schema | One DTO both directions (`PropertyDto`/`Channel`), so write bodies carry read-only fields (`PropertyDto.channels`) the server ignores. | +| Properties as a list | A channel's properties are `List<{name, owner, value}>` (`entity/Channel.java`): scan to read one, duplicates allowed, `owner` repeated per entry. | +| Definitions write channels | `PUT /properties/{propertyName}/{channelName}`, `POST /properties/{propertyName}`, and matching `DELETE` mutate channel membership from the definition side. | +| Casing | `camelCase` params (`tagName`, `channelName`, `withChannels`) and the `/ChannelFinder/resources/...` prefix on every path. | +| Opaque query input | Search is a single required `allRequestParams` `MultiValueMap` blob, not named parameters. | +| Boolean logic in magic chars | AND/OR/NOT encoded in punctuation — `!` negate, `\|,;` OR, distinct params AND — an undocumented DSL (`ChannelRepository.getBuiltQuery`). | +| Long work blocks the caller | Bulk writes and `/processors/process/all` run inside the request; no handle to observe, resume, or survive a timeout. | +| Processors as RPC + flag | `GET /processors/count`, `PUT /processors/process/*` (run), and `PUT /processor/{name}/enabled` (config) tangled on one controller. | +| Bespoke service-info endpoint | `GET /ChannelFinder` returns status from a hand-rolled `InfoController` instead of Spring Boot Actuator (already on the classpath). | + +--- + +## Consolidation targets + +1. **Drop the service prefix and version the API.** `/ChannelFinder/resources/channels` → `/channels` under a `v1` base. + +2. **Restore HTTP method semantics.** `POST` creates (`201` + `Location`), `PUT` replaces, `PATCH` partially updates, `GET` reads, `DELETE` removes (`204`). + +3. **One pagination story, engine-agnostic.** Offset (`size` + `from`) for browsing plus an opaque `cursor` for full traversal, returning `{ total_count, channels[], next_cursor? }`. The v0 `scroll` endpoints, `/channels/count`, and `/channels/combined` collapse into one `GET /channels`. + +4. **One wire vocabulary, `snake_case` everywhere.** `channel_name`, `property_name`, `tag_name`, `total_count`, `next_cursor`, `sort_id` — in paths, query params, and JSON alike. + +5. **Sub-resources for attachments.** `/channels/{channel_name}/properties[/{property_name}]` and `/channels/{channel_name}/tags[/{tag_name}]`, replacing the v0 `/properties/{propertyName}/{channelName}` paths that read parent-from-child. + +6. **A real error contract.** RFC 9457 Problem Details with `application/problem+json`, distinguishing `400` (syntax) from `422` (business-rule). + +7. **Stable, semantic operationIds and typed schemas.** Every operation gets a `{verb}{Resource}` id and a named schema; no `type: object` bodies; responses served as `application/json`. + +8. **Preserve the access model.** Keep v0's method-based split — `GET` open, writes authenticated and role-gated (`channel`/`property`/`tag`/`admin`) — and make failure codes explicit (`401` vs `403`). + +9. **Read property values, not just channels.** Add `GET /properties/{property_name}/values`, returning the distinct values a property takes and how many channels hold each, scoped by the same query as `GET /channels`. + +10. **Processors become a resource plus a command.** The v0 `PUT /processors/process/*` paths become the `POST /channels:process` command; `GET /processors`, `GET /processors/count`, and `PUT /processor/{name}/enabled` become the `/processors` resource (`GET` list/detail, `PATCH` config). + +11. **Application info moves to Actuator.** Drop `GET /ChannelFinder` in favour of `/actuator/info` + `/actuator/health` (metrics/Prometheus already there). + +--- + +## Design decisions and rationale + +**Channels, properties, and tags stay as three resources.** The domain model is unchanged; only the HTTP surface is. Properties and tags remain definable independently and attachable to channels with a per-channel value (properties) or as a bare label (tags). + +**Attachment is a sub-resource, not a two-segment path.** v0's `PUT /tags/{tagName}/{channelName}` puts the child's identity in the parent's path. `PUT /channels/{channel_name}/tags/{tag_name}` reads the way the relationship is owned — a channel has tags — and stays within two levels of nesting. + +**Add vs. replace is explicit per method.** On a channel's property/tag list, `PUT` replaces the whole list and `PATCH` adds or updates without removing the unmentioned. This makes the bulk-writer's intent (push a full set vs. patch a few) explicit instead of inferred from payload shape. + +**Bulk writes become one batch command.** A reconcile mixes creates, updates, and deletes — in v0, several separate calls with no shared outcome. v1 expresses it as a single `POST /channels:batch` whose body groups operations by HTTP method: one request, one job, one `Idempotency-Key`, one per-item result set. A mixed create/update/delete is not any single CRUD operation, which is why it is a colon-verb command; plain methods stay for single-resource writes. + +**Scroll → opaque cursor, not a server-held session.** v0 leaked Elasticsearch's `scroll`/`scrollId` into the URL, making an engine detail the public contract. The engine is moving on (`scroll` is deprecated upstream in favour of `search_after` + a point-in-time id), and reviving a server-held session would re-adopt the stateful model being retired. v1 exposes an opaque `cursor` the backend maps to whatever the engine currently uses, so future engine changes are invisible to clients. Folding count into the response (`total_count`) removes the separate `/count` and `/combined` round-trips. + +**`withChannels` toggle → split representations.** Instead of one schema whose heavy `channels[]` field is conditionally populated, list and detail projections are separate schemas. Clients get a predictable shape and collection reads stay lean. + +**Request and response are separate schemas.** v0 reuses one DTO both directions, so a write body carries read-only fields the server ignores. v1 gives each direction its own schema — a write body holds only what the caller may set, the response holds the full read projection. + +**A channel's properties are a name→value map, not a list.** v0's `List<{name, owner, value}>` forces a scan to read one property and permits duplicate names. v1 serializes properties as a map (`"properties": { "cell": "2" }`) and tags as a name list. The per-channel datum is just the value; `owner` lives on the definition, consistent with ownership being recorded, not enforced. The map makes a property lookup direct and removes duplicate-name ambiguity by construction. + +**Attachment is written only through the channel.** v0 lets channel membership be written from the definition side, a second write path parallel to writing the channel. v1 keeps `/properties` and `/tags` for defining and reading the vocabulary; membership is written only through the channel sub-resources. One fact, one write path. + +**Long-running work becomes a job.** In v0, unbounded operations (a `/processors/process/all` sweep, a tens-of-thousands-channel reconcile) run inside the request: they hit proxy/client timeouts with no honest "still going" status, offer no handle to observe progress or cancel, and duplicate work when a timed-out caller retries. v1 returns `202 Accepted` + a `/jobs/{job_id}` resource the client polls. Process commands are always async; bulk writes opt in with `Prefer: respond-async`. That turns the three failures into contract: a non-terminal `status`, a pollable resource with `progress` and per-item `result`, and an `Idempotency-Key` so a retry rejoins the same job. + +**Richer queries use RSQL.** v1's main search querying only includes simple AND-of-equality/glob query and drops v0's implicit boolean DSL (`!` negate, `|,;` OR, distinct-params AND) rather than carrying that undocumented punctuation forward. The richer layer restoring negation, OR, and grouped precedence is RSQL in a single `filter` parameter — a strict superset of v0's capability backed by a standard grammar and mature libraries, not a re-invented DSL. Alternatives include: OData, a JSON:API filter profile, and GraphQL. + +**Ownership is recorded, not enforced.** v0 records an `owner` on every resource but gates writes only on role membership; a caller with the right role can modify a resource owned by someone else. v1 keeps this: authorization is role-based only, and `owner` stays as provenance metadata, never the basis for a `403`. This matches v0's behaviour (no client breaks on a newly enforced barrier) and avoids per-resource checks on the bursty machine producers that do the heaviest writes. Owner-scoped authorization can be added later if a facility needs it. + +**Auth transport: Basic and Bearer.** v0 authenticates writes with HTTP Basic only. v1 keeps Basic for compatibility and adds Bearer tokens, with application/service tokens the recommended path for machine producers like RecCeiver. The two schemes are alternatives, not layers: a request carries exactly one — Basic or Bearer — never both in one `Authorization` header. Reads stay unauthenticated. + +**Aggregating property values.** v0's reads are channel-centric, so learning "what values does `ioc_id` take, and how common is each?" means paging the whole result and counting client-side. v1 adds `GET /properties/{property_name}/values`, returning distinct values and per-value channel counts, scoped by the same query as `GET /channels`. It stays behind an opaque contract for the same reason as the pagination cursor: an Elasticsearch `terms` (+ `cardinality`) aggregation today, exposing only values and counts so the engine can change without breaking clients. + +**Processors: a resource and a command.** v0 tangles inspecting, running, and configuring processors on one controller via RPC verb paths and a boolean-flag sub-path. v1 splits this like properties/tags split definition from attachment. Running is the side-effecting action — the colon-verb `POST /channels:process` command, always async because it is unbounded by directory size. Inspecting and configuring is the `/processors` resource: `GET` to read, `PATCH /processors/{name}` for mutable config (chiefly `{ "enabled": false }`). `enabled` is state on the noun, so it is a `PATCH`, not a `PUT .../enabled` flag path. + +**Application info lives in Actuator.** Spring Boot Actuator is already enabled and exposes `/actuator/{health,info,metrics,prometheus}` — the standard operational surface. v1 drops the hand-rolled `GET /ChannelFinder` (`InfoController`): service metadata and version go to `/actuator/info`, liveness/backend status to `/actuator/health`. These are not domain resources and stay outside the versioned API and its conventions. The one consumer to migrate is the bundled admin UI's "Service Info" panel (`cfmanage.js`), repointed at the Actuator endpoints; rewiring the same UI's complicated v0 queries is out of scope here.