From 9df8636c6298762b5fb3c646e04d6f6575463189 Mon Sep 17 00:00:00 2001 From: Chris Barber Date: Tue, 11 Aug 2026 02:30:57 -0500 Subject: [PATCH 1/4] docs(rest): pagination total count (Prefer: count=) and exactCount option MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Documents the REST pagination total-count feature shipping in Harper v5.3.0: - reference/rest/querying.md: new "Pagination and Total Count" section covering Prefer: count=exact|estimated, the Content-Range / Range-Unit / Preference-Applied response headers, the unavailable-total (.../*) case, HEAD pre-flight, CORS exposure, and disabling exact counts per mount. - reference/rest/overview.md: adds the `exactCount` rest-mount option. - reference/rest/headers.md: adds the Prefer request header and notes the count response headers. Version badges assume v5.3.0 (next minor after 5.2) — adjust if the feature lands in a different release. Pairs with the harper core branch feat/rest-pagination-total-count. Co-Authored-By: Claude Opus 4.8 (1M context) --- reference/rest/headers.md | 15 ++++++++++++ reference/rest/overview.md | 3 +++ reference/rest/querying.md | 49 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 67 insertions(+) diff --git a/reference/rest/headers.md b/reference/rest/headers.md index 818398eb..3b880d4a 100644 --- a/reference/rest/headers.md +++ b/reference/rest/headers.md @@ -20,6 +20,8 @@ These headers are included in all Harper REST API responses: | `etag` | `"abc123"` | Encoded version/last-modification time of the returned record. Used for conditional requests. | | `location` | `/MyTable/new-id` | Returned on `POST` responses. Contains the path to the newly created record. | +Collection responses to a [count request](./querying.md#pagination-and-total-count) additionally include `Content-Range`, `Range-Unit`, and `Preference-Applied` (). + ## Request Headers ### Content-Type @@ -67,6 +69,19 @@ Accept-Encoding: gzip, br Compression is particularly effective for JSON responses. For binary formats like CBOR, compression provides diminishing returns compared to the already-compact encoding. +### Prefer + + + +Opt in to a total match count on a collection `GET`/`HEAD`, returned via the `Content-Range` response header for pagination: + +```http +GET /Product/?category=software&limit(0,25) +Prefer: count=exact +``` + +Accepts `count=exact` (precise, scans the full matched set) or `count=estimated` (fast, approximate). See [Pagination and Total Count](./querying.md#pagination-and-total-count) for the full request/response contract. + ### Authorization Credentials for authenticating requests. See [Security Overview](../security/overview.md) for details on supported authentication mechanisms (Basic, JWT, mTLS). diff --git a/reference/rest/overview.md b/reference/rest/overview.md index 74b755f7..bbfff196 100644 --- a/reference/rest/overview.md +++ b/reference/rest/overview.md @@ -33,8 +33,11 @@ rest: true rest: lastModified: true # enables Last-Modified response header support webSocket: false # disables automatic WebSocket support (enabled by default) + exactCount: false # serve Prefer: count=exact requests as estimates instead of scanning ``` + `exactCount` (default `true`) controls whether the [pagination total-count](./querying.md#pagination-and-total-count) feature honors `Prefer: count=exact`. Because an exact count scans the full matched set, set this to `false` to serve those requests as cheaper estimates on a given mount. + ## URL Structure The REST interface follows a consistent URL structure: diff --git a/reference/rest/querying.md b/reference/rest/querying.md index b05a19c0..c3d2ffaa 100644 --- a/reference/rest/querying.md +++ b/reference/rest/querying.md @@ -173,6 +173,55 @@ GET /Product/?rating=gt=3&sort(+name) GET /Product/?sort(+rating,-price) ``` +## Pagination and Total Count + + + +Use `limit(start,end)` to page through a collection, and opt in to a total match count with the `Prefer: count=` request header so a client can render pagination (for example "1-25 of 1,234") without a second request. + +Counting is opt-in: without the header, no count is computed and no count headers are returned. + +### Requesting a count + +Send a `Prefer` header on a `GET` (or `HEAD`) request to a collection: + +| Value | Meaning | +| ----------------- | --------------------------------------------------------------------------------------------------------------- | +| `count=exact` | The exact number of matching records. Scans the full matched set, so it is more expensive than the page itself. | +| `count=estimated` | A fast planner/table estimate. Cheap, approximate. | + +```http +GET /Product/?category=software&limit(0,25) +Prefer: count=exact +``` + +### Count response headers + +The count is returned in [RFC 7233](https://datatracker.ietf.org/doc/html/rfc7233)-style response headers alongside the page body: + +| Header | Example | Description | +| -------------------- | ----------------- | --------------------------------------------------------------------------------------------- | +| `Content-Range` | `items 0-24/1234` | The 0-based, inclusive range of records returned (`start-end`) out of the total matching set. | +| `Range-Unit` | `items` | The unit used by `Content-Range`. | +| `Preference-Applied` | `count=exact` | The count mode the server applied (`exact` or `estimated`). | + +```http +HTTP/1.1 200 OK +Content-Range: items 0-24/1234 +Range-Unit: items +Preference-Applied: count=exact +``` + +The response status is always `200` — `Content-Range` is informational (Harper does not use `206 Partial Content`). A `HEAD` request with `Prefer: count=` returns the count headers with no body, a cheap way to ask "how many match?" without transferring the page. When CORS is enabled, these three headers are added to `Access-Control-Expose-Headers` so browser clients can read them cross-origin. + +### Unavailable totals + +The total is reported as `*` (for example `Content-Range: items 0-24/*`) when it cannot be produced — an exact scan that reaches its internal work limit, or a query with no cardinality estimate (for example a `!=` or `contains` condition). `Preference-Applied` still echoes the requested mode, so an unavailable total (`.../*`) is distinct from a request that asked for no count. + +### Disabling exact counts + +Because an exact count scans the full matched set, a deployment can disable it per REST mount via the [`exactCount` option](./overview.md#configuration). With `exactCount: false`, a `count=exact` request is served as an estimate instead (the response reports `Preference-Applied: count=estimated`). Estimated counts are always available; the default is `true`. + ## Relationships and Joins From ffb0d4fa19e5dbcde2bc8b1704a17292ba47f3c5 Mon Sep 17 00:00:00 2001 From: Chris Barber Date: Tue, 11 Aug 2026 18:34:18 -0500 Subject: [PATCH 2/4] docs(rest): note that pagination count requires a limit() A count request without a limit() is served normally with no count headers (the core feature falls through to streaming rather than counting the whole collection). Co-Authored-By: Claude Opus 4.8 (1M context) --- reference/rest/querying.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference/rest/querying.md b/reference/rest/querying.md index c3d2ffaa..f8445e8c 100644 --- a/reference/rest/querying.md +++ b/reference/rest/querying.md @@ -179,7 +179,7 @@ GET /Product/?sort(+rating,-price) Use `limit(start,end)` to page through a collection, and opt in to a total match count with the `Prefer: count=` request header so a client can render pagination (for example "1-25 of 1,234") without a second request. -Counting is opt-in: without the header, no count is computed and no count headers are returned. +Counting is opt-in: without the header, no count is computed and no count headers are returned. It also requires a `limit()` — a count request on an unbounded collection (no `limit()`) is served normally, with no count headers, since counting the whole collection would defeat the point of paging. ### Requesting a count From 94bdedeb3a747a09bd7aa84a6b423053b1c54262 Mon Sep 17 00:00:00 2001 From: Chris Barber Date: Tue, 11 Aug 2026 21:58:54 -0500 Subject: [PATCH 3/4] docs(rest): clarify exactCount scope wording; small querying nits MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses review feedback on #623: - Drop the confusing/inaccurate "per REST mount" / "on a given mount" scope wording for exactCount. It is not a global setting — it is read only from a component's `rest:` config (server/REST.ts), so describe it as configured "in an application's REST configuration" rather than a mount or a global option. - Refer to the header as the `Prefer` request header (`Prefer: count=exact`). - Use the `=ct=` operator spelling for the contains example, matching the operators table. Co-Authored-By: Claude Opus 4.8 (1M context) --- reference/rest/overview.md | 2 +- reference/rest/querying.md | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/reference/rest/overview.md b/reference/rest/overview.md index bbfff196..96a16019 100644 --- a/reference/rest/overview.md +++ b/reference/rest/overview.md @@ -36,7 +36,7 @@ rest: exactCount: false # serve Prefer: count=exact requests as estimates instead of scanning ``` - `exactCount` (default `true`) controls whether the [pagination total-count](./querying.md#pagination-and-total-count) feature honors `Prefer: count=exact`. Because an exact count scans the full matched set, set this to `false` to serve those requests as cheaper estimates on a given mount. + `exactCount` (default `true`) controls whether the [pagination total-count](./querying.md#pagination-and-total-count) feature honors `Prefer: count=exact`. Because an exact count scans the full matched set, set this to `false` to serve those requests as cheaper estimates for this application's REST interface. ## URL Structure diff --git a/reference/rest/querying.md b/reference/rest/querying.md index f8445e8c..902e8a3b 100644 --- a/reference/rest/querying.md +++ b/reference/rest/querying.md @@ -177,7 +177,7 @@ GET /Product/?sort(+rating,-price) -Use `limit(start,end)` to page through a collection, and opt in to a total match count with the `Prefer: count=` request header so a client can render pagination (for example "1-25 of 1,234") without a second request. +Use `limit(start,end)` to page through a collection, and opt in to a total match count with the `Prefer` request header (`Prefer: count=exact`) so a client can render pagination (for example "1-25 of 1,234") without a second request. Counting is opt-in: without the header, no count is computed and no count headers are returned. It also requires a `limit()` — a count request on an unbounded collection (no `limit()`) is served normally, with no count headers, since counting the whole collection would defeat the point of paging. @@ -216,11 +216,11 @@ The response status is always `200` — `Content-Range` is informational (Harper ### Unavailable totals -The total is reported as `*` (for example `Content-Range: items 0-24/*`) when it cannot be produced — an exact scan that reaches its internal work limit, or a query with no cardinality estimate (for example a `!=` or `contains` condition). `Preference-Applied` still echoes the requested mode, so an unavailable total (`.../*`) is distinct from a request that asked for no count. +The total is reported as `*` (for example `Content-Range: items 0-24/*`) when it cannot be produced — an exact scan that reaches its internal work limit, or a query with no cardinality estimate (for example a `!=` or `=ct=` (contains) condition). `Preference-Applied` still echoes the requested mode, so an unavailable total (`.../*`) is distinct from a request that asked for no count. ### Disabling exact counts -Because an exact count scans the full matched set, a deployment can disable it per REST mount via the [`exactCount` option](./overview.md#configuration). With `exactCount: false`, a `count=exact` request is served as an estimate instead (the response reports `Preference-Applied: count=estimated`). Estimated counts are always available; the default is `true`. +Because an exact count scans the full matched set, you can disable it in an application's REST configuration via the [`exactCount` option](./overview.md#configuration). With `exactCount: false`, a `count=exact` request is served as an estimate instead (the response reports `Preference-Applied: count=estimated`). Estimated counts are always available; the default is `true`. From 19c80d5f22f6e74c6533c2c502cb990ae42c9fb9 Mon Sep 17 00:00:00 2001 From: Chris Barber Date: Thu, 13 Aug 2026 11:43:33 -0500 Subject: [PATCH 4/4] docs(rest): exact counting is opt-in (default off); GET/HEAD + bounded limit Follows the harper #2147 review outcome: exact counting is now off by default and enabled per mount with `rest: { exactCount: true }`; count=exact is otherwise served as an estimate. Also note that counting applies to GET/HEAD only and requires a limit() within a supported page size. Co-Authored-By: Claude Opus 4.8 (1M context) --- reference/rest/overview.md | 4 ++-- reference/rest/querying.md | 21 ++++++++++++++------- 2 files changed, 16 insertions(+), 9 deletions(-) diff --git a/reference/rest/overview.md b/reference/rest/overview.md index 96a16019..37c94502 100644 --- a/reference/rest/overview.md +++ b/reference/rest/overview.md @@ -33,10 +33,10 @@ rest: true rest: lastModified: true # enables Last-Modified response header support webSocket: false # disables automatic WebSocket support (enabled by default) - exactCount: false # serve Prefer: count=exact requests as estimates instead of scanning + exactCount: true # opt in to Prefer: count=exact scans (off by default; otherwise served as estimates) ``` - `exactCount` (default `true`) controls whether the [pagination total-count](./querying.md#pagination-and-total-count) feature honors `Prefer: count=exact`. Because an exact count scans the full matched set, set this to `false` to serve those requests as cheaper estimates for this application's REST interface. + `exactCount` (default `false`) controls whether the [pagination total-count](./querying.md#pagination-and-total-count) feature honors `Prefer: count=exact`. Because an exact count scans the full matched set, it is off by default; set this to `true` to enable exact counts for this application's REST interface. A `count=exact` request is otherwise served as a cheaper estimate. ## URL Structure diff --git a/reference/rest/querying.md b/reference/rest/querying.md index 902e8a3b..139176ea 100644 --- a/reference/rest/querying.md +++ b/reference/rest/querying.md @@ -179,16 +179,16 @@ GET /Product/?sort(+rating,-price) Use `limit(start,end)` to page through a collection, and opt in to a total match count with the `Prefer` request header (`Prefer: count=exact`) so a client can render pagination (for example "1-25 of 1,234") without a second request. -Counting is opt-in: without the header, no count is computed and no count headers are returned. It also requires a `limit()` — a count request on an unbounded collection (no `limit()`) is served normally, with no count headers, since counting the whole collection would defeat the point of paging. +Counting is opt-in: without the header, no count is computed and no count headers are returned. It applies only to `GET`/`HEAD` requests and requires a `limit()` within a supported page size — a request with no `limit()`, an oversized one, or a non-numeric one is served normally with no count headers, since counting an unbounded page would defeat the point of paging. ### Requesting a count Send a `Prefer` header on a `GET` (or `HEAD`) request to a collection: -| Value | Meaning | -| ----------------- | --------------------------------------------------------------------------------------------------------------- | -| `count=exact` | The exact number of matching records. Scans the full matched set, so it is more expensive than the page itself. | -| `count=estimated` | A fast planner/table estimate. Cheap, approximate. | +| Value | Meaning | +| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | +| `count=exact` | The exact number of matching records. Scans the full matched set, so it is opt-in per mount (see below) and served as an estimate unless enabled. | +| `count=estimated` | A fast planner/table estimate. Cheap, approximate. | ```http GET /Product/?category=software&limit(0,25) @@ -218,9 +218,16 @@ The response status is always `200` — `Content-Range` is informational (Harper The total is reported as `*` (for example `Content-Range: items 0-24/*`) when it cannot be produced — an exact scan that reaches its internal work limit, or a query with no cardinality estimate (for example a `!=` or `=ct=` (contains) condition). `Preference-Applied` still echoes the requested mode, so an unavailable total (`.../*`) is distinct from a request that asked for no count. -### Disabling exact counts +### Enabling exact counts -Because an exact count scans the full matched set, you can disable it in an application's REST configuration via the [`exactCount` option](./overview.md#configuration). With `exactCount: false`, a `count=exact` request is served as an estimate instead (the response reports `Preference-Applied: count=estimated`). Estimated counts are always available; the default is `true`. +Because an exact count scans the full matched set, it is **off by default**. Enable it in an application's REST configuration via the [`exactCount` option](./overview.md#configuration): + +```yaml +rest: + exactCount: true +``` + +Without it, a `count=exact` request is served as an estimate (the response reports `Preference-Applied: count=estimated`). Estimated counts are always available.