From a9b1cb43c862deeca0520bde0b277d03f32edab6 Mon Sep 17 00:00:00 2001 From: piekstra Date: Tue, 11 Aug 2026 23:44:01 -0400 Subject: [PATCH] docs: record the internal auth boundary and why a 403 proves nothing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Tax-season tooling wanted `lofty documents list/download` over the two catalogued `taxdocuments/v2` routes. They cannot be reached from this CLI, and the reason is structural rather than a missing feature — so write down the finding instead of shipping a command that would 403 for everyone. - `/prod/taxdocuments/v2/*` lives on the internal surface, behind a website Cognito session. GET and POST, anonymous and with the SDK Bearer key, all return 403. The API key and the website session are unrelated credential systems, and the key's only permission axis is trading — there is no documents or tax scope to enable. - The public SDK has no document route at all: across `@loftyaicode/sdk` 0.2.0–0.2.4 there are 19 routes (properties, orders, account, amm, lp-rewards) and no tax/document/download route was ever published. Lofty's help center documents tax forms as a dashboard-only feature. - The trap that cost the most time: a gated route and a route that certainly does not exist return byte-identical `{"message":"Forbidden"}`, unchanged even by a malformed Authorization header (a Cognito authorizer would answer 401). So a 403 never means "exists but forbidden", and this API cannot be probed for route discovery. That last point also settles the catalog's status: it is a static harvest of the website bundle, records no HTTP method, and cannot be verified by probing. AGENTS.md and the README now say so. Follow-ups noted in the doc: bump cli-common off v0.1.2 to pick up the new documents/v1 profile (pk-cli-documents) before any document command is written, so this CLI adopts the family spelling rather than inventing one; and observing the real response shapes needs an interactive browser session, since fixtures must be scrubbed captures and never invented. --- AGENTS.md | 7 ++- README.md | 10 ++++- docs/api.md | 123 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 138 insertions(+), 2 deletions(-) create mode 100644 docs/api.md diff --git a/AGENTS.md b/AGENTS.md index 25938f2..7e22a07 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -37,8 +37,13 @@ Run `make verify` before considering a change done — it's exactly what CI runs - `src/client.rs` — HTTP against the SDK surface (`/public/v1`) and the internal website API (`--internal`, `/prod`). - `src/config.rs` — non-secret config; the API key is keychain-only. -- `src/catalog.rs` — the observed endpoint inventory (static). +- `src/catalog.rs` — the observed endpoint inventory (a static harvest of the + website bundle; never live-verified, and records no HTTP method). - `tests/` — offline contract/shape tests + `tests/fixtures/` (see its README). +- `docs/api.md` — the two API surfaces (SDK vs internal) and the auth boundary + between them, plus the traps: notably that a gated route and a nonexistent one + return identical `403`s, so this API cannot be probed for route discovery and + `catalog` entries cannot be confirmed that way. ## Conventions (do not break these) diff --git a/README.md b/README.md index 7f08405..b22dd04 100644 --- a/README.md +++ b/README.md @@ -70,6 +70,12 @@ $ lofty api --internal GET /properties/v2/marketplace # website API (open read $ lofty catalog --group exchange # observed endpoint inventory ``` +Not everything the website does is reachable from here. `lofty api --internal` +sends no credential, so only Lofty's genuinely open reads answer, and the +account-scoped routes behind a website login (tax documents among them) stay out +of reach — see [docs/api.md](docs/api.md) for the two API surfaces, the auth +boundary between them, and why a `403` from this API tells you nothing at all. + ### Example — LP-reward programs ```console @@ -349,7 +355,9 @@ $ cargo run -- rewards programs Tests are fully **offline**: raw captured API responses live in `tests/fixtures/` as independent files (see its README) and the contract tests load them from disk — no network, no credentials. See -[CONTRIBUTING.md](CONTRIBUTING.md) and [AGENTS.md](AGENTS.md). +[CONTRIBUTING.md](CONTRIBUTING.md) and [AGENTS.md](AGENTS.md); +[docs/api.md](docs/api.md) records the API surfaces, their auth boundaries, and +the traps worth knowing before you probe anything. ## License diff --git a/docs/api.md b/docs/api.md new file mode 100644 index 0000000..9669e8d --- /dev/null +++ b/docs/api.md @@ -0,0 +1,123 @@ +# API surfaces, auth boundaries, and traps + +What `lofty` talks to, what it deliberately cannot reach, and the ways this API +misleads you. Written down because two of these cost real debugging time and +are invisible from the outside. + +## The two surfaces + +### SDK surface — `https://api.lofty.ai/public/v1` + +Lofty's official, documented wire contract (the `@loftyaicode/sdk` package). +Auth is `Authorization: Bearer lofty_live_…`, a key minted in the dashboard and +stored only in the OS keychain. Every domain command in this CLI targets this +surface, and `lofty api` is its raw passthrough. + +The key's permission model has exactly **one** axis: read-only versus +read-and-trade. A trading-disabled key refuses mutations and nothing else — +there is no per-resource scope system, so there is no "documents" or "tax" +permission to switch on. If a route is missing from this surface, no key +setting will reveal it. + +### Internal platform surface — `https://api.lofty.ai/prod` + +The API the lofty.ai website's own front end calls. It authenticates with a +**Cognito idToken tied to a browser login session** (see also SigV4 paths); a +handful of its reads are open to the world. + +`lofty api --internal` attaches **no credential at all**, by design. That is +why its help text says only publicly open endpoints answer — for example +`GET /prod/properties/v2/marketplace` returns data anonymously. Everything +account-scoped does not. + +**The SDK API key does not authenticate this surface.** Routing an internal +path through the authenticated client (by passing the full URL, so the Bearer +header is attached) still fails. The two credential systems are unrelated: one +is an API key for the public SDK, the other is a website session. + +## Trap: a `403` here tells you nothing + +Verified live on 2026-08-11: + +| Request (unauthenticated) | Result | +| --- | --- | +| `GET /prod/properties/v2/marketplace` | `200` — an open read | +| `GET /prod/taxdocuments/v2/all` | `403 {"message":"Forbidden"}` | +| `GET /prod/zzz/v2/not-real` (a path that certainly does not exist) | `403 {"message":"Forbidden"}` | + +A route that is real but gated and a route that was never there return +**byte-identical** responses. Adding a malformed `Authorization` header changes +nothing — an AWS Cognito authorizer would normally answer `401 Unauthorized` +for a token it cannot parse, and this surface does not. The SDK surface behaves +the same way: an invented `/public/v1/…` path yields an auth error, not a `404`. + +Two consequences, both easy to get wrong: + +- **You cannot probe this API to discover routes.** Absence of a `200` is not + evidence of anything. +- **Never read a `403` as "the endpoint exists and I merely lack permission."** + It is equally consistent with the endpoint not existing. Confirming a route + requires a credential that can actually reach it, not a cleverer probe. + +Note also that the CLI collapses the upstream body into its own auth error, so +the raw `{"message":"Forbidden"}` above is only visible outside the client. + +## Tax documents: not reachable (blocked) + +`lofty catalog --group taxdocuments` lists two `Read` endpoints, +`/taxdocuments/v2/all` (`getAllDocuments`) and `/taxdocuments/v2/zip-documents` +(`getZipFile`). Neither is usable from this CLI today, and the reason is +structural rather than a missing feature: + +- Both exist only on the **internal** surface, behind the website Cognito + session. Every combination tried — `GET` and `POST`, anonymous and with the + SDK Bearer key — returns `403`. +- **The public SDK has no document route at all.** An audit of the published + `@loftyaicode/sdk` package across versions 0.2.0–0.2.4 found 19 routes, all + under `properties`, `orders`, `account`, `amm`, and `lp-rewards`. Searching + the bundle and its type definitions for `tax`, `1099`, `K-1`, `document`, + `download`, `statement`, `zip`, and `pdf` produces no functional hit — the + sole `document` match is a browser-environment check. Nothing was published + and later withdrawn. +- There is no OpenAPI spec and no developer documentation site (neither + `docs.lofty.ai` nor `developer.lofty.ai` resolves). +- Lofty's own help center describes tax forms as a **dashboard-only** feature: + log into the website, open the Taxes menu, click Download. No programmatic + path is advertised. + +So a `documents` command group cannot ship until `lofty` can present a website +session credential. Because of the `403` trap above, we cannot even confirm the +two catalog paths are live — only that nothing we can send reaches them. + +When that capture eventually happens, two rules apply. Response shapes here are +**entirely unobserved**, so fixtures must come from a real, scrubbed capture +(`tests/fixtures/README.md`) and must never be invented to fit an assumed +shape. And if `getZipFile` returns a pre-signed S3 URL, that URL is a **live +credential** — like document ids and file names, it never lands in a fixture, a +test, or a commit. + +## Catalog provenance + +`src/catalog.rs` is a **static harvest** of the endpoint registry in the +website's JS bundle, captured in the initial commit. It records each route's +path, name, group, and safety class — but **not its HTTP method** — and the +entries have never been live-verified. The `403` trap makes verification by +probing impossible in principle. + +Read `lofty catalog` as a map of what the website's front end refers to: useful +for orientation, not a guarantee that a route exists, is reachable with the +credentials this CLI holds, or accepts the method you assume. + +## Follow-ups + +- **Bump `cli-common`.** This repo pins `v0.1.2`. Upstream `HEAD` adds the + `documents/v1` profile (`pk-cli-documents`), which fixes the canonical + spelling — `documents list` plus `documents download -o ` — and + the `document-list/v1`, `document-download/v1`, and + `document-download-batch/v1` DTOs. Bump **before** any document support is + written here, so `lofty` adopts the family shape instead of inventing a + parallel one. +- **Observe the real shapes.** Reaching `/taxdocuments/v2/*` needs an + interactive session at a logged-in browser (a captured idToken, or a full + Cognito login flow in the CLI). Both need the account owner present and are + out of scope for automated work.