feat: upgrade to EQL v3 — cipherstash-client 0.42, typed jsonb domains, v3 typecheck & transform - #432
feat: upgrade to EQL v3 — cipherstash-client 0.42, typed jsonb domains, v3 typecheck & transform#432freshtonic wants to merge 56 commits into
Conversation
📝 WalkthroughWalkthroughThe PR migrates the proxy and EQL Mapper from EQL v2 to EQL v3, introducing domain-derived configuration, EQL v3 wire types, encrypted JSON rewrites, reshaped parameter plans, updated protocol handling, schemas, tests, and documentation. ChangesEQL v3 migration
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related issues
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Knowingly leaves the workspace not compiling. Fixing the fallout is deliberately deferred. - cipherstash-client / cipherstash-config / cts-common: 0.34.1-alpha.4 -> 0.42.0 - CS_EQL_VERSION: eql-2.3.0-pre.3 -> eql-3.0.1 (major, breaking) - Drop the CIP-3159 vendored stack-auth patch: 0.42.0 requires stack-auth ^0.42.0, which carries the CancelGuard fix upstream. vendor/stack-auth is left in place but is now unreferenced. Stable-Commit-Id: q-0c7qhchxzewgy0q472w3vcps2t
Switch the encrypt path to cipherstash-client 0.42.0's encrypt_eql_v3 and make v3 the only wire envelope Proxy speaks. Still does not compile; threading the new types through the protocol layer is deferred. - lib.rs re-exports the V3 types under the old names and deliberately does NOT re-export the v2 ones, so any remaining v2 use fails to resolve rather than silently writing v2 payloads. - EQL_SCHEMA_VERSION 2 -> 3; startup gate now queries eql_v3.version() (eql_v2 schema no longer exists in EQL 3). - Decrypt is now local: there is no decrypt_eql_v3 in the client, but a scalar payload's 'c' is an EncryptedRecord and EncryptedRecord is Decryptable, so it goes straight to the cipher. - SteVec (jsonb) decrypt returns SteVecV3DecryptUnsupported: v3 stores the key header once at the document root with raw AEAD bytes per entry, and reassembling that here would hard-code the envelope layout. Stable-Commit-Id: q-4n8w7v1b7dadsf27c98sz33kk1
Follows the approach protect-ffi uses (encrypted_record_from_value / v3_root_record): skip decrypt_eql entirely and hand an EncryptedRecord straight to the cipher. Replaces the SteVecV3DecryptUnsupported error added in the previous commit, which was wrong -- KeyHeader::record_with_selector is public and does exactly the reassembly needed. Scalar and SteVec records are NOT interchangeable, hence the V3Record enum: RecordWithNonce unconditionally reports a nonce override and an AAD selector, so wrapping a scalar record in one would decrypt against a nonce the value was never encrypted with. Note protect-ffi's SteVec branch does not port directly: it pins cipherstash-client =0.41.1, where sv entries are self-describing mp_base85 records. 0.42.0 moved to an envelope -- key material once in the document's 'h' header, raw AEAD bytes per entry, nonce and AAD derived from the entry selector. Stable-Commit-Id: q-2jsj9wjxvdj9s21xzpmp838x93
Review of the v3 representation adoption (#423) flagged that the three new SteVec errors had no entries in docs/errors.md: - SteVecOreModeUnsupported already linked #encrypt-ste-vec-ore-mode-unsupported but the anchor did not exist (dangling link) — add the entry. - SteVecMissingRootEntry and SteVecSelectorInvalid had no doc reference at all — add the 'For help visit ...' link (matching the KeysetId pattern) and a Decrypt-errors entry for each. Stable-Commit-Id: q-778vkvxxcgtewwbxpvzwg1hsjd
The queue targets EQL 3.0.2 (per the queue description) but the version was pinned to eql-3.0.1 in mise.toml and mise.local.example.toml. Bump both to eql-3.0.2 so the tests run and releases build against the intended version (#423 review). Stable-Commit-Id: q-13w6tzeqj9shdh0dc6xhsxwe8r
cipherstash-client 0.42.0 requires stack-auth ^0.42.0 (which carries the CIP-3159 CancelGuard token-refresh fix upstream), so the [patch.crates-io] override was already dropped. The vendored copy under vendor/stack-auth was left behind, unreferenced. Delete it and drop the workspace `exclude` entry that only existed to keep it out of the build (#423 review). Stable-Commit-Id: q-6x5j8fh1kcrr5hhawgfmhpyq0f
The four `to_ciphertext_*` tests in data_row.rs are pinned to captured PostgreSQL wire payloads from the EQL v2 era. Adopting the v3 representation (the following commit on this branch) reparses a column as a v3 `EqlCiphertextV3`, which the v2 composite/rowtype fixtures no longer satisfy — `as_ciphertext` deserialises them to `None` and the assertions fail. Valid v3 fixtures cannot be hand-authored; they need a real encrypt round-trip against a live ZeroKMS / EQL-v3 database. Ignore the four tests (with an explanatory note) so the suite is honest and green until the payloads are regenerated, rather than committing fabricated ciphertext. Stable-Commit-Id: q-4faw1qn0f5ac8srkt326at2ed2
Completes the v3 move. The workspace compiles again; cargo fmt and
clippy are clean.
The load-bearing change is in data_row.rs. EQL v2's eql_v2_encrypted was
a COMPOSITE type, so reads stripped a ('...') wrapper in text and a
12-byte rowtype header in binary. EQL v3's types are DOMAINS over jsonb
(95 CREATE DOMAIN, zero composites), and a domain is wire-identical to
its base type -- so both of those strips were wrong. Reads now take the
jsonb representation: bare JSON in text, a 1-byte version header plus
JSON in binary.
Also:
- Encrypt now carries EqlOutput, not EqlCiphertext, through the literal
and param paths: a query operand is not a ciphertext. Both v3 enums
are #[serde(untagged)], so the serialized wire form is unchanged.
- backend/bind/data_row imported cipherstash_client::eql::EqlCiphertext
directly, bypassing the crate alias and silently keeping the v2 type.
They now use crate::EqlCiphertext.
- EqlCiphertext::identifier is a method in 0.42.0, not a field.
- Map the new EqlError variants. UnsupportedSteVecOreInV3 gets its own
customer-facing error: v3 has no oc representation, so a ste_vec column
left in Standard (CLLW-ORE) mode cannot be written at all.
- cipherstash-config gained IndexType::SteVec { mode }. SteVecMode::Compat
(CLLW-OPE) is the default and the v3-compatible mode; Standard is
documented upstream as the legacy v2 protocol.
- CouldNotDecryptDataForKeyset gained a #[source]; Proxy's variant does
not carry it, so the chain stops at that boundary (noted in place).
Stable-Commit-Id: q-48c15nyszqvw09e8m6gydxqvfd
Completes the v3 move. The workspace compiles again; cargo fmt and
clippy are clean.
The load-bearing change is in data_row.rs. EQL v2's eql_v2_encrypted was
a COMPOSITE type, so reads stripped a ('...') wrapper in text and a
12-byte rowtype header in binary. EQL v3's types are DOMAINS over jsonb
(95 CREATE DOMAIN, zero composites), and a domain is wire-identical to
its base type -- so both of those strips were wrong. Reads now take the
jsonb representation: bare JSON in text, a 1-byte version header plus
JSON in binary.
The four `to_ciphertext_*` decoder tests are regenerated from real EQL
v3 wire captures (a live encrypt round-trip against ZeroKMS + EQL
v3.0.2), replacing the stale v2 composite/rowtype fixtures this change
invalidates.
Also:
- Encrypt now carries EqlOutput, not EqlCiphertext, through the literal
and param paths: a query operand is not a ciphertext. Both v3 enums
are #[serde(untagged)], so the serialized wire form is unchanged.
- backend/bind/data_row imported cipherstash_client::eql::EqlCiphertext
directly, bypassing the crate alias and silently keeping the v2 type.
They now use crate::EqlCiphertext.
- EqlCiphertext::identifier is a method in 0.42.0, not a field.
- Map the new EqlError variants. UnsupportedSteVecOreInV3 gets its own
customer-facing error: v3 has no oc representation, so a ste_vec column
left in Standard (CLLW-ORE) mode cannot be written at all.
- cipherstash-config gained IndexType::SteVec { mode }. SteVecMode::Compat
(CLLW-OPE) is the default and the v3-compatible mode; Standard is
documented upstream as the legacy v2 protocol.
- CouldNotDecryptDataForKeyset gained a #[source]; Proxy's variant does
not carry it, so the chain stops at that boundary (noted in place).
Stable-Commit-Id: q-48c15nyszqvw09e8m6gydxqvfd
Captures the EQL v3 domain model, three impact maps over eql-mapper (type system, SQL surface, declarations/macros), and the open design questions, so the design session starts fresh with full context. Records one corrected finding: literals are inference sinks (value.rs:19 unifies them with a fresh tvar, not Native), so a token type on EqlValue buys no literal checking. Two subagents disagreed on this; resolved by reading the code. Stable-Commit-Id: q-2r73hgg9ex95aba238a8fbe4kp
Captures the shared understanding from the grill-with-docs session on extending the EQL Mapper type checker for EQL v3. CONTEXT.md: refresh the glossary to the v3 target vocabulary — EqlValue now carries domain identity; EqlTrait drops Contain and is documented as coarse; add term-extraction function, functional-index rewrite, query operand, token type, and domain identity. The old 'known model gap' note becomes 'design decisions in flight' pointing at the ADRs. ADRs (new packages/eql-mapper/docs/adr/): - 0001 functional-index rewrite (term functions, not native operators) — forced by Supabase's lack of CREATE OPERATOR; term-fn selection is the capability check. - 0002 token type as inert identity, not a checked dimension — it buys no safety (encrypted cols never unify; literals are inference sinks). Assessed against EQL/eql-bindings 3.0.2 (released 2026-07-20): additive query-twin bindings only; none of the design decisions change. Stable-Commit-Id: q-06b9k91e7pnntayq3rs11tqznz
…rands Two latent bugs in the trait-bound machinery, harmless only while bound checking is dead code (every encrypted column currently gets EqlTraits::all()). Fix them before the EQL v3 loader makes bounds reachable. - EqlTraits::difference was implemented as XOR (symmetric difference), so UnsatisfiedBounds would list traits the type *has* alongside the ones it is missing. It is now a true set difference (self AND NOT other). - Type::must_implement passed its operands reversed relative to Unifier::satisfy_bounds (implemented.difference(required) instead of required.difference(implemented)). Harmless only because XOR is commutative; wrong the moment difference is corrected. Now consistent. Adds unit tests pinning the asymmetric set-difference semantics and that must_implement reports exactly the missing bounds. Refs CIP-3595. Stable-Commit-Id: q-0e9kbc03hrqef307fg7qb5sqj1
Groundwork for the EQL v3 type checker (ADR-0002). Introduces the token type and its inert per-column domain identity, threading it through the type carriers without making it a checked dimension of unification. - New `TokenType` (integer/text/timestamp/…) and `DomainIdentity` (token type + v3 domain typname, e.g. eql_v3_text_ord_ore). - `EqlValue` widens to (TableColumn, Option<DomainIdentity>, EqlTraits); `ColumnKind::Eql`, `SchemaTableColumn` and the new `Column::eql_with_identity` carry the identity. `Column::eql` stays as a back-compat constructor that defaults the identity to None. - The identity is `None` everywhere for now: the branch still emits the v2 surface and no loader populates it yet (CIP-3598 does). It rides through unification and the associated-type machinery untouched, exactly because it is never inspected there. - ColumnKind loses `Copy` (DomainIdentity is not Copy); the handful of by-value uses now clone. - Also fixes the never-invoked, latently-broken no-bounds arm of the concrete_ty!/EQL() macro (wrong EqlValue arity, missing .into()). eql-mapper suite 79 passing; workspace checks clean; fmt + clippy clean. Refs CIP-3597. Stable-Commit-Id: q-5dxadzsggt5seabqhg3sdj8ht5
Implements the SchemaManager side of the v3 type checker (ADR-0002): per-column capability becomes *observed* from the Postgres domain name instead of a hardcoded EqlTraits::all(). - New schema/eql_domains.rs inverts the eql-bindings v3 catalog (v3::all()) into `typname -> (TokenType, EqlTraits)`. Because the catalog is generated from the same source as the installed SQL domains, the mapping cannot drift from them. Term -> capability: hm->Eq, op/ob->Ord, bf->TokenMatch, empty->storage-only; the JSON SteVec domains (term_json_keys == None) map to JsonLike (provisional, see note). - load_schema now resolves each column's v3 domain identity and traits and builds the column via Column::eql_with_identity. The legacy eql_v2_encrypted composite-type arm is retained for reading pre-v3 schemas; unrecognised domains load as Native. - select_table_schemas.sql now also selects information_schema domain_name: v3 encrypted columns are jsonb-backed DOMAINs, so udt_name reports the base type (jsonb) and the domain typname is only available via domain_name (cf. CIP-3441). - Adds eql-bindings 3.0.1 as a workspace dependency, used only by the proxy's schema loader; eql-mapper stays wire-format-agnostic. Re-exports DomainIdentity / TokenType from eql-mapper's crate root. Unit-tested offline against the catalog (12 tests asserting the SEM-term table, incl. the text hm+ord exception and query-twin exclusion). End-to-end validation needs a database with EQL v3 installed and is deferred; the encrypt-config cross-check is a follow-up within CIP-3598. Refs CIP-3598. Stable-Commit-Id: q-6p9ykbew5b6sjaf5p7y9qza4ft
…+ Contain
Applies three confirmed design decisions on the EQL v3 type checker.
1. Domain identity is non-optional (strict ADR-0002). EqlValue is now
(TableColumn, DomainIdentity, EqlTraits) and ColumnKind::Eql carries a
mandatory DomainIdentity — no more Option. The schema loader always
supplies the real identity; there is no honest v3 identity for a legacy
eql_v2_encrypted column, so the loader now drops the v2 arm and logs a
warning instead of fabricating one (v2 is already retired on this build).
- New helpers: TokenType::{as_domain_str, from_domain_name},
DomainIdentity::{from_domain_name, canonical}, and a test-only
EqlValue::with_canonical_identity.
- The schema!/concrete_ty!/test-helper macros synthesise a canonical
text-token identity by default; schema! also gains an
EQL("<domain>") form to pin an explicit v3 domain (token + OPE/ORE
variant) for tests that care.
2. JSON domains map to JsonLike + Contain, verified against the installed
v3 SQL (cipherstash-encrypt.sql): -> / ->> and @> / <@ are real
operators on eql_v3_json_search (not raise-stubs). Corrects the earlier
JsonLike-only guess.
3. The encrypt-config cross-check is dropped: the domain name is the sole
authority for a column's capability. (ADR-0002 doc update to follow.)
The Contain trait is therefore retained (scoped to JSON), NOT deleted as
the handoff/glossary/ticket assumed — @>/<@ raise only on scalar encrypted
columns. Docs/tickets updated separately.
eql-mapper 79 passing; proxy eql_domains 12 passing; workspace check,
fmt, clippy all clean.
Refs CIP-3597, CIP-3598.
Stable-Commit-Id: q-31sxsrtyrptajjpj0n383cb690
…h a real domain Review of the v3 domain-identity work (#424) noted that canonical() can synthesise a name that is not merely approximate but identical to an unrelated real catalog domain — canonical(Text, {json_like}) yields eql_v3_text_search, whose terms are hm/op/bf (Eq+Ord+TokenMatch), nothing to do with JSON. Strengthen the doc comment so no caller mistakes the synthesised name for the column's real domain; only from_domain_name is authoritative. Stable-Commit-Id: q-1ewbmpx74jnyxfd7dyftv08ka9
Review of #424 (finding 5) noted the v2->native fallback should be hard to miss in ops, because a v2 column served as a native passthrough takes no encryption on the write path: a plaintext value written to it is stored in plaintext (the eql_v2_encrypted AS ASSIGNMENT cast does no validation). Spell that out in the warning — the column is served as PLAINTEXT, Proxy does no encrypt/decrypt on it, migrate before writing — instead of the softer "ignoring unsupported column". Stable-Commit-Id: q-4dmrmqgewe5mzj0fzb2qdk5dja
Review of #424 (finding 1) flagged that `eql_v3_json_entry` — the element a `->` traversal returns, whose per-entry terms are `hm` XOR `op` — resolved to JsonLike+Contain, because `ste_vec_domain_type!` leaves `term_json_keys()` at None. A column mistakenly declared with that type would have type-checked as supporting `->`/`@>` but not `=`/`<`, which is inverted. Exclude the SteVec sub-structural domains (`NON_COLUMN_DOMAINS`) from the column catalog so such a column falls through to a native (plaintext) column instead of a wrong capability set. The every_public_column_domain assertion skips the same list. Makes the misclassification impossible rather than latent. Stable-Commit-Id: q-4dg749ehycbdazr4etn9rk1fvm
Two design corrections found during implementation:
- Containment is NOT removed in v3. Verified against the installed
cipherstash-encrypt.sql: @>/<@ raise on scalar encrypted columns but are
real, supported operators on encrypted JSON (eql_v3_json_search). The
Contain trait is retained as a JSON-only capability. Fixes the glossary
("removed in v3") and adds a consequence to ADR-0001.
- ADR-0002 amended: the domain name is the sole authority for a column's
capability (no encrypt-config cross-check), and the domain identity is
non-optional (legacy eql_v2_encrypted columns are dropped with a warning
rather than given a fabricated identity).
Refs CIP-3597, CIP-3598, CIP-3599.
Stable-Commit-Id: q-4rd3p778qhjfc5dessqeeg1b2e
…ation
Starts the v3 functional-index rewrite (ADR-0001) by pinning the pipeline
design and landing its hardest, most correctness-sensitive piece: choosing
the eql_v3 term-extraction function for an operator on a given column.
- ADR-0003 records the v3 rewrite pipeline: the stored-value vs query-operand
context split (column domain vs query twin cast targets), operator ->
term-function rewriting, JSON containment retained (not deleted), and the
rule inventory. All operand/term details verified against the installed
cipherstash-encrypt.sql, not the docs.
- DomainIdentity gains stores_hm/op/ob/bf and eq_term_fn/ord_term_fn/
match_term_fn/query_twin. These encode the verified selection rules:
* eq_term only where hm is stored; =/<> otherwise fall back to the ordering
term (an ord-only scalar like integer_ord compares via ord_term, exactly
as eql_v3.eq does).
* ord_term (op) vs ord_term_ore (ob) chosen from the domain typname.
* text is the hm exception (text_ord* stores hm alongside its order term).
* query_twin names the term-only operand domain (eql_v3.query_<bare>).
Unit-tested against the SEM-term table. The rewrite rule that consumes these
lands next.
Refs CIP-3599.
Stable-Commit-Id: q-4fed4c4vg8sehf76wwrn5yaysf
…ex form
First working slice of the v3 rewrite pipeline (ADR-0003). Scalar
comparisons on encrypted columns now emit the functional-index form, and
all cast targets move from the opaque eql_v2_encrypted to real v3 domains.
- New RewriteEqlComparisonOps: `col <op> x` -> `eql_v3.<term>(col) <op>
eql_v3.<term>(x)`, the term function chosen from the column's domain
identity (eq_term / ord_term / ord_term_ore). A domain with no term for
the operator is a capability error.
- Cast rules retargeted (helpers::cast_to_v3_domain): a query operand casts
to the eql_v3.query_* twin, a stored value (INSERT/UPDATE) to the
public.eql_v3_* column domain. Role is detected from the AST context
(post-order traversal + full NodePath ancestry): a value under a
comparison predicate is an operand, otherwise stored.
- CastLiteralsAsEncrypted gains node_types so it can name the target domain;
EqlTerm gains eql_value(); helpers gain eql_v3_term_call / is_comparison_op.
Example:
WHERE salary > $1
-> WHERE eql_v3.ord_term(salary) > eql_v3.ord_term($1::JSONB::eql_v3.query_text_ord)
Transitional: JSON (->, @>, jsonb_*) and aggregate (min/max) rewriting still
emit the eql_v2.* function wrappers over v3-cast operands; those wrappers are
retargeted in the next slices. Tests updated to the current output; the
params fixture gains `EQL: Eq` since a bare (capability-less) v3 column is
storage-only and rightly rejects `=`.
eql-mapper 85 passing; workspace check, clippy, fmt clean.
Refs CIP-3599.
Stable-Commit-Id: q-62kqdp86gcbb66tvwhy69tgn5s
Continues the v3 rewrite (ADR-0003). min/max and the jsonb_* passthrough functions on encrypted columns now rewrite to their eql_v3 counterparts. - RewriteStandardSqlFnsOnEqlTypes resolves a pg_catalog function to its eql_v3 counterpart via the new get_eql_v3_function_name, and rewrites only when one is declared. This fixes the long-standing count bug: count has no eql_v3 counterpart (it runs natively on the encrypted value), so it is no longer rewritten to a non-existent eql_v3.count. - sql_decls: the min/max/jsonb_path_*/jsonb_array_* target declarations move from eql_v2 to eql_v3. The containment declarations (jsonb_array, jsonb_contains, jsonb_contained_by) stay under eql_v2 until the containment slice. min(salary) -> eql_v3.min(salary) jsonb_path_query(col, '$.x') -> eql_v3.jsonb_path_query(col, '$.x'::...) count(col) -> count(col) (unchanged) eql-mapper 85 passing; workspace check, clippy, fmt clean. Refs CIP-3599. Stable-Commit-Id: q-092hd8jb1d31ck4va4n7r2gzet
Continues the v3 rewrite (ADR-0003). `@>`/`<@` on encrypted JSON columns now rewrite to the eql_v3 containment functions. Containment is retained in v3, scoped to JSON (ADR-0002 amendment) — `@>`/`<@` on scalar encrypted columns still raise. - RewriteContainmentOps emits eql_v3.jsonb_contains / jsonb_contained_by. - sql_decls: the jsonb_array / jsonb_contains / jsonb_contained_by target declarations move from eql_v2 to eql_v3. notes @> needle -> eql_v3.jsonb_contains(notes, needle) needle <@ notes -> eql_v3.jsonb_contained_by(needle, notes) No eql_v2.* function names remain in the mapper's rewrite output. eql-mapper 85 passing; clippy, fmt clean. Refs CIP-3599. Stable-Commit-Id: q-6dne3hbwwfsj7q42yqf4da8rqv
Continues the v3 rewrite (ADR-0003). `->`/`->>` on encrypted JSON columns are functionalised, since managed Postgres (Supabase) forbids the CREATE OPERATOR DDL the native operators would need (ADR-0001). - RewriteContainmentOps (now the general encrypted-JSON binary-op rule) also rewrites `col -> sel` -> `eql_v3."->"(col, sel)` and `col ->> sel` -> `eql_v3."->>"(col, sel)`. Operator-symbol function names are quoted; ordinary names are not. - CastLiteralsAsEncrypted passes a JSON field selector (an EqlTerm::JsonAccessor, the RHS of ->/->>) as encrypted *text* rather than a jsonb-domain cast, to match eql_v3."->"(json_search, text). notes -> 'medications' -> eql_v3."->"(notes, '<encrypted-selector>') ASSUMPTION TO CONFIRM (flagged in code): the encrypted selector replacement is the selector string itself. If the encrypt pipeline instead produces a jsonb payload for JSON selectors, the selector must be extracted rather than emitted verbatim — a one-line change once confirmed. eql-mapper 88 passing; clippy, fmt clean. Refs CIP-3599. Stable-Commit-Id: q-03j8aeq7we1c74jhkrgrny3rt4
LIKE/ILIKE on an encrypted column previously only unified the result with Native, so it bypassed the trait system entirely — a column with no match capability silently accepted LIKE. They now apply the `~~`/`~~*` (and negated) operator rules, so the encrypted LHS must implement TokenMatch and the pattern takes its Tokenized associated type. Adds tests: LIKE on a TokenMatch column type-checks; LIKE on an Eq-only encrypted column is a capability error. (The v3 rewrite of LIKE/@@ to `eql_v3.match_term(a) @> eql_v3.match_term(b)` is a separate follow-up; this fixes the type-checking gap.) eql-mapper 87 passing; clippy, fmt clean. Refs CIP-3599. Stable-Commit-Id: q-429ag1vxhrefpb1vy71sbkzg34
Continues the v3 rewrite (ADR-0003). LIKE/ILIKE on encrypted columns now
rewrite to the fuzzy-match functional form.
- New RewriteEqlMatchOps: `col LIKE pat` -> `eql_v3.match_term(col) @>
eql_v3.match_term(pat)`; `NOT LIKE` wraps it in `NOT (...)`. Fuzzy match
compares bloom-filter terms with @> (containment), so the operator becomes
@> between the two match_term calls, mirroring eql_v3.matches. A column
whose domain stores no bloom filter has no match_term -> capability error.
- Operand role detection (helpers::is_query_operand) now also treats a value
under a LIKE/ILIKE predicate as a query operand, so the pattern casts to the
query twin.
email LIKE 'a%'
-> eql_v3.match_term(email) @> eql_v3.match_term('<ct>'::JSONB::eql_v3.query_text_match)
`@@` (BinaryOperator::AtAt) is the remaining fuzzy-match surface; it needs the
operator declaration to parse first (separate slice).
eql-mapper 88 passing; clippy, fmt clean.
Refs CIP-3599.
Stable-Commit-Id: q-0gdjjy9gdt34jne9yhvs1kjw9m
… double domain Fills the rewrite-output test gaps called out in review of the v3 rewrite pipeline (#428): - ILIKE and NOT LIKE / NOT ILIKE — the negated match arm in RewriteEqlMatchOps had no rewrite-output coverage (only positive LIKE/@@ did). - UPDATE SET stored-value cast — of ADR-0003's two stored-value contexts only INSERT had cast-target coverage; UPDATE now pins `SET salary = 'ENCRYPTED'::JSONB::public.eql_v3_text`. - native LIKE still type-checks — regression lock for routing LIKE/ILIKE through the TokenMatch-bounded rule (Native satisfies all bounds). - the `double` token does not swallow its capability suffix — pins the eql_v3_double_ord invariant that suffix()/stores_* only documents in a comment. Stable-Commit-Id: q-47bk2x9aqbw7j8v04j81ecbec4
Completes the fuzzy-match surface (ADR-0003). `@@` now parses in the
operator DSL and rewrites like LIKE/ILIKE.
- eql-mapper-macros: the binary-operator parser accepts `@@` (it previously
only accepted `@>` after `@`), so `@@` can be declared.
- sql_decls: declare `<T>(T @@ <T as TokenMatch>::Tokenized) -> Native where
T: TokenMatch` — `@@` requires the match capability.
- RewriteEqlMatchOps now also handles `BinaryOp::AtAt`:
col @@ pat -> eql_v3.match_term(col) @> eql_v3.match_term(pat)
and `is_query_operand` treats an `@@` operand as a query operand so the
pattern casts to the query twin.
eql-mapper 89 passing; eql-mapper-macros 3 passing; clippy, fmt clean.
Refs CIP-3599.
Stable-Commit-Id: q-7s39kd5jdyhfjh1qyhkg2s858f
`encrypted_jsonb -> 'field'` and `jsonb_path_query{,_first}(...)` rewrite to
`eql_v3."->"(...)` / `eql_v3.jsonb_path_query(...)`, which return a single
`eql_v3_json_entry`. Two proxy-side gaps left those queries returning NULL or
an undecrypted entry:
- Return path: a json_entry `{v,i,h,s,c,op}` carries a `c`, so it deserialised
as a scalar `Encrypted` payload whose ciphertext is not self-decryptable
(an entry ciphertext only opens with its selector-derived nonce). Detect the
bare entry by its root-level selector `s` and reshape it into a one-entry
SteVec document `{v,k:"sv",i,h,sv:[{s,c,op?}]}` so the existing SteVec
decrypt path recovers the field value.
- Param path: the tokenized selector is bound as `text`, but a binary Bind
unconditionally prepended the jsonb version byte (0x01), corrupting the
selector so `->` matched no stored entry. Bind it as bare text with no
header (the binary wire form of `text` is just its raw bytes).
Together these make encrypted JSON field access and path queries work over
both the simple and extended protocols (17 integration tests). The remaining
jsonb ordering, containment, array and term-filter gaps are tracked in
CIP-3630 / CIP-3631.
Stable-Commit-Id: q-2rq44tjqgd12sceeq283d6gk1s
Implements `WHERE col -> sel <op> value` (`<`, `<=`, `>`, `>=`, and the
`jsonb_path_query_first(col, sel) <op> value` form) over encrypted JSON, in
both the simple and extended protocols. Rewrites to the EQL v3 functional-index
form (matching the stack adapters' `eql_v3.gt(col -> $1, $2::query_*_ord)`):
eql_v3.ord_term(eql_v3."->"(col, <selector>))
> eql_v3.ord_term($2::jsonb::eql_v3.query_integer_ord)
- New `EqlTerm::JsonOrd` / `EqlTermVariant::JsonOrd` — the scalar ordering
operand of a JSON field comparison. Inferred (infer_type_impls/expr.rs) for
the non-`->` side of a `<`/`<=`/`>`/`>=` comparison whose other side is a
JsonLike field access. Equality is intentionally excluded (exact JSON equality
is value-selector containment, a separate feature).
- The operand casts to `eql_v3.query_integer_ord` — a shape-only, type-agnostic
ord twin. `eql_v3.ord_term` extracts the `op` bytes regardless of scalar type
and the twin's CHECK is shape-only (`{v,i,op}`), so ONE twin serves any JSON
leaf type. This is what makes it work in the extended protocol, where the
operand's scalar type is unknown at rewrite time.
- The proxy encrypts a JsonOrd operand via `QueryOp::SteVecTerm` (zerokms.rs)
and encodes the scalar to match the stored leaf's `for_json_value` SteVec `op`
(from_sql.rs): a JSON number → float (the 8-block CLLW-OPE encoding, NOT a
small int), a JSON string → its unquoted text. Both the bare-literal
(`> '4'` / `> 'C'`) and text-format jsonb-param renderings are parsed as JSON
to recover the scalar type/content.
Fixes the 8 `select_where_jsonb_{gt,gte,lt,lte}` integration tests (numeric and
string); no regressions (mapper 97 unit tests green, jsonb suite 54→62 passing).
Stable-Commit-Id: q-3ecs5hbqrk8vw4827atp3zp17n
Cleanup pass (CIP-3601), now that the v3 rewrite is in place. - eql-mapper-macros: the EqlTrait / operator parse-error messages listed only Eq, Ord, TokenMatch, JsonLike but Contain is a valid keyword (retained in v3 for JSON containment). Both messages now include Contain. - eql-mapper CONTEXT.md: the "migration in flight" note claimed the code still emits the v2 surface; it now emits v3 (no eql_v2.* names remain in output), with end-to-end validation still pending. Note on the default()/all() reconciliation (the other CIP-3601 item): no production change is needed. schema_delta's collect_ddl never fabricates EQL traits (new/added columns are Native), so its EqlTraits::default() only appears in test assertions, consistent with the schema! macro's bare `EQL` (a capability-less, storage-only v3 column). The former mismatch was the loader's hardcoded all(), which CIP-3598 replaced with domain-derived capability. eql-mapper 90 passing; eql-mapper-macros 3 passing; fmt clean. Refs CIP-3601. Stable-Commit-Id: q-56sd3v5nxbdh5b4kpepyc1mm31
The showcase's encrypted columns move from the EQL v2 opaque type +
per-column search config to EQL v3 self-configuring domain types.
- schema.sql: `pii/medication/procedure eql_v2_encrypted` +
`eql_v2.add_search_config('ste_vec', ...)` -> `eql_v3_json_search` columns
(the searchable encrypted-JSON / SteVec domain). The domain type alone
declares the encryption and searchability, so the add_search_config calls
are removed.
- data.rs::clear(): drop the `eql_v2_configuration` DELETE — v3 has no such
config table (self-configuring domains); clearing now just truncates.
- Cargo.toml / README.md / CLAUDE.md / model.rs / main.rs diagram: reworded
from EQL v2 to v3, incl. a capability -> v3-domain-suffix mapping table in
the showcase CLAUDE.md.
The demo queries are unchanged: they are plain SQL (->, ->>, @>, <@,
jsonb_path_*, comparisons, ORDER BY, aggregates) that Proxy's mapper now
rewrites to the v3 functional-index form. End-to-end validation still needs
a live Proxy + database with EQL v3 installed.
Compiles clean.
Refs CIP-3595.
Stable-Commit-Id: q-2sptjp0nj4f9wer95m8yfk4nxg
The proxy's encrypt config is now derived from the database schema instead
of the `eql_v2_configuration` table. EQL v3 columns are self-configuring
domain types, so `eql_v2.add_search_config` and the config table are
redundant — the schema is the single source of truth.
- New encrypt_config/from_domain.rs: `column_config_from_domain` builds a
ColumnConfig from a column's v3 domain typname — cast type from the token,
indexes from the stored SEM terms (verified against cipherstash-client's
indexers + eql-bindings v3::terms): hm→Unique, op→Ope, ob→Ore, bf→Match,
JSON SteVec→SteVec{Compat}. Scalar non-text `_ord` domains store only `op`
(a single Ope index, no HMAC); text carries `hm` alongside its ordering
term. Storage-only and non-EQL domains yield no indexes / None.
- load_encrypt_config now runs the shared SCHEMA_QUERY (which already returns
information_schema.domain_name) and maps each encrypted column via
column_config_from_domain. Removes the eql_v2_configuration query, the
canonical-JSON path, ENCRYPT_CONFIG_QUERY + select_config.sql, and the
MissingEncryptConfigTable startup handling (a load error is now a genuine
database failure; an empty encrypted schema is a successful empty config).
- The obsolete canonical-config parsing tests are dropped; from_domain.rs
carries the equivalent domain→config coverage (9 tests).
This unblocks running against a v3-only EQL install (which has no
eql_v2_configuration). proxy builds; from_domain 9 tests pass; workspace
check, clippy, fmt clean (pre-existing data_row::to_ciphertext_* failures
are unrelated).
Refs CIP-3595, CIP-3579.
Stable-Commit-Id: q-6xh7ejw90r1kxzdd0jmmmkzd79
Replaces the EQL v2 `eql_v2_encrypted` columns + `add_search_config` / `add_encrypted_constraint` / `eql_v2_configuration` truncate with self-configuring v3 domain types (the proxy now infers encrypt config from the schema). - Main `encrypted` / `encrypted_elixir`: scalars use the default CLLW-OPE ordering domain `_ord` (op; also supports equality); `encrypted_text` is `text_search` (eq+ord+match); jsonb -> `json_search`. - `encrypted_bool` is `eql_v3_boolean` (storage-only) — boolean has no searchable capability in v3; the bool search fixtures/columns are dropped. - ORE/OPE fixture tables select their ordering family by `kind`: `ore` -> block-ORE (`_ord_ore`, `text_search_ore`), `ope` -> CLLW-OPE (`_ord_ope`, `text_ord_ope`). The `encrypted_bool` column and the `*_where_bool` fixtures are removed. - `unconfigured` columns become storage-only `eql_v3_text`. - schema-uninstall drops the tables; there is no v2 config table to remove. Refs CIP-3595. Stable-Commit-Id: q-5mf9na2j8evysprjfqpn7a8587
Follows the v3 schema conversion. The test crate now compiles and is consistent with EQL v3 semantics (runtime validation still needs a live Proxy + database with EQL v3 installed). - disable_mapping: the EqlEncrypted struct maps to `eql_v3_text_search` (the encrypted_text column's v3 domain) instead of `eql_v2_encrypted`. - jsonb_containment_index: the explicit `eql_v2.jsonb_contains(...)` call and comments become `eql_v3.jsonb_contains(...)`. - indexing: the encrypted index is now the v3 functional form `CREATE INDEX ON encrypted (eql_v3.ord_term(encrypted_text))`. - Bool search tests dropped (v3 boolean is storage-only): map_unique_index_bool, map_ore_where_generic_bool, map_ope_where_generic_bool. Bool roundtrip tests (encrypt/decrypt of a storage-only bool) are kept. - eql_regression (v2->v3 backwards-compat, which is out of scope — prior releases are not in use) is #[ignore]d with a note; regenerate fixtures from a v3 baseline to re-enable. Compiles clean; fmt clean. Refs CIP-3595. Stable-Commit-Id: q-17gkjm4bew1pevb1j47f7z0jvb
Convert the last docs, comments, and example SQL that still described the EQL v2 model (opaque `eql_v2_encrypted` type + `eql_v2.add_search_config` + `eql_v2_configuration` table) to the v3 self-configuring domain-type model. - ARCHITECTURE.md: rewrite the transformation-rules table (v3 rule set incl. RewriteEqlComparisonOps / RewriteEqlMatchOps) and the schema-loading paragraph (config inferred from domain types, no config table). - CONTEXT-MAP.md: update context/glossary entries; rewrite the "capability across the seam" note — the v3 schema loader now derives real per-column traits from the domain type, so the old "currently broken" bug is resolved. - docs/how-to/index.md: teach the `eql_v3_<token>_<cap>` domain-type model in place of add_search_config index setup; eql_v3.version(). - docs/reference/index.md: domain type enforces the encrypted-payload constraint; no add_encrypted_constraint call. - docs/reference/searchable-json.md: eql_v3_json_search schema + self-config note; caveat the v2 add_search_config option examples. - docs/errors.md, docs/sql/schema-example.sql, benchmark-schema.sql, parse.rs, psql-passthrough.sh, CLAUDE.md: v3 domain-type phrasing. Intentional v2 mentions retained: contrastive prose, the legacy-warn arm in schema/manager.rs, and the #[ignore]'d backwards-compat regression tests. Stable-Commit-Id: q-76caz2g4z3fptnpp8dn70s8v6g
…ALL) `eql_v3_json_search` columns were configured with `ArrayIndexMode::NONE`, which stores an array as a single opaque entry and never indexes its elements — so `@>`/`<@` containment against an array, `jsonb_array_elements`, and `[@]`/`[*]` path queries over encrypted arrays all failed (array element selectors did not exist to match). Index JSON arrays with `ArrayIndexMode::ALL` (item `[@]`, position `[n]`, and wildcard `[*]` element entries) so encrypted arrays are searchable the same way scalars are. This is the storage/searchability tradeoff a searchable JSON column opts into; the mode is a per-column config knob if a lighter footprint is wanted. Fixes 8 array integration tests: `jsonb_contains`/`jsonb_contained_by` with array needles (numeric + string), `jsonb_array_elements`, and `jsonb_path_query_first` array-wildcard (numeric + string). No regressions (jsonb suite 62->70 passing). `jsonb_array_length` over `[@]` remains open — it needs the flattened array-element rows counted, which is a separate rewrite. Stable-Commit-Id: q-3hn5yd17zqm6v6k7xex60x4wwa
Encrypted JSON equality (`col -> sel = value`) is not a term comparison. In EQL v3 exact equality is *containment of a value selector*: a single keyed MAC over the path and the canonicalised value together (`QueryOp::SteVecValueSelector`). One needle, built from TWO SQL operands. This is the mapper half. It breaks the implicit 1:1 correspondence between input and output operands for the first time: - New `EqlTerm::JsonValueSelector` / `EqlTermVariant::JsonValueSelector` — the value operand of a JSON field equality. Inferred for the non-JSON side of `=`/`<>` whose other side is a genuine field ACCESS (`->`, `->>`, `jsonb_path_query_first`). A bare `col = $1` on a whole encrypted JSON column is document equality and keeps its ordinary typing. - New `JsonValueSelectors` on `TypeCheckedStatement` records the one relationship node types cannot express: which operand supplies the path for which value. The mapper holds no encryption key, so it emits the composition *input*, not the needle; the proxy fuses and encrypts. - New `RewriteJsonValueSelectorEq` rewrites the comparison to `eql_v3.jsonb_contains(col, <value>::eql_v3.query_json)` (`<>` negates), discarding the field access. `RewriteEqlComparisonOps` now skips these — `eql_v3.eq_term` has no unique overload for a JSON query operand. - The value operand casts to `eql_v3.query_json`, alongside the existing `JsonOrd` -> `query_integer_ord` rule (helper renamed to `json_query_operand_cast_target` to cover both). A discarded selector *placeholder* stays declared in Parse but unreferenced in the SQL. PostgreSQL permits that as long as its type is known, which is what lets input and output param numbering stay identical — so Bind, ParameterDescription and the encrypt pipeline all stay positional. Verified against PostgreSQL 17: `PREPARE p (text) AS SELECT $2::jsonb` prepares and executes; the same statement with no declared types fails with "could not determine data type of parameter $1". Mapper unit tests 97 -> 102, no regressions. Proxy-side composition (the literal and Bind paths) follows in subsequent commits. Stable-Commit-Id: q-7qfnv43maha2chq6shq55sc3m0
The proxy half of encrypted JSON field equality. The mapper types the
value operand `JsonValueSelector` and records where its path comes from;
this fuses the two SQL operands into one `{"path", "value"}` plaintext and
encrypts it with `QueryOp::SteVecValueSelector`, producing the one-entry
containment needle `eql_v3.jsonb_contains` matches.
Both protocols:
- Simple: `literals_to_plaintext` composes the needle for a value-selector
literal from its own JSON scalar and the selector literal's text. The
discarded selector literal is still encrypted but never placed, since the
rewrite removed its node.
- Extended: `Bind::to_plaintext` gains a second pass. A fused param is
skipped by the positional pass — its own bytes are only half a needle and
would not decode as a standalone operand for the column — then composed
from the path param (or literal) and its own value.
- `Parse::declare_unreferenced_param_types` declares TEXT for the selector
placeholder the rewrite dropped. Clients that let the server infer types
send no types at all, and PostgreSQL will not prepare a statement with an
unreferenced param of unknown type. Still-referenced slots are left at
OID 0 ("infer"), so this never overrides a type the query determines.
Param counts are unchanged end to end, so Bind, ParameterDescription and
the encrypt pipeline all stay positional.
Also folds the four copies of the `$.`-rooting of a JSON selector into one
`json_selector_path` helper — the fused path needs the same normalisation
the accessor and path-query operands already did.
Fixes `select_jsonb_where_{string,numeric}_eq` (both protocols, both the
`->` and `jsonb_path_query_first` spellings). jsonb suite 70 -> 72
passing. Verified zero regressions by running the full integration suite
against the base binary and diffing failure sets: the only differences are
these two tests plus known-flaky ORE ordering tests that fail identically
on base.
The 7 `jsonb_term_filter` tests remain red for an unrelated reason:
`encrypted_jsonb_filtered` is declared `eql_v3_json_search`, the same
domain as the unfiltered column, and `from_domain.rs` hardcodes
`term_filters: Vec::new()`. EQL v3 has no domain or config that encodes a
term filter, so the proxy cannot infer one. Equality itself works on that
column — it matches case-sensitively. Case-insensitive JSON needs an
EQL-side way to declare filters; that is a separate feature.
Stable-Commit-Id: q-5afqdbdnjhd299x8777mwdm4fj
Stable-Commit-Id: q-6658b8n0t0qbfkcksk6720x0b5
Replaces the dangling-param trick with the real shape of the problem: an output param may be derived from more than one input param, so there is no 1:1 correspondence to rely on. Previously the JSON-equality rewrite left the fused path operand in the SQL as an unreferenced `$n`, purely so that param numbering stayed positional. That kept the plumbing simple at the cost of lying about the statement: a param that means nothing, and a neighbouring param whose meaning has quietly become "composite". Both are invisible at the call sites that bind them. The correspondence is now explicit: - `TypeCheckedStatement::transform` returns a `TransformedStatement` — the rewritten SQL plus a `ParamPlan`. Each `OutputParam` names the input param(s) its value is built from (`Input`, or `JsonValueSelector` carrying both operands). - A `RenumberParams` pass runs after the rewrite rules and assigns `$1..$m` in SQL order, so rules may freely drop or duplicate placeholders without maintaining numbering themselves. Running it as a separate pass keeps `FailOnPlaceholderChange` governing the rules. - `ParamPlan::check_covers` enforces the invariant that replaces 1:1 — **coverage**: every input param must be consumed by some output. An input that feeds nothing has been silently dropped, which is a bug in a rewrite rule. The proxy binds against the plan rather than by position. `Statement` carries both shapes: `param_columns` (what the client binds, used to decode and to answer Describe) and `output_params` (what PostgreSQL receives). Bind builds each output from the inputs its source names; ParameterDescription is rebuilt from the input params, since the server describes the rewritten statement and would otherwise tell the client too few params to bind. Parse carries each client-declared type across to the output param that consumes it. Every output param is now referenced by the rewritten SQL, so `declare_unreferenced_param_types` and its unreferenced-placeholder workaround are gone. When the plan is positional — every statement that is not a JSON equality — Bind still patches values in place, leaving the client's wire framing byte-for-byte as sent. Mapper units 102 -> 104 (renumbering across a fusion, and identity plans); proxy units 116 -> 117. Integration unchanged: the 2 JSON equality tests pass, and the four ORE-ordering tests that differ from the previous run fail identically on the base binary when run in isolation. Stable-Commit-Id: q-67avvp278knhdy7xe192mq667s
Removes `CastLiteralsAsEncrypted` and `CastParamsAsEncrypted`. Both were context-blind: they fired on any encrypted literal or placeholder and then guessed which domain to cast it to by walking up the ancestor chain (`is_query_operand`) looking for a comparison or a LIKE. The rule that actually knew the context — the one rewriting the predicate — was a different rule entirely, running later. Now the rule that owns a construct casts the operands of that construct, where the context is a fact rather than an inference: - `RewriteEqlComparisonOps`, `RewriteEqlMatchOps` — query operands, cast to the term-only `eql_v3.query_*` twin. - `RewriteJsonValueSelectorEq` — the fused needle, `eql_v3.query_json`. - `RewriteContainmentOps` — a `@>`/`<@` needle is a whole document, so it casts to the column domain; a `->`/`->>` selector takes no cast. - `CastFullPayloadOperands` (new) — the contexts with no rewrite of their own: INSERT values, UPDATE assignments, and directly-written `eql_v3.jsonb_contains(col, $1)` calls, which clients on platforms without operator support write themselves and which need the column-domain cast for the GIN index over `eql_v3.jsonb_array(col)`. It fires on the enclosing `Values`/`Assignment`/`Function` node, not on the literal, so the context is again structural. Substitution is separated from casting: `SubstituteEncryptedLiterals` puts each ciphertext in place wherever it appears and casts nothing, since where a literal sits decides its domain but not whether it needs replacing. It keeps the postcondition that every encrypted literal was consumed. The two domain choices are now named for the distinction they encode — `query_operand_domain` (just the terms a predicate needs) and `full_payload_domain` (ciphertext plus every indexed term) — and `is_query_operand`, the ancestor walk, is gone. Every rule matches against the ORIGINAL node via `node_path`, so a construct already rewritten by an earlier rule cannot be cast twice. Mapper 104 unit tests and proxy 117 unchanged. Integration: no regressions — the only tests that differ from the previous run are in the flaky ORE/OPE ordering family (net +1 passing), and the one that regressed both appears in the base binary's failure list and flips between ok and failed across identical isolated runs. Stable-Commit-Id: q-3y30vhx105f1ms7h36tanr8k7m
…term
`ORDER BY col` on an encrypted column was passed through untouched. The
column is a domain over `jsonb`, so PostgreSQL compared whole payloads by
jsonb rules — and jsonb compares objects field by field starting at `c`,
the ciphertext, which is randomised per encryption. Results came back in
an order that was not merely wrong but different on every insert.
Rewrites to the ordering term, which is what makes the comparison
meaningful:
ORDER BY col -> ORDER BY eql_v3.ord_term(col)
ORDER BY t.col DESC -> ORDER BY eql_v3.ord_term(t.col) DESC
ORDER BY col NULLS FIRST-> ORDER BY eql_v3.ord_term(col) NULLS FIRST
(`ord_term_ore` for block-ORE domains.) Sort options are untouched, so
ASC/DESC and NULLS FIRST/LAST keep working — the term is a plain orderable
value and NULL stays NULL. `ord_term` yields `ope_cllw`, a `bytea` domain
ordered bytewise; `ord_term_ore` yields `ore_block_256`, ordered by its own
btree operator class. Verified directly against PostgreSQL: over four
values re-encrypted five times, the bare `ORDER BY` gave a different wrong
order every run while the term ordering was correct every run.
Ordering by a column whose domain carries no ordering term is now a
capability error rather than an arbitrary sort, matching how
`RewriteEqlComparisonOps` treats an unsupported operator.
This is the cause of the "flaky" ORE/OPE ordering tests. They were not
flaky: the ordering was random, so a test comparing two values passed
about half the time while one comparing five values essentially never did.
Integration: 236 -> 298 passing, 168 -> 106 failing, nothing newly
failing. Fixes all of map_ope_index_order (6), map_ore_index_order (18)
and select::order_by (42), and those families are now deterministic across
repeated runs. Mapper units 104 -> 107.
Stable-Commit-Id: q-7znx6wzcc63yyk8hnganvrbdce
Every scalar predicate on an encrypted column failed:
ERROR: value for domain eql_v3.query_integer_ord violates check
constraint "query_integer_ord_check"
A query operand carries the column's search terms but never a decryptable
ciphertext — the `eql_v3.query_*` domains enforce that with `NOT (VALUE ?
'c')`. The proxy encrypted every scalar operand with `EqlOperation::Store`
and sent the storage payload, `c` and all, into a query position.
It cannot simply encrypt differently: a single `EqlOperation::Query`
yields only ONE term, while an operand may need several (`{v,i,hm,ob}` for
`query_text_ord`). The client's answer is to encrypt in Store mode and
project — `EqlCiphertextV3::into_query_operand`, whose own test is named
`into_query_operand_scalar_keeps_all_terms_and_drops_c`. The proxy never
called it.
Projecting needs to know which operands are query operands, and nothing
carried that: the operand of `WHERE col = $1` and the value of
`INSERT ... VALUES ($1)` are both `EqlTerm::Full`. So the mapper now
records it, in the new `QueryOperands`, alongside the existing
`JsonValueSelectors`. Membership is decided syntactically by the predicate
an operand belongs to — comparisons, `LIKE`/`ILIKE`, `@@` — which is the
same set of contexts whose rewrite rules cast to a `eql_v3.query_*` twin.
Containment (`@>`/`<@`) is deliberately excluded: its needle is a whole
document and keeps its full payload, as do INSERT values and UPDATE
assignments.
The flag rides through `ParamPlan` to the proxy's `OutputParam`, so Bind
projects the params it sends, and the literal path projects at Parse.
Payloads the encryptor already produced query-shaped (the JSON selectors
and SteVec terms, which do go through `EqlOperation::Query`) are left
alone.
Fixes map_ope_index_where (6), map_ore_index_where (6), map_match_index
(1) and 7 of 8 map_unique_index — scalar encrypted search worked in
neither protocol before this. Integration 298 -> 317 passing, 106 -> 87
failing, nothing newly failing.
The remaining map_unique_index failure (`..._all_with_wildcard`, a
`SELECT *` with six encrypted params) fails with a protocol-level
UnexpectedMessage and is a separate, pre-existing bug.
Stable-Commit-Id: q-4jyyztp3wxv6ve8ndnwpp59r2y
…term
The third instance of the same trap. An encrypted column is a domain over
`jsonb`, so a bare `GROUP BY col` grouped on the whole payload — including
`c`, the ciphertext, which is randomised per encryption. Rows holding the
same plaintext landed in different groups, degrading `GROUP BY` into
`GROUP BY <every row>`: `select::group_by` saw 20 groups for 10 distinct
values.
Grouping is equality, so the key is the same term `=` uses —
`eql_v3.eq_term(col)`, or `ord_term` for a domain that stores no `hm`
(`eql_v3_integer_ord` groups through its ordering term, exactly as it
compares).
Once the key is `eq_term(col)`, PostgreSQL no longer sees the bare column
as functionally dependent on it and rejects `SELECT col`. So a projection
of a grouped column is lifted through an aggregate, as James specified:
SELECT col, COUNT(*) FROM t GROUP BY col
->
SELECT any_value(col) AS col, COUNT(*) FROM t GROUP BY eql_v3.eq_term(col)
Every row in a group holds the same plaintext, so any of them decrypts to
the right answer. `FIRST` does not exist in PostgreSQL and `min`/`max` are
not defined over `jsonb`; `any_value` is polymorphic and works on the
domain directly. It requires PostgreSQL 16+, but only for the projection
case — grouping without selecting the column works on any version.
The projection is matched on the resolved column rather than on syntax, so
`SELECT t.col ... GROUP BY col` — which PostgreSQL accepts today — keeps
working. The original projection name is preserved, so clients selecting
by name are unaffected: this is what keeps
`select::regression::select_with_order_by_in_subquery` (a
`SELECT encrypted_text ... GROUP BY encrypted_text`) passing, which a
group-key-only rewrite would have broken.
Grouping by a column whose domain carries no equality term is now a
capability error rather than a grouping on ciphertext.
Integration 317 -> 323 passing, 87 -> 81 failing, nothing newly failing.
Mapper units 107 -> 109.
Stable-Commit-Id: q-4080dw66nm5aqm7g0q717mqvmw
Replaces the `any_value` stand-in with the aggregate EQL provides for
exactly this case (CIP-3657, EQL PR 423):
SELECT col, COUNT(*) FROM t GROUP BY col
->
SELECT eql_v3.grouped_value(col) AS col, COUNT(*)
FROM t GROUP BY eql_v3.eq_term(col)
`grouped_value` returns one representative value per group without
decrypting or comparing anything, which is all that is needed: every row
in a group is an encryption of the same plaintext. It re-creates the
aggregate the eql_v2 surface provided and that 3.0.0 dropped, and it
accepts any encrypted-domain value since all of them are jsonb-backed.
Being EQL's own answer to this problem it is also the form EQL's docs
describe, so a hand-written query and a proxy-rewritten one now agree.
`any_value` worked but was the wrong dependency to take: it is a
PostgreSQL 16+ builtin, so it silently raised the proxy's minimum server
version for a case EQL already covers.
**Requires an EQL release later than the 3.0.2 pinned in `mise.toml`** —
`eql_v3.grouped_value` does not exist in 3.0.2, so `select::group_by` and
`select::regression::select_with_order_by_in_subquery` will fail on a
clean install until `CS_EQL_VERSION` is bumped. Verified here by
installing the aggregate from PR 423 into the dev database by hand: all 6
group_by tests and the regression test pass, and the full integration
suite is unchanged at 323 passing / 81 failing with nothing newly failing.
Stable-Commit-Id: q-4h7rjf4sncgcvk7v1cpy3ndxjd
`CS_EQL_VERSION` eql-3.0.2 -> eql-3.0.3, and the `eql-bindings` crate =3.0.1 -> =3.0.3. EQL 3.0.3 carries `eql_v3.grouped_value` (CIP-3657, EQL PR 423), which the GROUP BY rewrite emits. Until now that commit was red on a clean install — the aggregate did not exist in 3.0.2 and had to be applied to the dev database by hand to test. It is now installed by the release, so `select::group_by` and `select::regression` pass with no local patching. `eql-bindings` was pinned two releases behind the SQL at 3.0.1; 3.0.2 and 3.0.3 are both the same ts-rs serde-attribute warning fix, with the emitted TS/JSON byte-identical, so the jump carries no behaviour change. The proxy uses the crate only in its schema loader, to map v3 domains to capabilities. Integration suite unchanged at 323 passing / 81 failing, with nothing newly failing or passing. Mapper 109 and proxy 117 unit tests green. Stable-Commit-Id: q-67fwe28n0ab44t0zefphhnqd4p
060ebf7 to
6432499
Compare
Nine tests are ignored and three are corrected, all because v3 changed what a column can do — none of the proxy or mapper code is touched. Ignored, blocked on missing v3 features: - select::jsonb_term_filter (7 tests). A v3 column is configured entirely by its domain type, and eql_v3_json_search cannot express a term filter — column_config_from_domain builds its SteVec index with term_filters: Vec::new(). encrypted_jsonb_filtered therefore stores original case and a lowercase query matches 0 rows. Kept rather than deleted: they are the spec for the v3 term-filter feature. - select::jsonb_array_length (2 tests). jsonb_array_length over an encrypted array returns 1 rather than the element count; the array is indexed for search but its length is not recoverable from the encrypted form. Corrected, because the v2 behaviour they asserted is gone: - disable_mapping::select_with_set_unsafe_disable_mapping read the column as a struct pinned to the domain name. A v3 encrypted column is a DOMAIN over jsonb and PostgreSQL reports a domain's base type in RowDescription, so the client sees jsonb and rejects the mapping. Reads as Value and asserts the undecrypted payload still carries its column identifier. (Worked under v2 because eql_v2_encrypted was a composite type with its own OID.) - extended_protocol_error_messages::encrypted_column_with_no_configuration asserted that an encrypted-but-unconfigured column is rejected. v3 makes that state unreachable: a column is encrypted because it has an EQL domain, and every recognised domain yields a ColumnConfig — eql_v3_text simply yields one with no indexes. Rewritten as storage_only_encrypted_column_round_trips, which covers the behaviour that replaced it. - map_unique_index::map_unique_index_all_with_wildcard searched on encrypted_bool. v3 makes boolean storage-only (a two-value column leaks its distribution under any index), so equality on it is correctly rejected. Dropped from the WHERE clause; the column is still encrypted, decrypted and asserted on in the projection. Stable-Commit-Id: q-64yq0x0ngewks00bz5yy77tqwk
There was a problem hiding this comment.
Actionable comments posted: 11
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/cipherstash-proxy/src/proxy/mod.rs (1)
88-105: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winValidate the installed EQL version before startup
eql_versiondegrades failures toNone, and startup only prints a warning when the installed version differs fromEQL_VERSION_AT_BUILD_TIMEwithout comparing it against the required minimum (3.0.3). That can allow a v2 or older v3 database to continue and treat schema columns without EQL v3 domains as plaintext passthrough. Fail fast or block startup when the detected version is below the required minimum.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cipherstash-proxy/src/proxy/mod.rs` around lines 88 - 105, Update eql_version and its startup validation to parse the detected database version and enforce the minimum required EQL version 3.0.3, blocking startup when the version is missing, invalid, or below that minimum instead of allowing None to continue as a warning-only path. Preserve successful startup for compatible versions and retain the existing version-mismatch reporting for otherwise valid versions.
🧹 Nitpick comments (10)
docs/errors.md (1)
636-638: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd a language to the new fenced code blocks.
markdownlint flags these three new fenced blocks (Lines 636, 695, 718) for missing a language identifier (MD040).
📝 Suggested fix
-``` +```text An encrypted jsonb column is configured for ORE ordering, which EQL v3 does not support.(apply the same ` ```text ` fence to the blocks at Lines 695 and 718) </details> Also applies to: 695-697, 718-720 <details> <summary>🤖 Prompt for AI Agents</summary>Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.In
@docs/errors.mdaround lines 636 - 638, Update the three fenced code blocks
in docs/errors.md around the encrypted jsonb message and the blocks near lines
695 and 718 to specify the text language identifier; preserve each block’s
existing content and use the same text fence consistently.</details> <!-- cr-comment:v1:5c99a1de39a3531d959d5527 --> _Source: Linters/SAST tools_ </blockquote></details> <details> <summary>packages/cipherstash-proxy-integration/src/select/indexing.rs (1)</summary><blockquote> `37-38`: _📐 Maintainability & Code Quality_ | _🔵 Trivial_ | _⚡ Quick win_ **Don't swallow the `CREATE INDEX` result.** `let _ = client.simple_query(sql).await` hides a failed index creation, so this test would still pass if `eql_v3.ord_term` were the wrong term function for `eql_v3_text_search` — the one thing the v3 index migration needs to prove here. Since the table is shared, tolerate only the "already exists" case rather than all errors. <details> <summary>♻️ Assert index creation succeeds</summary> ```diff - let sql = "CREATE INDEX IF NOT EXISTS encrypted_text_ord_term_idx ON encrypted (eql_v3.ord_term(encrypted_text))"; - let _ = client.simple_query(sql).await; + let sql = "CREATE INDEX IF NOT EXISTS encrypted_text_ord_term_idx ON encrypted (eql_v3.ord_term(encrypted_text))"; + client.simple_query(sql).await.unwrap();🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cipherstash-proxy-integration/src/select/indexing.rs` around lines 37 - 38, Update the CREATE INDEX setup in the indexing test to inspect the result of client.simple_query(sql).await and fail on errors, while explicitly tolerating only the expected “already exists” error for the shared table. Do not discard other failures, so an incorrect eql_v3.ord_term index function causes the test to fail.packages/cipherstash-proxy/src/postgresql/context/statement.rs (1)
130-132: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low valueAssert the output-param mapping contract.
This is the only call site using
output_params_from_plan, andget_output_param_columnsis documented as positional overPlanPlan::outputs, butzip()still drops a trailing mapper output if that invariant ever breaks. A debug-level length assertion here makes the contract explicit without hiding future mismatches in rewrite output.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cipherstash-proxy/src/postgresql/context/statement.rs` around lines 130 - 132, Add a debug-level assertion before the positional zip in the output_params_from_plan flow, verifying that get_output_param_columns returns the same number of entries as plan.outputs(). Preserve the existing positional mapping while making any length mismatch explicit instead of silently dropping trailing mapper outputs.packages/cipherstash-proxy/src/error.rs (1)
377-381: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
UnsupportedV3QueryTermcollapses into a message with no friendly text or doc link.Both v3/v2 SteVec rejections map to
InvalidIndexTerm, whose#[error("InvalidIndexTerm")](Line 254-255) gives the customer an opaque identifier — unlike the new SteVec variants. SinceUnsupportedV3QueryTermis reachable from a real query shape, consider a dedicated variant (or at least a friendly message plus doc link onInvalidIndexTerm).As per coding guidelines: "Customer-facing errors must include friendly messages and documentation links".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cipherstash-proxy/src/error.rs` around lines 377 - 381, The EqlError mapping in the error conversion must give UnsupportedV3QueryTerm a customer-facing message and documentation link instead of the opaque InvalidIndexTerm text. Update the relevant error enum variant and its mapping, while preserving the existing UnsupportedSteVecInV2 behavior unless it requires the same customer-facing treatment.Source: Coding guidelines
packages/cipherstash-proxy/src/postgresql/data/from_sql.rs (1)
122-153: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd unit tests for the new JSON operand helpers.
json_selector_path,json_value_selector_plaintext,literal_json_valueandjson_ord_scalar_plaintextencode subtle contracts (rooting, scalar-only rejection, invalid-JSON-as-string fallback, number→f64). This file already has amod tests; a few table-driven cases here would pin the behaviour that the integration suite currently can't cover (several JSON tests are ignored under v3).Also applies to: 161-192, 226-235
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cipherstash-proxy/src/postgresql/data/from_sql.rs` around lines 122 - 153, Add focused table-driven unit tests in the existing tests module covering json_selector_path, json_value_selector_plaintext, literal_json_value, and json_ord_scalar_plaintext. Assert rooted selector paths, scalar acceptance and object/array rejection, invalid JSON string fallback, and numeric conversion to f64, including representative valid and edge cases.packages/eql-mapper/src/model/schema.rs (1)
44-53: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
Displayomits the newly added domain identity.
#[display("Eql({})", _0)]renders only theEqlTraits, so theDomainIdentitythis variant now carries never appears in diagnostics — which is the field the v3 loader populates and the one you'd want when debugging a domain mismatch.♻️ Include the domain
- #[display("Eql({})", _0)] + #[display("Eql({}, {})", _0, _1.domain.value)] Eql(EqlTraits, DomainIdentity),🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/eql-mapper/src/model/schema.rs` around lines 44 - 53, Update the Display format on ColumnKind::Eql to include both EqlTraits and DomainIdentity, ensuring diagnostics expose the domain identity populated by the v3 schema loader while preserving the Native formatting.packages/cipherstash-proxy/src/proxy/sql/select_table_schemas.sql (1)
4-9: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick winMake the aggregate ordering explicit so the three arrays stay positionally aligned.
packages/cipherstash-proxy/src/proxy/schema/manager.rs(Lines 140-144) andpackages/cipherstash-proxy/src/proxy/encrypt_config/manager.rs(Line 207) zipcolumns,column_type_names, andcolumn_domain_namespositionally. Aggregate input order is unspecified in SQL, so encode it here rather than relying on it.🛡️ Proposed fix
- array_agg(c.column_name)::text[] AS columns, - array_agg(c.udt_name)::text[] AS column_type_names, + array_agg(c.column_name ORDER BY c.ordinal_position)::text[] AS columns, + array_agg(c.udt_name ORDER BY c.ordinal_position)::text[] AS column_type_names, -- EQL v3 encrypted columns are jsonb-backed DOMAINs, so `udt_name` reports -- the base type (`jsonb`); the domain typname (e.g. `eql_v3_integer_ord`) -- is only available via `domain_name`. NULL for non-domain columns. - array_agg(c.domain_name)::text[] AS column_domain_names + array_agg(c.domain_name ORDER BY c.ordinal_position)::text[] AS column_domain_names🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cipherstash-proxy/src/proxy/sql/select_table_schemas.sql` around lines 4 - 9, Update the three array_agg expressions in the select-table-schemas query to use the same explicit ORDER BY key, such as the column ordinal position, so columns, column_type_names, and column_domain_names remain positionally aligned. Preserve the existing casts and selected values.tests/tasks/test/integration/psql-passthrough.sh (1)
25-28: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winPassthrough test lost its only EQL-specific assertion.
With the
eql_v2_configurationprobe gone, this script only checksSELECT 1twice. ASELECT eql_v3.version();through the proxy (same callProxy::eql_versionmakes inpackages/cipherstash-proxy/src/proxy/mod.rs) restores an equivalent v3 check for near-zero cost.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/tasks/test/integration/psql-passthrough.sh` around lines 25 - 28, Update the passthrough integration test near the EQL v3 configuration comment to add a proxied `SELECT eql_v3.version();` assertion, matching the call used by `Proxy::eql_version`. Retain the existing generic `SELECT 1` sanity checks and ensure the new query executes through the proxy.packages/cipherstash-proxy/src/proxy/schema/manager.rs (1)
172-174: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider rate-limiting or metricising the legacy-column warning.
load_schemaruns on every reload tick, so this warns once per legacy column per interval for the lifetime of the process. A counter/gauge (or a once-per-schema-change log) keeps the signal without flooding logs.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cipherstash-proxy/src/proxy/schema/manager.rs` around lines 172 - 174, Update the legacy-column warning in load_schema so repeated reload ticks do not emit an unbounded warning for the same unchanged column. Reuse an existing metric or schema-change tracking mechanism if available, recording the legacy-column condition while preserving the warning’s table and column context; otherwise emit the warning only when the schema state changes.packages/cipherstash-proxy/src/proxy/encrypt_config/from_domain.rs (1)
36-47: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick winSearchability of JSON domains is decided by a name suffix here, but by the eql-bindings catalog in
eql_domains.rs.
packages/cipherstash-proxy/src/proxy/schema/eql_domains.rsderives JSON capabilities fromterm_json_keys() == None(catalog-driven, cannot drift), while this path usesdomain.ends_with("_search"). Any future searchable JSON domain whose typname does not end in_searchwould type-check asJsonLike + Containyet be configured storage-only, so writes would silently carry no SteVec terms. Consider deriving this fromDomainIdentity/the catalog instead of the suffix.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cipherstash-proxy/src/proxy/encrypt_config/from_domain.rs` around lines 36 - 47, The JSON search-index decision in the identity.token == TokenType::Json branch currently relies on domain.ends_with("_search"), which can diverge from the catalog. Reuse DomainIdentity or the eql_domains catalog capability (the same term_json_keys() == None logic) to determine whether the domain is searchable, and add the SteVec index for every catalog-marked searchable JSON domain while leaving storage-only domains without one.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ARCHITECTURE.md`:
- Around line 117-126: Update the pipeline rules table to match the chain
assembled by type_checked_statement.rs::make_transformer: rename the casting
rows to SubstituteEncryptedLiterals and CastFullPayloadOperands, remove the
inaccurate CastParamsAsEncrypted entry, and add RewriteJsonValueSelectorEq,
RewriteEqlOrderBy, and RewriteEqlGroupBy with descriptions of their behavior.
Preserve the existing descriptions for rules that remain in the actual chain.
In `@docs/reference/searchable-json.md`:
- Line 150: Update the `->` and `->>` headings in searchable-json.md to include
`eql_v3_json_search` as the left-hand operand type, matching the operand-type
format used by sibling operator headings while preserving their existing return
descriptions.
In `@packages/cipherstash-proxy-integration/src/eql_regression.rs`:
- Line 157: The EQL regression fixtures remain disabled because generation and
insert_encrypted_directly still use v2 ciphertext and eql_v2_encrypted casts. In
packages/cipherstash-proxy-integration/src/eql_regression.rs:157-157, port
fixture generation and direct insertion to v3/jsonb payloads, regenerate
fixtures, and remove the ignore; then regenerate and re-enable text coverage at
268-268, int2 at 300-300, int4 at 330-330, int8 at 360-360, float8 at 390-390,
boolean at 420-420, JSONB decryption at 450-450, JSON field access at 480-480,
and JSON array coverage at 532-532.
In `@packages/cipherstash-proxy/src/postgresql/data/from_sql.rs`:
- Around line 109-120: Update json_selector_path to treat any selector beginning
with the JSON path root character '$' as already rooted, including '$', '$[0]',
'$["a"]', and '$[*].b'; only prepend '$.' for unrooted field names.
In `@packages/cipherstash-proxy/src/postgresql/messages/bind.rs`:
- Around line 195-212: Preserve the client-declared type for selector operands
instead of assigning them JSONB through output_params_from_plan. Add or
propagate a distinct selector marker so Parse::rewrite_param_types does not
apply the column-based JSONB rewrite, while apply_encrypted continues using
rewrite_text for the raw selector bytes.
In `@packages/cipherstash-proxy/src/postgresql/messages/data_row.rs`:
- Around line 302-307: Update the fixture provenance comment above the
to_ciphertext_* fixtures to reference EQL v3.0.3 instead of v3.0.2, matching the
version pinned by the migration. Leave the fixture descriptions and regeneration
guidance unchanged.
In `@packages/cipherstash-proxy/src/proxy/encrypt_config/manager.rs`:
- Around line 202-223: Update load_encrypt_config to preserve table_schema from
SCHEMA_QUERY when building each eql::Identifier, so configurations for
same-named tables in different schemas cannot overwrite one another. Carry the
schema through the table scan and include it in map.insert alongside table_name
and column, or restrict SCHEMA_QUERY to the served schema.
In `@packages/eql-mapper/docs/adr/0003-v3-rewrite-pipeline.md`:
- Around line 37-39: Add the text language identifier to both fenced code blocks
in the ADR, including the example around the operator transformation and the
block around the salary comparison, so the Markdown fences satisfy MD040 without
changing their contents.
In `@packages/eql-mapper/src/inference/infer_type_impls/expr.rs`:
- Around line 168-192: Propagate whether JSON-selector fusion succeeded from
infer_json_value_selector instead of treating every Ok result as handled. Update
the Eq/NotEq branches in the binary-expression inference flow to set handled
only when fusion unifies the value type; when fusion declines, leave handled
false so get_sql_binop_rule(op).apply_constraints(...) performs ordinary typing
and reports the capability error.
In `@packages/eql-mapper/src/inference/sql_types/sql_decls.rs`:
- Around line 113-123: Update get_eql_v3_function_name to accept v3 counterparts
only when fn_name is unqualified or explicitly qualified with pg_catalog; reject
other schemas before constructing the eql_v3 name. Preserve the existing
SQL_FUNCTION_TYPES lookup and return behavior for permitted function names.
In `@packages/eql-mapper/src/transformation_rules/rewrite_eql_match_ops.rs`:
- Around line 49-67: Update match_predicate to return whether the matched
identity is on the left side of the @@ expression, rather than only a boolean
negation flag. In apply, use that column_is_left result to normalize
original_col and original_pat from left/right or right/left, and build
match_term so the column is always the containment operand regardless of source
ordering.
---
Outside diff comments:
In `@packages/cipherstash-proxy/src/proxy/mod.rs`:
- Around line 88-105: Update eql_version and its startup validation to parse the
detected database version and enforce the minimum required EQL version 3.0.3,
blocking startup when the version is missing, invalid, or below that minimum
instead of allowing None to continue as a warning-only path. Preserve successful
startup for compatible versions and retain the existing version-mismatch
reporting for otherwise valid versions.
---
Nitpick comments:
In `@docs/errors.md`:
- Around line 636-638: Update the three fenced code blocks in docs/errors.md
around the encrypted jsonb message and the blocks near lines 695 and 718 to
specify the text language identifier; preserve each block’s existing content and
use the same text fence consistently.
In `@packages/cipherstash-proxy-integration/src/select/indexing.rs`:
- Around line 37-38: Update the CREATE INDEX setup in the indexing test to
inspect the result of client.simple_query(sql).await and fail on errors, while
explicitly tolerating only the expected “already exists” error for the shared
table. Do not discard other failures, so an incorrect eql_v3.ord_term index
function causes the test to fail.
In `@packages/cipherstash-proxy/src/error.rs`:
- Around line 377-381: The EqlError mapping in the error conversion must give
UnsupportedV3QueryTerm a customer-facing message and documentation link instead
of the opaque InvalidIndexTerm text. Update the relevant error enum variant and
its mapping, while preserving the existing UnsupportedSteVecInV2 behavior unless
it requires the same customer-facing treatment.
In `@packages/cipherstash-proxy/src/postgresql/context/statement.rs`:
- Around line 130-132: Add a debug-level assertion before the positional zip in
the output_params_from_plan flow, verifying that get_output_param_columns
returns the same number of entries as plan.outputs(). Preserve the existing
positional mapping while making any length mismatch explicit instead of silently
dropping trailing mapper outputs.
In `@packages/cipherstash-proxy/src/postgresql/data/from_sql.rs`:
- Around line 122-153: Add focused table-driven unit tests in the existing tests
module covering json_selector_path, json_value_selector_plaintext,
literal_json_value, and json_ord_scalar_plaintext. Assert rooted selector paths,
scalar acceptance and object/array rejection, invalid JSON string fallback, and
numeric conversion to f64, including representative valid and edge cases.
In `@packages/cipherstash-proxy/src/proxy/encrypt_config/from_domain.rs`:
- Around line 36-47: The JSON search-index decision in the identity.token ==
TokenType::Json branch currently relies on domain.ends_with("_search"), which
can diverge from the catalog. Reuse DomainIdentity or the eql_domains catalog
capability (the same term_json_keys() == None logic) to determine whether the
domain is searchable, and add the SteVec index for every catalog-marked
searchable JSON domain while leaving storage-only domains without one.
In `@packages/cipherstash-proxy/src/proxy/schema/manager.rs`:
- Around line 172-174: Update the legacy-column warning in load_schema so
repeated reload ticks do not emit an unbounded warning for the same unchanged
column. Reuse an existing metric or schema-change tracking mechanism if
available, recording the legacy-column condition while preserving the warning’s
table and column context; otherwise emit the warning only when the schema state
changes.
In `@packages/cipherstash-proxy/src/proxy/sql/select_table_schemas.sql`:
- Around line 4-9: Update the three array_agg expressions in the
select-table-schemas query to use the same explicit ORDER BY key, such as the
column ordinal position, so columns, column_type_names, and column_domain_names
remain positionally aligned. Preserve the existing casts and selected values.
In `@packages/eql-mapper/src/model/schema.rs`:
- Around line 44-53: Update the Display format on ColumnKind::Eql to include
both EqlTraits and DomainIdentity, ensuring diagnostics expose the domain
identity populated by the v3 schema loader while preserving the Native
formatting.
In `@tests/tasks/test/integration/psql-passthrough.sh`:
- Around line 25-28: Update the passthrough integration test near the EQL v3
configuration comment to add a proxied `SELECT eql_v3.version();` assertion,
matching the call used by `Proxy::eql_version`. Retain the existing generic
`SELECT 1` sanity checks and ensure the new query executes through the proxy.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: add8c498-7993-4275-b16e-6273a7a579b3
⛔ Files ignored due to path filters (2)
Cargo.lockis excluded by!**/*.lockvendor/stack-auth/Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (118)
ARCHITECTURE.mdCHANGELOG.mdCLAUDE.mdCONTEXT-MAP.mdCargo.tomldocs/errors.mddocs/how-to/index.mddocs/plans/2026-07-20-eql-v3-type-checker-handoff.mddocs/reference/index.mddocs/reference/searchable-json.mddocs/sql/schema-example.sqlmise.local.example.tomlmise.tomlpackages/cipherstash-proxy-integration/src/disable_mapping.rspackages/cipherstash-proxy-integration/src/eql_regression.rspackages/cipherstash-proxy-integration/src/extended_protocol_error_messages.rspackages/cipherstash-proxy-integration/src/map_ope_index_where.rspackages/cipherstash-proxy-integration/src/map_ore_index_where.rspackages/cipherstash-proxy-integration/src/map_unique_index.rspackages/cipherstash-proxy-integration/src/select/indexing.rspackages/cipherstash-proxy-integration/src/select/jsonb_array_length.rspackages/cipherstash-proxy-integration/src/select/jsonb_containment_index.rspackages/cipherstash-proxy-integration/src/select/jsonb_term_filter.rspackages/cipherstash-proxy/Cargo.tomlpackages/cipherstash-proxy/src/error.rspackages/cipherstash-proxy/src/lib.rspackages/cipherstash-proxy/src/postgresql/backend.rspackages/cipherstash-proxy/src/postgresql/column_mapper.rspackages/cipherstash-proxy/src/postgresql/context/mod.rspackages/cipherstash-proxy/src/postgresql/context/statement.rspackages/cipherstash-proxy/src/postgresql/data/from_sql.rspackages/cipherstash-proxy/src/postgresql/data/mod.rspackages/cipherstash-proxy/src/postgresql/frontend.rspackages/cipherstash-proxy/src/postgresql/messages/bind.rspackages/cipherstash-proxy/src/postgresql/messages/data_row.rspackages/cipherstash-proxy/src/postgresql/messages/mod.rspackages/cipherstash-proxy/src/postgresql/messages/param_description.rspackages/cipherstash-proxy/src/postgresql/messages/parse.rspackages/cipherstash-proxy/src/proxy/encrypt_config/from_domain.rspackages/cipherstash-proxy/src/proxy/encrypt_config/manager.rspackages/cipherstash-proxy/src/proxy/encrypt_config/mod.rspackages/cipherstash-proxy/src/proxy/mod.rspackages/cipherstash-proxy/src/proxy/schema/eql_domains.rspackages/cipherstash-proxy/src/proxy/schema/manager.rspackages/cipherstash-proxy/src/proxy/schema/mod.rspackages/cipherstash-proxy/src/proxy/sql/select_config.sqlpackages/cipherstash-proxy/src/proxy/sql/select_table_schemas.sqlpackages/cipherstash-proxy/src/proxy/zerokms/zerokms.rspackages/eql-mapper-macros/src/parse_type_decl.rspackages/eql-mapper/CONTEXT.mdpackages/eql-mapper/docs/adr/0001-functional-index-rewrite.mdpackages/eql-mapper/docs/adr/0002-token-type-as-inert-identity.mdpackages/eql-mapper/docs/adr/0003-v3-rewrite-pipeline.mdpackages/eql-mapper/src/eql_mapper.rspackages/eql-mapper/src/inference/infer_type_impls/expr.rspackages/eql-mapper/src/inference/infer_type_impls/insert_statement.rspackages/eql-mapper/src/inference/mod.rspackages/eql-mapper/src/inference/sql_types/sql_decls.rspackages/eql-mapper/src/inference/unifier/eql_traits.rspackages/eql-mapper/src/inference/unifier/type_env.rspackages/eql-mapper/src/inference/unifier/types.rspackages/eql-mapper/src/inference/unifier/unify_types.rspackages/eql-mapper/src/json_value_selector.rspackages/eql-mapper/src/lib.rspackages/eql-mapper/src/model/schema.rspackages/eql-mapper/src/model/schema_delta.rspackages/eql-mapper/src/param_plan.rspackages/eql-mapper/src/query_operands.rspackages/eql-mapper/src/renumber_params.rspackages/eql-mapper/src/test_helpers.rspackages/eql-mapper/src/transformation_rules/cast_full_payload_operands.rspackages/eql-mapper/src/transformation_rules/cast_literals_as_encrypted.rspackages/eql-mapper/src/transformation_rules/cast_params_as_encrypted.rspackages/eql-mapper/src/transformation_rules/helpers.rspackages/eql-mapper/src/transformation_rules/mod.rspackages/eql-mapper/src/transformation_rules/preserve_effective_aliases.rspackages/eql-mapper/src/transformation_rules/rewrite_containment_ops.rspackages/eql-mapper/src/transformation_rules/rewrite_eql_comparison_ops.rspackages/eql-mapper/src/transformation_rules/rewrite_eql_group_by.rspackages/eql-mapper/src/transformation_rules/rewrite_eql_match_ops.rspackages/eql-mapper/src/transformation_rules/rewrite_eql_order_by.rspackages/eql-mapper/src/transformation_rules/rewrite_json_value_selector_eq.rspackages/eql-mapper/src/transformation_rules/rewrite_standard_sql_fns_on_eql_types.rspackages/eql-mapper/src/transformation_rules/substitute_encrypted_literals.rspackages/eql-mapper/src/type_checked_statement.rspackages/showcase/CLAUDE.mdpackages/showcase/Cargo.tomlpackages/showcase/README.mdpackages/showcase/src/data.rspackages/showcase/src/main.rspackages/showcase/src/model.rspackages/showcase/src/schema.sqltests/benchmark/sql/benchmark-schema.sqltests/sql/schema-uninstall.sqltests/sql/schema.sqltests/tasks/test/integration/psql-passthrough.shvendor/stack-auth/.gitignorevendor/stack-auth/Cargo.tomlvendor/stack-auth/LICENSEvendor/stack-auth/README.mdvendor/stack-auth/examples/auto_strategy.rsvendor/stack-auth/examples/device_code.rsvendor/stack-auth/src/access_key.rsvendor/stack-auth/src/access_key_refresher.rsvendor/stack-auth/src/access_key_strategy.rsvendor/stack-auth/src/auto_refresh.rsvendor/stack-auth/src/auto_strategy.rsvendor/stack-auth/src/device_client.rsvendor/stack-auth/src/device_code/mod.rsvendor/stack-auth/src/device_code/protocol.rsvendor/stack-auth/src/device_code/tests.rsvendor/stack-auth/src/lib.rsvendor/stack-auth/src/oauth_refresher.rsvendor/stack-auth/src/oauth_strategy.rsvendor/stack-auth/src/refresher.rsvendor/stack-auth/src/service_token.rsvendor/stack-auth/src/static_token_strategy.rsvendor/stack-auth/src/token.rs
💤 Files with no reviewable changes (25)
- vendor/stack-auth/.gitignore
- vendor/stack-auth/README.md
- vendor/stack-auth/Cargo.toml
- packages/cipherstash-proxy/src/proxy/sql/select_config.sql
- vendor/stack-auth/src/static_token_strategy.rs
- vendor/stack-auth/src/access_key_strategy.rs
- packages/eql-mapper/src/transformation_rules/cast_literals_as_encrypted.rs
- vendor/stack-auth/examples/auto_strategy.rs
- packages/eql-mapper/src/transformation_rules/cast_params_as_encrypted.rs
- vendor/stack-auth/src/oauth_refresher.rs
- vendor/stack-auth/src/refresher.rs
- vendor/stack-auth/examples/device_code.rs
- vendor/stack-auth/LICENSE
- vendor/stack-auth/src/access_key.rs
- vendor/stack-auth/src/device_code/tests.rs
- vendor/stack-auth/src/oauth_strategy.rs
- vendor/stack-auth/src/device_client.rs
- vendor/stack-auth/src/auto_refresh.rs
- vendor/stack-auth/src/auto_strategy.rs
- vendor/stack-auth/src/token.rs
- vendor/stack-auth/src/access_key_refresher.rs
- vendor/stack-auth/src/device_code/mod.rs
- vendor/stack-auth/src/device_code/protocol.rs
- vendor/stack-auth/src/service_token.rs
- vendor/stack-auth/src/lib.rs
| | Rule | What it does | | ||
| |---|---| | ||
| | `CastLiteralsAsEncrypted` | Replaces plaintext literals with `eql_v2.cast_as_encrypted(ciphertext)` | | ||
| | `CastParamsAsEncrypted` | Wraps parameter placeholders (`$1`, `$2`, ...) with encrypted casts | | ||
| | `RewriteContainmentOps` | Transforms `col @> val` to `eql_v2.jsonb_contains(col, val)` | | ||
| | `RewriteStandardSqlFnsOnEqlTypes` | Rewrites `min()`, `max()`, `jsonb_path_query()` etc. to `eql_v2.*` equivalents | | ||
| | `RewriteEqlComparisonOps` | Rewrites scalar comparisons: `col <op> x` → `eql_v3.<term>(col) <op> eql_v3.<term>(x)` (term chosen from the column's domain: `eq_term`/`ord_term`/`ord_term_ore`) | | ||
| | `RewriteEqlMatchOps` | Rewrites `LIKE`/`ILIKE`/`@@` to `eql_v3.match_term(a) @> eql_v3.match_term(b)` | | ||
| | `RewriteContainmentOps` | Rewrites JSON `@>`/`<@` to `eql_v3.jsonb_contains`/`jsonb_contained_by`, and `->`/`->>` to `eql_v3."->"`/`"->>"` | | ||
| | `RewriteStandardSqlFnsOnEqlTypes` | Rewrites `min()`, `max()`, `jsonb_path_query()` etc. to their `eql_v3.*` counterparts (`count()` stays native) | | ||
| | `CastLiteralsAsEncrypted` | Replaces plaintext literals with the ciphertext cast to the column's v3 domain (`::public.eql_v3_*`) or, for a query operand, its query twin (`::eql_v3.query_*`) | | ||
| | `CastParamsAsEncrypted` | Wraps parameter placeholders (`$1`, `$2`, ...) with the same v3 domain casts | | ||
| | `PreserveEffectiveAliases` | Maintains column aliases through transformations | | ||
| | `FailOnPlaceholderChange` | Postcondition check that prepared statement placeholders weren't corrupted | |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Pipeline table doesn't match the actual rule set.
This table was updated for v3 but doesn't line up with the rules actually wired into the pipeline (type_checked_statement.rs::make_transformer, per this stack's own diff): the real chain is SubstituteEncryptedLiterals, RewriteStandardSqlFnsOnEqlTypes, RewriteContainmentOps, RewriteJsonValueSelectorEq, RewriteEqlComparisonOps, RewriteEqlMatchOps, RewriteEqlOrderBy, RewriteEqlGroupBy, CastFullPayloadOperands, PreserveEffectiveAliases, FailOnPlaceholderChange. The rows CastLiteralsAsEncrypted/CastParamsAsEncrypted don't match any rule struct name in that chain (the actual casting rules are SubstituteEncryptedLiterals and CastFullPayloadOperands), and three rules central to this PR — RewriteJsonValueSelectorEq, RewriteEqlOrderBy, RewriteEqlGroupBy — are missing from the table entirely.
📝 Suggested correction direction
-| `CastLiteralsAsEncrypted` | Replaces plaintext literals with the ciphertext cast to the column's v3 domain (`::public.eql_v3_*`) or, for a query operand, its query twin (`::eql_v3.query_*`) |
-| `CastParamsAsEncrypted` | Wraps parameter placeholders (`$1`, `$2`, ...) with the same v3 domain casts |
+| `SubstituteEncryptedLiterals` | Replaces plaintext literals with the ciphertext cast to the column's v3 domain (`::public.eql_v3_*`) or, for a query operand, its query twin (`::eql_v3.query_*`) |
+| `RewriteJsonValueSelectorEq` | Rewrites encrypted JSON field equality/inequality to value-selector containment |
+| `RewriteEqlOrderBy` | Rewrites `ORDER BY` on an encrypted column to order by its ordering term |
+| `RewriteEqlGroupBy` | Rewrites `GROUP BY` on an encrypted column to group by its equality term, lifting projections through `eql_v3.grouped_value` |
+| `CastFullPayloadOperands` | Wraps `INSERT`/`UPDATE` value placeholders (`$1`, `$2`, ...) with the column's full-payload v3 domain cast |📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| | Rule | What it does | | |
| |---|---| | |
| | `CastLiteralsAsEncrypted` | Replaces plaintext literals with `eql_v2.cast_as_encrypted(ciphertext)` | | |
| | `CastParamsAsEncrypted` | Wraps parameter placeholders (`$1`, `$2`, ...) with encrypted casts | | |
| | `RewriteContainmentOps` | Transforms `col @> val` to `eql_v2.jsonb_contains(col, val)` | | |
| | `RewriteStandardSqlFnsOnEqlTypes` | Rewrites `min()`, `max()`, `jsonb_path_query()` etc. to `eql_v2.*` equivalents | | |
| | `RewriteEqlComparisonOps` | Rewrites scalar comparisons: `col <op> x` → `eql_v3.<term>(col) <op> eql_v3.<term>(x)` (term chosen from the column's domain: `eq_term`/`ord_term`/`ord_term_ore`) | | |
| | `RewriteEqlMatchOps` | Rewrites `LIKE`/`ILIKE`/`@@` to `eql_v3.match_term(a) @> eql_v3.match_term(b)` | | |
| | `RewriteContainmentOps` | Rewrites JSON `@>`/`<@` to `eql_v3.jsonb_contains`/`jsonb_contained_by`, and `->`/`->>` to `eql_v3."->"`/`"->>"` | | |
| | `RewriteStandardSqlFnsOnEqlTypes` | Rewrites `min()`, `max()`, `jsonb_path_query()` etc. to their `eql_v3.*` counterparts (`count()` stays native) | | |
| | `CastLiteralsAsEncrypted` | Replaces plaintext literals with the ciphertext cast to the column's v3 domain (`::public.eql_v3_*`) or, for a query operand, its query twin (`::eql_v3.query_*`) | | |
| | `CastParamsAsEncrypted` | Wraps parameter placeholders (`$1`, `$2`, ...) with the same v3 domain casts | | |
| | `PreserveEffectiveAliases` | Maintains column aliases through transformations | | |
| | `FailOnPlaceholderChange` | Postcondition check that prepared statement placeholders weren't corrupted | | |
| | Rule | What it does | | |
| |---|---| | |
| | `RewriteEqlComparisonOps` | Rewrites scalar comparisons: `col <op> x` → `eql_v3.<term>(col) <op> eql_v3.<term>(x)` (term chosen from the column's domain: `eq_term`/`ord_term`/`ord_term_ore`) | | |
| | `RewriteEqlMatchOps` | Rewrites `LIKE`/`ILIKE`/`@@` to `eql_v3.match_term(a) @> eql_v3.match_term(b)` | | |
| | `RewriteContainmentOps` | Rewrites JSON `@>`/`<@` to `eql_v3.jsonb_contains`/`jsonb_contained_by`, and `->`/`->>` to `eql_v3."->"`/`"->>"` | | |
| | `RewriteStandardSqlFnsOnEqlTypes` | Rewrites `min()`, `max()`, `jsonb_path_query()` etc. to their `eql_v3.*` counterparts (`count()` stays native) | | |
| | `SubstituteEncryptedLiterals` | Replaces plaintext literals with the ciphertext cast to the column's v3 domain (`::public.eql_v3_*`) or, for a query operand, its query twin (`::eql_v3.query_*`) | | |
| | `RewriteJsonValueSelectorEq` | Rewrites encrypted JSON field equality/inequality to value-selector containment | | |
| | `RewriteEqlOrderBy` | Rewrites `ORDER BY` on an encrypted column to order by its ordering term | | |
| | `RewriteEqlGroupBy` | Rewrites `GROUP BY` on an encrypted column to group by its equality term, lifting projections through `eql_v3.grouped_value` | | |
| | `CastFullPayloadOperands` | Wraps `INSERT`/`UPDATE` value placeholders (`$1`, `$2`, ...) with the column's full-payload v3 domain cast | | |
| | `PreserveEffectiveAliases` | Maintains column aliases through transformations | | |
| | `FailOnPlaceholderChange` | Postcondition check that prepared statement placeholders weren't corrupted | |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ARCHITECTURE.md` around lines 117 - 126, Update the pipeline rules table to
match the chain assembled by type_checked_statement.rs::make_transformer: rename
the casting rows to SubstituteEncryptedLiterals and CastFullPayloadOperands,
remove the inaccurate CastParamsAsEncrypted entry, and add
RewriteJsonValueSelectorEq, RewriteEqlOrderBy, and RewriteEqlGroupBy with
descriptions of their behavior. Preserve the existing descriptions for rules
that remain in the actual chain.
|
|
||
| <a id='field_access_operator'></a> | ||
| ### `-> text returns eql_v2_encrypted decrypted as jsonb` | ||
| ### `-> text returns eql_v3_json_search decrypted as jsonb` |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
->/->> headings dropped the left operand's type.
Both headings read -> text returns eql_v3_json_search decrypted as jsonb / ->> text returns eql_v3_json_search decrypted as jsonb, omitting the left-hand operand's type entirely. Every sibling operator heading in this doc (e.g. @>/<@ at lines 270 and 325) explicitly names both operand types (eql_v3_json_search @> eql_v3_json_search returns boolean). This looks like a leftover from the v2→v3 heading rewrite (the old eql_v2_encrypted -> prefix was dropped but never replaced with eql_v3_json_search ->).
📝 Proposed fix
-### `-> text returns eql_v3_json_search decrypted as jsonb`
+### `eql_v3_json_search -> text returns eql_v3_json_search decrypted as jsonb`-### `->> text returns eql_v3_json_search decrypted as jsonb`
+### `eql_v3_json_search ->> text returns eql_v3_json_search decrypted as jsonb`Also applies to: 204-204
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/reference/searchable-json.md` at line 150, Update the `->` and `->>`
headings in searchable-json.md to include `eql_v3_json_search` as the left-hand
operand type, matching the operand-type format used by sibling operator headings
while preserving their existing return descriptions.
| /// | ||
| /// Set CS_GENERATE_EQL_FIXTURES=1 to enable fixture generation. | ||
| #[tokio::test] | ||
| #[ignore = "EQL v2->v3 backwards-compat regression: prior releases are not in use, and v2 ciphertext / ::eql_v2_encrypted casts do not exist under v3. Regenerate fixtures from a v3 baseline to re-enable."] |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Port the fixture path before disabling these regressions.
“Regenerate fixtures from a v3 baseline” cannot re-enable this suite: insert_encrypted_directly still casts fixture payloads to eql_v2_encrypted. V3-generated fixtures will therefore fail reinsertion, leaving encryption, decryption, and JSON regression coverage permanently skipped.
packages/cipherstash-proxy-integration/src/eql_regression.rs#L157-L157: port fixture generation and direct insertion to v3/jsonb payloads, then unignore generation.packages/cipherstash-proxy-integration/src/eql_regression.rs#L268-L268: regenerate and re-enable text regression coverage.packages/cipherstash-proxy-integration/src/eql_regression.rs#L300-L300: regenerate and re-enableint2coverage.packages/cipherstash-proxy-integration/src/eql_regression.rs#L330-L330: regenerate and re-enableint4coverage.packages/cipherstash-proxy-integration/src/eql_regression.rs#L360-L360: regenerate and re-enableint8coverage.packages/cipherstash-proxy-integration/src/eql_regression.rs#L390-L390: regenerate and re-enablefloat8coverage.packages/cipherstash-proxy-integration/src/eql_regression.rs#L420-L420: regenerate and re-enable boolean coverage.packages/cipherstash-proxy-integration/src/eql_regression.rs#L450-L450: regenerate and re-enable JSONB decryption coverage.packages/cipherstash-proxy-integration/src/eql_regression.rs#L480-L480: regenerate and re-enable JSON field-access coverage.packages/cipherstash-proxy-integration/src/eql_regression.rs#L532-L532: regenerate and re-enable JSON array coverage.
📍 Affects 1 file
packages/cipherstash-proxy-integration/src/eql_regression.rs#L157-L157(this comment)packages/cipherstash-proxy-integration/src/eql_regression.rs#L268-L268packages/cipherstash-proxy-integration/src/eql_regression.rs#L300-L300packages/cipherstash-proxy-integration/src/eql_regression.rs#L330-L330packages/cipherstash-proxy-integration/src/eql_regression.rs#L360-L360packages/cipherstash-proxy-integration/src/eql_regression.rs#L390-L390packages/cipherstash-proxy-integration/src/eql_regression.rs#L420-L420packages/cipherstash-proxy-integration/src/eql_regression.rs#L450-L450packages/cipherstash-proxy-integration/src/eql_regression.rs#L480-L480packages/cipherstash-proxy-integration/src/eql_regression.rs#L532-L532
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/cipherstash-proxy-integration/src/eql_regression.rs` at line 157,
The EQL regression fixtures remain disabled because generation and
insert_encrypted_directly still use v2 ciphertext and eql_v2_encrypted casts. In
packages/cipherstash-proxy-integration/src/eql_regression.rs:157-157, port
fixture generation and direct insertion to v3/jsonb payloads, regenerate
fixtures, and remove the ignore; then regenerate and re-enable text coverage at
268-268, int2 at 300-300, int4 at 330-330, int8 at 360-360, float8 at 390-390,
boolean at 420-420, JSONB decryption at 450-450, JSON field access at 480-480,
and JSON array coverage at 532-532.
| /// Normalises a JSON field selector to an eJSONPath rooted at `$`. | ||
| /// | ||
| /// `->`/`->>` take a bare field name (`name`), `jsonb_path_query*` takes a path | ||
| /// (`nested.title`, or already-rooted `$.nested.title`). The client's | ||
| /// `Selector::parse` only accepts the rooted form. | ||
| pub fn json_selector_path(val: &str) -> String { | ||
| if val.starts_with("$.") { | ||
| val.to_string() | ||
| } else { | ||
| format!("$.{val}") | ||
| } | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Rooted-path detection only recognises the $. prefix.
jsonb_path_query accepts rooted paths that don't begin with $. — '$', '$[0]', '$["a"]', '$[*].b'. Those get re-rooted into $.$, $.$[0], etc., producing a selector that silently matches nothing rather than erroring.
🐛 Proposed fix
pub fn json_selector_path(val: &str) -> String {
- if val.starts_with("$.") {
+ if val.starts_with('$') {
val.to_string()
} else {
format!("$.{val}")
}
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| /// Normalises a JSON field selector to an eJSONPath rooted at `$`. | |
| /// | |
| /// `->`/`->>` take a bare field name (`name`), `jsonb_path_query*` takes a path | |
| /// (`nested.title`, or already-rooted `$.nested.title`). The client's | |
| /// `Selector::parse` only accepts the rooted form. | |
| pub fn json_selector_path(val: &str) -> String { | |
| if val.starts_with("$.") { | |
| val.to_string() | |
| } else { | |
| format!("$.{val}") | |
| } | |
| } | |
| /// Normalises a JSON field selector to an eJSONPath rooted at `$`. | |
| /// | |
| /// `->`/`->>` take a bare field name (`name`), `jsonb_path_query*` takes a path | |
| /// (`nested.title`, or already-rooted `$.nested.title`). The client's | |
| /// `Selector::parse` only accepts the rooted form. | |
| pub fn json_selector_path(val: &str) -> String { | |
| if val.starts_with('$') { | |
| val.to_string() | |
| } else { | |
| format!("$.{val}") | |
| } | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/cipherstash-proxy/src/postgresql/data/from_sql.rs` around lines 109
- 120, Update json_selector_path to treat any selector beginning with the JSON
path root character '$' as already rooted, including '$', '$[0]', '$["a"]', and
'$[*].b'; only prepend '$.' for unrooted field names.
| for table in tables { | ||
| let table_name: String = table.get("table_name"); | ||
| let columns: Vec<String> = table.get("columns"); | ||
| let column_domain_names: Vec<Option<String>> = table.get("column_domain_names"); | ||
|
|
||
| Ok(encrypt_config) | ||
| } | ||
| Err(err) => { | ||
| if configuration_table_not_found(&err) { | ||
| return Err(ConfigError::MissingEncryptConfigTable.into()); | ||
| for (column, domain) in columns.iter().zip(column_domain_names) { | ||
| let Some(domain) = domain else { continue }; | ||
| if let Some(column_config) = column_config_from_domain(&table_name, column, &domain) { | ||
| debug!( | ||
| target: ENCRYPT_CONFIG, | ||
| msg = "Encrypted column", | ||
| table = table_name, | ||
| column = column, | ||
| domain = domain | ||
| ); | ||
| map.insert( | ||
| eql::Identifier::new(table_name.clone(), column.clone()), | ||
| column_config, | ||
| ); | ||
| } | ||
| Err(ConfigError::Database(err).into()) | ||
| } | ||
| } |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
rg -n 'table_schema' --type=sql packages/cipherstash-proxy/src/proxy/sql
ast-grep run --pattern 'Identifier::new($$$)' --lang rust packages/cipherstash-proxy/src | head -50
rg -nP -C3 'fn new\s*\(' packages/cipherstash-proxy/src --glob '*identifier*'Repository: cipherstash/proxy
Length of output: 1735
🏁 Script executed:
#!/bin/bash
set -e
printf '--- manager relevant sections ---\n'
sed -n '1,260p' packages/cipherstash-proxy/src/proxy/encrypt_config/manager.rs
printf '\n--- schema SQL ---\n'
cat -n packages/cipherstash-proxy/src/proxy/sql/select_table_schemas.sql
printf '\n--- eql identifier occurrences ---\n'
rg -n 'pub struct Identifier|impl .*Identifier|Identifier|schema' packages/cipherstash-proxy/src --glob '*.rs'Repository: cipherstash/proxy
Length of output: 246
🏁 Script executed:
#!/bin/bash
set -e
echo '--- manager relevant sections ---'
sed -n '1,280p' packages/cipherstash-proxy/src/proxy/encrypt_config/manager.rs
echo
echo '--- schema SQL ---'
cat -n packages/cipherstash-proxy/src/proxy/sql/select_table_schemas.sql
echo
echo '--- eql identifier occurrences ---'
rg -n 'pub struct Identifier|impl .*Identifier|Identifier|schema' packages/cipherstash-proxy/src --glob '*.rs'Repository: cipherstash/proxy
Length of output: 30354
🏁 Script executed:
#!/bin/bash
set -e
echo '--- schema manager / load_schema relevant sections ---'
sed -n '1,225p' packages/cipherstash-proxy/src/proxy/schema/manager.rs
echo
echo '--- find package manifests for cipherstash-client source / dependency ---'
fd -i 'Cargo.toml|Cargo.lock' . -d 6 | while read -r f; do
grep -n 'cipherstash-client\|cipherstash_proxy\|cipherstash-client\|packages=cipherstash-proxy' "$f" || true
done
echo
echo '--- identifier definitions and constructor usage in codebase ---'
rg -n 'struct Identifier|enum Identifier|pub fn new|impl .*Identifier' --glob '*.rs' --glob '!packages/cipherstash-proxy/src/postgresql/data/from_sql.rs'Repository: cipherstash/proxy
Length of output: 7721
🌐 Web query:
cipherstash-client 0.42.0 eql Identifier::new source
💡 Result:
In the cipherstash-client Rust SDK, the Identifier::new method is used to construct a new identifier, which represents the specific table and column associated with an encrypted payload [1][2]. This identifier is a core component within the eql module, which handles the types and encryption/decryption logic required to interact with CipherStash's Encrypt Query Language (EQL) [3][4]. The cipherstash-client SDK (version 0.42.0 is the current version on crates.io as of July 2026) [5] provides the necessary abstractions to work with EQL. While specific method signatures can evolve between versions, the Identifier struct is utilized within the EncryptedPayload structure to uniquely identify the data's origin schema [1]. You can access the official documentation for the latest version of the crate on docs.rs to view the precise signature and implementation details for Identifier::new and related types [3][6].
Citations:
- 1: https://docs.rs/cipherstash-client/latest/cipherstash_client/eql/struct.EncryptedPayload.html
- 2: https://docs.rs/cipherstash-client/latest/cipherstash_client/eql/enum.EqlCiphertext.html
- 3: https://docs.rs/cipherstash-client/latest/cipherstash_client/
- 4: https://docs.rs/cipherstash-client/latest/cipherstash_client/index.html
- 5: https://crates.io/crates/cipherstash-client
- 6: https://docs.rs/cipherstash-client/latest/cipherstash_client/all.html
🌐 Web query:
github.com/cipherstash/proxy src/eql/schema.rs cipherstash-client Identifier new
💡 Result:
In the context of the CipherStash Proxy codebase, particularly within src/eql/schema.rs, Identifier::new is a method used to construct an Identifier object, which represents a SQL identifier (such as a table or column name) [1]. Historically, this has been used in the IndexResolver to handle schema-qualified table names [1]. For instance, it allows the proxy to correctly resolve TableColumn lookups by using a fully qualified table name rather than just a bare table name, ensuring that schema-qualified EncryptConfig keys are matched correctly [1]. While cipherstash-client is a distinct Rust crate providing SDK functionality (such as managing keysets, workspace access, and authentication), the Identifier type found in cipherstash/proxy is typically part of the proxy's internal logic for parsing and mapping SQL statements to EQL (Encrypt Query Language) [1][2]. It should not be confused with the Name struct or other resource identifiers found in the cipherstash-client crate, which are used for managing CipherStash service resources [3][4].
Citations:
- 1: feat: client 0.38.0 migration — drop vendored stack-auth + EQL v2.3 mapper support (CIP-3233) #409
- 2: https://crates.io/crates/cipherstash-client
- 3: https://docs.rs/cipherstash-client/latest/cipherstash_client/
- 4: https://docs.rs/cipherstash-client/latest/cipherstash_client/struct.Name.html
🌐 Web query:
crates.io cipherstash-client 0.42.0 eql.rs Identifier fn new
💡 Result:
In the cipherstash-client crate (version 0.42.0), the Identifier type located in the cipherstash_client::eql module is used to represent a specific database table and column pair [1][2]. While the public documentation for cipherstash_client confirms that Identifier is a key component for EQL payloads and internal configuration lookups [1][2][3], it does not explicitly document a public new constructor method for the Identifier struct. Typically, in this SDK, identifiers are managed through internal resolution paths or are implicitly handled by the types that contain them, such as EncryptedPayload [2]. If you are attempting to construct an identifier, note that internal codebase discussions suggest that when resolving configuration, the SDK performs lookups that rely on fully qualified table names [3]. Developers should rely on the documented interfaces for EQL operations rather than manually constructing internal structures unless those methods are explicitly exposed as public API [1][2].
Citations:
- 1: https://docs.rs/cipherstash-client/latest/cipherstash_client/eql/index.html
- 2: https://docs.rs/cipherstash-client/latest/cipherstash_client/eql/struct.EncryptedPayload.html
- 3: feat: client 0.38.0 migration — drop vendored stack-auth + EQL v2.3 mapper support (CIP-3233) #409
🌐 Web query:
ghcr.io crates.io source cipherstash-client 0.42.0 eql identifier
💡 Result:
The cipherstash-client crate is an official Rust SDK provided by CipherStash for interacting with their services, such as ZeroKMS, the CipherStash Token Service (CTS), and Audit logging [1][2]. As of July 28, 2026, version 0.42.0 is the latest published version on crates.io [1]. Regarding the Encrypt Query Language (EQL) components in the client: 1. Identifier: Within the cipherstash_client::eql module, Identifier is a struct used to identify specific database table and column pairs associated with encrypted payloads [3][4]. The EqlCiphertext enum provides a method identifier which returns the identifier for a given payload [5]. 2. EQL Payload Format: In the context of EQL, the identifier is part of the EncryptedPayload structure, which includes fields such as the table identifier (i.t) and column identifier (i.c) [4][6]. 3. Container/Registry: While you mentioned ghcr.io, cipherstash-client is primarily distributed as a Rust crate via crates.io [1]. The CipherStash suite source code is maintained on GitHub (https://github.com/cipherstash/cipherstash-suite) [1], and container images for related infrastructure (like CipherStash Proxy) may be hosted on ghcr.io, but the client library itself is consumed as a dependency in Rust projects via Cargo [1][2].
Citations:
- 1: https://crates.io/crates/cipherstash-client
- 2: https://docs.rs/cipherstash-client/latest/cipherstash_client/index.html
- 3: https://docs.rs/cipherstash-client/latest/cipherstash_client/eql/index.html
- 4: https://docs.rs/cipherstash-client/latest/cipherstash_client/eql/struct.EncryptedPayload.html
- 5: https://docs.rs/cipherstash-client/latest/cipherstash_client/eql/enum.EqlCiphertext.html
- 6: https://github.com/cipherstash/encrypt-query-language/blob/main/docs/reference/PAYLOAD.md
🌐 Web query:
site:docs.rs/cipherstash-client/0.42.0 cipherstash_client::eql::Identifier
💡 Result:
There is no publicly indexed documentation or code reference for a specific identifier named cipherstash_client::eql::Identifier. CipherStash is a data security platform that provides searchable encryption for PostgreSQL [1]. Its architecture relies on the Encrypt Query Language (EQL), which defines the abstractions, JSONB payload formats, and indexing functions required to perform operations like equality, range, and free-text search directly over encrypted data [1][2]. While CipherStash utilizes Rust for its proxy and core encryption libraries [2][3], the specific path or struct cipherstash_client::eql::Identifier does not appear in the available project repositories [1][2][3][4][5]. It is possible that this refers to an internal, non-public, or deprecated API, or a component of a specific implementation that is not exposed in the main documentation.
Citations:
- 1: https://github.com/cipherstash/cipherstash
- 2: https://github.com/cipherstash/proxy?tab=readme-ov-file
- 3: https://www.github.com/cipherstash
- 4: https://github.com/cipherstash/protectjs
- 5: https://github.com/cipherstash/stack
🌐 Web query:
github.com/cipherstash/cipherstash-src 0.42.0 e Ql Identifier::new table column
💡 Result:
The reference to Identifier::new in the context of CipherStash's internal codebases (specifically within the cipherstash/proxy repository) relates to the construction of identifiers for table-column lookups during EQL (Encrypt Query Language) processing [1]. In internal development (such as the migration work cited in cipherstash/proxy pull requests), Identifier::new is used to construct identifiers that map data rows to specific schema-qualified table columns [1]. Developers have been advised to ensure that Identifier::new is called with fully qualified table names to prevent lookup failures, particularly for schema-qualified tables that might otherwise default to an incorrect lookup path [1]. This identifier structure is fundamental to how CipherStash handles encrypted data, where each piece of ciphertext is associated with a specific table and column (often represented as an object with t for table and c for column) to enable correct decryption and index utilization [2]. Please note that cipherstash-src is not a standard public repository name; CipherStash repositories are typically organized by their specific components, such as proxy, stack, encrypt-query-language, or cipherstash-client [3]. The usage described pertains to internal Rust-based infrastructure within the cipherstash/proxy project [1].
Citations:
- 1: feat: client 0.38.0 migration — drop vendored stack-auth + EQL v2.3 mapper support (CIP-3233) #409
- 2: https://cipherstash.com/docs/stack/reference/cipher-cell
- 3: https://github.com/cipherstash/cipherstash
🌐 Web query:
crates.io source cipherstash-client 0.42.0 identifier.rs New Identifier table column
💡 Result:
In the cipherstash-client crate, specifically within the cipherstash_client::eql module, the Identifier struct is used to represent the table and column to which an encrypted payload belongs [1][2]. The EncryptedPayload struct, which is defined in cipherstash_client::eql, contains an identifier field of type Identifier [2]. According to the documentation for this struct, this identifier serves to identify the "Table and column this payload belongs to" [2]. While there is no public file named identifier.rs exposed as a top-level module in the crate documentation, the Identifier type is a core component within the eql module, which handles CipherStash encryption schemes and indexers [3][4][2]. The version 0.42.0 of cipherstash-client was published on July 19, 2026 [5]. You can review the specific implementation details by accessing the source code for the eql module in the cipherstash/cipherstash-suite repository on GitHub [5].
Citations:
- 1: https://docs.rs/cipherstash-client/latest/cipherstash_client/all.html
- 2: https://docs.rs/cipherstash-client/latest/cipherstash_client/eql/struct.EncryptedPayload.html
- 3: https://docs.rs/cipherstash-client/latest/cipherstash_client/
- 4: https://docs.rs/cipherstash-client/latest/cipherstash_client/index.html
- 5: https://crates.io/crates/cipherstash-client
🌐 Web query:
rust docs.rs source cipherstash-client 0.42.0 Identifier::new table column Identifier struct
💡 Result:
In the cipherstash-client crate (version 0.42.0), the Identifier struct is located within the cipherstash_client::eql module [1]. This struct is used to uniquely identify a database table and column pair [2][3][1]. While the documentation confirms the existence of the Identifier struct and its role in EQL payloads [2][3][1], specific public constructor methods such as Identifier::new are not explicitly documented in the available crate documentation. References in development discussions suggest that Identifier construction may be used during index resolution to target fully qualified table and column names for configuration lookups [4]. For further implementation details, you may need to inspect the source code directly or examine the private module visibility if the constructor is not exposed in the public API.
Citations:
- 1: https://docs.rs/cipherstash-client/latest/cipherstash_client/eql/index.html
- 2: https://docs.rs/cipherstash-client/latest/cipherstash_client/eql/struct.EncryptedPayload.html
- 3: https://docs.rs/cipherstash-client/latest/cipherstash_client/eql/struct.SteVecPayload.html
- 4: feat: client 0.38.0 migration — drop vendored stack-auth + EQL v2.3 mapper support (CIP-3233) #409
Include the schema when loading encrypt config
SCHEMA_QUERY groups by table_schema, table_name but load_encrypt_config inserts into HashMap<eql::Identifier> with only table_name and column. Same-named tables in different schemas overwrite each other, so a column can receive the wrong domain config or drop encryption mapping entirely. Restrict the query/scanner to the served schema, or carry the loaded schema into the config key before inserting.
[low_effort_and_HIGH_reward not applicable]
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/cipherstash-proxy/src/proxy/encrypt_config/manager.rs` around lines
202 - 223, Update load_encrypt_config to preserve table_schema from SCHEMA_QUERY
when building each eql::Identifier, so configurations for same-named tables in
different schemas cannot overwrite one another. Carry the schema through the
table scan and include it in map.insert alongside table_name and column, or
restrict SCHEMA_QUERY to the served schema.
| ``` | ||
| col <op> operand → eql_v3.<term>(col) <op> eql_v3.<term>(operand) | ||
| ``` |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Add language identifiers to fenced code blocks.
Static analysis flags both fenced blocks as missing a language tag (MD040).
📝 Proposed fix
-```
+```text
col <op> operand → eql_v3.<term>(col) <op> eql_v3.<term>(operand)
```diff
-```
+```text
salary > 'x' → eql_v3.ord_term(salary) > eql_v3.ord_term('<ct>'::jsonb::eql_v3.query_numeric_ord)
</details>
Also applies to: 72-74
<details>
<summary>🧰 Tools</summary>
<details>
<summary>🪛 markdownlint-cli2 (0.23.0)</summary>
[warning] 37-37: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
</details>
</details>
<details>
<summary>🤖 Prompt for AI Agents</summary>
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @packages/eql-mapper/docs/adr/0003-v3-rewrite-pipeline.md around lines 37 -
39, Add the text language identifier to both fenced code blocks in the ADR,
including the example around the operator transformation and the block around
the salary comparison, so the Markdown fences satisfy MD040 without changing
their contents.
</details>
<!-- fingerprinting:phantom:medusa:beignet -->
<!-- cr-indicator-types:potential_issue -->
<!-- cr-comment:v1:1c13aebd952dcac2f6b57278 -->
_Source: Linters/SAST tools_
<!-- This is an auto-generated comment by CodeRabbit -->
| let handled = handled | ||
| || if matches!(op, BinaryOperator::Eq | BinaryOperator::NotEq) { | ||
| match ( | ||
| self.eql_json_field_access(left), | ||
| self.eql_json_field_access(right), | ||
| ) { | ||
| (Some((json, selector)), None) => { | ||
| self.infer_json_value_selector(json, selector, right)?; | ||
| self.unify_node_with_type(expr_val, Type::native())?; | ||
| true | ||
| } | ||
| (None, Some((json, selector))) => { | ||
| self.infer_json_value_selector(json, selector, left)?; | ||
| self.unify_node_with_type(expr_val, Type::native())?; | ||
| true | ||
| } | ||
| _ => false, | ||
| } | ||
| } else { | ||
| false | ||
| }; | ||
|
|
||
| if !handled { | ||
| get_sql_binop_rule(op).apply_constraints(self, left, right, expr_val)?; | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Declined JSON-selector fusion never falls through to ordinary typing as documented.
infer_json_value_selector's doc comment says an unresolvable selector "declines" fusion so the comparison "falls through to ordinary typing" to produce a clear capability error. But the function just returns Ok(()) without unifying value's type, and the caller unconditionally sets handled = true afterward — so get_sql_binop_rule(op).apply_constraints(...) (the promised fallback) is never invoked. When the selector is a column reference or function call, value's type variable is left completely unconstrained instead of failing the capability check, likely surfacing a confusing TypeError::Incomplete (unresolved type variable) rather than the intended, clearer bounds error.
🐛 Proposed fix: propagate fusion success back to the caller
fn infer_json_value_selector(
&self,
json: EqlValue,
selector: &'ast Expr,
value: &'ast Expr,
- ) -> Result<(), TypeError> {
+ ) -> Result<bool, TypeError> {
let Some(source) = Self::json_selector_source(selector) else {
- return Ok(());
+ return Ok(false);
};
self.unify_node_with_type(
value,
Type::Value(Value::Eql(EqlTerm::JsonValueSelector(json))),
)?;
match Self::as_ast_value(value) {
Some(ast::Value::Placeholder(placeholder)) => {
if let Ok(param) = Param::try_from(placeholder) {
self.record_json_value_selector_param(param, source);
}
}
Some(node) => self.record_json_value_selector_literal(node, source),
None => {}
}
- Ok(())
+ Ok(true)
} (Some((json, selector)), None) => {
- self.infer_json_value_selector(json, selector, right)?;
- self.unify_node_with_type(expr_val, Type::native())?;
- true
+ if self.infer_json_value_selector(json, selector, right)? {
+ self.unify_node_with_type(expr_val, Type::native())?;
+ true
+ } else {
+ false
+ }
}
(None, Some((json, selector))) => {
- self.infer_json_value_selector(json, selector, left)?;
- self.unify_node_with_type(expr_val, Type::native())?;
- true
+ if self.infer_json_value_selector(json, selector, left)? {
+ self.unify_node_with_type(expr_val, Type::native())?;
+ true
+ } else {
+ false
+ }
}Want me to open a Linear issue (CIP-) to track this?
Based on learnings, issues in this repo are tracked in Linear under the Product Engineering team using CIP- keys.
Also applies to: 633-667
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/eql-mapper/src/inference/infer_type_impls/expr.rs` around lines 168
- 192, Propagate whether JSON-selector fusion succeeded from
infer_json_value_selector instead of treating every Ok result as handled. Update
the Eq/NotEq branches in the binary-expression inference flow to set handled
only when fusion unifies the value type; when fusion declines, leave handled
false so get_sql_binop_rule(op).apply_constraints(...) performs ordinary typing
and reports the capability error.
Source: Learnings
| pub(crate) fn get_eql_v3_function_name(fn_name: &ObjectName) -> Option<ObjectName> { | ||
| let bare = fn_name.0.last()?; | ||
| let eql_v3_name = ObjectName(vec![ | ||
| ObjectNamePart::Identifier(Ident::new("eql_v3")), | ||
| bare.clone(), | ||
| ]); | ||
| if SQL_FUNCTION_TYPES.contains_key(&IdentCase(eql_v3_name.clone())) { | ||
| Some(eql_v3_name) | ||
| } else { | ||
| None | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
rg -n -C3 'get_eql_v3_function_name\s*\(' packages/eql-mapper
rg -n -C3 'RewriteStandardSqlFnsOnEqlTypes' packages/eql-mapperRepository: cipherstash/proxy
Length of output: 7534
🏁 Script executed:
#!/bin/bash
sed -n '1,150p' packages/eql-mapper/src/inference/sql_types/sql_decls.rs
printf '\n--- transformer ---\n'
sed -n '1,110p' packages/eql-mapper/src/transformation_rules/rewrite_standard_sql_fns_on_eql_types.rsRepository: cipherstash/proxy
Length of output: 9219
Restrict v3 counterpart lookup to pg_catalog functions.
fn_name.0.last() maps qualified calls like custom_schema.min(...) to eql_v3.min(...) because the lookup only inspects the final identifier. The v3 rewriting should apply only to unqualified functions and explicitly pg_catalog.<name> calls.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/eql-mapper/src/inference/sql_types/sql_decls.rs` around lines 113 -
123, Update get_eql_v3_function_name to accept v3 counterparts only when fn_name
is unqualified or explicitly qualified with pg_catalog; reject other schemas
before constructing the eql_v3 name. Preserve the existing SQL_FUNCTION_TYPES
lookup and return behavior for permitted function names.
| fn match_predicate(&self, expr: &'ast Expr) -> Option<(DomainIdentity, bool)> { | ||
| match expr { | ||
| Expr::Like { | ||
| expr, negated, any, .. | ||
| } | ||
| | Expr::ILike { | ||
| expr, negated, any, .. | ||
| } if !*any => self.eql_identity_of(expr).map(|id| (id, *negated)), | ||
| Expr::BinaryOp { | ||
| left, | ||
| op: BinaryOperator::AtAt, | ||
| right, | ||
| } => self | ||
| .eql_identity_of(left) | ||
| .or_else(|| self.eql_identity_of(right)) | ||
| .map(|id| (id, false)), | ||
| _ => None, | ||
| } | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Reversed @@ operand order isn't normalized, producing backwards match_term containment.
match_predicate's .or_else fallback anticipates the encrypted column appearing on either side of @@, but apply always binds original_col = left, original_pat = right positionally (Lines 98-104) and builds the output as match_term(left) @> match_term(right). For 'pattern' @@ email_col (a syntactically valid, if unusual, ordering that Postgres's own @@ operator supports symmetrically), this emits match_term(pattern) @> match_term(col) instead of the intended match_term(col) @> match_term(pattern) — a silently wrong containment check rather than an error.
🐛 Proposed fix: track which side matched and normalize (col, pattern) order
- fn match_predicate(&self, expr: &'ast Expr) -> Option<(DomainIdentity, bool)> {
+ /// Returns `(domain identity, negated, column_is_left)`.
+ fn match_predicate(&self, expr: &'ast Expr) -> Option<(DomainIdentity, bool, bool)> {
match expr {
Expr::Like {
expr, negated, any, ..
}
| Expr::ILike {
expr, negated, any, ..
- } if !*any => self.eql_identity_of(expr).map(|id| (id, *negated)),
+ } if !*any => self.eql_identity_of(expr).map(|id| (id, *negated, true)),
Expr::BinaryOp {
left,
op: BinaryOperator::AtAt,
right,
- } => self
- .eql_identity_of(left)
- .or_else(|| self.eql_identity_of(right))
- .map(|id| (id, false)),
+ } => {
+ if let Some(id) = self.eql_identity_of(left) {
+ Some((id, false, true))
+ } else {
+ self.eql_identity_of(right).map(|id| (id, false, false))
+ }
+ }
_ => None,
}
}Then in apply, use the column_is_left flag to pick (original_col, original_pat) from (left, right) or (right, left) accordingly, so the output always builds match_term(col) @> match_term(pattern) regardless of source ordering.
Also applies to: 98-134, 136-141
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/eql-mapper/src/transformation_rules/rewrite_eql_match_ops.rs` around
lines 49 - 67, Update match_predicate to return whether the matched identity is
on the left side of the @@ expression, rather than only a boolean negation flag.
In apply, use that column_is_left result to normalize original_col and
original_pat from left/right or right/left, and build match_term so the column
is always the containment operand regardless of source ordering.
coderdan
left a comment
There was a problem hiding this comment.
Reviewer: Codex (GPT-5.6 Sol).
Five correctness findings follow.
| param: Param((idx + 1) as u16), | ||
| value, | ||
| source, | ||
| query_operand: self.query_operands.contains_param(input), |
There was a problem hiding this comment.
[P1] Track query roles per placeholder occurrence
When one placeholder is reused for storage and querying, e.g. UPDATE t SET enc = $1 WHERE enc = $1, this marks both renumbered outputs query-only. The proxy strips ciphertext from the SET value, so its stored-domain cast fails. Track the role per AST occurrence/output instead of per input parameter.
| let param_types = output_params | ||
| .iter() | ||
| .map(|output| match &output.column { | ||
| Some(_) => Type::JSONB.oid() as i32, |
There was a problem hiding this comment.
[P2] Declare selector-only parameters as text
When a client supplies Parse OIDs for json_col -> $1 or jsonb_path_query(json_col, $1), the selector still has an encrypted column configuration, but the rewritten EQL function expects bare text. Forcing JSONB here makes PostgreSQL reject the rewritten Parse. Special-case JsonAccessor/JsonPath outputs as text.
| .unwrap_or(serde_json::Value::String(text)) | ||
| } | ||
| FormatCode::Binary => { | ||
| parse_bytes_from_sql::<serde_json::Value>(¶m.bytes, postgres_type) |
There was a problem hiding this comment.
[P2] Decode binary text JSON-field operands as strings
For extended ->> equality with a binary TEXT/VARCHAR RHS, normal bytes such as Alice are passed directly to serde_json::Value::from_sql and rejected instead of receiving the text-format string fallback. The same issue affects JsonOrd. Decode textual PostgreSQL types as String before JSON parsing.
| fn projects_grouped_column(&self, item: &'ast SelectItem, grouped: &[EqlValue]) -> bool { | ||
| Self::select_item_expr(item) | ||
| .and_then(|expr| self.eql_value_of(expr)) | ||
| .is_some_and(|value| grouped.contains(&value)) |
There was a problem hiding this comment.
[P2] Avoid wrapping aggregate projections in grouped_value
For SELECT MIN(enc) FROM t GROUP BY enc, MIN(enc) resolves to the same EqlValue as the grouped column, so this predicate wraps it as grouped_value(eql_v3.min(enc)). PostgreSQL rejects that nested aggregate. Only direct grouped-column projections should be lifted.
| fn select_item_expr(item: &'ast SelectItem) -> Option<&'ast Expr> { | ||
| match item { | ||
| SelectItem::UnnamedExpr(expr) | SelectItem::ExprWithAlias { expr, .. } => Some(expr), | ||
| _ => None, |
There was a problem hiding this comment.
[P2] Handle wildcard projections in GROUP BY rewrites
For a valid query such as SELECT * FROM t GROUP BY id, enc, the encrypted key becomes eq_term(enc), but the wildcard still projects bare enc, causing PostgreSQL's “must appear in GROUP BY” error. Expand/lift encrypted wildcard columns or reject this shape before emitting invalid SQL.
coderdan
left a comment
There was a problem hiding this comment.
Nice work. Findings from the Codex review should be addressed, particularly the P1.
`ORDER BY` on an encrypted column must order by its ordering term —
ordering on the bare column compares whole jsonb payloads starting at the
randomised ciphertext, which is silently wrong and differs on every
insert. But PostgreSQL requires every `ORDER BY` expression under
`SELECT DISTINCT` to appear in the select list, and the term does not:
ERROR: for SELECT DISTINCT, ORDER BY expressions must appear in select list
So the two rewrites are individually correct and jointly invalid. This
restructures the query entirely within the transformation stage — no
proxy-side handling — by pushing the select into a subquery that also
projects the ordering term, and ordering the outer query by it:
SELECT DISTINCT a, enc FROM t ORDER BY enc
-- becomes
SELECT __eql_col_0 AS a, __eql_col_1 AS enc
FROM (
SELECT DISTINCT a AS __eql_col_0, enc AS __eql_col_1,
eql_v3.ord_term(enc) AS __eql_ord_0
FROM t
) AS __eql_distinct
ORDER BY __eql_ord_0
The outer query is not DISTINCT, so its ORDER BY may reference a subquery
column it does not project — which is what keeps the ordering term out of
the client's result set.
Adding the term to the DISTINCT list cannot change which rows are
distinct: `ord_term(enc)` is a deterministic function of `enc`, which is
itself in the list, so it can only tie where the originals tie.
Every projected column goes through a synthetic `__eql_col_N` name and is
re-aliased to its original effective name on the way out, so a projection
naming the same column twice does not make the outer reference ambiguous.
ORDER BY, LIMIT, OFFSET, locks and CTEs stay on the wrapping query, so
LIMIT still applies to the ordered result. Ordinals are left alone — the
outer projection preserves column order and count.
Two shapes cannot be expressed this way and now report that rather than
failing as a bare PostgreSQL error: `DISTINCT ON`, whose ORDER BY must
begin with its own expressions, and a wildcard projection, which cannot
be named column for column.
This was found by the showcase, whose complex-nested-queries test had been
failing on exactly this shape.
Integration tests assert the ordering itself, in both protocols, plus that
the ordering term does not leak into the result and that aliases survive.
Docs follow-up for the website and encrypt-query-language: CIP-3677.
Stable-Commit-Id: q-4e7j1a6d5y99a8carmvc04wh15
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (2)
packages/eql-mapper/src/transformation_rules/rewrite_eql_distinct_order_by.rs (1)
259-287: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win
zipsilently truncates if a prior rule changed the projection length.
original_select.projectionand the already-rewritteninner_select.projectionare assumed to be positionally identical. If any earlier rule ever adds or removes a projection item (the group-by rewrite rewrites items in place today, but that is an implicit invariant), the zip drops the trailing columns fromouter_projectionand the client silently loses columns. A length assertion turning this into an explicitEqlMapperError::Transformis cheap insurance.🛡️ Proposed guard
+ if original_select.projection.len() != inner_select.projection.len() { + return Err(EqlMapperError::Transform( + "SELECT DISTINCT projection changed length during rewriting; cannot wrap in a \ + subquery for ORDER BY on an encrypted column" + .to_string(), + )); + } + let mut outer_projection = Vec::with_capacity(inner_select.projection.len());🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/eql-mapper/src/transformation_rules/rewrite_eql_distinct_order_by.rs` around lines 259 - 287, Validate that original_select.projection and inner_select.projection have equal lengths before the zip loop, returning an EqlMapperError::Transform when they differ. Keep the existing positional rewrite unchanged once the lengths match, preventing zip from silently dropping trailing projection items.packages/eql-mapper/src/lib.rs (1)
2601-2655: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winConsider covering
LIMIT/OFFSETretention and the unnamed-projection case.The rule keeps
limit_clauseon the wrapping query and falls back to"?column?"whenderive_effective_aliasreturnsNone(rewrite_eql_distinct_order_by.rslines 284-285, 318-329). Neither branch is exercised here; the integration suite only coversLIMIT. Two extra table rows (e.g.... ORDER BY salary LIMIT 2and a projection likeSELECT DISTINCT id + 1, salary ...) would pin both.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/eql-mapper/src/lib.rs` around lines 2601 - 2655, Extend the table-driven tests around the existing DISTINCT ORDER BY cases to cover both untested branches: add a query with ORDER BY salary and LIMIT 2, asserting the rewritten query retains LIMIT 2 on the outer wrapper, and add a query with an unnamed projection such as id + 1, asserting the output uses the ?column? fallback alias from derive_effective_alias. Keep the expected SQL normalized consistently with the existing assertions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@packages/eql-mapper/src/transformation_rules/rewrite_eql_distinct_order_by.rs`:
- Around line 239-254: Prevent the rewrite path from returning Ok(false) after
mutating target.body. Validate the target shape and confirm target.body is a
SetExpr::Select before replacing it, or restore the original body on any
subsequent rejection; update the related branches around inner_select and
plan_order_sources so unexpected shapes return a proper error or leave the
statement unchanged.
- Around line 289-313: Update the synthetic alias generation and references used
by the rewrite rule, including the symbols creating __eql_col_* and __eql_ord_*
identifiers, so generated names cannot collide with literal user columns. Apply
the same reserved or unambiguous naming consistently to inner projection aliases
and outer ORDER BY identifiers, while preserving the existing mapping between
hoisted terms and their aliases.
---
Nitpick comments:
In `@packages/eql-mapper/src/lib.rs`:
- Around line 2601-2655: Extend the table-driven tests around the existing
DISTINCT ORDER BY cases to cover both untested branches: add a query with ORDER
BY salary and LIMIT 2, asserting the rewritten query retains LIMIT 2 on the
outer wrapper, and add a query with an unnamed projection such as id + 1,
asserting the output uses the ?column? fallback alias from
derive_effective_alias. Keep the expected SQL normalized consistently with the
existing assertions.
In
`@packages/eql-mapper/src/transformation_rules/rewrite_eql_distinct_order_by.rs`:
- Around line 259-287: Validate that original_select.projection and
inner_select.projection have equal lengths before the zip loop, returning an
EqlMapperError::Transform when they differ. Keep the existing positional rewrite
unchanged once the lengths match, preventing zip from silently dropping trailing
projection items.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: d611b051-533c-4900-a983-28a092b9d182
📒 Files selected for processing (7)
CHANGELOG.mdpackages/cipherstash-proxy-integration/src/select/distinct_order_by.rspackages/cipherstash-proxy-integration/src/select/mod.rspackages/eql-mapper/src/lib.rspackages/eql-mapper/src/transformation_rules/mod.rspackages/eql-mapper/src/transformation_rules/rewrite_eql_distinct_order_by.rspackages/eql-mapper/src/type_checked_statement.rs
🚧 Files skipped from review as they are similar to previous changes (3)
- packages/eql-mapper/src/transformation_rules/mod.rs
- CHANGELOG.md
- packages/eql-mapper/src/type_checked_statement.rs
| let Some(target) = target_node.downcast_mut::<Query>() else { | ||
| return Ok(false); | ||
| }; | ||
|
|
||
| // Move the rewritten body out; it becomes the subquery. | ||
| let body = mem::replace( | ||
| &mut target.body, | ||
| Box::new(SetExpr::Values(Values { | ||
| explicit_row: false, | ||
| rows: vec![], | ||
| })), | ||
| ); | ||
|
|
||
| let SetExpr::Select(mut inner_select) = *body else { | ||
| return Ok(false); | ||
| }; |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Silent Ok(false) on shapes applies_to/plan_order_sources already accepted.
Lines 240, 253 and 274 return Ok(false) after the rule has committed to rewriting — line 253 in particular happens after target.body was swapped out for an empty Values, leaving the query mutated into VALUES () with the original body dropped. These branches should be unreachable given applies_to (checked on the original) and the wildcard rejection, but if they ever fire the failure mode is a corrupted statement rather than an error.
🛡️ Proposed fix for the body-swap path
- let SetExpr::Select(mut inner_select) = *body else {
- return Ok(false);
- };
+ let SetExpr::Select(mut inner_select) = *body else {
+ return Err(EqlMapperError::Transform(
+ "SELECT DISTINCT body was not a SELECT after rewriting".to_string(),
+ ));
+ };Also applies to: 269-275
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@packages/eql-mapper/src/transformation_rules/rewrite_eql_distinct_order_by.rs`
around lines 239 - 254, Prevent the rewrite path from returning Ok(false) after
mutating target.body. Validate the target shape and confirm target.body is a
SetExpr::Select before replacing it, or restore the original body on any
subsequent rejection; update the related branches around inner_select and
plan_order_sources so unexpected shapes return a proper error or leave the
statement unchanged.
| // Hoist each encrypted ordering term into the subquery, and repoint the | ||
| // outer ORDER BY at it. | ||
| if let Some(OrderBy { | ||
| kind: OrderByKind::Expressions(order_by), | ||
| .. | ||
| }) = &mut target.order_by | ||
| { | ||
| for (obe, source) in order_by.iter_mut().zip(sources.iter()) { | ||
| match source { | ||
| OrderSource::Hoisted(alias) => { | ||
| let term = | ||
| mem::replace(&mut obe.expr, Expr::Value(SqltkValue::Null.into())); | ||
| inner_select.projection.push(SelectItem::ExprWithAlias { | ||
| expr: term, | ||
| alias: alias.clone(), | ||
| }); | ||
| obe.expr = Expr::Identifier(alias.clone()); | ||
| } | ||
| OrderSource::Column(alias) => { | ||
| obe.expr = Expr::Identifier(alias.clone()); | ||
| } | ||
| OrderSource::Ordinal => {} | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
rg -n '__eql' packages/eql-mapper/src packages/cipherstash-proxy/src | head -40Repository: cipherstash/proxy
Length of output: 3474
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== candidate files =="
git ls-files | rg '(\.rs$|manifest|Cargo\.toml)$' | rg 'packages/eql-mapper' | head -80
echo
echo "== relevant source outline =="
ast-grep outline packages/eql-mapper/src/transformation_rules/rewrite_eql_distinct_order_by.rs --view condensed || true
echo
echo "== relevant source section =="
sed -n '1,260p' packages/eql-mapper/src/transformation_rules/rewrite_eql_distinct_order_by.rs | nl -ba
echo
echo "== identifier-related usages in mapper =="
rg -n '\bIdent::new\b|order_by|OrderSource|rewrite.*distinct|distinct.*order|distinct' packages/eql-mapper/src -S | head -200Repository: cipherstash/proxy
Length of output: 4226
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== relevant source section =="
awk '{print NR": "$0}' packages/eql-mapper/src/transformation_rules/rewrite_eql_distinct_order_by.rs | sed -n '1,270p'
echo
echo "== imports and tests mentioning reserved names or collision handling =="
rg -n '__eql|Ident::new|reserved|shadow|collision|alias|OrderBySource|OrderSource|hoist' packages/eql-mapper/src/transformation_rules/rewrite_eql_distinct_order_by.rs packages/eql-mapper/src -S | head -240
echo
echo "== file size =="
wc -l packages/eql-mapper/src/transformation_rules/rewrite_eql_distinct_order_by.rsRepository: cipherstash/proxy
Length of output: 38983
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== sqltk Ident::new and parser docs/definition if local =="
rg -n 'struct Ident|fn new\(|impl Ident|Ident::new|unquoted|Identifier|quote' Cargo.lock Cargo.toml packages -S \
| grep -E 'sqltk|Ident::new|struct Ident|fn new\(|unquoted|Identifier' | head -200
echo
echo "== Cargo.toml deps for sqltk =="
rg -n 'sqltk|postgres|sqlx|sql_' Cargo.toml packages/eql-mapper/Cargo.toml -S || true
echo
echo "== local sqltk source in target/cache only, read-only =="
find $HOME/.cargo/registry/src $HOME/.cargo/git/checkouts 2>/dev/null -path '*sqltk*' \( -name '*.rs' -o -name 'Cargo.toml' \) | head -50Repository: cipherstash/proxy
Length of output: 24308
🌐 Web query:
sqltk Ident::new to_string quoted parser Ident unquoted
💡 Result:
In the context of the sqltk-parser crate, which is a derivative of sqlparser-rs, the Ident struct represents a SQL identifier and tracks its value and optional quote style [1][2][3]. When using Ident, the following concepts apply regarding construction and representation: Ident::new: This constructor creates a new identifier with the specified value, defaulting to no quotes (None for quote_style) and an empty span [1][4]. Quoting vs. Unquoted: The Ident struct includes a quote_style field (Option). An identifier is considered "quoted" if it was delimited by characters like double quotes in the source SQL, and "unquoted" (or plain) if no such delimiters were present [1][5]. Other methods such as with_quote exist to explicitly create identifiers with specific quote characters [1]. to_string: The Ident struct implements the to_string method (or via the Display trait), which reconstructs the identifier's string representation, typically including the original quote style if present [1][2]. This allows the parser to preserve identifier metadata during AST-to-SQL round trips [2]. While some external projects or custom implementations (such as dbt-xdbc) may use their own Ident structures with explicit Plain/Unquoted variants or custom constructor logic [6], the standard sqlparser/sqltk-parser AST Ident is defined by its value and an optional quote_style [1][7].
Citations:
- 1: https://docs.rs/sqlparser/latest/sqlparser/ast/struct.Ident.html
- 2: https://docs.rs/sqltk-parser/latest/sqltk_parser/
- 3: https://github.com/cipherstash/sqltk
- 4: https://docs.rs/datafusion/latest/datafusion/sql/sqlparser/ast/struct.Ident.html
- 5: https://docs.rs/sqlparser/latest/src/sqlparser/tokenizer.rs.html
- 6: https://github.com/dbt-labs/dbt-fusion/blob/ed90a8ea/crates/dbt-xdbc/src/sql/ident.rs
- 7: https://docs.rs/sqltk-parser/latest/sqltk_parser/ast/index.html
Avoid colliding with literal __eql_* columns.
This rule creates unquoted synthetic identifiers with Ident::new("...__eql_col_0") and Ident::new("...__eql_ord_0"). If the original query selects or orders by a column literally named __eql_col_0, the wrapping subquery’s ORDER BY __eql_col_0 can resolve to the user column instead of the reserved synthetic projected column. Use a reserved/unambiguous name for both outer ORDER BY references and inner aliases, or reject/use quoted variants that preserve the original effective alias.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@packages/eql-mapper/src/transformation_rules/rewrite_eql_distinct_order_by.rs`
around lines 289 - 313, Update the synthetic alias generation and references
used by the rewrite rule, including the symbols creating __eql_col_* and
__eql_ord_* identifiers, so generated names cannot collide with literal user
columns. Apply the same reserved or unambiguous naming consistently to inner
projection aliases and outer ORDER BY identifiers, while preserving the existing
mapping between hoisted terms and their aliases.
`SELECT DISTINCT enc` compared whole jsonb payloads — including `c`, the
ciphertext, which is randomised per encryption. Two rows holding the same
plaintext never collapsed, so DISTINCT quietly returned duplicates:
inserting `apple` twice and selecting DISTINCT returned `apple` twice.
Deduplication is equality, so it is now keyed on the same `eq_term` an `=`
comparison uses (`ord_term` for a domain that stores no `hm`):
SELECT DISTINCT enc FROM t
-- becomes
SELECT DISTINCT ON (eql_v3.eq_term(enc)) enc FROM t
DISTINCT ON is what lets the key differ from the projection: the column is
still returned in full, one row per distinct plaintext, while the grouping
happens on the term. A plaintext column in the same projection keys on
itself. Which row of each group is returned is unspecified, which is fine —
every row in a group is an encryption of the same plaintext.
This composes with the subquery wrapping added for ORDER BY. PostgreSQL
requires an ORDER BY to *begin with* the DISTINCT ON expressions, which an
arbitrary ORDER BY will not, so the wrapping rule now also fires when the
projection dedupes an encrypted column — not only when the ORDER BY names
one. The DISTINCT ON travels into the subquery, where the absence of an
ORDER BY leaves PostgreSQL free to pick any row per group, and the outer
query keeps the ORDER BY with no DISTINCT to constrain it.
Also constrains DISTINCT in the unifier: every expression `DISTINCT` or
`DISTINCT ON (…)` dedupes on must implement `Eq`, so a storage-only column
is a capability error instead of a silent no-op.
Enforcing that surfaced a hole in `unify_with_type_var`. Bounds were only
checked when the type variable was *already* substituted, so a bound was
enforced on the second type unified with a variable and never on the
first. A constraint applied to a single expression (`SELECT DISTINCT enc`)
was therefore never checked at all, while the same constraint applied to a
pair (`a IS DISTINCT FROM b`) was. A concrete type must now prove the
bound before it may stand in for the variable.
That fix unmasked two test fixtures that had been relying on the hole:
`select_with_multiple_placeholder_and_wildcard_expansion` compared with
`=` on columns declaring no capabilities, and
`modify_aggregate_when_eql_column_affected_by_group_by_of_other_column`
used `min`/`max` on a column declaring no ordering. Both schemas now
declare what they use.
Integration tests assert that equal plaintexts collapse, with and without
an ORDER BY.
Docs follow-up: CIP-3677.
Stable-Commit-Id: q-0926azhtqpfqe1md3pe37fjw18
Five correctness findings from coderdan's review of #432. Each was reproduced before being fixed; the reproductions are kept as tests. P1 — a param bound as both a stored value and a query operand UPDATE t SET enc = $1 WHERE enc = $1 failed with `value for domain eql_v3_text_search violates check constraint`. The two occurrences need different payloads — the stored one carries the ciphertext, the query one only search terms — but the role was keyed on the *input* param, so one occurrence being a query operand marked both, stripping the ciphertext from the value being stored. The role now comes from the rewritten statement, per occurrence. The rewrite has already cast each operand to the domain its position requires, so a cast to an `eql_v3.query_*` twin is the statement itself saying "this one is a query operand". `RenumberParams` records that while it walks the output, and `param_plan` keys on the output param. JSON selectors take no cast at all and are still query operands, so they stay special-cased. P2 — JSON selector params when the client declares its own types A selector is passed to the rewritten function as bare text, but was declared `jsonb` like every other encrypted operand, so PostgreSQL could not find the function: ERROR: function eql_v3.jsonb_path_exists(eql_v3_json_search, jsonb) does not exist Invisible unless the client sends its own Parse OIDs — which is exactly what pgx does in `cache_describe` mode, and why the Go tests failed while every Rust test passed. `JsonAccessor`/`JsonPath` outputs are now declared TEXT. P2 — binary-format text operands on encrypted JSON fields A TEXT/VARCHAR operand in binary format went straight to `serde_json::Value::from_sql` and was rejected, though the same value in text format was accepted. Textual types are now read as a string first and given the text format's treatment, so `Alice` behaves like `"Alice"`. The `JsonOrd` arm matched any incoming type and had the same hole. This one could not be reproduced end to end: tokio-postgres honours the ParameterDescription the proxy sends and refuses to serialise a &str for a jsonb param, so the bad bytes never reach the wire. Covered by unit tests on the decoder instead. P2 — aggregates over a grouped encrypted column `SELECT MIN(enc) FROM t GROUP BY enc` produced `grouped_value(eql_v3.min(enc))`, an aggregate inside an aggregate. `MIN(enc)` resolves to the same EqlValue as `enc`, so it matched the lifting predicate. Only a *direct* column reference is lifted now; an aggregate already yields one value per group. P2 — SELECT * with GROUP BY on an encrypted column The wildcard still projected the bare column while the key became `eq_term(col)`, so PostgreSQL rejected it with "must appear in the GROUP BY clause". Now refused with an error naming the query shape, matching how `SELECT DISTINCT *` is already handled. Stable-Commit-Id: q-473y885g8td399qgh2kc02gcvy
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
packages/eql-mapper/src/renumber_params.rs (1)
80-100: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick winPin the query-operand cast shape for
wrapped_placeholder.
wrapped_placeholderis the only decision point that turns aneql_v3.query_*cast intoquery_operands; it silently returnsNonewhenever the cast body is not a single placeholder. Add a same-crate test that asserts helper-built query operands emit<placeholder>::JSONB::eql_v3.query_*, so a future change tocast_expr_to_v3_domainfails loudly instead of making the mapped payload choose the stored shape by accident.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/eql-mapper/src/renumber_params.rs` around lines 80 - 100, Add a same-crate test covering the query-operand construction path through wrapped_placeholder and cast_expr_to_v3_domain. Build query operands with the existing helper, then assert the emitted operand cast has the exact <placeholder>::JSONB::eql_v3.query_* shape, ensuring the placeholder remains the cast body expected by wrapped_placeholder.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@packages/eql-mapper/src/renumber_params.rs`:
- Around line 80-100: Add a same-crate test covering the query-operand
construction path through wrapped_placeholder and cast_expr_to_v3_domain. Build
query operands with the existing helper, then assert the emitted operand cast
has the exact <placeholder>::JSONB::eql_v3.query_* shape, ensuring the
placeholder remains the cast body expected by wrapped_placeholder.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 3b44d0db-00aa-49c5-848b-c900b0a5681e
📒 Files selected for processing (17)
CHANGELOG.mdpackages/cipherstash-proxy-integration/src/select/distinct_order_by.rspackages/cipherstash-proxy-integration/src/select/jsonb_selector_param_types.rspackages/cipherstash-proxy-integration/src/select/mod.rspackages/cipherstash-proxy-integration/src/update/mod.rspackages/cipherstash-proxy-integration/src/update/update_with_reused_param.rspackages/cipherstash-proxy/src/postgresql/data/from_sql.rspackages/cipherstash-proxy/src/postgresql/messages/parse.rspackages/eql-mapper/src/inference/infer_type_impls/select.rspackages/eql-mapper/src/inference/unifier/mod.rspackages/eql-mapper/src/lib.rspackages/eql-mapper/src/renumber_params.rspackages/eql-mapper/src/transformation_rules/mod.rspackages/eql-mapper/src/transformation_rules/rewrite_eql_distinct.rspackages/eql-mapper/src/transformation_rules/rewrite_eql_distinct_order_by.rspackages/eql-mapper/src/transformation_rules/rewrite_eql_group_by.rspackages/eql-mapper/src/type_checked_statement.rs
🚧 Files skipped from review as they are similar to previous changes (9)
- packages/cipherstash-proxy-integration/src/select/mod.rs
- packages/eql-mapper/src/transformation_rules/rewrite_eql_group_by.rs
- CHANGELOG.md
- packages/cipherstash-proxy-integration/src/select/distinct_order_by.rs
- packages/eql-mapper/src/type_checked_statement.rs
- packages/eql-mapper/src/transformation_rules/rewrite_eql_distinct_order_by.rs
- packages/cipherstash-proxy/src/postgresql/messages/parse.rs
- packages/cipherstash-proxy/src/postgresql/data/from_sql.rs
- packages/eql-mapper/src/lib.rs
| /// The encrypted columns a plain `DISTINCT` would deduplicate on, positionally. | ||
| fn distinct_eql_values(&self, select: &'ast Select) -> Vec<Option<EqlValue>> { | ||
| if !matches!(select.distinct, Some(Distinct::Distinct)) { | ||
| return vec![]; | ||
| } | ||
|
|
||
| select | ||
| .projection | ||
| .iter() | ||
| .map(|item| Self::select_item_expr(item).and_then(|expr| self.eql_value_of(expr))) | ||
| .collect() | ||
| } | ||
| } | ||
|
|
||
| impl<'ast> TransformationRule<'ast> for RewriteEqlDistinct<'ast> { | ||
| fn apply<N: Visitable>( | ||
| &mut self, | ||
| node_path: &NodePath<'ast>, | ||
| target_node: &mut N, | ||
| ) -> Result<bool, EqlMapperError> { | ||
| // Read the encrypted columns from the ORIGINAL select — `node_types` is | ||
| // keyed by it, and the target's children are already rewritten. | ||
| let Some((original,)) = node_path.last_1_as::<Select>() else { | ||
| return Ok(false); | ||
| }; | ||
|
|
||
| let distinct = self.distinct_eql_values(original); | ||
| if distinct.iter().all(Option::is_none) { | ||
| return Ok(false); | ||
| } | ||
|
|
||
| // A wildcard hides the columns being deduplicated on, so the key list | ||
| // cannot be built and the encrypted columns would dedupe on ciphertext. | ||
| if original | ||
| .projection | ||
| .iter() | ||
| .any(|item| Self::select_item_expr(item).is_none()) | ||
| { | ||
| return Err(EqlMapperError::Transform( | ||
| "SELECT DISTINCT with a wildcard cannot deduplicate an encrypted column: list the \ | ||
| columns explicitly so each one can be keyed on its equality term" | ||
| .to_string(), | ||
| )); | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Files:"
git ls-files | rg 'packages/eql-mapper/(src/transformation_rules/rewrite_eql_distinct\.rs|src/inference/infer_type_impls/select\.rs|src/renumber_params\.rs|src/.)' | head -100
echo
echo "rewrite_eql_distinct outline:"
ast-grep outline packages/eql-mapper/src/transformation_rules/rewrite_eql_distinct.rs 2>/dev/null || true
echo
echo "relevant rewrite_eql_distinct lines 1-180:"
sed -n '1,180p' packages/eql-mapper/src/transformation_rules/rewrite_eql_distinct.rs
echo
echo "relevant select inference lines 1-100:"
sed -n '1,100p' packages/eql-mapper/src/inference/infer_type_impls/select.rsRepository: cipherstash/proxy
Length of output: 13047
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Locate SelectItem-related code:"
git ls-files | rg 'packages/eql-mapper/src/.*(select.*item|select_items)|packages/eql-mapper/src/transformation_rules/rewrite_eql_distinct\.rs|packages/eql-mapper/src/inference/infer_type_impls/select_items\.rs|packages/eql-mapper/src/inference/infer_type_impls/select_item\.rs'
echo
echo "select_item inference:"
sed -n '1,180p' packages/eql-mapper/src/inference/infer_type_impls/select_item.rs
echo
echo "select_items inference:"
sed -n '1,180p' packages/eql-mapper/src/inference/infer_type_impls/select_items.rs
echo
echo "Search for wildcard resolution/usages:"
rg -n "SelectItem::Wildcard|Wildcard|wildcard|expand|projection" packages/eql-mapper/src -S
echo
echo "Struct definitions:"
rg -n "struct Select|enum SelectItem|pub enum SelectItem|Unresolved|Star" packages/eql-mapper/src -SRepository: cipherstash/proxy
Length of output: 34625
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Type env projection access:"
sed -n '200,280p' packages/eql-mapper/src/inference/unifier/type_env.rs
sed -n '1,220p' packages/eql-mapper/src/inference/unifier/types.rs | sed -n '/impl Projection/,/pub fn flatten/p'
sed -n '130,200p' packages/eql-mapper/src/inference/unifier/types.rs
echo
echo "Scope tracker wildcard implementation:"
sed -n '90,155p' packages/eql-mapper/src/scope_tracker.rs
sed -n '190,230p' packages/eql-mapper/src/inference/mod.rs
echo
echo "Relevant existing tests:"
sed -n '715,780p' packages/eql-mapper/src/lib.rs
sed -n '795,825p' packages/eql-mapper/src/lib.rs
sed -n '1550,1565p' packages/eql-mapper/src/lib.rs
sed -n '2817,2895p' packages/eql-mapper/src/lib.rs
sed -n '2735,2763p' packages/eql-mapper/src/lib.rsRepository: cipherstash/proxy
Length of output: 16842
Resolve wildcard projection columns before skipping them. Both the DISTINCT rewrite and the DISTINCT type bound skip SelectItem::Wildcard, so SELECT DISTINCT * FROM t bypasses the equality-term protection for encrypted columns hidden behind the wildcard. In the rewrite, a lone-wildcard projection makes distinct all None, returning Ok(false) before the wildcard rejection; in type inference, the wildcard item is never passed to unify_node_with_eq_bound. Check or expand the wildcard columns first, matching the explicit rejection already used for wildcards alongside named encrypted columns.
📍 Affects 2 files
packages/eql-mapper/src/transformation_rules/rewrite_eql_distinct.rs#L72-L115(this comment)packages/eql-mapper/src/inference/infer_type_impls/select.rs#L23-L34
Migrates CipherStash Proxy from EQL v2 to EQL v3.
The central change is how an encrypted column is declared. Under v2 every encrypted column had the same opaque
eql_v2_encryptedcomposite type, and its searchable capabilities lived somewhere else entirely — rows in aneql_v2_configurationtable, populated byeql_v2.add_search_configcalls. Under v3 the column type is the configuration: a typedjsonbdomain such aseql_v3_text_searchoreql_v3_integer_ordencodes both the scalar type and the searches the column supports.That single change ripples through the whole stack, which is why this PR is large: the proxy stops reading a config table and infers encrypt config from the schema, the type checker gains a domain identity per column, and every rewrite rule has to emit the v3 functional term form rather than v2's bare operators.
Breaking changes
eql_v2_encryptedcolumn is served as a plaintext passthrough column with a loud ops-visible warning: Proxy performs no encryption on writes or decryption on reads for it.eql_v2.add_search_configandeql_v2.add_encrypted_constraintare gone, as is theeql_v2_configurationtable. Declare the domain type instead.Dependencies
cipherstash-client/cipherstash-configeql-bindingsCS_EQL_VERSION)Upgrading to
cipherstash-client0.42.0 also lets the vendoredstack-authcopy go — it was the CIP-3159 hotfix patch and is no longer referenced. That deletion accounts for most of the ~12k removed lines; the substantive diff is far smaller than the headline numbers suggest.New capabilities
WHERE col -> 'field' = 'value', in both the simple and extended protocols, and in the->>andjsonb_path_query_first(col, path) = valuespellings, with<>as the negation. The field and the value are fused into a single encrypted value-selector needle and matched by containment, so a query never reveals the field and the value separately. Exact and case-sensitive; the value must be a JSON scalar.@@on text columns whose domain carries a bloom-filter term.ArrayIndexMode::ALL).Fixed along the way
LIKE/ILIKEon an encrypted column is now gated by the column's token-match capability. It previously bypassed capability checking and was silently accepted on columns that cannot do fuzzy match.ORDER BYon an encrypted column now orders by its ordering term, andGROUP BYgroups by its equality term, projecting grouped columns througheql_v3.grouped_value. Both previously fell through to rawjsonbsemantics over the ciphertext — which is non-deterministic, so the results were wrong rather than merely unsorted.Reviewer's map
The commits are ordered so the story reads top to bottom; reviewing by commit is likely easier than by file.
packages/eql-mapper(+3672/-312)inference/unifier/types.rsforDomainIdentity/TokenType;transformation_rules/for the rewrite pipeline —cast_full_payload_operands,rewrite_eql_comparison_ops,rewrite_eql_match_ops,rewrite_eql_order_by,rewrite_eql_group_by,rewrite_json_value_selector_eqare new, replacingcast_literals_as_encrypted/cast_params_as_encryptedpackages/cipherstash-proxy(+1684/-804)proxy/encrypt_config/from_domain.rs(config inferred from the domain typname),proxy/schema/eql_domains.rs, and the v3 encrypt/decrypt paths inproxy/zerokms/andpostgresql/messages/data_row.rsdocs/docs/plans/2026-07-20-eql-v3-type-checker-handoff.mdis the design record; ADRs cover the domain-identity and rewrite-pipeline decisionstests/,packages/showcaseTesting
tests/sql/schema.sql, the benchmark schema and the showcase schema are all v3 now, and the integration suite is v3-shaped.Ten backwards-compatibility tests in
packages/cipherstash-proxy-integration/src/eql_regression.rsare#[ignore]d: they pin captured v2 ciphertext and::eql_v2_encryptedcasts, neither of which exists under v3. They need fixtures regenerated from a v3 baseline before they can be re-enabled.Acknowledgment
By submitting this pull request, I confirm that CipherStash can use, modify, copy, and redistribute this contribution, under the terms of CipherStash's choice.
Summary by CodeRabbit
@@, richer JSON equality/inequality and containment (including<>,->/->>,jsonb_path_query_first), plusSELECT DISTINCT ... ORDER BYsupport for encrypted sorts.LIKE/ILIKE; improved encrypted JSON selector typing and error clarity.GROUP BYhandling (including rejectingSELECT *with encryptedGROUP BY).