From 2d70a17c30e5ec35c14ada85832a249be6901c4b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Sun, 12 Jul 2026 17:29:11 +0300 Subject: [PATCH 01/11] Add architecture evaluation with REST-VKG comparison and improvement plan Evaluates the composed-operations premise, the formal semantics, the JSON DSL, the Python codebase and the rdflib data model; diffs the operation set and execution semantics against the Java/XML sibling (REST-VKG) and lays out a three-tier plan: spec unification, two-way operation parity, and Python code health. Co-Authored-By: Claude Fable 5 --- architecture-evaluation.md | 390 +++++++++++++++++++++++++++++++++++++ 1 file changed, 390 insertions(+) create mode 100644 architecture-evaluation.md diff --git a/architecture-evaluation.md b/architecture-evaluation.md new file mode 100644 index 0000000..0aa0092 --- /dev/null +++ b/architecture-evaluation.md @@ -0,0 +1,390 @@ +# Web-Algebra Architecture Evaluation + +*Scope: Web-Algebra (this repo, v1.5.0, Python/rdflib) evaluated on the conceptual and +code level, and compared against its Java/XML sibling `../REST-VKG` (webalgebra module +v1.12.0-SNAPSHOT, Jena/Saxon). Written 2026-07-12.* + +## Verdict in brief + +The premise — an LLM compiling a whole Linked Data workflow into a declarative, +composable operation document instead of issuing step-by-step tool calls — is sound, +and was ahead of its time. The weak points are not the idea but its contracts: +`formal-semantics.md` is a type *catalog* rather than a semantics, the two sibling +implementations have quietly forked (both in operations and in execution model), and +the Python codebase has a handful of structural debts (mutable shared state, a +god-class `Operation`, a three-way execution surface) that are cheap to fix now and +expensive later. The rdflib-based data model is the right choice and its JSON-LD +boundary discipline is genuinely well designed. Part II proposes a three-tier plan: +unify the spec, sync the operation set both ways, then pay down the Python debts. + +--- + +# Part I — Evaluation + +## 1. The premise: composed operations as LLM-emitted bytecode + +**Sound? Yes.** The core bet (README: agents "compile entire workflows into optimized +JSON 'bytecode' that executes atomically") separates *planning* from *execution*: + +- One LLM turn produces the whole program; execution is then deterministic, cheap, and + free of per-step model round-trips. For N-row ForEach workloads this is the + difference between 1 LLM call and O(N) calls. +- The artifact is inspectable and replayable — a JSON document you can review, diff, + version, and re-run, which per-step tool calling can never give you. +- The value domain is RDF-native (`URIRef`/`Literal`/`Graph`/`Result`), so data flows + between operations *as RDF*, not as strings squeezed through generic tool-call JSON. + This is the part most "agent + SPARQL endpoint" designs get wrong. + +**Innovative? Yes, with context.** The now-mainstream pattern of "have the model emit +a program over the tool surface instead of chaining tool calls" (CodeAct-style agents, +code-mode MCP execution) arrived after this design. Web-Algebra's distinctive +contributions beyond that pattern are: + +1. **A domain algebra, not a general-purpose language.** The constrained operation set + is what makes documents verifiable, replayable, and safe — an LLM emitting Python + can do anything; an LLM emitting Web-Algebra can only do what the algebra permits. +2. **XSLT lineage for control flow** — `ForEach`/`Value`/`Current`/`Variable` map to + `xsl:for-each`/`xsl:value-of`/`current()`/`xsl:variable`, a proven declarative + iteration model rather than an invented one. +3. **JSON-LD as both code and data carrier** (§3) — the same document embeds RDF + payloads and operations, discriminated by `@`-keys. + +**Where the premise is under-delivered** (these are contract gaps, not design flaws): + +- **No validation phase.** The "bytecode" is never typechecked before running. A type + error in operation 7 surfaces after operations 1–6 have already POSTed/PUT to live + servers; there is no dry-run, and HTTP effects are not transactional. A compiler + metaphor implies a checker; the checker is missing. +- **Quotation is implicit.** `process_json` evaluates nested `@op` eagerly, + depth-first — except `ForEach`, which receives its `operation` argument raw and + evaluates it once per row (`for_each.py:53`). That makes `ForEach` a special form in + the Lisp sense, but nothing in the spec or the schema declares which arguments are + evaluated and which are quoted. Any future conditional/short-circuit op will hit the + same wall. +- **Effects are typed as pure functions.** `POST : URI × Graph → Result` reads like + arithmetic; nothing distinguishes effectful operations, their ordering guarantees + (currently: top-level JSON array order + ForEach row order), or their failure + behavior. + +## 2. The formal definitions (`formal-semantics.md`) + +**What it does well.** Every operation gets a dual signature (abstract + +concrete Python), the abstract type language is compact (`Term = URI + Literal + +BNode`, `Maybe`, `Sequence α`), and the Strict Type Checking property is stated +explicitly. As a catalog it is mostly accurate against the code. + +**What it is not: a semantics.** There are no evaluation rules. The document never +defines: + +- what nesting means (evaluation order, eager vs quoted arguments); +- what a top-level JSON array means (sequencing? variable-stack accumulation? — + `operation.py:130-137` implements accumulation, the spec is silent); +- how ForEach context propagates and how `Value` resolves the name (`$var` stack vs + context lookup is stated informally at best); +- what happens on error (class, propagation, partial results). + +The type system also gives up exactly where the DSL is most interesting: +`Context = Any` (`formal-semantics.md:20`), `VariableStack = [Dict[String, Any]]`, +and the control-flow ops are typed `Any → Any`. The algebra's *data* operations are +precisely typed; its *composition* operations are untyped. + +Concrete defects, all cheap to fix: + +- `Variable : String × Any × VariableStack → ⊥` (`formal-semantics.md:69`) — `⊥` means + non-termination; the intended type is `Unit`. +- Filter's sequence case reads `Sequence α × Expression → α` (`formal-semantics.md:99`) + — should be `→ Sequence α`. +- `Concat` and `ExtractOntology` are implemented but absent from the catalog. +- `tests/SPEC_GAPS.md` tracks **30+ places where the spec underdetermines the + implementation** — datatype of `Str` results, `Substitute`'s variable syntax and + term-serialization rules, ForEach output shape for `None`/sequence inner results, + the entire error-semantics column, and the JSON arg-key names for ~20 operations. + The spec-driven test suite is the best evidence of the gap: dozens of tests are + `pytest.skip("UNCLEAR(spec)")`. + +**The deeper problem: the spec has forked.** REST-VKG carries its own +`docs/WEB-ALGEBRA.md` — 994 lines of *operational* semantics (execution model, +patterns, examples) versus this repo's 360-line type catalog. Neither references the +other; each has drifted toward its host implementation. For two projects that "should +be in sync in terms of the operations and their signatures", the single highest-value +move is one shared spec (Part II, Tier 1). + +## 3. The JSON DSL + +**Strengths.** + +- `@op`/`args` rides JSON-LD's established `@`-keyword convention, so one document + carries both operations and RDF data with unambiguous discrimination. +- The best design element in the codebase is the **quasi-quotation of JSON-LD + bodies**: a dict carrying `@context`/`@graph`/`@id`/`@type` is treated as data, but + `_resolve_jsonld` (`operation.py:144-175`) walks it and evaluates embedded `@op` + holes in place — e.g. an `@id` computed from a runtime binding — while deliberately + *not* parsing to a `Graph`, so the consuming operation can parse with its own base + IRI. The reasoning is documented in the code (`operation.py:102-119`) and it is + correct: central parsing would freeze unresolved holes into blank nodes. +- Flat, schema-describable JSON is arguably *easier* for an LLM to emit correctly than + free-form code, and trivially validatable — once a validator exists. + +**Weaknesses.** + +- **Verbosity.** Every URI-valued argument needs a nested + `{"@op": "URI", "args": {"input": ...}}` cast, every string interpolation a + `Concat`/`Value` tree. In `examples/united-kingdom-cities.json`, a two-step workflow + costs 104 lines and 14 operation nodes, roughly a third of which are casts and + variable plumbing. This is a *type-system choice* (§5) — plain JSON strings become + `xsd:string` literals (`operation.py:275-277`), so URIs must be cast explicitly — + but the cost lands on every document and every LLM emission. Sugar is available + without weakening the typing: `{"@id": "..."}` is already valid JSON-LD for "this is + a URI" and could be accepted anywhere a URI is expected. +- **No envelope.** Documents have no version, no namespace, no name — just a bare + array. The Java XML side has `xmlns="https://w3id.org/atomgraph/web-algebra"`; the + JSON side has nothing to dispatch or validate against. +- **The JSON-LD sniff** (`_JSONLD_KEYS`, `operation.py:17`) is a heuristic: any dict + containing `@type` is data. The code comment argues these keys are unambiguous, and + within RDF workflows that mostly holds, but it is a global reserved-word rule the + spec never states. +- **Arrays are overloaded**: top-level array = sequential program with shared variable + stack; array under `ForEach.operation` = per-row sequence where only the last + non-`None` result is kept (`for_each.py:84-97`); array elsewhere = plain argument + list. Three meanings, zero spec lines. +- **JSON arg keys are folklore.** The spec gives Python parameter names; the JSON + layer uses different keys (`select` vs `select_data`), confirmed only by fixtures + (SPEC_GAPS "JSON dispatch surface"). + +## 4. The Python codebase + +Ranked by how much they matter: + +1. **Mutable default arguments** — `execute_json(self, arguments, variable_stack: + list = [])` (`operation.py:56`), `process_json(..., context: dict = {}, + variable_stack: list = [])` (`operation.py:83-84`), and the class field + `context: Any = {}` (`operation.py:31`). Python evaluates these once; every call + that omits the argument shares one list/dict across the process. Today the + single-threaded interpreter mostly masks it; the day two documents run in one + process (the MCP server is exactly that), variables leak between executions. This + is the one outright *bug class* in the core. +2. **`Operation` is a god class.** Registry, the whole interpreter (`process_json` + + `_resolve_jsonld`), variable-stack management, and five type-conversion helpers all + live in the ABC every operation inherits (`operation.py`, 345 lines). The + interpreter is not an operation concern; it should be a separate module holding an + execution context — which is also the precondition for parallelism (§6). +3. **The `execute()` contract is violated by its own flagship op.** + `ForEach.execute()` raises `NotImplementedError` (`for_each.py:42-44`) because the + pure layer has no way to run an operation with context — i.e. the algebra's central + higher-order operation has no pure form, only a JSON form. `Bindings` and + `ldh-List` return `list[dict]`, many `ldh-*` ops return `Any`. The abstract + signature `execute(*args)` is variadic while every implementation is fixed-arity, + so type checkers verify nothing. +4. **pydantic is decorative.** `extra="allow"`, no field validation, hand-written + `inputSchema()` dicts that often omit `"type"` (`for_each.py:24-34`) and are never + used to validate anything. The DSL's missing validator (§1) could be generated from + real pydantic models; today neither exists. +5. **Three execution surfaces per operation** (`execute` / `execute_json` / + `mcp_run`) is a triple maintenance burden, and it shows: `ForEach.mcp_run` returns + the static string "ForEach operation completed" (`for_each.py:112-114`). 21 of 43 + ops are MCP-exposed; the boundary between "MCP tool" and "DSL-only" is undocumented. +6. **No exception taxonomy.** `TypeError` vs `ValueError` vs raw + `urllib.error.HTTPError` varies by op; SPEC_GAPS' error-semantics section exists + because callers cannot classify failures. +7. **Duplication.** Seven HTTP-backed ops repeat the same `LinkedDataClient` + construction in `model_post_init`; isinstance-check boilerplate opens every + `execute()`. +8. **Minor:** `_serialize_for_json_context` (`operation.py:177-188`) is dead code; no + retry on transient network failures (only 429 `Retry-After` is honored). + +**What is genuinely good** — worth saying plainly, because it should be preserved +through any refactor: + +- The **JSON-LD → Graph boundary** is exactly right: `to_graph()` + (`operation.py:215-242`) is the single parse point, each op applies its own base + IRI, and the design rationale is written down where it matters. +- `client.py` handles 308 redirects and 429 backoff correctly, with client-cert auth + cleanly isolated in settings. +- The registry + auto-discovery pattern is clean and scales. +- The **spec-driven test discipline** (203 tests authored from the spec alone, + `SPEC_GAPS.md` recording every ambiguity with a proposed spec edit) is a practice + most projects never reach. The gaps it found are the spec's problem, not the suite's. + +## 5. The rdflib data model + +Right choice, well executed at the boundaries. `Node`/`Graph`/`Result` as the value +domain keeps datatype and language-tag fidelity end-to-end; `JSONResult` +(`json_result.py`) is a clean adapter between rdflib results and the SPARQL JSON +wire format; `POST`/`PUT` returning a `Result` of `{status, url}` bindings is a nice +touch — HTTP responses become queryable data, and it happens to match REST-VKG's +`ResultSet` shape exactly. + +Two observations: + +- **Leaf typing is principled and expensive.** Every plain JSON string becomes + `Literal(..., datatype=xsd:string)` — never a URI (`operation.py:275-277`). That + strictness is defensible RDF hygiene (Java's `String`-typed leaves lose + datatype/lang information, see §6) and it is what forces the `URI` cast operation + and much of the DSL's verbosity. Keep the typing; add sugar at the JSON boundary. +- The stated convention "execute() is pure rdflib" holds for the data-plane ops but + not the control plane (§4.3). Either the convention gets a carve-out for + interpreter-level forms (ForEach, Execute, Variable, Value, Current, Filter, + Bindings) — which is honest, they are special forms, not term functions — or those + need pure formulations. The spec should say which. + +## 6. Python ↔ Java: parity and divergence + +### Operation parity (registry names) + +| | Operations | +|---|---| +| **Shared core (20)** | GET, POST, PUT, SELECT, DESCRIBE, CONSTRUCT, Merge, ForEach, Value, Str, Concat, Replace, EncodeForURI, ResolveURI, Substitute, Variable, Current, Execute, STRUUID/StrUUID, SPARQLString | +| **Java-only (1)** | Iterate — stateful pagination: `param` initialization, `next-iteration` passing, `break` conditions (`IterateOperation.java`, 244 lines) | +| **Python-only, generic (9)** | PATCH, Values, URI, Filter, Bindings, ExtractClasses, ExtractDatatypeProperties, ExtractObjectProperties, ExtractOntology | +| **Python-only, product-specific (14)** | the `ldh-*` LinkedDataHub operations | + +(Java's GRDDL is superseded by the client-side response filter and excluded.) + +### Three deep divergences — and per-op verdicts + +**a. ForEach: map → List (Python) vs parallel map-merge → Model (Java).** +Python's ForEach returns the list of per-row results (`for_each.py:79-110`); Java's +requires the select to yield a `ResultSet`, requires the inner operation to return a +`Model`, executes rows via `parallelStream()`, and merges into one `Model` +(`ForEachOperation.java:61-100`). **Verdict: Python's shape is the better algebra; +Java's execution model is the better runtime.** `Sequence α × (α → β) → Sequence β` +is more general — Java's fused map-merge cannot express "PUT each row's document and +give me the statuses" (the UK-cities example) without contortion, and its two +`instanceof` gates are exactly the kind of restriction a spec should not bake in. +Java's merge is `Merge(ForEach(...))` — an explicit composition Python already has. +Conversely, Java's per-row **immutable context clone enabling parallel iteration** is +strictly better than Python's shared mutable stack. Sync direction: spec ForEach as +sequence-returning with the map-merge documented as a fused specialization Java may +keep; Python adopts context isolation (and then parallelism) from Java. + +**b. Variables/context: mutable stack (Python) vs immutable `ExecutionContext` +(Java).** Java's context is a persistent structure — `withVariable`/`withBinding` +return new instances (`ExecutionContext.java:81-98`), with a progress emitter riding +along (`start:`/`complete:`/`error:` events). **Verdict: Java wins outright.** This is +thread safety, ForEach-row isolation, and observability in one move, and it is the +enabler for fixing Python issues §4.1 and §4.2 in a way that converges the two +codebases instead of diverging them further. + +**c. Leaf typing: RDF terms (Python) vs Strings (Java).** Java string ops return +`String` and `Value` stringifies RDF nodes; Python returns typed `Literal`s and keeps +`URIRef`/`Literal` distinct end-to-end. **Verdict: Python wins.** A `String`-typed +data plane silently drops datatypes and language tags — the exact failure RDF systems +exist to avoid. Long-term, Java should move its operation returns toward +`RDFNode`-typed values; the shared spec should define signatures in abstract RDF terms +(as `formal-semantics.md` already does) either way. + +### Maturity gaps (Java ahead, no controversy) + +- **Iterate** — no Python equivalent for paginated APIs; the biggest functional gap. +- **Parallel ForEach** — blocked in Python only by the mutable context. +- **Progress events** — Python has `logging.info` only; no structured lifecycle. +- **SPARQLString hardening** — Java takes question + endpoint, injects the service's + AGENTS.md plus date/timezone into the prompt, and retries 3× with exponential + backoff; Python takes a bare question with no retries. +- **Hybrid SELECT** — Java's SELECT accepts a remote endpoint *or* a local graph; + Python is endpoint-only, which blocks querying intermediate in-memory results. +- Deployment/observability (Docker, health endpoints, timing metrics) — product-level + rather than algebra-level, but worth noting. + +--- + +# Part II — Implementation plan + +Three tiers, independently executable, in value order. Decisions already made: +divergences are resolved per-op as recommended above; parity scope is the generic +core in both directions (`ldh-*` stays Python-only, declared as a product extension); +all three tiers are in scope. + +## Tier 1 — One spec, made whole (highest value, zero code risk) + +1. **Unify the fork.** Merge this repo's `formal-semantics.md` (type catalog) and + REST-VKG's `docs/WEB-ALGEBRA.md` (operational semantics) into a single canonical + spec shared by both repos (one home, the other references it — natural candidate: + a spec file under the `w3id.org/atomgraph/web-algebra` namespace both already + implicitly claim). Structure: type system → **evaluation rules** → operation + catalog → error semantics → serializations (JSON and XML as two concrete syntaxes + of one abstract syntax). +2. **Write the missing evaluation rules** (the §2 list): eager depth-first argument + evaluation; *quoted operands declared per-op* (ForEach.operation, Execute's body); + top-level array sequencing incl. variable-stack accumulation; ForEach context + propagation and `Value` resolution order (`$var` stack lookup vs context binding); + the three meanings of arrays; effect annotation for GET/POST/PUT/PATCH/SELECT/ + CONSTRUCT/DESCRIBE and their ordering guarantees. +3. **Resolve `tests/SPEC_GAPS.md` item by item** — it already contains proposed edits + for nearly every entry; most are one-line decisions (Str result datatype, + EncodeForURI's RFC, Merge set-semantics, Bindings ordering, error classes, + JSON arg-key table). Fix the two catalog defects (`⊥` → `Unit`, + Filter → `Sequence α`) and add the two missing entries (Concat, ExtractOntology). +4. **Declare the extension model**: `ldh-*` as a named product-specific extension + namespace; Iterate added to the core catalog (from Java). + +*Verification:* every resolved item un-skips its `UNCLEAR(spec)` tests; +`uv run pytest -m 'not network and not sparql and not ldh'` green with strictly fewer +skips than today. + +## Tier 2 — Operation & signature sync (generic core, both directions) + +**Python gains:** + +| Item | Notes | +|---|---| +| `Iterate` | Port from `IterateOperation.java` (params, next-iteration, break condition). Spec first (Tier 1), then implement + spec-driven tests. | +| Hybrid `SELECT` | Accept a `Graph` argument alternative to `endpoint` (query local/intermediate results via rdflib). Mirrors `SelectOperation.java`. | +| `SPARQLString` parity | Add endpoint parameter and context injection; add bounded retry with backoff. | +| Parallel ForEach | After Tier 3.2 (immutable context). Row isolation semantics per unified spec. | + +**Java gains** (tracked here, implemented in REST-VKG): `PATCH`, `Values`, `Filter`, +`Bindings`, `URI`, and the four `Extract*` schema ops — signatures taken verbatim from +the unified catalog. + +**Harmonizations:** + +- ForEach per verdict §6a: spec is sequence-returning; Java either generalizes or its + map-merge is documented as a fused `Merge ∘ ForEach` specialization. +- Registry-name alignment: `STRUUID` vs `StrUUID` — recommend `STRUUID` (matches the + SPARQL function name, which is the naming rule the other ops already follow). +- POST/PUT `{status, url}` result shape: already aligned; codify it in the catalog. + +*Verification:* a parity table in the unified spec, asserted by a test in each repo +that diffs its registry against the spec catalog (Python: registry names vs a +committed list; the SPEC_GAPS re-verify note at `tests/SPEC_GAPS.md:23` already asks +for exactly this). + +## Tier 3 — Python code health + +Ordered so each step stands alone: + +1. **Kill mutable defaults.** `variable_stack: list = None` → `if None: []` (or + required-arg) in `execute_json`/`process_json`/all ops; `context` field default via + pydantic `default_factory`. Mechanical, high value. (`operation.py:31,56,83-84` + and every operation's `execute_json`.) +2. **Extract the interpreter.** Move `process_json`, `_resolve_jsonld`, and + variable-stack handling out of `Operation` into an `interpreter.py` with an + immutable `ExecutionContext` (variables + current binding + progress callback) — + deliberately the same shape as `ExecutionContext.java`, converging the two + codebases. Operations keep `execute`/`execute_json`; `self.context` and the stack + parameter are replaced by the context object. +3. **Exception taxonomy.** `WebAlgebraError` base; `UnknownOperationError`, + `OperationTypeError`, `VariableNotFoundError`, `HttpOperationError(status, url)`. + Raise-sites updated; spec's error-semantics section (Tier 1.2) is the contract. +4. **Deduplicate the HTTP plumbing.** One client factory/mixin for the 7 ops that + construct `LinkedDataClient` in `model_post_init`; add transient-failure retry at + the client, not per-op. +5. **Honest contracts.** Delete `_serialize_for_json_context`; declare the + interpreter-level special forms (ForEach, Execute, Variable, Value, Current) as + such instead of pretending at a pure `execute()` (drop the `NotImplementedError` + stub per the unified spec); give ForEach a real `mcp_run` or remove its `MCPTool` + claim; replace hand-written `inputSchema()` dicts with schemas generated from + pydantic argument models — which also yields the missing pre-execution document + validator (§1) nearly for free. + +*Verification:* full suite `uv run pytest` after each step; step 2 additionally +verified by the integration fixtures in `tests/integration/` (behavior-preserving +refactor); step 5's validator gets new negative fixtures (malformed documents rejected +before any HTTP effect). + +## Suggested sequencing + +Tier 1 first (it unblocks skipped tests and is the sync keystone). Tier 3.1 anytime +(it is a bug fix). Tier 3.2 before the parallel-ForEach item of Tier 2. Everything +else is independent. From 601b1aab5ca088401c5efc7001ef271178068c39 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Sun, 12 Jul 2026 17:29:33 +0300 Subject: [PATCH 02/11] Make formal-semantics.md a real semantics and align implementation and tests Spec (formal-semantics.md rewrite): - Add evaluation semantics: eager depth-first evaluation, quoted operands (ForEach.operation, Execute.operation), sequence forms with fresh variable scopes, context rules, effect classes and ordering guarantees, and a normative error taxonomy. - Add the document model: optional envelope ({"@web-algebra": "1", "program": [...]}), the URI reference form ({"@id": ...}), normative JSON-LD discrimination keys, scalar coercion table, base-IRI rule. - Rebuild the catalog with per-operation JSON argument tables; add missing Concat and ExtractOntology entries; fix Unit vs bottom on Variable; pin every decision tracked in tests/SPEC_GAPS.md (Str datatype, EncodeForURI charset, Replace dialect, STRUUID format, Substitute and Values rules, Merge set-union, Bindings order, Filter positional signature, ForEach output shape, Value/Current context rules, Execute narrative, Extract* endpoint role). ldh-* moves to an informative appendix. Implementation alignment: - Fix mutable default arguments across the interpreter and all operations (variable_stack/context leak between documents in one process, e.g. the MCP server). - Sequence forms and ForEach iterations now push/pop a variable scope (replacing copy() semantics that leaked writes when an outer scope existed). - Implement the URI reference form and Operation.unwrap_document; unwrap the envelope in main.py. - Execute threads the variable environment (was silently dropped); Current errors without an iteration context; Value supports mapping context items; URI and Substitute reject BNodes (Substitute previously emitted invalid "_: N" syntax); Filter raises TypeError for non-integer expressions; boolean scalars coerce to xsd:boolean (bool-before-int); null forms raise TypeError; remove dead _serialize_for_json_context. Tests: 42 UNCLEAR(spec) skips un-skipped and authored from the new spec; new test_document.py covers the envelope, URI reference form, coercions and scoping. 221 passed, 8 skipped (was 145/50). Co-Authored-By: Claude Fable 5 --- formal-semantics.md | 771 ++++++++++++------ prompts/system.md | 12 +- src/web_algebra/main.py | 3 +- src/web_algebra/operation.py | 89 +- src/web_algebra/operations/bindings.py | 2 +- src/web_algebra/operations/current.py | 12 +- src/web_algebra/operations/execute.py | 16 +- src/web_algebra/operations/filter.py | 9 +- src/web_algebra/operations/for_each.py | 59 +- src/web_algebra/operations/linked_data/get.py | 2 +- .../operations/linked_data/patch.py | 2 +- .../operations/linked_data/post.py | 2 +- src/web_algebra/operations/linked_data/put.py | 2 +- .../operations/linkeddatahub/add_file.py | 2 +- .../linkeddatahub/add_generic_service.py | 2 +- .../linkeddatahub/add_result_set_chart.py | 2 +- .../operations/linkeddatahub/add_select.py | 2 +- .../operations/linkeddatahub/add_view.py | 2 +- .../linkeddatahub/content/add_object_block.py | 2 +- .../linkeddatahub/content/add_xhtml_block.py | 2 +- .../content/generate_class_containers.py | 2 +- .../content/generate_ontology_views.py | 2 +- .../linkeddatahub/content/generate_portal.py | 2 +- .../linkeddatahub/content/remove_block.py | 2 +- .../linkeddatahub/create_container.py | 2 +- .../operations/linkeddatahub/create_item.py | 2 +- .../operations/linkeddatahub/list.py | 2 +- src/web_algebra/operations/merge.py | 2 +- src/web_algebra/operations/resolve_uri.py | 2 +- .../operations/schema/extract_classes.py | 2 +- .../schema/extract_datatype_properties.py | 2 +- .../schema/extract_object_properties.py | 2 +- .../operations/schema/extract_ontology.py | 2 +- .../operations/sparql/construct.py | 2 +- src/web_algebra/operations/sparql/describe.py | 2 +- src/web_algebra/operations/sparql/select.py | 2 +- .../operations/sparql/substitute.py | 11 +- src/web_algebra/operations/sparql/values.py | 2 +- src/web_algebra/operations/sparql_string.py | 2 +- src/web_algebra/operations/str.py | 2 +- src/web_algebra/operations/string/concat.py | 2 +- .../operations/string/encode_for_uri.py | 2 +- src/web_algebra/operations/string/replace.py | 2 +- src/web_algebra/operations/struuid.py | 2 +- src/web_algebra/operations/uri.py | 7 +- src/web_algebra/operations/value.py | 12 +- src/web_algebra/operations/variable.py | 4 +- tests/SPEC_GAPS.md | 201 ++--- tests/unit/test_bindings.py | 46 +- tests/unit/test_current.py | 16 +- tests/unit/test_describe.py | 14 +- tests/unit/test_document.py | 138 ++++ tests/unit/test_encode_for_uri.py | 16 +- tests/unit/test_execute.py | 54 +- tests/unit/test_extract_classes.py | 13 +- .../unit/test_extract_datatype_properties.py | 13 +- tests/unit/test_extract_object_properties.py | 13 +- tests/unit/test_extract_ontology.py | 13 +- tests/unit/test_filter.py | 73 +- tests/unit/test_for_each.py | 127 ++- tests/unit/test_get.py | 9 +- tests/unit/test_merge.py | 27 +- tests/unit/test_patch.py | 14 +- tests/unit/test_post.py | 14 +- tests/unit/test_replace.py | 8 +- tests/unit/test_resolve_uri.py | 20 +- tests/unit/test_select.py | 25 +- tests/unit/test_sparql_string.py | 14 +- tests/unit/test_str.py | 40 +- tests/unit/test_struuid.py | 15 +- tests/unit/test_substitute.py | 73 +- tests/unit/test_uri.py | 25 +- tests/unit/test_value.py | 61 +- tests/unit/test_variable.py | 52 +- 74 files changed, 1525 insertions(+), 684 deletions(-) create mode 100644 tests/unit/test_document.py diff --git a/formal-semantics.md b/formal-semantics.md index 485ce09..30662db 100644 --- a/formal-semantics.md +++ b/formal-semantics.md @@ -1,360 +1,619 @@ # Web Algebra Formal Semantics -## Abstract Type System +This document is the normative specification of Web Algebra: its type system, its +JSON serialization, its evaluation semantics, and the catalog of core operations. +The Python implementation in this repository and the test suite under `tests/` +conform to it; where the two disagree, this document wins and the code is wrong. -### Primitive Types -``` -URI = Abstract URI reference -Literal = Abstract literal value with optional datatype and language -BNode = Abstract blank node identifier -Graph = Abstract RDF graph -Term = URI + Literal + BNode -``` +The LinkedDataHub-specific `ldh-*` operations are a product extension and are +described in the informative Appendix A. A sibling implementation (REST-VKG, +Java) serializes the same algebra as XML under the namespace +`https://w3id.org/atomgraph/web-algebra`; this document specifies the abstract +algebra and its JSON serialization. -### Collection Types -``` -Sequence α = [α] -- Ordered list of elements -Result = SPARQL SELECT result with variables and bindings -ResultRow = Single binding row from Result -VariableStack = [Dict[String, Any]] -- Stack of variable scopes -Context = Any -- Current execution context (varies by operation) -``` +## 1. Type System + +### 1.1 Abstract types -### Operation Types ``` -Operation = Abstract operation that can be executed -Expression = Operation + Literal + Integer -- Expressions for filtering +URI = URI reference +Literal = literal value with optional datatype IRI or language tag +BNode = blank node +Term = URI + Literal + BNode +Graph = RDF graph (set of triples) +Result = SPARQL SELECT result: a variable list and an ordered sequence + of Bindings +Binding = one solution row: a partial mapping from variable names to Terms +Sequence α = ordered list of values of type α +Position = integer ≥ 1 (XSLT-style 1-based index) +Unit = no meaningful value (an operation executed for its effect) +Context = the current iteration item (see §3.5); one of + Binding + Term + Graph + JSON value +Environment = stack of variable scopes; each scope maps names to values +Operation = an unevaluated operation form (see quoting, §3.3) ``` -## Concrete Python Type System - -### RDFLib Types -```python -URI = rdflib.URIRef -Literal = rdflib.Literal -BNode = rdflib.BNode -Graph = rdflib.Graph -Term = Union[rdflib.URIRef, rdflib.Literal, rdflib.BNode] -``` +### 1.2 Concrete Python types -### Collection Types ```python -Sequence = List[Any] -Result = rdflib.query.Result -ResultRow = rdflib.query.ResultRow -VariableStack = List[Dict[str, Any]] -Context = Any -``` +URI = rdflib.URIRef +Literal = rdflib.Literal +BNode = rdflib.BNode +Term = Union[rdflib.URIRef, rdflib.Literal, rdflib.BNode] +Graph = rdflib.Graph +Result = rdflib.query.Result # typically web_algebra.json_result.JSONResult +Binding = rdflib.query.ResultRow # or Dict[str, Term] via Bindings +Sequence = list +Unit = None +Environment = list[dict[str, Any]] # the "variable stack" +``` + +## 2. Document Model (JSON serialization) + +### 2.1 Documents + +A Web Algebra document is a JSON document in one of three shapes: + +1. **A single form** — most commonly an operation call object. +2. **A program** — a JSON array of forms, evaluated in order (§3.2, *sequence + form*). +3. **An envelope** — a JSON object whose `@web-algebra` member identifies the + dialect version and whose `program` member holds a form or program: + +```json +{ + "@web-algebra": "1", + "name": "united-kingdom-cities", + "description": "Create a container and load UK city data into it", + "program": [ ... ] +} +``` + +The envelope is optional; the bare forms remain valid. `@web-algebra` is the +version of this specification the document targets (currently `"1"`). `name` +and `description` are optional and informative. Consumers ignore unknown +envelope members (forward compatibility). An envelope without a `program` +member is invalid (`ValueError`). The envelope is recognized at the document +top level only; it is not a form and cannot be nested. -### Execution Architecture -```python -# Triple execution pattern - all operations implement: -def execute(*args: RDFLib_types) -> RDFLib_type # Pure function with RDFLib terms -def execute_json(arguments: dict, variable_stack: list) -> Any # JSON processing -def mcp_run(arguments: dict, context: Any = None) -> Any # MCP interface -``` +### 2.2 Forms -## Operation Catalog +Every JSON value in (the program of) a document is a **form**. The kind of a +form is decided *syntactically*, by the first matching rule: -### Core System Operations +| # | Syntax | Form kind | +|---|--------|-----------| +| 1 | object with an `@op` member | **operation call** | +| 2 | object whose only member is `@id` | **URI reference** | +| 3 | object with any of `@context`, `@graph`, `@id`, `@type` | **RDF data** (JSON-LD) | +| 4 | any other object | **generic object** | +| 5 | array | **sequence** | +| 6 | string, number, boolean | **scalar** | +| 7 | null | invalid (`TypeError`) | -**Value** - Access variables and context values -``` -Abstract: String × Context × VariableStack → Any -Python: def execute(self, name: str, context: Any, variable_stack: List[Dict[str, Any]]) -> Any +**Operation call.** `{"@op": Name, "args": {key: form, ...}}`. `Name` must be a +registered operation name (§4, Appendix A); otherwise the document is invalid +(`ValueError`). `args` may be omitted when the operation takes no arguments. +Argument keys are operation-specific and normative (§4); a missing required +argument raises `KeyError`. + +**URI reference.** `{"@id": form}` — an object with *exactly one* member named +`@id` — evaluates to a `URI`. The inner form may be a string or any form that +evaluates to a Term; the URI is its lexical form. This is deliberately the +JSON-LD node-reference syntax: a bare node reference carries no triples, so +reusing it as the URI form is unambiguous. It replaces the verbose +`{"@op": "URI", "args": {"input": ...}}` cast in the common case: + +```json +{"@op": "GET", "args": {"url": {"@id": "https://dbpedia.org/resource/London"}}} ``` -**Variable** - Set variables in current scope (XSLT-style) -``` -Abstract: String × Any × VariableStack → ⊥ -Python: def execute(self, name: str, value: Any, variable_stack: List[Dict[str, Any]]) -> None -``` +Inside an *RDF data* form, `@id` keeps its JSON-LD meaning and is **not** +subject to this rule (rule 3 wins because the discrimination happens on the +enclosing document, whose other members mark it as data). + +**RDF data.** An object carrying any of the four reserved JSON-LD keys +`@context`, `@graph`, `@id`, `@type` is RDF data (a JSON-LD document or +fragment), not a structure to evaluate. These are JSON-LD reserved terms with +no meaning in non-RDF JSON; this list is normative and closed. Within an RDF +data form, embedded operation-call objects ("holes", e.g. an `@id` computed at +runtime) are evaluated in place and replaced by their results; every other +value is left untouched, and the form as a whole remains a JSON structure. It +is parsed into a `Graph` only by the consuming operation, which supplies the +correct base IRI (§2.3). + +**Generic object.** Evaluated member-wise: each value is evaluated as a form, +keys are preserved. This is how, e.g., SPARQL JSON term objects (§2.4) with +computed values are written. + +**Sequence.** Evaluated element-wise, in order, in a fresh variable scope +(§3.4). The value is the Sequence of element values. At document top level +this is the *program* shape. + +**Scalar.** Coerced to a Term: + +| JSON | Term | +|------|------| +| string | `Literal` with datatype `xsd:string` | +| integer | `Literal` with datatype `xsd:integer` | +| number with fraction | `Literal` with datatype `xsd:double` | +| boolean | `Literal` with datatype `xsd:boolean` | + +A plain string is *always* a string literal, never a URI; URIs are written +with the URI reference form or produced by `URI`/`ResolveURI`. `null` is not a +valid form. -**Current** - Return current context item -``` -Abstract: Any → Any -Python: def execute(self, current_item: Any) -> Any -``` +### 2.3 Base IRI -**Execute** - Execute nested operation -``` -Abstract: Operation → Any -Python: def execute(self, operation: Any) -> Any -``` +RDF data forms may contain relative IRIs. The consuming operation — the one +that turns the JSON-LD into a `Graph` — resolves them against its *target* +URI (e.g. `PUT`'s `url`), by parsing with that URI as base. There is no +document-global base IRI. Fragment references like `#service` therefore +resolve against the document being written, which is the intended semantics. + +### 2.4 SPARQL JSON term form + +Where an operation expects a Term argument (noted in its catalog entry), the +SPARQL 1.1 Query Results JSON term object is accepted: + +```json +{"type": "uri" | "literal" | "bnode", "value": "...", + "datatype": "...IRI...", "xml:lang": "..."} +``` + +`datatype` and `xml:lang` are optional and only meaningful for `literal`. An +unknown `type` raises `ValueError`. This is a generic-object form whose +member values may themselves be computed by nested operation calls. + +## 3. Evaluation Semantics + +### 3.1 Values + +Evaluation maps forms to values in the domain + +``` +Value = Term + Graph + Result + Sequence Value + Binding + Unit + JSON +``` + +(`JSON` covers RDF data forms and generic objects, which evaluate to JSON +structures with their holes filled; they become `Graph`s only at operation +boundaries.) + +### 3.2 Evaluation rules + +Evaluation is **eager and depth-first**: when an operation call is evaluated, +its argument forms are evaluated first, in document order, and the operation +is then applied to the resulting values. The exceptions are **quoted +operands** (§3.3). + +- *Operation call*: evaluate non-quoted arguments (document order), apply the + operation, yield its result. +- *URI reference*: evaluate the inner form; yield `URIRef` of its lexical + form. +- *RDF data*: walk the structure; evaluate embedded operation calls in place; + yield the resulting JSON structure. +- *Generic object*: evaluate each member value; yield the object. +- *Sequence*: push a fresh variable scope; evaluate elements in order; pop the + scope; yield the Sequence of element values. Elements are evaluated for both + value and effect — an element that is an effectful operation call (§3.6) + executes even if its value is never consumed. +- *Scalar*: yield the coerced Term (§2.2). + +### 3.3 Quoted operands + +Some operations receive an *unevaluated* form — they are special forms in the +Lisp sense, and their quoted operands are evaluated under a different regime +(later, repeatedly, or in a different context). Quoted operands are marked +**quoted** in the catalog. They are exactly: + +| Operation | Operand | Evaluation regime | +|-----------|---------|-------------------| +| `ForEach` | `operation` | once per iteration item, with that item as context | +| `Execute` | `operation` | once, in the current context and environment | + +All other arguments of all operations are eagerly evaluated. An operation not +in this table never sees an unevaluated form. + +### 3.4 Variable environment + +The environment is a stack of scopes. + +- **Binding**: `Variable` binds a name to a value in the *current* (innermost) + scope, creating one if the stack is empty. Rebinding a name in the same + scope overwrites it. +- **Lookup**: `Value` with a `$`-prefixed name searches scopes innermost to + outermost; a miss raises `ValueError`. +- **Scope creation**: a fresh scope is pushed for the duration of + (a) every sequence form, and (b) every `ForEach` iteration. Consequently a + variable bound in a program step is visible to *subsequent* steps of the + same program and to forms nested within them, and ceases to exist after the + sequence ends; a variable bound inside a `ForEach` iteration does not leak + into the next iteration. + +The `$` sigil belongs to the reference syntax of `Value`, not to the variable +name: `Variable` binds `name`, `Value` reads `$name`. Because the sigil +decides the lookup domain, variable and context lookups never shadow each +other. + +### 3.5 Context -**URI** - Convert term to URI reference -``` -Abstract: Term → URI -Python: def execute(self, term: rdflib.term.Node) -> rdflib.URIRef -``` +The context is the current iteration item. It is established *only* by +`ForEach`, which evaluates its quoted `operation` once per item with that item +as context; nested `ForEach` shadows the outer context for the extent of its +own operand. Outside any iteration there is no context, and operations that +require one (`Current`, context-lookup `Value`) raise `ValueError`. -**ForEach** - Map operation over sequence (sequence → sequence semantics) -``` -Abstract: Sequence α × Operation → Sequence β -Python: def execute(self, select_data: Union[List[Any], rdflib.query.Result], operation: Any) -> List[Any] -``` +- `Current` yields the context item itself. +- `Value` with an unprefixed name looks the name up *in* the context item: + - `Binding` (SPARQL row): the term bound to that variable name; + - mapping (e.g. a JSON object item): the member value; + - any other object: the attribute of that name; + - a miss, or an item supporting none of these, raises `ValueError`. -**Filter** - Filter sequences or select from results -``` -Abstract: (Sequence α × Expression → α) + (Result × Expression → Result) -Python: def execute(self, input_data: Any, expression: Any) -> Union[list, Any] -``` +### 3.6 Effects and ordering -**Bindings** - Extract binding sequence from SPARQL results -``` -Abstract: Result → Sequence ResultRow -Python: def execute(self, table: rdflib.query.Result) -> List[Dict[str, Any]] -``` +Operations are marked in the catalog as **pure** (no observable effect), +**query** (reads external state: HTTP GET, SPARQL query), or **update** +(writes external state: HTTP POST/PUT/PATCH). `SPARQLString` calls an external +LLM service and is additionally **non-deterministic**, as is `STRUUID`. -### String Operations +Ordering guarantees: -**Str** - Convert any term to string literal -``` -Abstract: Term → Literal -Python: def execute(self, term: rdflib.term.Node) -> rdflib.Literal -``` +- Argument evaluation is depth-first in document order (§3.2); an operation's + effects happen after all its arguments' effects. +- Sequence elements evaluate in order: all effects of element *n* happen + before any effect of element *n+1*. +- `ForEach` yields its result sequence in item order. Whether iterations + execute sequentially or concurrently is implementation-defined; a program + must not rely on effect ordering *across* iterations (within one iteration, + sequence ordering applies). The Python implementation is currently + sequential. -**Replace** - Replace patterns in strings using regex -``` -Abstract: Literal × Literal × Literal → Literal -Python: def execute(self, input_str: rdflib.Literal, pattern: rdflib.Literal, replacement: rdflib.Literal) -> rdflib.Literal -``` +There are no transactions: if evaluation fails midway, effects already +performed are not rolled back. + +### 3.7 Errors + +Failures raise Python exceptions per this table (normative): + +| Condition | Exception | +|-----------|-----------| +| unknown operation name in `@op` | `ValueError` | +| envelope without `program` | `ValueError` | +| `null` form | `TypeError` | +| missing required argument key | `KeyError` | +| argument or operand of the wrong type (any layer) | `TypeError` | +| unknown variable in `$name` lookup | `ValueError` | +| context lookup miss, or no context established | `ValueError` | +| `Filter` position < 1 or > length | `ValueError` | +| unknown `type` in SPARQL JSON term form | `ValueError` | +| blank node where SPARQL syntax forbids it (`Values` data) | `ValueError` | +| HTTP/SPARQL transport failure | `urllib.error.HTTPError` / `URLError`, unwrapped | + +Type checking is strict: operations validate their inputs and raise `TypeError` +*before* performing any effect. No implicit casting is performed between Term +kinds; the only implicit conversion anywhere is scalar coercion (§2.2) and +string-compatibility (§4.2). + +## 4. Operation Catalog (normative) -**EncodeForURI** - URL-encode strings for URI usage +Catalog entry conventions: the *Abstract* signature is in the type language of +§1.1; *JSON args* lists the argument keys of the JSON serialization with their +expected value types after evaluation; ⟨quoted⟩ marks quoted operands (§3.3). +`Maybe τ` marks optional arguments. Effects per §3.6 are noted when not pure. + +### 4.1 Control flow, variables, context + +**ForEach** — evaluate an operation once per item of a sequence or per row of +a SPARQL result; the item is the context (§3.5). +``` +Abstract: (Sequence α + Result) × Operation⟨quoted⟩ → Sequence β +Python: execute_json only (interpreter-level special form) +JSON: select: Sequence α + Result · operation⟨quoted⟩: form or array of forms ``` -Abstract: Literal → Literal -Python: def execute(self, input_str: Literal) -> Literal +- Iterates a `Sequence` item-by-item, a `Result` row-by-row in result order. + Any other `select` value raises `TypeError`. +- Each iteration runs in a fresh variable scope with the item as context. +- If `operation` is an array, its forms evaluate in order within the + iteration's scope and the iteration's value is the *last* non-Unit value. +- Iteration values that are Unit (`None`) are dropped from the output; + sequence-valued iteration results are kept nested (no flattening). Output + length therefore equals input length minus Unit-valued iterations. + +**Filter** — positional selection from a sequence, XSLT-style. +``` +Abstract: (Sequence α + Result) × Position → α +Python: def execute(self, input_data: Any, expression: Any) -> Any +JSON: input: Sequence α + Result · expression: Position ``` +- 1-based. A `Result` input is treated as its row sequence (yields a + `Binding`). Position < 1 or > length raises `ValueError`; a non-integer + expression raises `TypeError`. Only positional expressions are defined in + this version of the algebra. -**STRUUID** - Generate random UUID string +**Bindings** — project a SPARQL result to its row sequence. ``` -Abstract: () → Literal -Python: def execute(self) -> Literal +Abstract: Result → Sequence Binding +Python: def execute(self, table: Result) -> List[Dict[str, Node]] +JSON: table: Result ``` +- Order-preserving; an empty result yields the empty sequence. -### SPARQL Operations - -**SELECT** - Execute SPARQL SELECT query +**Variable** — bind a name in the current scope (like `xsl:variable`). ``` -Abstract: URI × Literal → Result -Python: def execute(self, endpoint: rdflib.URIRef, query: rdflib.Literal) -> rdflib.query.Result +Abstract: String × Any → Unit +Python: def execute(self, name: str, value: Any, variable_stack: list) -> None +JSON: name: String (plain JSON string, not a form) · value: any form ``` +- Binds in the innermost scope (§3.4). The JSON layer returns Unit (`None`); + in a `ForEach` operation array, Unit values do not become the iteration's + value. -**CONSTRUCT** - Execute SPARQL CONSTRUCT query +**Value** — read a variable (`$name`) or a context member (`name`). §3.4–3.5. ``` -Abstract: URI × Literal → Graph -Python: def execute(self, endpoint: rdflib.URIRef, query: rdflib.Literal) -> rdflib.Graph +Abstract: String → Any +Python: def execute(self, name: str, context: Any, variable_stack: list) -> Any +JSON: name: String (plain JSON string; `$` prefix selects variable lookup) ``` -**DESCRIBE** - Execute SPARQL DESCRIBE query -``` -Abstract: URI × Literal → Graph -Python: def execute(self, endpoint: rdflib.URIRef, query: rdflib.Literal) -> rdflib.Graph -``` - -**Substitute** - Replace variables in SPARQL queries -``` -Abstract: Literal × Literal × Term → Literal -Python: def execute(self, query: Literal, var: Literal, binding_value: Any) -> Literal -``` - -**Values** - Append a VALUES data block from a result set to a SPARQL query -``` -Abstract: Literal × Result × Maybe (Sequence Literal) → Literal -Python: def execute(self, query: Literal, data: Result, vars: Optional[List[str]] = None) -> Literal +**Current** — the context item itself (like XSLT `current()`). ``` - -**SPARQLString** - Generate SPARQL queries from natural language -``` -Abstract: Literal → Literal -Python: def execute(self, question: Literal) -> Literal +Abstract: () → Context +Python: def execute(self, current_item: Any) -> Any +JSON: (no arguments) ``` +- Raises `ValueError` when no context is established (§3.5). -### HTTP Operations - -**GET** - Retrieve RDF data via HTTP GET +**Execute** — evaluate a quoted operation form in the current context and +environment. ``` -Abstract: URI → Graph -Python: def execute(self, url: rdflib.URIRef) -> Graph +Abstract: Operation⟨quoted⟩ → Any +Python: def execute(self, operation: Any) -> Any +JSON: operation⟨quoted⟩: an operation-call form ``` +- The operand must be an operation-call object (`TypeError` otherwise). Its + purpose is indirection: the operand may be assembled or selected at runtime + (e.g. read from a variable) before being evaluated. -**POST** - Submit RDF data via HTTP POST -``` -Abstract: URI × Graph → Result -Python: def execute(self, url: rdflib.URIRef, data: rdflib.Graph) -> Result -``` +### 4.2 String and term operations -**PUT** - Replace RDF data via HTTP PUT -``` -Abstract: URI × Graph → Result -Python: def execute(self, url: rdflib.URIRef, data: rdflib.Graph) -> Result -``` +String-compatibility rule: where an operation is documented as accepting a +*string-compatible* Literal it accepts `xsd:string` literals, language-tagged +literals, and plain literals; any other Term raises `TypeError` (use `Str` to +cast explicitly). -**PATCH** - Update RDF data via HTTP PATCH with SPARQL Update +**Str** — cast a Term to a string literal. ``` -Abstract: URI × Literal → Result -Python: def execute(self, url: URIRef, update: Literal) -> Result +Abstract: Term → Literal +Python: def execute(self, term: Node) -> Literal +JSON: input: Term ``` +- String-compatible literals pass through unchanged (a language tag is + *preserved*). Any other Term yields `Literal` of its lexical/IRI form with + datatype `xsd:string`. Non-Terms raise `TypeError`. +- *Known divergence from SPARQL:* SPARQL `STR()` returns a simple literal and + drops language tags; Web Algebra `Str` returns `xsd:string` and preserves + tags on passthrough. -### LinkedDataHub Operations - -**ldh-CreateContainer** - Create LinkedDataHub container document +**Concat** — concatenate string literals. ``` -Abstract: URI × Literal × Maybe Literal × Maybe Literal → Result -Python: def execute(self, parent_uri: rdflib.URIRef, title: rdflib.Literal, slug: rdflib.Literal = None, description: rdflib.Literal = None) -> Result +Abstract: Sequence Literal → Literal +Python: def execute(self, inputs: List[Literal]) -> Literal +JSON: inputs: array of string-compatible Literal forms ``` +- Result datatype `xsd:string`. -**ldh-CreateItem** - Create LinkedDataHub item document +**Replace** — regular-expression replacement, in the spirit of SPARQL +`REPLACE()`. ``` -Abstract: URI × Literal × Maybe Literal → Result -Python: def execute(self, container_uri: rdflib.URIRef, title: rdflib.Literal, slug: Optional[rdflib.Literal] = None) -> Result +Abstract: Literal × Literal × Literal → Literal +Python: def execute(self, input_str, pattern, replacement) -> Literal +JSON: input · pattern · replacement: string-compatible Literals ``` +- Result datatype `xsd:string`. All three inputs must be string-compatible. +- *Known divergence:* the pattern dialect is Python `re`, not the XPath/XQuery + regular expressions SPARQL specifies. Patterns using shared syntax behave + identically. -**ldh-List** - List LinkedDataHub resources +**EncodeForURI** — percent-encode a string for use inside a URI, per SPARQL +`ENCODE_FOR_URI` / XPath `fn:encode-for-uri`. ``` -Abstract: URI × URI → List[Dict] -Python: def execute(self, url: URIRef, endpoint: URIRef) -> list[dict] +Abstract: Literal → Literal +Python: def execute(self, input_str: Literal) -> Literal +JSON: input: string-compatible Literal ``` +- Every character except the RFC 3986 unreserved set + (`A–Z a–z 0–9 - . _ ~`) is percent-encoded (UTF-8). Result `xsd:string`. -**ldh-AddView** - Add view to LinkedDataHub document +**STRUUID** — fresh UUID string, per SPARQL `STRUUID()`. Non-deterministic. ``` -Abstract: URI × URI × Literal × Maybe Literal × Maybe Literal × Maybe URI → Any -Python: def execute(self, url: URIRef, query: URIRef, title: Literal, description: Literal = None, fragment: Literal = None, mode: URIRef = None) -> Any +Abstract: () → Literal +Python: def execute(self) -> Literal +JSON: (no arguments) ``` +- An RFC 4122 version-4 UUID in lowercase hyphenated form, datatype + `xsd:string`. Successive invocations differ. -**ldh-AddResultSetChart** - Add result set chart to LinkedDataHub document +**URI** — cast a Term to a URI, like SPARQL `URI()`/`IRI()`. ``` -Abstract: URI × URI × Literal × URI × Literal × Literal × Maybe Literal × Maybe Literal → Any -Python: def execute(self, url: URIRef, query: URIRef, title: Literal, chart_type: URIRef, category_var_name: Literal, series_var_name: Literal, description: Literal = None, fragment: Literal = None) -> Any +Abstract: (URI + Literal) → URI +Python: def execute(self, term: Node) -> URIRef +JSON: input: URI + Literal ``` +- A URI input is returned as-is; a Literal yields the URI of its lexical + form. A `BNode` raises `TypeError` (a blank node has no IRI). The lexical + form is *not* validated against RFC 3986; garbage in, garbage out. -**ldh-AddSelect** - Add SPARQL SELECT service to LinkedDataHub +**ResolveURI** — RFC 3986 reference resolution. ``` -Abstract: URI × Literal × Literal × Maybe Literal × Maybe Literal × Maybe URI → Any -Python: def execute(self, url: URIRef, query: Literal, title: Literal, description: Literal = None, fragment: Literal = None, service: URIRef = None) -> Any +Abstract: URI × Literal → URI +Python: def execute(self, base: URIRef, relative: Literal) -> URIRef +JSON: base: URI · relative: string-compatible Literal ``` +- Standard §5 resolution semantics (as by `urljoin`): if `relative` is itself + an absolute URI, the result is `relative`. -**ldh-AddGenericService** - Add generic SPARQL service to LinkedDataHub -``` -Abstract: URI × URI × Literal × Maybe Literal × Maybe Literal × Maybe URI × Maybe Literal × Maybe Literal → Any -Python: def execute(self, url: URIRef, endpoint: URIRef, title: Literal, description: Literal = None, fragment: Literal = None, graph_store: URIRef = None, auth_user: Literal = None, auth_pwd: Literal = None) -> Any -``` +### 4.3 SPARQL operations -**ldh-AddObjectBlock** - Add object content block to LinkedDataHub document +**SELECT** — execute a SPARQL SELECT query against an endpoint. *Query* effect. ``` -Abstract: URI × URI × Maybe Literal × Maybe Literal × Maybe Literal × Maybe URI → Any -Python: def execute(self, url: URIRef, value: URIRef, title: Literal = None, description: Literal = None, fragment: Literal = None, mode: URIRef = None) -> Any +Abstract: URI × Literal → Result +Python: def execute(self, endpoint: URIRef, query: Literal) -> Result +JSON: endpoint: URI · query: Literal (xsd:string) ``` +- Types are validated before any network I/O. -**ldh-AddXHTMLBlock** - Add XHTML content block to LinkedDataHub document +**CONSTRUCT** — execute a SPARQL CONSTRUCT query. *Query* effect. ``` -Abstract: URI × Literal × Maybe Literal × Maybe Literal × Maybe Literal → Any -Python: def execute(self, url: URIRef, value: Literal, title: Literal = None, description: Literal = None, fragment: Literal = None) -> Any +Abstract: URI × Literal → Graph +Python: def execute(self, endpoint: URIRef, query: Literal) -> Graph +JSON: endpoint: URI · query: Literal (xsd:string) ``` -**ldh-AddFile** - Add file (binary) to LinkedDataHub document via multipart RDF/POST +**DESCRIBE** — execute a SPARQL DESCRIBE query. *Query* effect. ``` -Abstract: URI × Literal × Literal × Maybe Literal × Maybe Literal → Any -Python: def execute(self, url: URIRef, file_path: Literal, title: Literal, description: Literal = None, content_type: Literal = None) -> Any +Abstract: URI × Literal → Graph +Python: def execute(self, endpoint: URIRef, query: Literal) -> Graph +JSON: endpoint: URI · query: Literal (xsd:string) ``` -**ldh-RemoveBlock** - Remove content block from LinkedDataHub document +**Substitute** — textually substitute one SPARQL variable with a Term. ``` -Abstract: URI × Maybe URI → Any -Python: def execute(self, url: URIRef, block: URIRef = None) -> Any +Abstract: Literal × Literal × (URI + Literal) → Literal +Python: def execute(self, query, var, binding_value) -> Literal +JSON: query: Literal · var: Literal (variable name, with or without `?`) + · binding: URI + Literal (Term or SPARQL JSON term form, §2.4) ``` +- Matches both `?var` and `$var` occurrences at token boundaries. A URI value + serializes as ``; a Literal as a quoted literal with its language tag + or datatype. A `BNode` value raises `TypeError` (a blank-node label in a + query is a fresh variable, not a reference — substitution would be + meaningless). +- The substitution is textual, not parse-aware; it can produce an invalid + query if `var` collides with content inside string literals of the query. + Prefer `Values` where applicable. -**ldh-GenerateOntologyViews** - Generate LDH views (`ldh:view`) and SPIN `sp:Select` queries for each non-`owl:FunctionalProperty` `owl:ObjectProperty` in an ontology graph; `owl:DatatypeProperty` is excluded because literal values are displayed inline in LDH and do not benefit from a table view +**Values** — append a SPARQL `VALUES` data block built from a result set. ``` -Abstract: Graph × URI × URI → Graph -Python: def execute(self, ontology: rdflib.Graph, base_uri: URIRef, service_uri: URIRef) -> rdflib.Graph +Abstract: Literal × Result × Maybe (Sequence String) → Literal +Python: def execute(self, query: Literal, data: Result, + vars: Optional[List[str]] = None) -> Literal +JSON: query: Literal · data: Result · vars: Maybe (array of String) ``` +- Columns default to the result's variables; `vars` selects/reorders them + (names given with or without `?`). Missing values render as `UNDEF`. Terms + serialize per SPARQL syntax with correct escaping. Blank nodes raise + `ValueError` (forbidden in `VALUES`). -**ldh-GenerateClassContainers** - Create an LDH container per `owl:Class` in an ontology graph (each with a SPARQL service and instance-list view) +**SPARQLString** — generate a SPARQL query string from natural language via an +LLM. *Non-deterministic*; external service call. ``` -Abstract: Graph × URI × URI → Result -Python: def execute(self, ontology: rdflib.Graph, parent_container: URIRef, endpoint: URIRef) -> Result +Abstract: Literal → Literal +Python: def execute(self, question: Literal) -> Literal +JSON: question: Literal ``` +- Only the type contract is normative: string-compatible Literal in, Literal + out. The generated query text is not specified. -**ldh-GeneratePortal** - End-to-end portal generation; composes `ExtractOntology`, `ldh-GenerateOntologyViews`, `POST`, and `ldh-GenerateClassContainers` -``` -Abstract: URI × URI × URI → Result -Python: def execute(self, endpoint: URIRef, ontology_namespace: URIRef, parent_container: URIRef) -> Result -``` +### 4.4 Linked Data (HTTP) operations -### Schema Operations +`POST`, `PUT` and `PATCH` return a single-row `Result` with variables +`status` (`xsd:integer` HTTP status) and `url` (the effective request URI). +Transport failures propagate per §3.7. -**ExtractClasses** - Extract RDF classes from graph +**GET** — dereference a URI to an RDF graph. *Query* effect. ``` Abstract: URI → Graph -Python: def execute(self, endpoint: URIRef) -> rdflib.Graph +Python: def execute(self, url: URIRef) -> Graph +JSON: url: URI ``` -**ExtractDatatypeProperties** - Extract datatype properties from graph +**POST** — append RDF data to a resource. *Update* effect. ``` -Abstract: URI → Graph -Python: def execute(self, endpoint: URIRef) -> rdflib.Graph +Abstract: URI × Graph → Result +Python: def execute(self, url: URIRef, data: Graph) -> Result +JSON: url: URI · data: Graph or RDF data form (parsed with base = url) ``` -**ExtractObjectProperties** - Extract object properties from instance data; infers `owl:FunctionalProperty` when global max objects-per-subject = 1 (closed-world assumption over present triples, ignores formal ontology at `/ns`) +**PUT** — replace a resource's RDF representation. *Update* effect. ``` -Abstract: URI → Graph -Python: def execute(self, endpoint: URIRef) -> rdflib.Graph +Abstract: URI × Graph → Result +Python: def execute(self, url: URIRef, data: Graph) -> Result +JSON: url: URI · data: Graph or RDF data form (parsed with base = url) ``` -**ExtractOntology** - Extract a full ontology (classes + datatype + object properties) from a SPARQL endpoint as a single graph +**PATCH** — apply a SPARQL Update to a resource. *Update* effect. ``` -Abstract: URI → Graph -Python: def execute(self, endpoint: URIRef) -> rdflib.Graph +Abstract: URI × Literal → Result +Python: def execute(self, url: URIRef, update: Literal) -> Result +JSON: url: URI · update: Literal (SPARQL Update string) ``` -### Utility Operations +### 4.5 Graph operations -**Merge** - Merge multiple RDF graphs into one +**Merge** — union of graphs. ``` Abstract: Sequence Graph → Graph -Python: def execute(self, graphs: List[rdflib.Graph]) -> rdflib.Graph -``` - -**ResolveURI** - Resolve relative URI against base URI -``` -Abstract: URI × Literal → URI -Python: def execute(self, base: URIRef, relative: Literal) -> URIRef -``` - -## Type System Properties - -### Strict Type Checking -- All operations enforce strict input type checking -- TypeError raised for mismatched input types with informative messages -- No automatic type casting or conversion -- RDFLib types must match exactly as specified in signatures - -### Execution Architecture -- **execute()**: Pure functions operating on RDFLib types only -- **execute_json()**: JSON processing layer that calls execute() with type validation -- **mcp_run()**: MCP interface layer that calls execute() with plain arguments - -### Sequence Semantics -- **ForEach**: Maps from sequences to sequences (sequence → sequence) -- **Context**: In ForEach, context is the current sequence item (ResultRow for JSONResult) -- **Filter**: Can operate on both sequences and JSONResult tables -- **Automatic Application**: Single-item operations applied element-wise to sequences in execution layer - -### Variable System -- **Lexical Scoping**: Variables follow XSLT-style lexical scoping rules -- **Variable Stack**: Maintains nested scopes for variable resolution -- **Syntax**: `$variableName` for variable references, `variableName` for context access -- **Variable**: Sets variables in current scope, Variable operation manages the stack - -### Context System -- **Context Type**: `Any` - varies by operation and execution context -- **ForEach Context**: Current sequence item (ResultRow for SPARQL results) -- **Current Operation**: Returns the current context item unchanged -- **Value Operation**: Accesses both context values and variables from stack - -### URI Resolution -- **JSON-LD Parsing**: All operations that parse JSON-LD use `base` parameter to resolve relative URIs -- **Fragment URIs**: Fragment identifiers like `#service` resolve against target document URI -- **Base URI**: Set to the target document URI for correct resolution of relative references -- **Implementation**: Uses `rdflib.Graph.parse(data=json_data, format="json-ld", base=target_url)` \ No newline at end of file +Python: def execute(self, graphs: List[Graph]) -> Graph +JSON: graphs: array of Graph or RDF data forms +``` +- Set union of triples: duplicate triples collapse. Blank-node labels are + taken as-is (this is graph union, not RDF merge — graphs sharing a label + will coalesce on it). Input order is irrelevant to the result. + +### 4.6 Schema operations + +All take `endpoint`, the URI of a **SPARQL endpoint**, query instance data +there, and return an ontology `Graph`. *Query* effect. + +**ExtractClasses** — `URI → Graph`. Classes present in the data +(`owl:Class` candidates), from `rdf:type` usage. JSON: `endpoint: URI`. + +**ExtractDatatypeProperties** — `URI → Graph`. `owl:DatatypeProperty` +candidates from literal-valued predicates. JSON: `endpoint: URI`. + +**ExtractObjectProperties** — `URI → Graph`. `owl:ObjectProperty` candidates +from IRI-valued predicates; infers `owl:FunctionalProperty` when the maximum +number of objects per subject is 1 (closed-world over the present triples). +JSON: `endpoint: URI`. + +**ExtractOntology** — `URI → Graph`. The union of the three extractions +above: classes plus datatype and object properties, as one graph. +JSON: `endpoint: URI`. + +## 5. Conformance notes + +- The JSON serialization here and the XML serialization used by REST-VKG are + two concrete syntaxes of the same abstract algebra; operation names and + abstract signatures are shared. Operations currently exclusive to one + implementation (e.g. `Iterate` in REST-VKG; `PATCH`, `Values`, `Filter`, + `Bindings`, `URI` and the schema operations here) are slated for parity. +- MCP exposure (`mcp_run`) is an interface adapter, not part of the algebra; + its plain-JSON conversions are implementation detail. + +--- + +## Appendix A — LinkedDataHub extension operations (informative) + +The `ldh-*` operations target a LinkedDataHub instance and compose the core +operations above (mostly `PUT`/`POST`/`PATCH` with LDH vocabularies). Their +return contracts are intentionally loose in this revision and are *not* +normative; they will be pinned in a later revision. All are *update* effects +unless noted. + +| Operation | JSON args (`Maybe` = optional) | +|-----------|--------------------------------| +| `ldh-CreateContainer` | `parent: URI · title: Literal · slug: Maybe Literal · description: Maybe Literal` | +| `ldh-CreateItem` | `container: URI · title: Literal · slug: Maybe Literal` | +| `ldh-List` *(query)* | `url: URI · endpoint: URI` (or `base: Literal`, from which `endpoint` = `base` + `sparql`) | +| `ldh-AddFile` | `url: URI · file: Literal (path) · title: Literal · description: Maybe Literal · content_type: Maybe Literal` | +| `ldh-AddGenericService` | `url: URI · endpoint: URI · title: Literal · description/fragment: Maybe Literal · graph_store: Maybe URI · auth_user/auth_pwd: Maybe Literal` | +| `ldh-AddResultSetChart` | `url: URI · query: URI · title: Literal · chart_type: URI · category_var_name: Literal · series_var_name: Literal · description/fragment: Maybe Literal` | +| `ldh-AddSelect` | `url: URI · query: Literal · title: Literal · description/fragment: Maybe Literal · service: Maybe URI` | +| `ldh-AddView` | `url: URI · query: URI · title: Literal · description/fragment: Maybe Literal · mode: Maybe URI` | +| `ldh-AddObjectBlock` | `url: URI · value: URI · title/description/fragment: Maybe Literal · mode: Maybe URI` | +| `ldh-AddXHTMLBlock` | `url: URI · value: Literal (XHTML) · title/description/fragment: Maybe Literal` | +| `ldh-RemoveBlock` | `url: URI · block: Maybe URI` | +| `ldh-GenerateOntologyViews` | `ontology: Graph · base_uri: URI · service_uri: URI` | +| `ldh-GenerateClassContainers` | `ontology: Graph · parent_container: URI · endpoint: URI · service_uri: Maybe URI` | +| `ldh-GeneratePortal` | `endpoint: URI · ontology_namespace: URI · parent_container: URI` | diff --git a/prompts/system.md b/prompts/system.md index fe2a594..535d7b5 100644 --- a/prompts/system.md +++ b/prompts/system.md @@ -9,9 +9,11 @@ Your output must be a **JSON-formatted structure** of operation calls, where **o - **Operations must be represented as JSON objects**. Each operation corresponds to a function call with a specific signature. - **Operations may be nested inside arguments** to indicate dependencies. - **A result can be used directly as an argument in another operation** instead of requiring explicit intermediate variables. +- **URIs are written with the URI reference form** `{"@id": "https://..."}` — an object whose only member is `@id`. A plain JSON string is **always a string literal, never a URI**. To produce a URI from a computed value, use `URI` or `ResolveURI`. - **ForEach supports executing multiple operations sequentially** when provided with a list of operations. Each operation in the list is executed for every row in the table before moving to the next row. - **Where an operation returns or expects RDF data, it is handled internally as an `rdflib.Graph`, but is represented as JSON-LD in the JSON structure.** - **SPARQL tabular data** (e.g., from `SELECT`) can be provided inline as a list of bindings, while **RDF Graph data** (e.g., from `GET`, `CONSTRUCT`, or merges) can be provided inline as JSON-LD objects. +- **A document may optionally be wrapped in an envelope** `{"@web-algebra": "1", "name": "...", "description": "...", "program": [...]}`; the bare operation object or array remains valid. ## Example JSON Output @@ -28,7 +30,9 @@ would produce this JSON output: "select": { "@op": "SELECT", "args": { - "endpoint": "https://dbpedia.org/sparql", + "endpoint": { + "@id": "https://dbpedia.org/sparql" + }, "query": { "@op": "SPARQLString", "args": { @@ -43,7 +47,9 @@ would produce this JSON output: "url": { "@op": "ResolveURI", "args": { - "base": "http://localhost/denmark/", + "base": { + "@id": "http://localhost/denmark/" + }, "relative": { "@op": "Value", "args": { @@ -56,7 +62,7 @@ would produce this JSON output: "@op": "GET", "args": { "url": { - "@op": "Str", + "@op": "URI", "args": { "input": { "@op": "Value", diff --git a/src/web_algebra/main.py b/src/web_algebra/main.py index 7cca750..14b79f2 100644 --- a/src/web_algebra/main.py +++ b/src/web_algebra/main.py @@ -65,7 +65,8 @@ def main(settings: BaseSettings, json_data: Optional[str]): with open(json_data) as json_file: json_input = json.load(json_file) - # Execute the JSON input + # Unwrap the optional document envelope, then execute the JSON input + json_input = Operation.unwrap_document(json_input) result = Operation.process_json(settings, json_input) # Serialize final result for output diff --git a/src/web_algebra/operation.py b/src/web_algebra/operation.py index 2cae5b5..5635ed5 100644 --- a/src/web_algebra/operation.py +++ b/src/web_algebra/operation.py @@ -53,7 +53,7 @@ def execute(self, *args) -> Union[Node, Result, Graph]: @abstractmethod def execute_json( - self, arguments: dict, variable_stack: list = [] + self, arguments: dict, variable_stack: list = None ) -> Union[Node, Result, Graph]: """JSON execution: processes JSON args, returns RDFLib objects""" pass @@ -75,15 +75,37 @@ def list_operations(cls) -> List[Type["Operation"]]: def get(cls, name: str) -> Optional[Type["Operation"]]: return cls.registry.get(name) + @staticmethod + def unwrap_document(json_data: Any) -> Any: + """Unwrap an optional document envelope (formal-semantics.md §2.1). + + An envelope is a top-level object whose `@web-algebra` member names + the dialect version and whose `program` member holds the form(s) to + evaluate; other members (`name`, `description`, ...) are informative + and ignored. Bare documents pass through unchanged. The envelope is + recognized at the document top level only. + """ + if isinstance(json_data, dict) and "@web-algebra" in json_data: + if "program" not in json_data: + raise ValueError( + "Web Algebra envelope is missing its 'program' member" + ) + return json_data["program"] + return json_data + @classmethod def process_json( cls, settings: BaseSettings, json_data: Any, - context: dict = {}, - variable_stack: list = [], + context: dict = None, + variable_stack: list = None, ) -> Any: """Class method for processing JSON with @op structures""" + if context is None: + context = {} + if variable_stack is None: + variable_stack = [] if isinstance(json_data, dict): if "@op" in json_data: op_name = json_data["@op"] @@ -99,6 +121,18 @@ def process_json( # Return RDFLib objects as-is for operation chaining return result + # URI reference form (formal-semantics.md §2.2 rule 2): an object + # whose ONLY member is `@id` evaluates to a URI. This is JSON-LD's + # node-reference syntax — a bare node reference carries no triples, + # so reusing it as the URI form is unambiguous. Inside an RDF data + # form this rule never fires: `_resolve_jsonld` walks those without + # re-entering this dispatch for non-`@op` objects. + if set(json_data.keys()) == {"@id"}: + inner = cls.process_json( + settings, json_data["@id"], context, variable_stack + ) + return URIRef(str(inner)) + # JSON-LD shape recognition — a dict carrying any JSON-LD reserved # key is RDF data (a JSON-LD document or fragment), not generic # JSON to recurse into. It may still embed `@op` operations or @@ -128,13 +162,18 @@ def process_json( } elif isinstance(json_data, list): - # For sequential operations, share variable stack to allow accumulation - results = [] - current_stack = variable_stack.copy() - for item in json_data: - result = cls.process_json(settings, item, context, current_stack) - results.append(result) - return results + # Sequence form (formal-semantics.md §3.2): elements evaluate in + # order in a fresh variable scope, so a Variable bound in step N is + # visible to steps N+1.. and to nested forms, and goes out of scope + # when the sequence ends. + variable_stack.append({}) + try: + return [ + cls.process_json(settings, item, context, variable_stack) + for item in json_data + ] + finally: + variable_stack.pop() else: # Convert plain values to RDFLib terms @@ -145,8 +184,8 @@ def _resolve_jsonld( cls, settings: BaseSettings, json_data: Any, - context: dict = {}, - variable_stack: list = [], + context: dict = None, + variable_stack: list = None, ) -> Any: """Resolve embedded `@op` nodes inside a JSON-LD document in place. @@ -174,19 +213,6 @@ def _resolve_jsonld( else: return json_data - @staticmethod - def _serialize_for_json_context(obj) -> Any: - """Convert RDFLib objects to appropriate format for JSON consumption""" - if isinstance(obj, (URIRef, Literal, BNode)): - return str(obj) # Convert RDFLib terms to strings for JSON-LD - elif hasattr(obj, "to_json") and callable(obj.to_json): - return obj.to_json() # Convert Result to SPARQL JSON format - elif isinstance(obj, Graph): - # Keep graphs as-is for now - they'll be serialized by HTTP operations - return obj - else: - return obj - # Variable stack management methods def push_variable_scope(self, variable_stack: list): """Create a new variable scope (like entering a new XSLT template).""" @@ -244,6 +270,9 @@ def to_graph(data: Any, *, base: Optional[str] = None) -> Graph: @staticmethod def json_to_rdflib(data) -> Node: """Convert JSON/binding objects to RDFLib terms""" + if data is None: + # formal-semantics.md §2.2: null is not a valid form. + raise TypeError("null is not a valid Web Algebra form") if isinstance(data, dict) and "type" in data and "value" in data: # SPARQL binding object - values may have been processed to RDFLib terms type_str = str(data["type"]) # Convert potential Literal to string @@ -275,12 +304,13 @@ def json_to_rdflib(data) -> Node: elif isinstance(data, str): # Plain string → always convert to string literal return Literal(data, datatype=XSD.string) + elif isinstance(data, bool): + # bool before int — bool is an int subclass in Python + return Literal(data, datatype=XSD.boolean) elif isinstance(data, int): return Literal(data, datatype=XSD.integer) elif isinstance(data, float): return Literal(data, datatype=XSD.double) - elif isinstance(data, bool): - return Literal(data, datatype=XSD.boolean) else: # Default: convert to string literal return Literal(str(data), datatype=XSD.string) @@ -291,12 +321,13 @@ def plain_to_rdflib(value: Any) -> Node: if isinstance(value, str): # Plain string → always convert to string literal return Literal(value, datatype=XSD.string) + elif isinstance(value, bool): + # bool before int — bool is an int subclass in Python + return Literal(value, datatype=XSD.boolean) elif isinstance(value, int): return Literal(value, datatype=XSD.integer) elif isinstance(value, float): return Literal(value, datatype=XSD.double) - elif isinstance(value, bool): - return Literal(value, datatype=XSD.boolean) else: return Literal(str(value), datatype=XSD.string) diff --git a/src/web_algebra/operations/bindings.py b/src/web_algebra/operations/bindings.py index ca221fe..69451df 100644 --- a/src/web_algebra/operations/bindings.py +++ b/src/web_algebra/operations/bindings.py @@ -35,7 +35,7 @@ def execute(self, table: Result) -> List[Dict[str, Node]]: return table.bindings def execute_json( - self, arguments: dict, variable_stack: list = [] + self, arguments: dict, variable_stack: list = None ) -> List[Dict[str, Node]]: """JSON execution: process arguments with strict type checking""" # Process table diff --git a/src/web_algebra/operations/current.py b/src/web_algebra/operations/current.py index 6f890a0..9467200 100644 --- a/src/web_algebra/operations/current.py +++ b/src/web_algebra/operations/current.py @@ -27,10 +27,16 @@ def execute(self, current_item: Any) -> Any: """Pure function: return current sequence item""" return current_item - def execute_json(self, arguments: dict, variable_stack: list = []) -> Any: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Any: """JSON execution: return current context item""" - if self.context is None: - raise ValueError("Current operation requires context") + # No iteration context established (formal-semantics.md §3.5) — the + # interpreter's default context is an empty dict. + if self.context is None or ( + isinstance(self.context, dict) and not self.context + ): + raise ValueError( + "Current requires an iteration context (only ForEach establishes one)" + ) return self.execute(self.context) diff --git a/src/web_algebra/operations/execute.py b/src/web_algebra/operations/execute.py index 69c962c..b42ee7f 100644 --- a/src/web_algebra/operations/execute.py +++ b/src/web_algebra/operations/execute.py @@ -45,10 +45,18 @@ def execute(self, operation: Any) -> Any: # Delegate to Operation.process_json for nested operation execution return Operation.process_json(self.settings, operation, self.context) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Any: - """JSON execution: pass raw operation to execute""" - # Don't process the operation argument - execute() expects raw operation dict - return self.execute(arguments["operation"]) + def execute_json(self, arguments: dict, variable_stack: list = None) -> Any: + """JSON execution: evaluate the quoted operation form in the current + context AND the current variable environment (formal-semantics.md + §4.1).""" + operation = arguments["operation"] + if not isinstance(operation, dict) or "@op" not in operation: + raise TypeError( + f"Execute expects 'operation' to be an operation-call form, got {type(operation)}" + ) + return Operation.process_json( + self.settings, operation, self.context, variable_stack + ) def mcp_run(self, arguments: dict, context: Any = None) -> Any: """MCP execution: plain args → plain results""" diff --git a/src/web_algebra/operations/filter.py b/src/web_algebra/operations/filter.py index 8c66ce4..3ab2f06 100644 --- a/src/web_algebra/operations/filter.py +++ b/src/web_algebra/operations/filter.py @@ -46,9 +46,10 @@ def execute(self, input_data: Any, expression: Any) -> Union[list, Any]: # Positional filtering (current implementation) filtered_items = self._apply_positional_filter(items, expression) else: - # Future: could support other expression types (boolean expressions, etc.) - raise NotImplementedError( - f"Filter expression type {type(expression)} not yet supported" + # formal-semantics.md §4.1: only positional (integer) expressions + # are defined in this version of the algebra. + raise TypeError( + f"Filter expects an integer position expression, got {type(expression)}" ) # Return single item directly if only one result (XSLT semantics) @@ -57,7 +58,7 @@ def execute(self, input_data: Any, expression: Any) -> Union[list, Any]: return filtered_items def execute_json( - self, arguments: dict, variable_stack: list = [] + self, arguments: dict, variable_stack: list = None ) -> Union[list, Any]: """JSON execution: process arguments with support for both Result and sequence""" # Process input diff --git a/src/web_algebra/operations/for_each.py b/src/web_algebra/operations/for_each.py index 4f47ce0..bd1236b 100644 --- a/src/web_algebra/operations/for_each.py +++ b/src/web_algebra/operations/for_each.py @@ -43,8 +43,10 @@ def execute( "ForEach pure function needs operation execution context" ) - def execute_json(self, arguments: dict, variable_stack: list = []) -> List[Any]: + def execute_json(self, arguments: dict, variable_stack: list = None) -> List[Any]: """JSON execution: apply operations to each item in sequence or SPARQL results""" + if variable_stack is None: + variable_stack = [] # Get the select data (sequence or Result) select_data = Operation.process_json( self.settings, arguments["select"], self.context, variable_stack @@ -80,32 +82,43 @@ def execute_json(self, arguments: dict, variable_stack: list = []) -> List[Any]: for item in items: logging.info("Processing item: %s", item) - # Handle list of operations or single operation - if isinstance(operation, list): - # Execute operations in sequence, with item as context - last_result = None + # Each iteration runs in a fresh variable scope + # (formal-semantics.md §3.4): bindings made inside one iteration + # do not leak into the next. + variable_stack.append({}) + try: + # Handle list of operations or single operation + if isinstance(operation, list): + # Execute operations in sequence, with item as context; + # the iteration's value is the last non-Unit result. + last_result = None - for op in operation: + for op in operation: + result = Operation.process_json( + self.settings, + op, + context=item, + variable_stack=variable_stack, + ) + if result is not None: + last_result = result + + # Only collect the last non-None result + if last_result is not None: + results.append(last_result) + else: + # Single operation result = Operation.process_json( - self.settings, op, context=item, variable_stack=variable_stack + self.settings, + operation, + context=item, + variable_stack=variable_stack, ) + # Only collect non-None results if result is not None: - last_result = result - - # Only collect the last non-None result - if last_result is not None: - results.append(last_result) - else: - # Single operation - result = Operation.process_json( - self.settings, - operation, - context=item, - variable_stack=variable_stack, - ) - # Only collect non-None results - if result is not None: - results.append(result) + results.append(result) + finally: + variable_stack.pop() return results diff --git a/src/web_algebra/operations/linked_data/get.py b/src/web_algebra/operations/linked_data/get.py index 3b3d16d..a29ed5f 100644 --- a/src/web_algebra/operations/linked_data/get.py +++ b/src/web_algebra/operations/linked_data/get.py @@ -53,7 +53,7 @@ def execute(self, url: URIRef) -> Graph: return graph - def execute_json(self, arguments: dict, variable_stack: list = []) -> Graph: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Graph: """JSON execution: process arguments and delegate to execute()""" url_data = Operation.process_json( self.settings, arguments["url"], self.context, variable_stack diff --git a/src/web_algebra/operations/linked_data/patch.py b/src/web_algebra/operations/linked_data/patch.py index f969a11..8ae5583 100644 --- a/src/web_algebra/operations/linked_data/patch.py +++ b/src/web_algebra/operations/linked_data/patch.py @@ -89,7 +89,7 @@ def execute(self, url: URIRef, update: Literal) -> Result: ], ) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Result: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Result: """JSON execution: process arguments and call pure function""" # Process URL url_data = Operation.process_json( diff --git a/src/web_algebra/operations/linked_data/post.py b/src/web_algebra/operations/linked_data/post.py index 40242c5..50b8f59 100644 --- a/src/web_algebra/operations/linked_data/post.py +++ b/src/web_algebra/operations/linked_data/post.py @@ -79,7 +79,7 @@ def execute(self, url: URIRef, data: Graph) -> Result: ], ) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Result: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Result: """JSON execution: process arguments with strict type checking""" # Process URL url_data = Operation.process_json( diff --git a/src/web_algebra/operations/linked_data/put.py b/src/web_algebra/operations/linked_data/put.py index 16a5760..e016e97 100644 --- a/src/web_algebra/operations/linked_data/put.py +++ b/src/web_algebra/operations/linked_data/put.py @@ -79,7 +79,7 @@ def execute(self, url: URIRef, data: Graph) -> Result: ], ) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Result: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Result: """JSON execution: process arguments with strict type checking""" # Process URL url_data = Operation.process_json( diff --git a/src/web_algebra/operations/linkeddatahub/add_file.py b/src/web_algebra/operations/linkeddatahub/add_file.py index b10bd47..75ae753 100644 --- a/src/web_algebra/operations/linkeddatahub/add_file.py +++ b/src/web_algebra/operations/linkeddatahub/add_file.py @@ -165,7 +165,7 @@ def execute( ], ) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Result: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Result: """JSON execution: process arguments with strict type checking.""" url_data = Operation.process_json( self.settings, arguments["url"], self.context, variable_stack diff --git a/src/web_algebra/operations/linkeddatahub/add_generic_service.py b/src/web_algebra/operations/linkeddatahub/add_generic_service.py index 403943d..9a7eabf 100644 --- a/src/web_algebra/operations/linkeddatahub/add_generic_service.py +++ b/src/web_algebra/operations/linkeddatahub/add_generic_service.py @@ -62,7 +62,7 @@ def inputSchema(cls) -> dict: "required": ["url", "endpoint", "title"], } - def execute_json(self, arguments: dict, variable_stack: list = []) -> Any: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Any: """JSON execution: process arguments and delegate to execute()""" # Process required arguments url_data = Operation.process_json( diff --git a/src/web_algebra/operations/linkeddatahub/add_result_set_chart.py b/src/web_algebra/operations/linkeddatahub/add_result_set_chart.py index eac7a3a..81c5da4 100644 --- a/src/web_algebra/operations/linkeddatahub/add_result_set_chart.py +++ b/src/web_algebra/operations/linkeddatahub/add_result_set_chart.py @@ -86,7 +86,7 @@ def inputSchema(cls) -> dict: ], } - def execute_json(self, arguments: dict[str, str], variable_stack: list = []) -> Any: + def execute_json(self, arguments: dict[str, str], variable_stack: list = None) -> Any: """JSON execution: process arguments and delegate to execute()""" # Process required arguments url_data = Operation.process_json( diff --git a/src/web_algebra/operations/linkeddatahub/add_select.py b/src/web_algebra/operations/linkeddatahub/add_select.py index d1bf1c4..e40548e 100644 --- a/src/web_algebra/operations/linkeddatahub/add_select.py +++ b/src/web_algebra/operations/linkeddatahub/add_select.py @@ -59,7 +59,7 @@ def inputSchema(cls) -> dict: "required": ["url", "query", "title"], } - def execute_json(self, arguments: dict[str, str], variable_stack: list = []) -> Any: + def execute_json(self, arguments: dict[str, str], variable_stack: list = None) -> Any: """JSON execution: process arguments and delegate to execute()""" # Process required arguments url_data = Operation.process_json( diff --git a/src/web_algebra/operations/linkeddatahub/add_view.py b/src/web_algebra/operations/linkeddatahub/add_view.py index 44eab5a..015c0b5 100644 --- a/src/web_algebra/operations/linkeddatahub/add_view.py +++ b/src/web_algebra/operations/linkeddatahub/add_view.py @@ -76,7 +76,7 @@ def inputSchema(cls) -> dict: "required": ["url", "query", "title"], } - def execute_json(self, arguments: dict[str, str], variable_stack: list = []) -> Any: + def execute_json(self, arguments: dict[str, str], variable_stack: list = None) -> Any: """JSON execution: process arguments and delegate to execute()""" # Process required arguments url_data = Operation.process_json( diff --git a/src/web_algebra/operations/linkeddatahub/content/add_object_block.py b/src/web_algebra/operations/linkeddatahub/content/add_object_block.py index e281efd..fc5073a 100644 --- a/src/web_algebra/operations/linkeddatahub/content/add_object_block.py +++ b/src/web_algebra/operations/linkeddatahub/content/add_object_block.py @@ -77,7 +77,7 @@ def inputSchema(cls) -> dict: "required": ["url", "value"], } - def execute_json(self, arguments: dict[str, str], variable_stack: list = []) -> Any: + def execute_json(self, arguments: dict[str, str], variable_stack: list = None) -> Any: """JSON execution: process arguments and delegate to execute()""" # Process required arguments url_data = Operation.process_json( diff --git a/src/web_algebra/operations/linkeddatahub/content/add_xhtml_block.py b/src/web_algebra/operations/linkeddatahub/content/add_xhtml_block.py index 9f96992..b258603 100644 --- a/src/web_algebra/operations/linkeddatahub/content/add_xhtml_block.py +++ b/src/web_algebra/operations/linkeddatahub/content/add_xhtml_block.py @@ -68,7 +68,7 @@ def inputSchema(cls) -> dict: "required": ["url", "value"], } - def execute_json(self, arguments: dict[str, str], variable_stack: list = []) -> Any: + def execute_json(self, arguments: dict[str, str], variable_stack: list = None) -> Any: """JSON execution: process arguments and delegate to execute()""" # Process required arguments url_data = Operation.process_json( diff --git a/src/web_algebra/operations/linkeddatahub/content/generate_class_containers.py b/src/web_algebra/operations/linkeddatahub/content/generate_class_containers.py index 0051aa0..160b526 100644 --- a/src/web_algebra/operations/linkeddatahub/content/generate_class_containers.py +++ b/src/web_algebra/operations/linkeddatahub/content/generate_class_containers.py @@ -216,7 +216,7 @@ def _build_view_graph(self, view_uri: URIRef, class_local: str, query_uri: URIRe return g - def execute_json(self, arguments: dict, variable_stack: list = []) -> Result: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Result: """JSON execution: process arguments with type checking""" # Process ontology graph — accept a Graph (e.g. from CONSTRUCT) or a # JSON-LD document, converting the latter to a Graph diff --git a/src/web_algebra/operations/linkeddatahub/content/generate_ontology_views.py b/src/web_algebra/operations/linkeddatahub/content/generate_ontology_views.py index f202c16..bb94c16 100644 --- a/src/web_algebra/operations/linkeddatahub/content/generate_ontology_views.py +++ b/src/web_algebra/operations/linkeddatahub/content/generate_ontology_views.py @@ -164,7 +164,7 @@ def _generate_sparql_query(self, property_uri: URIRef) -> str: return sparql - def execute_json(self, arguments: dict, variable_stack: list = []) -> Graph: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Graph: """JSON execution: process arguments with type checking""" # Process ontology graph — accept a Graph (e.g. from CONSTRUCT) or a # JSON-LD document, converting the latter to a Graph diff --git a/src/web_algebra/operations/linkeddatahub/content/generate_portal.py b/src/web_algebra/operations/linkeddatahub/content/generate_portal.py index ceb9507..a1fe899 100644 --- a/src/web_algebra/operations/linkeddatahub/content/generate_portal.py +++ b/src/web_algebra/operations/linkeddatahub/content/generate_portal.py @@ -115,7 +115,7 @@ def execute(self, endpoint: URIRef, ontology_namespace: URIRef, parent_container return JSONResult(list(all_vars), all_bindings) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Result: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Result: """JSON execution: process arguments with strict type checking""" # Process endpoint endpoint_data = Operation.process_json( diff --git a/src/web_algebra/operations/linkeddatahub/content/remove_block.py b/src/web_algebra/operations/linkeddatahub/content/remove_block.py index 241f51b..b37fd7a 100644 --- a/src/web_algebra/operations/linkeddatahub/content/remove_block.py +++ b/src/web_algebra/operations/linkeddatahub/content/remove_block.py @@ -40,7 +40,7 @@ def inputSchema(cls) -> dict: "required": ["url"], } - def execute_json(self, arguments: dict[str, str], variable_stack: list = []) -> Any: + def execute_json(self, arguments: dict[str, str], variable_stack: list = None) -> Any: """JSON execution: process arguments and delegate to execute()""" # Process required arguments url_data = Operation.process_json( diff --git a/src/web_algebra/operations/linkeddatahub/create_container.py b/src/web_algebra/operations/linkeddatahub/create_container.py index 59ff34b..7ac40db 100644 --- a/src/web_algebra/operations/linkeddatahub/create_container.py +++ b/src/web_algebra/operations/linkeddatahub/create_container.py @@ -113,7 +113,7 @@ def execute( # Call parent PUT execute method return super().execute(URIRef(url), graph) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Result: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Result: """JSON execution: process arguments and call pure function""" # Process parent URI parent_data = Operation.process_json( diff --git a/src/web_algebra/operations/linkeddatahub/create_item.py b/src/web_algebra/operations/linkeddatahub/create_item.py index 1065e9f..cf5c27e 100644 --- a/src/web_algebra/operations/linkeddatahub/create_item.py +++ b/src/web_algebra/operations/linkeddatahub/create_item.py @@ -96,7 +96,7 @@ def execute( # Call parent PUT execute method return super().execute(URIRef(url), graph) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Result: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Result: """JSON execution: process arguments with strict type checking""" # Process container URI container_data = Operation.process_json( diff --git a/src/web_algebra/operations/linkeddatahub/list.py b/src/web_algebra/operations/linkeddatahub/list.py index 67e394c..f1971ce 100644 --- a/src/web_algebra/operations/linkeddatahub/list.py +++ b/src/web_algebra/operations/linkeddatahub/list.py @@ -102,7 +102,7 @@ def execute( return result def execute_json( - self, arguments: dict[str, str], variable_stack: list = [] + self, arguments: dict[str, str], variable_stack: list = None ) -> list[dict]: """JSON execution: process arguments and delegate to execute()""" # Process required arguments diff --git a/src/web_algebra/operations/merge.py b/src/web_algebra/operations/merge.py index 278cfe8..749f8cc 100644 --- a/src/web_algebra/operations/merge.py +++ b/src/web_algebra/operations/merge.py @@ -54,7 +54,7 @@ def execute(self, graphs: List[Graph]) -> Graph: logging.info("Merged RDF data (%s triple(s))", len(merged_graph)) return merged_graph - def execute_json(self, arguments: dict, variable_stack: list = []) -> Graph: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Graph: """JSON execution: process arguments and delegate to execute()""" # Process graphs argument - may return dicts with processed nested operations graphs_data = Operation.process_json( diff --git a/src/web_algebra/operations/resolve_uri.py b/src/web_algebra/operations/resolve_uri.py index fdf1d34..3873fca 100644 --- a/src/web_algebra/operations/resolve_uri.py +++ b/src/web_algebra/operations/resolve_uri.py @@ -51,7 +51,7 @@ def execute(self, base: URIRef, relative: Literal) -> URIRef: resolved_uri = urljoin(base_str, relative_str) return URIRef(resolved_uri) - def execute_json(self, arguments: dict, variable_stack: list = []) -> URIRef: + def execute_json(self, arguments: dict, variable_stack: list = None) -> URIRef: """JSON execution: process arguments and call pure function""" # Process base URI base_data = Operation.process_json( diff --git a/src/web_algebra/operations/schema/extract_classes.py b/src/web_algebra/operations/schema/extract_classes.py index 9a68470..2d00b71 100644 --- a/src/web_algebra/operations/schema/extract_classes.py +++ b/src/web_algebra/operations/schema/extract_classes.py @@ -41,7 +41,7 @@ def execute(self, endpoint: URIRef) -> Graph: """, datatype=XSD.string) return super().execute(endpoint, query) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Graph: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Graph: """JSON execution: process arguments with strict type checking""" # Process endpoint endpoint_data = Operation.process_json( diff --git a/src/web_algebra/operations/schema/extract_datatype_properties.py b/src/web_algebra/operations/schema/extract_datatype_properties.py index a034b20..46d1138 100644 --- a/src/web_algebra/operations/schema/extract_datatype_properties.py +++ b/src/web_algebra/operations/schema/extract_datatype_properties.py @@ -105,7 +105,7 @@ def execute(self, endpoint: URIRef) -> Graph: """, datatype=XSD.string) return super().execute(endpoint, query) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Graph: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Graph: """JSON execution: process arguments with strict type checking""" # Process endpoint endpoint_data = Operation.process_json( diff --git a/src/web_algebra/operations/schema/extract_object_properties.py b/src/web_algebra/operations/schema/extract_object_properties.py index fe5b2b7..294eba8 100644 --- a/src/web_algebra/operations/schema/extract_object_properties.py +++ b/src/web_algebra/operations/schema/extract_object_properties.py @@ -95,7 +95,7 @@ def execute(self, endpoint: URIRef) -> Graph: """, datatype=XSD.string) return super().execute(endpoint, query) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Graph: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Graph: """JSON execution: process arguments with strict type checking""" # Process endpoint endpoint_data = Operation.process_json( diff --git a/src/web_algebra/operations/schema/extract_ontology.py b/src/web_algebra/operations/schema/extract_ontology.py index 055fff7..d2ba345 100644 --- a/src/web_algebra/operations/schema/extract_ontology.py +++ b/src/web_algebra/operations/schema/extract_ontology.py @@ -47,7 +47,7 @@ def execute(self, endpoint: URIRef) -> Graph: return ontology_graph - def execute_json(self, arguments: dict, variable_stack: list = []) -> Graph: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Graph: """JSON execution: process arguments with strict type checking""" # Process endpoint endpoint_data = Operation.process_json( diff --git a/src/web_algebra/operations/sparql/construct.py b/src/web_algebra/operations/sparql/construct.py index a54dbab..e4c1100 100644 --- a/src/web_algebra/operations/sparql/construct.py +++ b/src/web_algebra/operations/sparql/construct.py @@ -59,7 +59,7 @@ def execute(self, endpoint: URIRef, query: Literal) -> Graph: # Convert JSON-LD response to RDF Graph return self.to_graph(json_ld_response) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Graph: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Graph: """JSON execution: process arguments and return Graph (same as execute)""" # Process endpoint endpoint_data = Operation.process_json( diff --git a/src/web_algebra/operations/sparql/describe.py b/src/web_algebra/operations/sparql/describe.py index be82c9a..c9a6930 100644 --- a/src/web_algebra/operations/sparql/describe.py +++ b/src/web_algebra/operations/sparql/describe.py @@ -59,7 +59,7 @@ def execute(self, endpoint: URIRef, query: Literal) -> Graph: # Convert JSON-LD response to RDF Graph return self.to_graph(json_ld_response) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Graph: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Graph: """JSON execution: process arguments and return Graph (same as execute)""" # Process endpoint endpoint_data = Operation.process_json( diff --git a/src/web_algebra/operations/sparql/select.py b/src/web_algebra/operations/sparql/select.py index 2bf9550..40e9b39 100644 --- a/src/web_algebra/operations/sparql/select.py +++ b/src/web_algebra/operations/sparql/select.py @@ -67,7 +67,7 @@ def execute(self, endpoint: URIRef, query: Literal) -> Result: return JSONResult.from_json(sparql_json) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Result: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Result: """JSON execution: process arguments with strict type checking""" # Process endpoint endpoint_data = Operation.process_json( diff --git a/src/web_algebra/operations/sparql/substitute.py b/src/web_algebra/operations/sparql/substitute.py index 0141c39..55edea7 100644 --- a/src/web_algebra/operations/sparql/substitute.py +++ b/src/web_algebra/operations/sparql/substitute.py @@ -70,9 +70,12 @@ def execute(self, query: Literal, var: Literal, binding_value: Any) -> Literal: raise TypeError( f"Substitute.execute expects var to be Literal, got {type(var)}" ) - if not isinstance(binding_value, (URIRef, Literal, BNode)): + if not isinstance(binding_value, (URIRef, Literal)): + # formal-semantics.md §4.3: a blank-node label in a query is a + # fresh variable, not a reference — substituting one is + # meaningless, so BNode is rejected along with non-Terms. raise TypeError( - f"Substitute.execute expects binding_value to be URIRef, Literal, or BNode, got {type(binding_value)}" + f"Substitute.execute expects binding_value to be URIRef or Literal, got {type(binding_value)}" ) query_str = str(query) @@ -86,7 +89,7 @@ def execute(self, query: Literal, var: Literal, binding_value: Any) -> Literal: return Literal(substituted_query, datatype=XSD.string) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Literal: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Literal: """JSON execution: process arguments and call pure function""" # Process query query_data = Operation.process_json( @@ -185,8 +188,6 @@ def _format_node(self, node): return f'"{node}"^^<{node.datatype}>' else: return f'"{node}"' - elif isinstance(node, BNode): - return f"_: {node}" else: raise ValueError("Unsupported RDFLib node type") diff --git a/src/web_algebra/operations/sparql/values.py b/src/web_algebra/operations/sparql/values.py index f1f9cb6..77dbd5f 100644 --- a/src/web_algebra/operations/sparql/values.py +++ b/src/web_algebra/operations/sparql/values.py @@ -115,7 +115,7 @@ def _format_term(term: Optional[Node]) -> str: # "lex"^^
, and bare numeric/boolean forms. return term.n3() - def execute_json(self, arguments: dict, variable_stack: list = []) -> Literal: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Literal: """JSON execution: process arguments and delegate to execute().""" query_data = Operation.process_json( self.settings, arguments["query"], self.context, variable_stack diff --git a/src/web_algebra/operations/sparql_string.py b/src/web_algebra/operations/sparql_string.py index ebf799a..d689a58 100644 --- a/src/web_algebra/operations/sparql_string.py +++ b/src/web_algebra/operations/sparql_string.py @@ -67,7 +67,7 @@ def execute(self, question: Literal) -> Literal: logging.info("Generated SPARQL query: %s", result) return Literal(result, datatype=XSD.string) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Literal: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Literal: """JSON execution: process arguments and delegate to execute()""" question_data = Operation.process_json( self.settings, arguments["question"], self.context, variable_stack diff --git a/src/web_algebra/operations/str.py b/src/web_algebra/operations/str.py index f810fad..ce90270 100644 --- a/src/web_algebra/operations/str.py +++ b/src/web_algebra/operations/str.py @@ -46,7 +46,7 @@ def execute(self, term: Node) -> Literal: return Literal(str(term), datatype=XSD.string) def execute_json( - self, arguments: dict, variable_stack: list = [] + self, arguments: dict, variable_stack: list = None ) -> Literal: """JSON execution: processes JSON args, returns RDFLib string literal""" # Process the input argument through the JSON system diff --git a/src/web_algebra/operations/string/concat.py b/src/web_algebra/operations/string/concat.py index 0182b59..586f4ed 100644 --- a/src/web_algebra/operations/string/concat.py +++ b/src/web_algebra/operations/string/concat.py @@ -43,7 +43,7 @@ def execute(self, inputs: List[Literal]) -> Literal: return Literal(result_str, datatype=XSD.string) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Literal: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Literal: """JSON execution: process arguments and call pure function""" inputs_data = arguments["inputs"] diff --git a/src/web_algebra/operations/string/encode_for_uri.py b/src/web_algebra/operations/string/encode_for_uri.py index 0e1dba5..2e9adb2 100644 --- a/src/web_algebra/operations/string/encode_for_uri.py +++ b/src/web_algebra/operations/string/encode_for_uri.py @@ -46,7 +46,7 @@ def execute(self, input_str: Literal) -> Literal: logging.info("Encoded URI: %s", encoded_value) return Literal(encoded_value, datatype=XSD.string) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Literal: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Literal: """JSON execution: process arguments and call pure function""" input_data = Operation.process_json( self.settings, arguments["input"], self.context, variable_stack diff --git a/src/web_algebra/operations/string/replace.py b/src/web_algebra/operations/string/replace.py index 8b2eb46..a61b404 100644 --- a/src/web_algebra/operations/string/replace.py +++ b/src/web_algebra/operations/string/replace.py @@ -97,7 +97,7 @@ def is_string_compatible(lit): return Literal(formatted_string, datatype=XSD.string) def execute_json( - self, arguments: dict, variable_stack: list = [] + self, arguments: dict, variable_stack: list = None ) -> Literal: """JSON execution: process arguments with strict type checking""" # Process input - allow implicit string conversion diff --git a/src/web_algebra/operations/struuid.py b/src/web_algebra/operations/struuid.py index 5443e18..36f4380 100644 --- a/src/web_algebra/operations/struuid.py +++ b/src/web_algebra/operations/struuid.py @@ -29,7 +29,7 @@ def execute(self) -> Literal: logging.info("Generated UUID: %s", generated_uuid) return Literal(generated_uuid, datatype=XSD.string) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Literal: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Literal: """JSON execution: process arguments and call pure function""" return self.execute() diff --git a/src/web_algebra/operations/uri.py b/src/web_algebra/operations/uri.py index cf94ec2..3f90f51 100644 --- a/src/web_algebra/operations/uri.py +++ b/src/web_algebra/operations/uri.py @@ -1,5 +1,5 @@ from typing import Any -from rdflib import URIRef +from rdflib import BNode, URIRef from rdflib.term import Node from mcp import types from web_algebra.operation import Operation @@ -30,10 +30,13 @@ def execute(self, term: Node) -> URIRef: raise TypeError( f"URI operation expects input to be RDFLib term, got {type(term)}" ) + if isinstance(term, BNode): + # formal-semantics.md §4.2: a blank node has no IRI to cast to. + raise TypeError("URI cannot cast a BNode — blank nodes have no IRI") return URIRef(str(term)) - def execute_json(self, arguments: dict, variable_stack: list = []) -> URIRef: + def execute_json(self, arguments: dict, variable_stack: list = None) -> URIRef: """JSON execution: processes JSON args, returns RDFLib URI reference""" # Process the input argument through the JSON system input_data = Operation.process_json( diff --git a/src/web_algebra/operations/value.py b/src/web_algebra/operations/value.py index b7e0125..75e2c84 100644 --- a/src/web_algebra/operations/value.py +++ b/src/web_algebra/operations/value.py @@ -1,3 +1,4 @@ +from collections.abc import Mapping from typing import Any import logging from rdflib.query import ResultRow @@ -38,13 +39,18 @@ def execute(self, name: str, context: Any, variable_stack: list) -> Any: ) return result else: - # Context binding reference + # Context lookup (formal-semantics.md §3.5): Binding → bound term, + # mapping → member value, other object → attribute. if isinstance(context, ResultRow): # SPARQL result row - access by variable name try: return context[name] # Already RDFLib term except KeyError: raise ValueError(f"Variable '{name}' not found in ResultRow") + elif isinstance(context, Mapping): + if name in context: + return context[name] + raise ValueError(f"Context member '{name}' not found in mapping") else: # Other context types if hasattr(context, name): @@ -53,8 +59,10 @@ def execute(self, name: str, context: Any, variable_stack: list) -> Any: f"Context variable '{name}' not found in {type(context)}" ) - def execute_json(self, arguments: dict, variable_stack: list = []) -> Any: + def execute_json(self, arguments: dict, variable_stack: list = None) -> Any: """JSON execution: processes JSON args, returns value (RDFLib term or raw value)""" + if variable_stack is None: + variable_stack = [] var_name: str = arguments["name"] logging.info("Resolving Value variable: %s", var_name) diff --git a/src/web_algebra/operations/variable.py b/src/web_algebra/operations/variable.py index d09274e..891f422 100644 --- a/src/web_algebra/operations/variable.py +++ b/src/web_algebra/operations/variable.py @@ -35,8 +35,10 @@ def execute(self, name: str, value: Any, variable_stack: list) -> None: self.set_variable(name, value, variable_stack) return None - def execute_json(self, arguments: dict, variable_stack: list = []) -> None: + def execute_json(self, arguments: dict, variable_stack: list = None) -> None: """JSON execution: evaluate value expression and store variable""" + if variable_stack is None: + variable_stack = [] name: str = arguments["name"] value_expr = arguments["value"] diff --git a/tests/SPEC_GAPS.md b/tests/SPEC_GAPS.md index 412579b..497dc63 100644 --- a/tests/SPEC_GAPS.md +++ b/tests/SPEC_GAPS.md @@ -1,136 +1,77 @@ # Web Algebra spec gaps -This file tracks ambiguities and omissions in `formal-semantics.md` discovered while authoring the test suite. Tests that depend on an unresolved item are marked `pytest.skip("UNCLEAR(spec): ...")` until the spec settles the question. +This file tracks ambiguities and omissions in `formal-semantics.md` discovered while +authoring the test suite. Tests that depend on an unresolved item are marked +`pytest.skip(...)` until the spec settles the question. -Format per entry: -- **Operation / property** — what's unclear - - Assumed for now: ... - - Proposed spec edit: ... +The 2026-07 spec rewrite (evaluation semantics, error taxonomy, per-operation JSON +argument tables) resolved the bulk of the original entries; the resolution record is +kept below so the decisions stay traceable. Only the **Remaining gaps** section is +live. --- -## Operations present in the implementation but absent from the spec catalog - -The spec should either add these to the catalog or the impl should drop them. - -- **Concat** (`operations/string/concat.py`) — no spec entry; signature unknown. - - Assumed for now: no test file authored. - - Proposed spec edit: add to "String Operations" with signature `Sequence Literal → Literal` (or whatever the intended shape is). -- **ExtractOntology** (`operations/schema/extract_ontology.py`) — no spec entry; sibling `Extract*` ops are spec'd. - - Assumed for now: no test file authored. - - Proposed spec edit: add to "Schema Operations" with concrete input/output types. - -(Re-verify the full list during implementation by `ls`-ing `src/web_algebra/operations/` and diffing names against the spec catalog at `formal-semantics.md` lines 57-287.) - -## Result-type and behavior ambiguities - -- **Str** (`Term → Literal`) — what datatype does the result Literal carry? `xsd:string`? simple literal (no datatype)? passthrough for an already-string Literal? SPARQL `STR()` returns a simple literal; spec should pin one. - - Assumed for now: tests assert `isinstance(result, Literal)` and lexical-form equality only; datatype assertions are skipped. - - Proposed spec edit: state result datatype explicitly. -- **URI** (`Term → URI`) — behavior on `Literal` whose lexical form isn't a valid URI? On a `BNode`? Spec lists `BNode` as a `Term` but says nothing about `URI(BNode)`. - - Assumed for now: only URIRef/Literal happy paths exercised; BNode and invalid-URI cases skipped. - - Proposed spec edit: enumerate behavior across all three Term subtypes. -- **EncodeForURI** (`Literal → Literal`) — which character set / RFC? RFC 3986 unreserved? SPARQL `ENCODE_FOR_URI`? They differ on `~`, `*`, etc. - - Assumed for now: only the uncontroversial space-to-`%20` case is asserted. - - Proposed spec edit: cite the RFC or SPARQL function explicitly. -- **Replace** (`Literal × Literal × Literal → Literal`) — regex pattern or literal pattern? SPARQL `REPLACE` is XQuery regex; the existing fixture uses `pattern: "%20"` which works either way. - - Assumed for now: only patterns that are valid as both literal and regex are tested. - - Proposed spec edit: state which. -- **STRUUID** (`() → Literal`) — UUID format? UUID4? Hyphenated? Case? - - Assumed for now: only `isinstance(result, Literal)` and "two consecutive calls differ" are asserted. - - Proposed spec edit: state format. -- **Substitute** (`Literal × Literal × Term → Literal`) — SPARQL variable syntax matched: `?var`, `$var`, or both? How are Term values serialized into the query (URIRef → `<...>`, Literal → `"..."` with datatype? lang tag?). - - Assumed for now: tests skipped pending spec. - - Proposed spec edit: define accepted variable syntax and term serialization rules. -- **Merge** (`Sequence Graph → Graph`) — duplicate triples deduplicated? RDF semantics implies set union; spec is silent. - - Assumed for now: tests assert union behavior; deduplication test marked skip. - - Proposed spec edit: state set vs multiset semantics. -- **Bindings** (`Result → Sequence ResultRow`) — order preservation? Empty Result → empty list? - - Assumed for now: length only is asserted; order assertions skipped. - - Proposed spec edit: state ordering contract. - -## Error semantics - -The Strict Type Checking property (`formal-semantics.md` lines 291-295) says "TypeError raised for mismatched input types" but doesn't extend to other error classes: - -- Missing required argument in JSON dispatch — TypeError? KeyError? ValueError? -- Unknown `@op` — ValueError? Custom exception? -- Live-service operations on network/endpoint failure — propagate? wrap? what type? -- **Variable / Value** lookup on a missing name — error or `None`? - - Assumed for now: tests assert `pytest.raises(Exception)` (broad) for these paths; specific exception class skipped. - - Proposed spec edit: state exception classes. - -## Sequence semantics - -- **ForEach** (`Sequence α × Operation → Sequence β`) — when the inner operation returns `None` or itself a sequence, what's the output shape? Filter `None`s? Flatten? The Sequence Semantics section (lines 302-306) says "Single-item operations applied element-wise" which doesn't answer the multi-item case. - - Assumed for now: only "input length = output length" is asserted, with inner ops that return single items. - - Proposed spec edit: define output shape across each inner-op return shape (None, single, sequence). -- ForEach over a SPARQL `Result` — is iteration order part of the contract? - - Assumed for now: order-sensitive assertions skipped. - -## Filter - -- **Filter signature typo** — `formal-semantics.md` line 99: `(Sequence α × Expression → α) + (Result × Expression → Result)`. The sequence case almost certainly should return `Sequence α`, not `α`. - - Assumed for now: all Filter tests skipped pending correction. - - Proposed spec edit: change `→ α` to `→ Sequence α` in the sequence case. -- **Expression type** — line 27 says `Expression = Operation + Literal + Integer`, but how each kind evaluates as a predicate is undefined. - - Proposed spec edit: define evaluation rules per Expression variant. - -## Variable system - -- **Variable** (`String × Any × VariableStack → ⊥`) — `⊥` (bottom) means non-terminating in type theory; presumably means "no meaningful return". But `execute_json` on the JSON layer must return *something* — what? - - Assumed for now: return value not asserted. - - Proposed spec edit: state JSON-layer return value (`None`? the bound value?). -- **Variable System property** (line 311) is internally contradictory: "Sets variables in current scope, Variable operation manages the stack." Sets-in-current vs manages-the-stack are different operations. - - Proposed spec edit: split into two sentences clarifying which operation is responsible for scope creation vs assignment. -- **Value lookup precedence** — when a name exists both in the variable stack and in the context, which wins? - - Assumed for now: precedence-collision tests skipped. - -## Context system - -- **Current** (`Any → Any`) — behavior when context is unset (default `{}` per the abstract type signature)? Returns the empty dict? Errors? - - Assumed for now: only the "context-set" happy path is tested. -- **Value** — which context container shapes are supported? Spec line 315 says context is `Any` and "varies by operation"; line 318 says Value "accesses context values and variables from stack" without enumerating shapes. The impl supports `ResultRow` (`context[name]`) and any object with `getattr(context, name)`, but not plain `dict`. The default `Operation.context: Any = {}` is a dict, which suggests dict should be valid — but the spec doesn't make that explicit. - - Assumed for now: dict-context test skipped pending spec. - - Proposed spec edit: enumerate the supported context container shapes for Value (ResultRow only? + dict? + arbitrary objects?). -- **Execute** (`Operation → Any`) — narrative description is missing entirely. What does Execute do that JSON dispatch doesn't already? - - Assumed for now: all Execute tests skipped pending spec narrative. - - Proposed spec edit: add narrative description. - -## Schema operations - -- **ExtractClasses / ExtractDatatypeProperties / ExtractObjectProperties** (`URI → Graph`) — what does the URI parameter denote? A SPARQL endpoint, a document URL, or an ontology IRI? Spec narrative is silent. - - Assumed for now: only TypeError-on-non-URIRef case is exercised. - - Proposed spec edit: name the URI's role explicitly. - -## JSON dispatch surface - -The `formal-semantics.md` Execution Architecture section (lines 49-55) declares `execute_json(arguments: dict, variable_stack: list) -> Any` but never specifies the keys that each operation expects in `arguments`. In practice the existing positive fixtures confirm key names for a subset of operations (Str/URI/EncodeForURI: `input`; Replace: `input`/`pattern`/`replacement`; CONSTRUCT: `query`/`endpoint`; PUT: `url`/`data`; ldh-CreateContainer: `parent`/`title`/`slug`; ldh-AddSelect: `url`/`query`/`title`; SPARQLString: `question`). - -The remaining operations (ResolveURI, Merge, Substitute, Variable, Value, Bindings, ForEach, Filter, Execute, GET, POST, PATCH, SELECT, DESCRIBE, schema and most LDH ops) have unverified JSON arg shapes. Tests for those JSON layers are skipped with `UNCLEAR(spec)`. - -- Assumed for now: ForEach uses `{select, operation}` (Python param `select_data` shortened to `select`) — used by `tests/fixtures/positive/for-each-sequence.json`. -- Proposed spec edit: per-operation JSON arg key documentation, or a stated rule (e.g. "JSON arg keys equal Python parameter names"). - -Pending fixtures (will be added once spec confirms key shapes): -- `tests/fixtures/positive/nested-resolve-uri.json` — ResolveURI keys. -- `tests/fixtures/positive/variable-and-value.json` — Variable + Value keys. -- `tests/fixtures/positive/substitute-template.json` — Substitute keys. -- `tests/fixtures/positive/merge-two-graphs.json` — Merge keys. - -## Spec/impl divergences observed on first run - -These four assertions were written from `formal-semantics.md` and failed against the implementation. They are not harness bugs — each is a place where the spec and code disagree, and the team needs to decide which side moves. - -- **`tests/unit/test_str.py::TestStrPure::test_non_term_raises_type_error`** — Spec's Strict Type Checking property mandates TypeError on mismatched input. `Str.execute([1, 2, 3])` returns a Literal instead of raising. Either Str should validate `term` is a `URIRef | Literal | BNode`, or the spec should carve out an exception for Str ("accepts any value, casts via `str(...)`"). -- **`tests/unit/test_select.py::TestSELECTPure::test_wrong_endpoint_type_raises`** — Spec: `URI × Literal → Result`. `SELECT.execute(Literal(...), Literal(...))` does not raise TypeError; it proceeds to an HTTP call. Same conflict between Strict Type Checking and the implementation, scaled to a network side effect. -- **`tests/unit/test_select.py::TestSELECTPure::test_wrong_query_type_raises`** — Same as above with `query=URIRef(...)`. - -## Live-service operations - -- **GET, POST, PUT, PATCH** — return types are spec'd, but content negotiation, headers, status-code handling, redirects, timeouts are all silent. -- **SELECT, CONSTRUCT, DESCRIBE** — same: behavior on 4xx/5xx, malformed query, network failure unspecified. -- **ldh-*** — most return `Any`. What is the meaningful assertion for tests against a live LDH instance? -- **SPARQLString** (`Literal → Literal`) — generates SPARQL "from natural language". Non-deterministic (LLM); no testable invariant beyond return type. - - Assumed for now: pure-layer tests cover input-type validation only; live tests assert return types under the relevant marker. - - Proposed spec edit: define error-handling contract for each I/O op. +## Remaining gaps + +- **`ldh-*` operations** — Appendix A of the spec is explicitly informative: JSON arg + keys are documented, but return contracts are deliberately loose (`Any`, or + `Result` with unspecified shape). Unit tests for these operations stay skipped + until a later spec revision pins them; behavior is exercised by the `ldh`-marked + integration fixtures against a live LinkedDataHub. +- **SPARQLString** — §4.3 pins the type contract only (string-compatible Literal → + Literal); the operation is non-deterministic (LLM) and needs an OpenAI client, so + even the type contract is only exercised in live runs. +- **Live-service behavior** — §3.7 pins transport failures to + `urllib.error.HTTPError`/`URLError` propagating unwrapped, but content negotiation, + redirects (beyond 308), timeouts, and retry policy remain unspecified. + +## Resolved in the 2026-07 spec revision + +Each item below is now normative in `formal-semantics.md` (section in parentheses), +and the corresponding tests are un-skipped. + +- **Catalog omissions**: `Concat` (§4.2) and `ExtractOntology` (§4.6) added. +- **Str result datatype** (§4.2): string-compatible literals pass through unchanged + (language tags preserved — documented divergence from SPARQL `STR()`); other Terms + → `xsd:string` of the lexical/IRI form. +- **URI on BNode / invalid lexical form** (§4.2): BNode → `TypeError`; lexical forms + are not validated against RFC 3986. +- **EncodeForURI character set** (§4.2): percent-encode everything outside the + RFC 3986 unreserved set (`A–Z a–z 0–9 - . _ ~`), per XPath `fn:encode-for-uri`. +- **Replace pattern dialect** (§4.2): regular expression, Python `re` dialect + (documented divergence from SPARQL's XPath/XQuery regex). +- **STRUUID format** (§4.2): RFC 4122 version-4, lowercase hyphenated, `xsd:string`. +- **Substitute** (§4.3): matches `?var` and `$var` at token boundaries; URI → ``, + Literal → quoted with lang/datatype; BNode → `TypeError`; substitution is textual + and documented as not parse-aware. +- **Merge duplicate semantics** (§4.5): set union; duplicates collapse; blank-node + labels taken as-is (graph union, not RDF merge). +- **Bindings order/empty** (§4.1): order-preserving; empty result → empty sequence. +- **Filter signature** (§4.1): `(Sequence α + Result) × Position → α` — the old + `→ α` was correct for the positional case, which is the only expression kind this + version defines; non-integer expressions → `TypeError`, out-of-range → `ValueError`. +- **ForEach output shape** (§4.1): Unit-valued iterations dropped; sequence-valued + results stay nested; operation arrays yield the last non-Unit value; Result rows + iterate in result order; fresh variable scope per iteration. +- **ForEach pure layer** (§4.1): declared an interpreter-level special form — + `execute_json` only; no pure `execute()` contract. +- **Variable return type** (§4.1): `⊥` corrected to `Unit`; JSON layer returns + `None`; binds in the innermost scope, rebinding overwrites; scope creation belongs + to sequences and ForEach iterations (§3.4), not to Variable. +- **Value context shapes & precedence** (§3.5, §4.1): Binding → bound term, mapping → + member value, other object → attribute; the `$` sigil selects the lookup domain, so + variables and context never shadow; misses → `ValueError`. +- **Current on unset context** (§3.5): `ValueError` — only ForEach establishes a + context. +- **Execute narrative** (§4.1): evaluates a quoted operation form in the current + context and the current variable environment. +- **Extract\* URI role** (§4.6): the URI names a SPARQL endpoint. +- **Error semantics** (§3.7): normative exception table — unknown `@op` → + `ValueError`, type mismatch → `TypeError`, missing required argument → `KeyError`, + unknown variable / context miss → `ValueError`, `null` form → `TypeError`, + transport failures propagate unwrapped. +- **JSON dispatch surface**: every core operation's argument keys are now normative + (§4 catalog, per-entry `JSON:` line); `ldh-*` keys documented informatively + (Appendix A). +- **Strict-typing divergences observed on first run** (Str, SELECT): resolved on the + implementation side — both validate input types before any effect (§3.7). diff --git a/tests/unit/test_bindings.py b/tests/unit/test_bindings.py index e502ea5..872af98 100644 --- a/tests/unit/test_bindings.py +++ b/tests/unit/test_bindings.py @@ -1,10 +1,8 @@ -"""Spec: formal-semantics.md "Bindings - Extract binding sequence from SPARQL results" -Abstract: Result → Sequence ResultRow -Python: def execute(self, table: rdflib.query.Result) -> List[Dict[str, Any]] - -Note: the abstract signature says `Sequence ResultRow`, but the Python signature -returns `List[Dict[str, Any]]`. The spec is internally inconsistent here; tests -assert only the abstract sequence shape (length / non-empty / iterable). +"""Spec: formal-semantics.md §4.1 "Bindings — project a SPARQL result to its +row sequence" +Abstract: Result → Sequence Binding +- Order-preserving; an empty result yields the empty sequence. +- A Binding is a partial mapping from variable names to Terms (§1.1). """ from __future__ import annotations @@ -47,12 +45,38 @@ def test_non_result_input_raises(self, settings): with pytest.raises(TypeError): op.execute([1, 2, 3]) - @pytest.mark.skip(reason="UNCLEAR(spec): order preservation not stated") def test_order_preserved(self, settings): - pass + # §4.1: order-preserving + from web_algebra.json_result import JSONResult + + op = Operation.get("Bindings")(settings=settings) + table = JSONResult.from_json( + { + "head": {"vars": ["x"]}, + "results": { + "bindings": [ + {"x": {"type": "literal", "value": v}} + for v in ("a", "b", "c") + ] + }, + } + ) + rows = op.execute(table) + assert [str(row["x"]) for row in rows] == ["a", "b", "c"] class TestBindingsJson: - @pytest.mark.skip(reason="UNCLEAR(spec): JSON arg key for Bindings not given by spec or existing fixtures") def test_json_dispatch(self, settings): - pass + # §4.1 JSON: table: Result + from web_algebra.json_result import JSONResult + + op = Operation.get("Bindings")(settings=settings) + table = JSONResult.from_json( + { + "head": {"vars": ["x"]}, + "results": {"bindings": [{"x": {"type": "literal", "value": "a"}}]}, + } + ) + rows = op.execute_json({"table": table}) + assert len(rows) == 1 + assert rows[0]["x"] == Literal("a") diff --git a/tests/unit/test_current.py b/tests/unit/test_current.py index c8ea998..5f672df 100644 --- a/tests/unit/test_current.py +++ b/tests/unit/test_current.py @@ -1,7 +1,7 @@ -"""Spec: formal-semantics.md "Current - Return current context item" -Abstract: Any → Any -Python: def execute(self, current_item: Any) -> Any -Plus Context System property: "Current Operation: Returns the current context item unchanged" (line 317). +"""Spec: formal-semantics.md §4.1 "Current — the context item itself" +Abstract: () → Context +- Yields the context item; raises ValueError when no context is + established (§3.5: only ForEach establishes one). """ from __future__ import annotations @@ -19,9 +19,11 @@ def test_returns_argument_unchanged(self, settings): result = op.execute(sentinel) assert result is sentinel or result == sentinel - @pytest.mark.skip(reason="UNCLEAR(spec): behavior when context is unset (default `{}` per the abstract type signature)") - def test_unset_context(self, settings): - pass + def test_unset_context_raises_value_error(self, settings): + # §3.5/§3.7: no context established → ValueError + op = Operation.get("Current")(settings=settings) + with pytest.raises(ValueError): + op.execute_json({}) class TestCurrentJson: diff --git a/tests/unit/test_describe.py b/tests/unit/test_describe.py index fba2580..02780b7 100644 --- a/tests/unit/test_describe.py +++ b/tests/unit/test_describe.py @@ -37,6 +37,14 @@ def test_returns_graph(self, settings): class TestDESCRIBEJson: - @pytest.mark.skip(reason="UNCLEAR(spec): DESCRIBE JSON arg shape not exemplified by existing fixtures") - def test_json_dispatch(self, settings): - pass + def test_wrong_endpoint_type_raises_before_network(self, settings): + # §4.3 JSON: endpoint: URI · query: Literal (xsd:string). + # §3.7: strict typing before any effect. + op = Operation.get("DESCRIBE")(settings=settings) + with pytest.raises(TypeError): + op.execute_json( + { + "endpoint": "http://example.org/sparql", + "query": "DESCRIBE ", + } + ) diff --git a/tests/unit/test_document.py b/tests/unit/test_document.py new file mode 100644 index 0000000..107d286 --- /dev/null +++ b/tests/unit/test_document.py @@ -0,0 +1,138 @@ +"""Spec: formal-semantics.md §2 (Document Model) and §3 (Evaluation Semantics). + +Covers the envelope (§2.1), form discrimination and scalar coercion (§2.2), +the URI reference form, and sequence/variable scoping (§3.2, §3.4). +""" + +from __future__ import annotations + +import pytest +from rdflib import Literal, URIRef +from rdflib.namespace import XSD + +from web_algebra.operation import Operation + + +class TestEnvelope: + def test_envelope_unwraps_program(self): + # §2.1: the `program` member holds the form(s) to evaluate + doc = {"@web-algebra": "1", "program": [{"@op": "STRUUID"}]} + assert Operation.unwrap_document(doc) == [{"@op": "STRUUID"}] + + def test_envelope_without_program_raises(self): + # §2.1/§3.7: an envelope without a `program` member is invalid + with pytest.raises(ValueError): + Operation.unwrap_document({"@web-algebra": "1"}) + + def test_informative_members_are_ignored(self): + # §2.1: `name`/`description` are informative; unknown members ignored + doc = { + "@web-algebra": "1", + "name": "x", + "description": "y", + "future-member": True, + "program": [], + } + assert Operation.unwrap_document(doc) == [] + + def test_bare_document_passes_through(self): + # §2.1: the envelope is optional; bare forms remain valid + bare = [{"@op": "STRUUID"}] + assert Operation.unwrap_document(bare) is bare + + +class TestURIReferenceForm: + def test_id_string_evaluates_to_uri(self, settings): + # §2.2 rule 2: an object whose only member is `@id` evaluates to a URI + result = Operation.process_json(settings, {"@id": "http://example.org/x"}) + assert result == URIRef("http://example.org/x") + + def test_id_with_nested_operation(self, settings): + # §2.2: the inner form may be any form that evaluates to a Term + result = Operation.process_json( + settings, + {"@id": {"@op": "Concat", "args": {"inputs": ["http://ex/", "x"]}}}, + ) + assert result == URIRef("http://ex/x") + + def test_object_with_id_and_other_members_is_rdf_data(self, settings): + # §2.2 rule 3 wins when other members are present: the object is an + # RDF data form and stays a JSON structure + form = {"@id": "http://ex/s", "http://ex/p": "v"} + result = Operation.process_json(settings, form) + assert result == form + + +class TestScalarForms: + def test_string_coerces_to_xsd_string(self, settings): + result = Operation.process_json(settings, "hello") + assert result == Literal("hello", datatype=XSD.string) + + def test_integer_coerces_to_xsd_integer(self, settings): + result = Operation.process_json(settings, 42) + assert result.datatype == XSD.integer + + def test_fractional_number_coerces_to_xsd_double(self, settings): + result = Operation.process_json(settings, 3.14) + assert result.datatype == XSD.double + + def test_boolean_coerces_to_xsd_boolean(self, settings): + result = Operation.process_json(settings, True) + assert result.datatype == XSD.boolean + + def test_null_is_invalid(self, settings): + # §2.2 rule 7 / §3.7: null is not a valid form + with pytest.raises(TypeError): + Operation.process_json(settings, None) + + +class TestOperationCallForm: + def test_unknown_operation_raises_value_error(self, settings): + # §3.7: unknown operation name in `@op` → ValueError + with pytest.raises(ValueError): + Operation.process_json(settings, {"@op": "NoSuchOperation"}) + + +class TestSequenceScoping: + def test_variable_visible_to_later_steps(self, settings): + # §3.4: a variable bound in a program step is visible to subsequent + # steps of the same sequence + program = [ + {"@op": "Variable", "args": {"name": "x", "value": "v"}}, + {"@op": "Value", "args": {"name": "$x"}}, + ] + result = Operation.process_json(settings, program) + assert result == [None, Literal("v", datatype=XSD.string)] + + def test_variable_visible_in_nested_sequence(self, settings): + # §3.4: ...and to forms nested within them + program = [ + {"@op": "Variable", "args": {"name": "x", "value": "v"}}, + [{"@op": "Value", "args": {"name": "$x"}}], + ] + result = Operation.process_json(settings, program) + assert result[1] == [Literal("v", datatype=XSD.string)] + + def test_binding_ceases_after_sequence_ends(self, settings): + # §3.4: the binding ceases to exist after the sequence ends + stack: list = [] + Operation.process_json( + settings, + [{"@op": "Variable", "args": {"name": "x", "value": "v"}}], + variable_stack=stack, + ) + assert stack == [] + with pytest.raises(ValueError): + Operation.process_json( + settings, + {"@op": "Value", "args": {"name": "$x"}}, + variable_stack=stack, + ) + + def test_sequence_value_is_list_of_element_values(self, settings): + # §3.2: the sequence's value is the Sequence of element values + result = Operation.process_json(settings, ["a", 1]) + assert result == [ + Literal("a", datatype=XSD.string), + Literal(1, datatype=XSD.integer), + ] diff --git a/tests/unit/test_encode_for_uri.py b/tests/unit/test_encode_for_uri.py index 2a2a752..3fb1f2d 100644 --- a/tests/unit/test_encode_for_uri.py +++ b/tests/unit/test_encode_for_uri.py @@ -31,9 +31,19 @@ def test_uri_input_raises(self, settings): with pytest.raises(TypeError): op.execute(URIRef("http://example.org/x")) - @pytest.mark.skip(reason="UNCLEAR(spec): which character set / RFC — `~`, `*`, `'`, etc. differ across RFC 3986 and SPARQL ENCODE_FOR_URI") - def test_reserved_character_set(self, settings): - pass + def test_rfc3986_unreserved_set_passes_through(self, settings): + # §4.2: everything except A–Z a–z 0–9 - . _ ~ is percent-encoded + op = Operation.get("EncodeForURI")(settings=settings) + result = op.execute(Literal("AZaz09-._~")) + assert str(result) == "AZaz09-._~" + + def test_reserved_characters_are_encoded(self, settings): + # §4.2: reserved characters like / : * ' are encoded (UTF-8) + op = Operation.get("EncodeForURI")(settings=settings) + assert str(op.execute(Literal("a/b"))) == "a%2Fb" + assert str(op.execute(Literal("a:b"))) == "a%3Ab" + assert str(op.execute(Literal("a*b"))) == "a%2Ab" + assert str(op.execute(Literal("a'b"))) == "a%27b" class TestEncodeForURIJson: diff --git a/tests/unit/test_execute.py b/tests/unit/test_execute.py index 8e5b066..f0a3479 100644 --- a/tests/unit/test_execute.py +++ b/tests/unit/test_execute.py @@ -1,25 +1,57 @@ -"""Spec: formal-semantics.md "Execute - Execute nested operation" -Abstract: Operation → Any -Python: def execute(self, operation: Any) -> Any - -The spec entry has only a signature; no narrative description is given. All -behavioral cases are blocked until the spec adds one. +"""Spec: formal-semantics.md §4.1 "Execute — evaluate a quoted operation form +in the current context and environment" +Abstract: Operation⟨quoted⟩ → Any +- The operand must be an operation-call form (TypeError otherwise). +- It is evaluated in the current context AND the current variable environment. """ from __future__ import annotations import pytest +from rdflib import Literal +from rdflib.namespace import XSD from web_algebra.operation import Operation class TestExecutePure: - @pytest.mark.skip(reason="UNCLEAR(spec): Execute has no narrative description in formal-semantics.md — what does it do that JSON dispatch doesn't already?") - def test_basic(self, settings): - pass + def test_evaluates_operation_form(self, settings): + op = Operation.get("Execute")(settings=settings) + result = op.execute({"@op": "Str", "args": {"input": "hi"}}) + # §2.2: the scalar "hi" coerces to an xsd:string Literal + assert result == Literal("hi", datatype=XSD.string) + + def test_non_operation_form_raises_type_error(self, settings): + # §4.1: the operand must be an operation-call form + op = Operation.get("Execute")(settings=settings) + with pytest.raises(TypeError): + op.execute({"not-an-op": 1}) + with pytest.raises(TypeError): + op.execute("just a string") class TestExecuteJson: - @pytest.mark.skip(reason="UNCLEAR(spec): Execute has no narrative description in formal-semantics.md") def test_json_dispatch(self, settings): - pass + # §4.1 JSON: operation⟨quoted⟩: an operation-call form + op = Operation.get("Execute")(settings=settings) + result = op.execute_json( + {"operation": {"@op": "Str", "args": {"input": "hi"}}} + ) + assert result == Literal("hi", datatype=XSD.string) + + def test_operand_sees_current_environment(self, settings): + # §4.1/§3.3: the quoted operand evaluates in the current variable + # environment — a variable bound outside Execute is visible inside. + op = Operation.get("Execute")(settings=settings) + stack = [{"x": Literal("bound")}] + result = op.execute_json( + {"operation": {"@op": "Value", "args": {"name": "$x"}}}, stack + ) + assert result == Literal("bound") + + def test_operand_sees_current_context(self, settings): + # §4.1/§3.3: the quoted operand evaluates in the current context. + ctx = Literal("ctx-item") + op = Operation.get("Execute")(settings=settings, context=ctx) + result = op.execute_json({"operation": {"@op": "Current", "args": {}}}) + assert result == ctx diff --git a/tests/unit/test_extract_classes.py b/tests/unit/test_extract_classes.py index d71024c..7b4829a 100644 --- a/tests/unit/test_extract_classes.py +++ b/tests/unit/test_extract_classes.py @@ -17,12 +17,13 @@ def test_wrong_input_type_raises(self, settings): with pytest.raises(TypeError): op.execute(Literal("not-a-uri")) - @pytest.mark.skip(reason="UNCLEAR(spec): is the URI a SPARQL endpoint, document URL, or ontology IRI? — narrative omits this") - def test_happy_path(self, settings): - pass + # §4.6: the URI names a SPARQL endpoint; the happy path queries it and + # is covered under the `sparql` marker via the live suite. class TestExtractClassesJson: - @pytest.mark.skip(reason="UNCLEAR(spec): JSON arg key for ExtractClasses not given by spec or existing fixtures") - def test_json_dispatch(self, settings): - pass + def test_wrong_endpoint_type_raises_before_network(self, settings): + # §4.6 JSON: endpoint: URI. §3.7: strict typing before any effect. + op = Operation.get("ExtractClasses")(settings=settings) + with pytest.raises(TypeError): + op.execute_json({"endpoint": "http://example.org/sparql"}) diff --git a/tests/unit/test_extract_datatype_properties.py b/tests/unit/test_extract_datatype_properties.py index d21dddc..84b7e2d 100644 --- a/tests/unit/test_extract_datatype_properties.py +++ b/tests/unit/test_extract_datatype_properties.py @@ -17,12 +17,13 @@ def test_wrong_input_type_raises(self, settings): with pytest.raises(TypeError): op.execute(Literal("not-a-uri")) - @pytest.mark.skip(reason="UNCLEAR(spec): is the URI a SPARQL endpoint, document URL, or ontology IRI?") - def test_happy_path(self, settings): - pass + # §4.6: the URI names a SPARQL endpoint; the happy path queries it and + # is covered under the `sparql` marker via the live suite. class TestExtractDatatypePropertiesJson: - @pytest.mark.skip(reason="UNCLEAR(spec): JSON arg key not given by spec or existing fixtures") - def test_json_dispatch(self, settings): - pass + def test_wrong_endpoint_type_raises_before_network(self, settings): + # §4.6 JSON: endpoint: URI. §3.7: strict typing before any effect. + op = Operation.get("ExtractDatatypeProperties")(settings=settings) + with pytest.raises(TypeError): + op.execute_json({"endpoint": "http://example.org/sparql"}) diff --git a/tests/unit/test_extract_object_properties.py b/tests/unit/test_extract_object_properties.py index ef1d8b3..b923682 100644 --- a/tests/unit/test_extract_object_properties.py +++ b/tests/unit/test_extract_object_properties.py @@ -17,12 +17,13 @@ def test_wrong_input_type_raises(self, settings): with pytest.raises(TypeError): op.execute(Literal("not-a-uri")) - @pytest.mark.skip(reason="UNCLEAR(spec): is the URI a SPARQL endpoint, document URL, or ontology IRI?") - def test_happy_path(self, settings): - pass + # §4.6: the URI names a SPARQL endpoint; the happy path queries it and + # is covered under the `sparql` marker via the live suite. class TestExtractObjectPropertiesJson: - @pytest.mark.skip(reason="UNCLEAR(spec): JSON arg key not given by spec or existing fixtures") - def test_json_dispatch(self, settings): - pass + def test_wrong_endpoint_type_raises_before_network(self, settings): + # §4.6 JSON: endpoint: URI. §3.7: strict typing before any effect. + op = Operation.get("ExtractObjectProperties")(settings=settings) + with pytest.raises(TypeError): + op.execute_json({"endpoint": "http://example.org/sparql"}) diff --git a/tests/unit/test_extract_ontology.py b/tests/unit/test_extract_ontology.py index 52c9fd7..10f792b 100644 --- a/tests/unit/test_extract_ontology.py +++ b/tests/unit/test_extract_ontology.py @@ -18,12 +18,13 @@ def test_wrong_input_type_raises(self, settings): with pytest.raises(TypeError): op.execute(Literal("not-a-uri")) - @pytest.mark.skip(reason="UNCLEAR(spec): is the URI a SPARQL endpoint, document URL, or ontology IRI? — narrative omits this") - def test_happy_path(self, settings): - pass + # §4.6: the URI names a SPARQL endpoint; the happy path queries it and + # is covered under the `sparql` marker via the live suite. class TestExtractOntologyJson: - @pytest.mark.skip(reason="UNCLEAR(spec): JSON arg key for ExtractOntology not given by spec or existing fixtures") - def test_json_dispatch(self, settings): - pass + def test_wrong_endpoint_type_raises_before_network(self, settings): + # §4.6 JSON: endpoint: URI. §3.7: strict typing before any effect. + op = Operation.get("ExtractOntology")(settings=settings) + with pytest.raises(TypeError): + op.execute_json({"endpoint": "http://example.org/sparql"}) diff --git a/tests/unit/test_filter.py b/tests/unit/test_filter.py index 8429391..c1361dd 100644 --- a/tests/unit/test_filter.py +++ b/tests/unit/test_filter.py @@ -1,28 +1,73 @@ -"""Spec: formal-semantics.md "Filter - Filter sequences or select from results" -Abstract: (Sequence α × Expression → α) + (Result × Expression → Result) -Python: def execute(self, input_data: Any, expression: Any) -> Union[list, Any] - -The sequence case in the abstract signature has a typo (returns `α`, a single -item, instead of `Sequence α`). Until the spec is corrected and the Expression -semantics are defined (line 27 declares `Expression = Operation + Literal + -Integer` but doesn't define how each kind acts as a predicate), tests are -blocked. +"""Spec: formal-semantics.md §4.1 "Filter — positional selection from a +sequence, XSLT-style" +Abstract: (Sequence α + Result) × Position → α +- 1-based; a Result input is treated as its row sequence (yields a Binding). +- Position < 1 or > length raises ValueError; a non-integer expression raises + TypeError. Only positional expressions are defined in this version. """ from __future__ import annotations import pytest +from rdflib import Literal +from rdflib.namespace import XSD +from web_algebra.json_result import JSONResult from web_algebra.operation import Operation +def _result_of(*values: str) -> JSONResult: + return JSONResult.from_json( + { + "head": {"vars": ["x"]}, + "results": { + "bindings": [ + {"x": {"type": "literal", "value": v}} for v in values + ] + }, + } + ) + + class TestFilterPure: - @pytest.mark.skip(reason="UNCLEAR(spec): line 99 sequence case has typo (`→ α` should be `→ Sequence α`) and Expression evaluation is undefined") - def test_basic(self, settings): - pass + def test_positional_selection_returns_item(self, settings): + # §4.1: 1-based positional selection yields the item itself + op = Operation.get("Filter")(settings=settings) + items = [Literal("a"), Literal("b"), Literal("c")] + assert op.execute(items, 1) == Literal("a") + assert op.execute(items, 3) == Literal("c") + + def test_result_input_yields_binding(self, settings): + # §4.1: a Result input is treated as its row sequence + op = Operation.get("Filter")(settings=settings) + row = op.execute(_result_of("a", "b"), 2) + assert row["x"] == Literal("b") + + def test_position_out_of_range_raises_value_error(self, settings): + # §3.7: Filter position < 1 or > length → ValueError + op = Operation.get("Filter")(settings=settings) + items = [Literal("a")] + with pytest.raises(ValueError): + op.execute(items, 0) + with pytest.raises(ValueError): + op.execute(items, 2) + + def test_non_integer_expression_raises_type_error(self, settings): + # §4.1: a non-integer expression raises TypeError + op = Operation.get("Filter")(settings=settings) + with pytest.raises(TypeError): + op.execute([Literal("a")], "1") class TestFilterJson: - @pytest.mark.skip(reason="UNCLEAR(spec): see TestFilterPure") def test_json_dispatch(self, settings): - pass + # §4.1 JSON: input: Sequence + Result · expression: Position + op = Operation.get("Filter")(settings=settings) + result = op.execute_json({"input": ["a", "b", "c"], "expression": 2}) + # §2.2: the scalar "b" coerces to an xsd:string Literal + assert result == Literal("b", datatype=XSD.string) + + def test_non_integer_expression_raises_type_error(self, settings): + op = Operation.get("Filter")(settings=settings) + with pytest.raises(TypeError): + op.execute_json({"input": ["a"], "expression": "not-an-int"}) diff --git a/tests/unit/test_for_each.py b/tests/unit/test_for_each.py index 1b8c80b..90d3262 100644 --- a/tests/unit/test_for_each.py +++ b/tests/unit/test_for_each.py @@ -1,30 +1,27 @@ -"""Spec: formal-semantics.md "ForEach - Map operation over sequence (sequence → sequence semantics)" -Abstract: Sequence α × Operation → Sequence β -Python: def execute(self, select_data: Union[List[Any], rdflib.query.Result], - operation: Any) -> List[Any] -Plus Sequence Semantics property (lines 302-306). +"""Spec: formal-semantics.md §4.1 "ForEach — evaluate an operation once per +item of a sequence or per row of a SPARQL result" +Abstract: (Sequence α + Result) × Operation⟨quoted⟩ → Sequence β +- Interpreter-level special form: execute_json only, no pure layer (§4.1). +- Iterates a Sequence item-by-item, a Result row-by-row in result order. +- Each iteration runs in a fresh variable scope with the item as context. +- Operation arrays evaluate in order; the iteration's value is the last + non-Unit result. Unit-valued iterations are dropped; sequence-valued + results stay nested (no flattening). """ from __future__ import annotations import pytest from rdflib import Literal +from rdflib.namespace import XSD +from web_algebra.json_result import JSONResult from web_algebra.operation import Operation -class TestForEachPure: - @pytest.mark.skip(reason="UNCLEAR(spec): ForEach pure execute() requires an Operation value plus dispatcher context — abstract signature is testable only via execute_json") - def test_pure(self, settings): - pass - - class TestForEachJson: def test_empty_sequence(self, settings): - # JSON arg keys derived from Python parameter names: select_data → "select" by convention. - # The existing fixture set has no ForEach example; flagged in SPEC_GAPS for confirmation. op = Operation.get("ForEach")(settings=settings) - # Use the JSON dispatcher: an inner Str on each item. result = op.execute_json( { "select": [], @@ -47,7 +44,7 @@ def test_length_matches_input(self, settings): assert [str(item) for item in result] == ["a", "b", "c"] def test_non_iterable_select_raises(self, settings): - # Strict Type Checking property: select must be a Sequence or Result. + # §4.1: any other `select` value raises TypeError op = Operation.get("ForEach")(settings=settings) with pytest.raises(TypeError): op.execute_json( @@ -57,10 +54,98 @@ def test_non_iterable_select_raises(self, settings): } ) - @pytest.mark.skip(reason="UNCLEAR(spec): output shape when inner op returns None or a sequence — flatten? filter Nones?") - def test_inner_op_none_handling(self, settings): - pass + def test_unit_valued_iterations_are_dropped(self, settings): + # §4.1: iteration values that are Unit (None) are dropped — Variable + # returns Unit, so an all-Variable operation yields the empty sequence. + op = Operation.get("ForEach")(settings=settings) + result = op.execute_json( + { + "select": ["a", "b"], + "operation": { + "@op": "Variable", + "args": {"name": "x", "value": {"@op": "Current", "args": {}}}, + }, + } + ) + assert result == [] + + def test_sequence_results_stay_nested(self, settings): + # §4.1: sequence-valued iteration results are kept nested (no + # flattening) — a nested ForEach yields a sequence per outer item. + op = Operation.get("ForEach")(settings=settings) + result = op.execute_json( + { + "select": [["a", "b"]], + "operation": { + "@op": "ForEach", + "args": { + "select": {"@op": "Current", "args": {}}, + "operation": { + "@op": "Str", + "args": {"input": {"@op": "Current", "args": {}}}, + }, + }, + }, + } + ) + # §2.2: scalars coerce to xsd:string Literals + assert result == [ + [Literal("a", datatype=XSD.string), Literal("b", datatype=XSD.string)] + ] + + def test_operation_array_yields_last_non_unit(self, settings): + # §4.1: operation arrays evaluate in order within the iteration's + # scope; the iteration's value is the last non-Unit result. + op = Operation.get("ForEach")(settings=settings) + result = op.execute_json( + { + "select": ["a"], + "operation": [ + { + "@op": "Variable", + "args": {"name": "x", "value": {"@op": "Current", "args": {}}}, + }, + {"@op": "Str", "args": {"input": {"@op": "Value", "args": {"name": "$x"}}}}, + ], + } + ) + assert result == [Literal("a", datatype=XSD.string)] + + def test_iteration_scope_does_not_leak(self, settings): + # §3.4: each iteration runs in a fresh scope — bindings made inside + # do not survive the ForEach. + op = Operation.get("ForEach")(settings=settings) + stack = [{}] + op.execute_json( + { + "select": ["a"], + "operation": [ + {"@op": "Variable", "args": {"name": "x", "value": "v"}}, + {"@op": "Current", "args": {}}, + ], + }, + stack, + ) + assert stack == [{}] - @pytest.mark.skip(reason="UNCLEAR(spec): SPARQL Result iteration order") - def test_result_iteration_order(self, settings): - pass + def test_result_rows_iterate_in_result_order(self, settings): + # §4.1: a Result iterates row-by-row in result order + op = Operation.get("ForEach")(settings=settings) + table = JSONResult.from_json( + { + "head": {"vars": ["x"]}, + "results": { + "bindings": [ + {"x": {"type": "literal", "value": v}} + for v in ("a", "b", "c") + ] + }, + } + ) + result = op.execute_json( + { + "select": table, + "operation": {"@op": "Value", "args": {"name": "x"}}, + } + ) + assert [str(v) for v in result] == ["a", "b", "c"] diff --git a/tests/unit/test_get.py b/tests/unit/test_get.py index cc3ebd2..e4ac0ff 100644 --- a/tests/unit/test_get.py +++ b/tests/unit/test_get.py @@ -32,6 +32,9 @@ def test_returns_graph(self, settings): class TestGETJson: - @pytest.mark.skip(reason="UNCLEAR(spec): GET JSON arg shape not exemplified by existing fixtures (presumed `{url}`)") - def test_json_dispatch(self, settings): - pass + def test_wrong_url_type_raises_before_network(self, settings): + # §4.4 JSON: url: URI. §3.7: strict typing before any effect — + # a plain string coerces to a string Literal, not a URI. + op = Operation.get("GET")(settings=settings) + with pytest.raises(TypeError): + op.execute_json({"url": "http://example.org/x"}) diff --git a/tests/unit/test_merge.py b/tests/unit/test_merge.py index cbce6eb..0cdf241 100644 --- a/tests/unit/test_merge.py +++ b/tests/unit/test_merge.py @@ -1,11 +1,11 @@ -"""Spec: formal-semantics.md "Merge - Merge multiple RDF graphs into one" +"""Spec: formal-semantics.md §4.5 "Merge — union of graphs" Abstract: Sequence Graph → Graph -Python: def execute(self, graphs: List[rdflib.Graph]) -> rdflib.Graph +- Set union of triples: duplicate triples collapse. +- JSON: graphs: array of Graph or RDF data forms. """ from __future__ import annotations -import pytest from rdflib import Graph, Literal, URIRef from web_algebra.operation import Operation @@ -44,12 +44,25 @@ def test_two_graphs_union(self, settings): assert t1 in result assert t2 in result - @pytest.mark.skip(reason="UNCLEAR(spec): duplicate-triple semantics (set union vs multiset) not stated") def test_duplicate_triples_deduplicated(self, settings): - pass + # §4.5: set union — duplicate triples collapse + op = Operation.get("Merge")(settings=settings) + triple = (URIRef("http://ex/s"), URIRef("http://ex/p"), Literal("o")) + result = op.execute([_graph_with([triple]), _graph_with([triple])]) + assert len(result) == 1 class TestMergeJson: - @pytest.mark.skip(reason="UNCLEAR(spec): JSON arg key for Merge ('graphs'? 'input'?) not given by spec or existing fixtures") def test_json_dispatch(self, settings): - pass + # §4.5 JSON: graphs: array of Graph or RDF data forms + op = Operation.get("Merge")(settings=settings) + result = op.execute_json( + { + "graphs": [ + {"@id": "http://ex/s1", "http://ex/p": "a"}, + {"@id": "http://ex/s2", "http://ex/p": "b"}, + ] + } + ) + assert (URIRef("http://ex/s1"), URIRef("http://ex/p"), Literal("a")) in result + assert (URIRef("http://ex/s2"), URIRef("http://ex/p"), Literal("b")) in result diff --git a/tests/unit/test_patch.py b/tests/unit/test_patch.py index 6305aca..fcc8e47 100644 --- a/tests/unit/test_patch.py +++ b/tests/unit/test_patch.py @@ -31,6 +31,14 @@ def test_returns_result(self, settings): class TestPATCHJson: - @pytest.mark.skip(reason="UNCLEAR(spec): PATCH JSON arg shape not exemplified by existing fixtures (presumed `{url, update}`)") - def test_json_dispatch(self, settings): - pass + def test_wrong_url_type_raises_before_network(self, settings): + # §4.4 JSON: url: URI · update: Literal (SPARQL Update string). + # §3.7: strict typing before any effect. + op = Operation.get("PATCH")(settings=settings) + with pytest.raises(TypeError): + op.execute_json( + { + "url": "http://example.org/x", + "update": "DELETE WHERE { ?s ?p ?o }", + } + ) diff --git a/tests/unit/test_post.py b/tests/unit/test_post.py index b75ef9b..86b06bf 100644 --- a/tests/unit/test_post.py +++ b/tests/unit/test_post.py @@ -31,6 +31,14 @@ def test_returns_result(self, settings): class TestPOSTJson: - @pytest.mark.skip(reason="UNCLEAR(spec): POST JSON arg shape not exemplified by existing fixtures (presumed `{url, data}`)") - def test_json_dispatch(self, settings): - pass + def test_wrong_url_type_raises_before_network(self, settings): + # §4.4 JSON: url: URI · data: Graph or RDF data form. + # §3.7: strict typing before any effect. + op = Operation.get("POST")(settings=settings) + with pytest.raises(TypeError): + op.execute_json( + { + "url": "http://example.org/x", + "data": {"@id": "http://ex/s", "@type": "http://ex/T"}, + } + ) diff --git a/tests/unit/test_replace.py b/tests/unit/test_replace.py index efed367..8672555 100644 --- a/tests/unit/test_replace.py +++ b/tests/unit/test_replace.py @@ -38,9 +38,11 @@ def test_uri_replacement_raises(self, settings): with pytest.raises(TypeError): op.execute(Literal("Hello"), Literal("e"), URIRef("http://example.org/x")) - @pytest.mark.skip(reason="UNCLEAR(spec): regex vs literal pattern semantics — class name and SPARQL parallel suggest regex but spec is silent") - def test_regex_metacharacter_treated_as_regex(self, settings): - pass + def test_pattern_is_a_regular_expression(self, settings): + # §4.2: the pattern is a regular expression (Python re dialect) + op = Operation.get("Replace")(settings=settings) + result = op.execute(Literal("a1b2c3"), Literal("[0-9]"), Literal("#")) + assert str(result) == "a#b#c#" class TestReplaceJson: diff --git a/tests/unit/test_resolve_uri.py b/tests/unit/test_resolve_uri.py index 7d181d4..e1da593 100644 --- a/tests/unit/test_resolve_uri.py +++ b/tests/unit/test_resolve_uri.py @@ -43,12 +43,22 @@ def test_wrong_relative_type_raises(self, settings): with pytest.raises(TypeError): op.execute(URIRef("http://example.org/base/"), URIRef("foo")) - @pytest.mark.skip(reason="UNCLEAR(spec): behavior when relative is itself an absolute URI") - def test_absolute_relative(self, settings): - pass + def test_absolute_relative_returns_itself(self, settings): + # §4.2: RFC 3986 §5 — if `relative` is itself an absolute URI, the + # result is `relative` + op = Operation.get("ResolveURI")(settings=settings) + result = op.execute( + URIRef("http://example.org/base/"), Literal("https://other.example/x") + ) + assert str(result) == "https://other.example/x" class TestResolveURIJson: - @pytest.mark.skip(reason="UNCLEAR(spec): JSON arg key names for ResolveURI not given by spec or existing fixtures") def test_json_dispatch(self, settings): - pass + # §4.2 JSON: base: URI · relative: string-compatible Literal + op = Operation.get("ResolveURI")(settings=settings) + result = op.execute_json( + {"base": {"@id": "http://example.org/base/"}, "relative": "foo"} + ) + assert isinstance(result, URIRef) + assert str(result) == "http://example.org/base/foo" diff --git a/tests/unit/test_select.py b/tests/unit/test_select.py index 4556d18..5e040ba 100644 --- a/tests/unit/test_select.py +++ b/tests/unit/test_select.py @@ -38,6 +38,25 @@ def test_returns_result(self, settings): class TestSELECTJson: - @pytest.mark.skip(reason="UNCLEAR(spec): SELECT JSON arg shape — existing fixtures show `{query, endpoint}` for CONSTRUCT but SELECT is not exemplified") - def test_json_dispatch(self, settings): - pass + def test_wrong_endpoint_type_raises_before_network(self, settings): + # §4.3 JSON: endpoint: URI · query: Literal (xsd:string). + # §3.7: TypeError raised before any effect — a plain string is a + # string Literal (§2.2), not a URI. + op = Operation.get("SELECT")(settings=settings) + with pytest.raises(TypeError): + op.execute_json( + { + "endpoint": "http://example.org/sparql", + "query": "SELECT * WHERE { ?s ?p ?o }", + } + ) + + def test_wrong_query_type_raises_before_network(self, settings): + op = Operation.get("SELECT")(settings=settings) + with pytest.raises(TypeError): + op.execute_json( + { + "endpoint": {"@id": "http://example.org/sparql"}, + "query": {"@id": "http://example.org/not-a-query"}, + } + ) diff --git a/tests/unit/test_sparql_string.py b/tests/unit/test_sparql_string.py index f5b3dcf..c505df2 100644 --- a/tests/unit/test_sparql_string.py +++ b/tests/unit/test_sparql_string.py @@ -1,25 +1,23 @@ -"""Spec: formal-semantics.md "SPARQLString - Generate SPARQL queries from natural language" +"""Spec: formal-semantics.md §4.3 "SPARQLString — generate a SPARQL query +string from natural language via an LLM" Abstract: Literal → Literal -Python: def execute(self, question: Literal) -> Literal - -This operation calls an LLM and is non-deterministic — there is no testable -invariant beyond return type, and even that requires an OpenAI client. +- Non-deterministic; only the type contract is normative (§4.3). Exercising + it requires an OpenAI client, so behavior is covered by live runs only. """ from __future__ import annotations import pytest -from web_algebra.operation import Operation class TestSPARQLStringPure: - @pytest.mark.skip(reason="UNCLEAR(spec): operation depends on an LLM; no deterministic testable invariant in spec") + @pytest.mark.skip(reason="§4.3: non-deterministic (LLM); type-only contract needs a live OpenAI client to exercise") def test_basic(self, settings): pass class TestSPARQLStringJson: - @pytest.mark.skip(reason="UNCLEAR(spec): same as TestSPARQLStringPure") + @pytest.mark.skip(reason="§4.3: same as TestSPARQLStringPure") def test_json_dispatch(self, settings): pass diff --git a/tests/unit/test_str.py b/tests/unit/test_str.py index 96c157b..05e2306 100644 --- a/tests/unit/test_str.py +++ b/tests/unit/test_str.py @@ -1,13 +1,15 @@ -"""Spec: formal-semantics.md "Str - Convert any term to string literal" +"""Spec: formal-semantics.md §4.2 "Str — cast a Term to a string literal" Abstract: Term → Literal -Python: def execute(self, term: rdflib.term.Node) -> rdflib.Literal -Plus the Strict Type Checking property (lines 291-295). +- String-compatible literals pass through unchanged (language tag preserved). +- Any other Term yields a Literal of its lexical/IRI form, datatype xsd:string. +- Non-Terms raise TypeError (§3.7 strict typing). """ from __future__ import annotations import pytest from rdflib import BNode, Literal, URIRef +from rdflib.namespace import XSD from web_algebra.operation import Operation @@ -38,9 +40,35 @@ def test_non_term_raises_type_error(self, settings): with pytest.raises(TypeError): op.execute([1, 2, 3]) - @pytest.mark.skip(reason="UNCLEAR(spec): result Literal datatype (xsd:string vs simple literal) unspecified") - def test_result_datatype(self, settings): - pass + def test_uri_input_yields_xsd_string(self, settings): + # §4.2: a non-string-compatible Term yields xsd:string of its IRI form + op = Operation.get("Str")(settings=settings) + result = op.execute(URIRef("http://example.org/foo")) + assert result.datatype == XSD.string + + def test_plain_literal_passes_through_unchanged(self, settings): + # §4.2: string-compatible literals pass through unchanged + op = Operation.get("Str")(settings=settings) + term = Literal("hello") + result = op.execute(term) + assert result == term + assert result.datatype is None + + def test_lang_tagged_literal_preserves_tag(self, settings): + # §4.2: language tag is preserved on passthrough (documented + # divergence from SPARQL STR(), which drops it) + op = Operation.get("Str")(settings=settings) + term = Literal("hallo", lang="de") + result = op.execute(term) + assert result == term + assert result.language == "de" + + def test_typed_literal_yields_xsd_string_of_lexical_form(self, settings): + # §4.2: any other Term → xsd:string of its lexical form + op = Operation.get("Str")(settings=settings) + result = op.execute(Literal(42)) + assert result.datatype == XSD.string + assert str(result) == "42" class TestStrJson: diff --git a/tests/unit/test_struuid.py b/tests/unit/test_struuid.py index 899624f..74da80f 100644 --- a/tests/unit/test_struuid.py +++ b/tests/unit/test_struuid.py @@ -5,7 +5,6 @@ from __future__ import annotations -import pytest from rdflib import Literal from web_algebra.operation import Operation @@ -23,9 +22,19 @@ def test_two_calls_differ(self, settings): b = op.execute() assert str(a) != str(b) - @pytest.mark.skip(reason="UNCLEAR(spec): UUID format (UUID4? hyphenated? case?) not specified") def test_uuid_format(self, settings): - pass + # §4.2: RFC 4122 version-4 UUID, lowercase hyphenated, xsd:string + import re + + from rdflib.namespace import XSD + + op = Operation.get("STRUUID")(settings=settings) + result = op.execute() + assert result.datatype == XSD.string + assert re.fullmatch( + r"[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}", + str(result), + ) class TestSTRUUIDJson: diff --git a/tests/unit/test_substitute.py b/tests/unit/test_substitute.py index 2725d56..a33d3b4 100644 --- a/tests/unit/test_substitute.py +++ b/tests/unit/test_substitute.py @@ -1,12 +1,15 @@ -"""Spec: formal-semantics.md "Substitute - Replace variables in SPARQL queries" -Abstract: Literal × Literal × Term → Literal -Python: def execute(self, query: Literal, var: Literal, binding_value: Any) -> Literal +"""Spec: formal-semantics.md §4.3 "Substitute — textually substitute one +SPARQL variable with a Term" +Abstract: Literal × Literal × (URI + Literal) → Literal +- Matches both `?var` and `$var` at token boundaries. +- URI serializes as ``; Literal as a quoted literal with its language + tag or datatype. BNode raises TypeError. """ from __future__ import annotations import pytest -from rdflib import Literal, URIRef +from rdflib import BNode, Literal, URIRef from web_algebra.operation import Operation @@ -27,12 +30,64 @@ def test_wrong_query_type_raises(self, settings): with pytest.raises(TypeError): op.execute(URIRef("not-a-query"), Literal("x"), URIRef("http://example.org/foo")) - @pytest.mark.skip(reason="UNCLEAR(spec): SPARQL variable syntax — `?var`, `$var`, or both? How are URIRef/Literal binding values serialized into the query?") - def test_replacement_form(self, settings): - pass + def test_question_mark_variable_replaced_with_iri(self, settings): + # §4.3: `?var` matched; URI serializes as + op = Operation.get("Substitute")(settings=settings) + result = op.execute( + Literal("DESCRIBE ?x"), Literal("x"), URIRef("http://example.org/foo") + ) + assert "" in str(result) + assert "?x" not in str(result) + + def test_dollar_variable_replaced(self, settings): + # §4.3: `$var` matched too + op = Operation.get("Substitute")(settings=settings) + result = op.execute( + Literal("DESCRIBE $x"), Literal("x"), URIRef("http://example.org/foo") + ) + assert "" in str(result) + assert "$x" not in str(result) + + def test_token_boundary_respected(self, settings): + # §4.3: matches at token boundaries — ?xy must not be touched when + # substituting ?x + op = Operation.get("Substitute")(settings=settings) + result = op.execute( + Literal("SELECT ?xy WHERE { ?x ?p ?xy }"), + Literal("x"), + URIRef("http://example.org/foo"), + ) + assert "?xy" in str(result) + + def test_literal_value_serialized_with_datatype(self, settings): + # §4.3: Literal serializes as a quoted literal with its datatype + op = Operation.get("Substitute")(settings=settings) + result = op.execute( + Literal("SELECT * WHERE { ?s ?p ?x }"), + Literal("x"), + Literal("42", datatype=URIRef("http://www.w3.org/2001/XMLSchema#integer")), + ) + assert '"42"' in str(result) + assert "http://www.w3.org/2001/XMLSchema#integer" in str(result) + + def test_bnode_value_raises_type_error(self, settings): + # §4.3: a blank-node label in a query is a fresh variable, not a + # reference — BNode raises TypeError + op = Operation.get("Substitute")(settings=settings) + with pytest.raises(TypeError): + op.execute(Literal("DESCRIBE ?x"), Literal("x"), BNode("b1")) class TestSubstituteJson: - @pytest.mark.skip(reason="UNCLEAR(spec): JSON arg keys for Substitute not given by spec or existing fixtures") def test_json_dispatch(self, settings): - pass + # §4.3 JSON: query · var · binding (Term or SPARQL JSON term form §2.4) + op = Operation.get("Substitute")(settings=settings) + result = op.execute_json( + { + "query": "DESCRIBE ?x", + "var": "x", + "binding": {"type": "uri", "value": "http://example.org/foo"}, + } + ) + assert isinstance(result, Literal) + assert "" in str(result) diff --git a/tests/unit/test_uri.py b/tests/unit/test_uri.py index 9ceceea..a25eeda 100644 --- a/tests/unit/test_uri.py +++ b/tests/unit/test_uri.py @@ -1,7 +1,8 @@ -"""Spec: formal-semantics.md "URI - Convert term to URI reference" -Abstract: Term → URI -Python: def execute(self, term: rdflib.term.Node) -> rdflib.URIRef -Plus the Strict Type Checking property (lines 291-295). +"""Spec: formal-semantics.md §4.2 "URI — cast a Term to a URI" +Abstract: (URI + Literal) → URI +- URI input returned as-is; Literal yields the URI of its lexical form. +- BNode raises TypeError (a blank node has no IRI). +- The lexical form is NOT validated against RFC 3986. """ from __future__ import annotations @@ -31,15 +32,19 @@ def test_non_term_raises_type_error(self, settings): with pytest.raises(TypeError): op.execute(42) - @pytest.mark.skip(reason="UNCLEAR(spec): URI(BNode) — spec lists BNode as a Term but doesn't define this case") - def test_bnode_input(self, settings): + def test_bnode_raises_type_error(self, settings): + # §4.2: a BNode raises TypeError — a blank node has no IRI op = Operation.get("URI")(settings=settings) - op.execute(BNode("b1")) + with pytest.raises(TypeError): + op.execute(BNode("b1")) - @pytest.mark.skip(reason="UNCLEAR(spec): URI(Literal whose lexical form is not a valid URI) unspecified") - def test_invalid_uri_literal(self, settings): + def test_invalid_uri_literal_is_not_validated(self, settings): + # §4.2: the lexical form is not validated against RFC 3986 — + # garbage in, garbage out op = Operation.get("URI")(settings=settings) - op.execute(Literal("not a uri")) + result = op.execute(Literal("not a uri")) + assert isinstance(result, URIRef) + assert str(result) == "not a uri" class TestURIJson: diff --git a/tests/unit/test_value.py b/tests/unit/test_value.py index 3696a25..d642a4d 100644 --- a/tests/unit/test_value.py +++ b/tests/unit/test_value.py @@ -1,7 +1,10 @@ -"""Spec: formal-semantics.md "Value - Access variables and context values" -Abstract: String × Context × VariableStack → Any -Python: def execute(self, name: str, context: Any, variable_stack: List[Dict[str, Any]]) -> Any -Plus Variable System property (lines 308-311) and Context System property (lines 314-318). +"""Spec: formal-semantics.md §4.1 "Value" with §3.4 (variable environment) +and §3.5 (context). +Abstract: String → Any +- `$name` searches variable scopes innermost to outermost; miss → ValueError. +- Unprefixed `name` looks up in the context item: Binding → bound term, + mapping → member value, other object → attribute; miss → ValueError. +- The `$` sigil decides the lookup domain, so the two never shadow each other. """ from __future__ import annotations @@ -26,20 +29,48 @@ def test_lookup_falls_back_to_outer_scope(self, settings): result = op.execute("$x", {}, stack) assert result == Literal("outer") - @pytest.mark.skip(reason="UNCLEAR(spec): which context container shapes does Value support? Context type is `Any` (line 315) and the narrative names ResultRow but doesn't enumerate other shapes (dict? attribute-bearing object? both?)") - def test_context_lookup(self, settings): - pass + def test_mapping_context_lookup(self, settings): + # §3.5: mapping context item → member value + op = Operation.get("Value")(settings=settings) + result = op.execute("city", {"city": Literal("Vilnius")}, []) + assert result == Literal("Vilnius") - @pytest.mark.skip(reason="UNCLEAR(spec): precedence when same name appears in both context and stack") - def test_context_vs_stack_precedence(self, settings): - pass + def test_attribute_context_lookup(self, settings): + # §3.5: any other object → the attribute of that name + class Item: + city = Literal("Kaunas") - @pytest.mark.skip(reason="UNCLEAR(spec): behavior on missing name — error class unspecified") - def test_missing_name(self, settings): - pass + op = Operation.get("Value")(settings=settings) + result = op.execute("city", Item(), []) + assert result == Literal("Kaunas") + + def test_sigil_selects_lookup_domain(self, settings): + # §3.4: `$name` reads the variable stack, plain `name` the context — + # the same name in both never shadows. + op = Operation.get("Value")(settings=settings) + context = {"x": Literal("from-context")} + stack = [{"x": Literal("from-stack")}] + assert op.execute("$x", context, stack) == Literal("from-stack") + assert op.execute("x", context, stack) == Literal("from-context") + + def test_missing_variable_raises_value_error(self, settings): + # §3.7: unknown variable in `$name` lookup → ValueError + op = Operation.get("Value")(settings=settings) + with pytest.raises(ValueError): + op.execute("$missing", {}, []) + + def test_missing_context_member_raises_value_error(self, settings): + # §3.7: context lookup miss → ValueError + op = Operation.get("Value")(settings=settings) + with pytest.raises(ValueError): + op.execute("missing", {"other": Literal("v")}, []) class TestValueJson: - @pytest.mark.skip(reason="UNCLEAR(spec): JSON arg key for Value not given by spec or existing fixtures") def test_json_dispatch(self, settings): - pass + # §4.1 JSON: name: String (plain JSON string, `$` prefix for variables) + op = Operation.get("Value")(settings=settings) + result = op.execute_json( + {"name": "$x"}, [{"x": Literal("bound")}] + ) + assert result == Literal("bound") diff --git a/tests/unit/test_variable.py b/tests/unit/test_variable.py index 8df6244..f4b5a94 100644 --- a/tests/unit/test_variable.py +++ b/tests/unit/test_variable.py @@ -1,13 +1,14 @@ -"""Spec: formal-semantics.md "Variable - Set variables in current scope (XSLT-style)" -Abstract: String × Any × VariableStack → ⊥ -Python: def execute(self, name: str, value: Any, variable_stack: List[Dict[str, Any]]) -> None -Plus Variable System property (lines 308-311). +"""Spec: formal-semantics.md §4.1 "Variable — bind a name in the current scope" +Abstract: String × Any → Unit +- Binds in the innermost scope; rebinding the same name overwrites (§3.4). +- The JSON layer returns Unit (None). +- Scope creation belongs to sequences and ForEach iterations, not to Variable. """ from __future__ import annotations -import pytest from rdflib import Literal +from rdflib.namespace import XSD from web_algebra.operation import Operation @@ -22,16 +23,39 @@ def test_binds_into_current_scope(self, settings): result = value_op.execute("$x", {}, stack) assert result == Literal("v") - @pytest.mark.skip(reason="UNCLEAR(spec): `⊥` (bottom) return type — what does execute_json return on the JSON layer?") - def test_return_value(self, settings): - pass - - @pytest.mark.skip(reason="UNCLEAR(spec): line 311 self-contradiction — does Variable push a new scope or write into the current one?") - def test_scope_management(self, settings): - pass + def test_returns_unit(self, settings): + # §4.1: Abstract String × Any → Unit; JSON layer returns None + op = Operation.get("Variable")(settings=settings) + assert op.execute("x", Literal("v"), [{}]) is None + + def test_binds_into_innermost_scope_only(self, settings): + # §3.4: Variable binds in the innermost scope; it does not push or + # pop scopes itself. + op = Operation.get("Variable")(settings=settings) + stack = [{}, {}] + op.execute("x", Literal("v"), stack) + assert len(stack) == 2 + assert "x" not in stack[0] + assert stack[1]["x"] == Literal("v") + + def test_rebinding_overwrites(self, settings): + # §3.4: rebinding a name in the same scope overwrites it + op = Operation.get("Variable")(settings=settings) + stack = [{}] + op.execute("x", Literal("first"), stack) + op.execute("x", Literal("second"), stack) + assert stack[0]["x"] == Literal("second") class TestVariableJson: - @pytest.mark.skip(reason="UNCLEAR(spec): JSON arg keys for Variable not given by spec or existing fixtures") def test_json_dispatch(self, settings): - pass + # §4.1 JSON: name: String · value: any form; returns Unit (None) + op = Operation.get("Variable")(settings=settings) + stack = [{}] + result = op.execute_json({"name": "x", "value": "v"}, stack) + assert result is None + value_op = Operation.get("Value")(settings=settings) + # §2.2: the scalar "v" coerces to an xsd:string Literal + assert value_op.execute_json({"name": "$x"}, stack) == Literal( + "v", datatype=XSD.string + ) From 708386e9cf75d5cc7fbe01b0bf8fe452538bd839 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Sun, 12 Jul 2026 17:44:41 +0300 Subject: [PATCH 03/11] =?UTF-8?q?Conform=20string=20operations=20to=20SPAR?= =?UTF-8?q?QL=201.1=20/=20XPath=20F&O=20=E2=80=94=20no=20divergences?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The spec no longer documents divergences from W3C functions; operations named after SPARQL/XPath functions follow those definitions by normative reference, signatures included. Simple literals are materialized as plain rdflib literals (no datatype) exactly as rdflib's own SPARQL engine does. - Str: per `simple literal STR(literal ltrl)` / `STR(IRI rsrc)` — returns the lexical form / codepoint representation as a simple literal; language tags are no longer carried over; BNode is a type error. - Replace: per REPLACE()/fn:replace — adds the optional flags argument (s m i x q), XPath replacement syntax ($N group references, \$ and \\ escapes), err:FORX000* conditions as ValueError (invalid flags/pattern/ replacement, zero-length-matching pattern), result kind follows the first argument, and pattern/replacement/flags must be simple literals. - Concat: per CONCAT() result-kind rules — all xsd:string inputs yield xsd:string, a shared language tag is carried, anything else yields a simple literal. - EncodeForURI, STRUUID: return simple literals per their signatures. - SELECT/CONSTRUCT/DESCRIBE accept simple-literal queries via a new Operation.is_string_literal predicate (RDF 1.1 equivalence), so Str output composes into query arguments. tests/SPEC_GAPS.md records the one honest implementation gap: XPath-only regex constructs unsupported by Python re surface as ValueError. New test_concat.py; Str/Replace suites rewritten against the W3C behavior. 238 passed, 8 skipped. Co-Authored-By: Claude Fable 5 --- formal-semantics.md | 84 ++++++--- src/web_algebra/operation.py | 10 + .../operations/sparql/construct.py | 4 +- src/web_algebra/operations/sparql/describe.py | 4 +- src/web_algebra/operations/sparql/select.py | 2 +- src/web_algebra/operations/str.py | 29 ++- src/web_algebra/operations/string/concat.py | 22 ++- .../operations/string/encode_for_uri.py | 3 +- src/web_algebra/operations/string/replace.py | 178 ++++++++++++++---- src/web_algebra/operations/struuid.py | 4 +- tests/SPEC_GAPS.md | 30 ++- tests/unit/test_concat.py | 81 ++++++++ tests/unit/test_encode_for_uri.py | 6 + tests/unit/test_execute.py | 7 +- tests/unit/test_for_each.py | 9 +- tests/unit/test_replace.py | 105 +++++++++-- tests/unit/test_str.py | 88 ++++----- tests/unit/test_struuid.py | 7 +- 18 files changed, 490 insertions(+), 183 deletions(-) create mode 100644 tests/unit/test_concat.py diff --git a/formal-semantics.md b/formal-semantics.md index 30662db..fdfa1c7 100644 --- a/formal-semantics.md +++ b/formal-semantics.md @@ -286,6 +286,7 @@ Failures raise Python exceptions per this table (normative): | unknown variable in `$name` lookup | `ValueError` | | context lookup miss, or no context established | `ValueError` | | `Filter` position < 1 or > length | `ValueError` | +| regular-expression errors in `Replace` — invalid pattern or flags, zero-length-matching pattern, invalid replacement (XPath `err:FORX000*`) | `ValueError` | | unknown `type` in SPARQL JSON term form | `ValueError` | | blank node where SPARQL syntax forbids it (`Values` data) | `ValueError` | | HTTP/SPARQL transport failure | `urllib.error.HTTPError` / `URLError`, unwrapped | @@ -377,43 +378,65 @@ JSON: operation⟨quoted⟩: an operation-call form ### 4.2 String and term operations +Operations in this section are named after SPARQL 1.1 / XPath F&O functions +and follow those definitions exactly — including their signatures (e.g. +`simple literal STR(literal ltrl)` / `simple literal STR(IRI rsrc)`); the +summaries below are paraphrases, and where they fall short the W3C text is +normative. A SPARQL *simple literal* is materialized as an rdflib `Literal` +with no datatype and no language tag — exactly as rdflib's own SPARQL engine +does — and under RDF 1.1 denotes the same value as the corresponding +`xsd:string` literal. + String-compatibility rule: where an operation is documented as accepting a *string-compatible* Literal it accepts `xsd:string` literals, language-tagged literals, and plain literals; any other Term raises `TypeError` (use `Str` to cast explicitly). -**Str** — cast a Term to a string literal. +**Str** — the lexical form of a Term, per SPARQL 1.1 `STR()`: +`simple literal STR(literal ltrl)` / `simple literal STR(IRI rsrc)`. ``` -Abstract: Term → Literal +Abstract: (URI + Literal) → Literal Python: def execute(self, term: Node) -> Literal -JSON: input: Term +JSON: input: URI + Literal ``` -- String-compatible literals pass through unchanged (a language tag is - *preserved*). Any other Term yields `Literal` of its lexical/IRI form with - datatype `xsd:string`. Non-Terms raise `TypeError`. -- *Known divergence from SPARQL:* SPARQL `STR()` returns a simple literal and - drops language tags; Web Algebra `Str` returns `xsd:string` and preserves - tags on passthrough. +- Returns the lexical form of a Literal, or the codepoint representation of a + URI, as a **simple literal**. As in SPARQL, the language tag is **not** + carried over. A `BNode` raises `TypeError` (a SPARQL type error), as does + any non-Term. -**Concat** — concatenate string literals. +**Concat** — per SPARQL 1.1 `CONCAT()`: +`string literal CONCAT(string literal ltrl1 ... string literal ltrln)`. ``` Abstract: Sequence Literal → Literal Python: def execute(self, inputs: List[Literal]) -> Literal JSON: inputs: array of string-compatible Literal forms ``` -- Result datatype `xsd:string`. - -**Replace** — regular-expression replacement, in the spirit of SPARQL -`REPLACE()`. -``` -Abstract: Literal × Literal × Literal → Literal -Python: def execute(self, input_str, pattern, replacement) -> Literal -JSON: input · pattern · replacement: string-compatible Literals -``` -- Result datatype `xsd:string`. All three inputs must be string-compatible. -- *Known divergence:* the pattern dialect is Python `re`, not the XPath/XQuery - regular expressions SPARQL specifies. Patterns using shared syntax behave - identically. +- Result kind per SPARQL: if all inputs are typed `xsd:string`, so is the + result; if all inputs carry the *same* language tag, the result carries it + too; in all other cases (including the empty input sequence) the result is + a simple literal. + +**Replace** — per SPARQL 1.1 `REPLACE()` / XPath `fn:replace`: +`string literal REPLACE(string literal arg, simple literal pattern, +simple literal replacement [, simple literal flags])`. +``` +Abstract: Literal × Literal × Literal × Maybe Literal → Literal +Python: def execute(self, input_str, pattern, replacement, flags=None) -> Literal +JSON: input: string-compatible Literal · pattern · replacement · flags: + language-tag-free string Literals (simple literals) +``` +- Per the signature, `pattern`, `replacement` and `flags` are simple + literals: a language-tagged value there raises `TypeError`. `input` may be + any string literal. +- Pattern and `flags` (`s`, `m`, `i`, `x`, `q`) per XPath `fn:replace`. In the + replacement string, `$N` references capture group *N*, `\$` is a literal + dollar, and `\\` is a literal backslash; any other use of `\` or `$` is an + error. +- Per the SPARQL string-function convention, the result is a string literal + of the same kind as `arg` (its datatype and language tag are carried over). +- Errors (`ValueError`, mirroring XPath `err:FORX000*`): invalid flags, an + invalid pattern, a pattern that matches the zero-length string, or an + invalid replacement string. **EncodeForURI** — percent-encode a string for use inside a URI, per SPARQL `ENCODE_FOR_URI` / XPath `fn:encode-for-uri`. @@ -423,7 +446,9 @@ Python: def execute(self, input_str: Literal) -> Literal JSON: input: string-compatible Literal ``` - Every character except the RFC 3986 unreserved set - (`A–Z a–z 0–9 - . _ ~`) is percent-encoded (UTF-8). Result `xsd:string`. + (`A–Z a–z 0–9 - . _ ~`) is percent-encoded (UTF-8). Per the signature + `simple literal ENCODE_FOR_URI(string literal ltrl)`, the result is a + simple literal. **STRUUID** — fresh UUID string, per SPARQL `STRUUID()`. Non-deterministic. ``` @@ -431,8 +456,9 @@ Abstract: () → Literal Python: def execute(self) -> Literal JSON: (no arguments) ``` -- An RFC 4122 version-4 UUID in lowercase hyphenated form, datatype - `xsd:string`. Successive invocations differ. +- A simple literal (per the signature `simple literal STRUUID()`) holding an + RFC 4122 version-4 UUID in lowercase hyphenated form. Successive + invocations differ. **URI** — cast a Term to a URI, like SPARQL `URI()`/`IRI()`. ``` @@ -459,7 +485,7 @@ JSON: base: URI · relative: string-compatible Literal ``` Abstract: URI × Literal → Result Python: def execute(self, endpoint: URIRef, query: Literal) -> Result -JSON: endpoint: URI · query: Literal (xsd:string) +JSON: endpoint: URI · query: string Literal (simple or xsd:string) ``` - Types are validated before any network I/O. @@ -467,14 +493,14 @@ JSON: endpoint: URI · query: Literal (xsd:string) ``` Abstract: URI × Literal → Graph Python: def execute(self, endpoint: URIRef, query: Literal) -> Graph -JSON: endpoint: URI · query: Literal (xsd:string) +JSON: endpoint: URI · query: string Literal (simple or xsd:string) ``` **DESCRIBE** — execute a SPARQL DESCRIBE query. *Query* effect. ``` Abstract: URI × Literal → Graph Python: def execute(self, endpoint: URIRef, query: Literal) -> Graph -JSON: endpoint: URI · query: Literal (xsd:string) +JSON: endpoint: URI · query: string Literal (simple or xsd:string) ``` **Substitute** — textually substitute one SPARQL variable with a Term. diff --git a/src/web_algebra/operation.py b/src/web_algebra/operation.py index 5635ed5..6498408 100644 --- a/src/web_algebra/operation.py +++ b/src/web_algebra/operation.py @@ -331,6 +331,16 @@ def plain_to_rdflib(value: Any) -> Node: else: return Literal(str(value), datatype=XSD.string) + @staticmethod + def is_string_literal(term: Any) -> bool: + """True for a language-tag-free string literal — a SPARQL simple + literal or its RDF 1.1 equivalent, an xsd:string literal.""" + return ( + isinstance(term, Literal) + and term.language is None + and (term.datatype is None or term.datatype == XSD.string) + ) + @staticmethod def to_string_literal(term: Node) -> Literal: """Convert Literal terms to string-compatible literals, following SPARQL semantics""" diff --git a/src/web_algebra/operations/sparql/construct.py b/src/web_algebra/operations/sparql/construct.py index e4c1100..dc344cb 100644 --- a/src/web_algebra/operations/sparql/construct.py +++ b/src/web_algebra/operations/sparql/construct.py @@ -41,7 +41,7 @@ def execute(self, endpoint: URIRef, query: Literal) -> Graph: raise TypeError( f"CONSTRUCT operation expects endpoint to be URIRef, got {type(endpoint)}" ) - if not isinstance(query, Literal) or query.datatype != XSD.string: + if not Operation.is_string_literal(query): raise TypeError( f"CONSTRUCT operation expects query to be string Literal, got {type(query)}" ) @@ -74,7 +74,7 @@ def execute_json(self, arguments: dict, variable_stack: list = None) -> Graph: query_data = Operation.process_json( self.settings, arguments["query"], self.context, variable_stack ) - if not isinstance(query_data, Literal) or query_data.datatype != XSD.string: + if not Operation.is_string_literal(query_data): raise TypeError( f"CONSTRUCT operation expects 'query' to be string Literal, got {type(query_data)}" ) diff --git a/src/web_algebra/operations/sparql/describe.py b/src/web_algebra/operations/sparql/describe.py index c9a6930..d43f745 100644 --- a/src/web_algebra/operations/sparql/describe.py +++ b/src/web_algebra/operations/sparql/describe.py @@ -41,7 +41,7 @@ def execute(self, endpoint: URIRef, query: Literal) -> Graph: raise TypeError( f"DESCRIBE operation expects endpoint to be URIRef, got {type(endpoint)}" ) - if not isinstance(query, Literal) or query.datatype != XSD.string: + if not Operation.is_string_literal(query): raise TypeError( f"DESCRIBE operation expects query to be string Literal, got {type(query)}" ) @@ -74,7 +74,7 @@ def execute_json(self, arguments: dict, variable_stack: list = None) -> Graph: query_data = Operation.process_json( self.settings, arguments["query"], self.context, variable_stack ) - if not isinstance(query_data, Literal) or query_data.datatype != XSD.string: + if not Operation.is_string_literal(query_data): raise TypeError( f"DESCRIBE operation expects 'query' to be string Literal, got {type(query_data)}" ) diff --git a/src/web_algebra/operations/sparql/select.py b/src/web_algebra/operations/sparql/select.py index 40e9b39..45e3bff 100644 --- a/src/web_algebra/operations/sparql/select.py +++ b/src/web_algebra/operations/sparql/select.py @@ -82,7 +82,7 @@ def execute_json(self, arguments: dict, variable_stack: list = None) -> Result: query_data = Operation.process_json( self.settings, arguments["query"], self.context, variable_stack ) - if not isinstance(query_data, Literal) or query_data.datatype != XSD.string: + if not Operation.is_string_literal(query_data): raise TypeError( f"SELECT operation expects 'query' to be string Literal, got {type(query_data)}" ) diff --git a/src/web_algebra/operations/str.py b/src/web_algebra/operations/str.py index ce90270..9863251 100644 --- a/src/web_algebra/operations/str.py +++ b/src/web_algebra/operations/str.py @@ -1,19 +1,19 @@ from typing import Any from rdflib.term import Node -from rdflib import BNode, Literal, URIRef -from rdflib.namespace import XSD +from rdflib import Literal, URIRef from mcp import types from web_algebra.operation import Operation class Str(Operation): """ - Converts any RDF term to a string literal + Returns the lexical form of a Literal or the codepoint representation of + a URI as a simple literal, per SPARQL 1.1 STR(). """ @classmethod def description(cls) -> str: - return "Converts any RDF term to a string literal" + return "Returns the lexical form of a Literal or the string representation of a URI, per SPARQL's STR() function. The language tag, if any, is not carried over." @classmethod def inputSchema(cls) -> dict: @@ -27,23 +27,16 @@ def inputSchema(cls) -> dict: def execute(self, term: Node) -> Literal: """Pure function: RDFLib term → string literal""" - # Strict Type Checking: spec defines Str as Term → Literal where Term = URI + Literal + BNode. - if not isinstance(term, (URIRef, Literal, BNode)): + # SPARQL 1.1 STR() accepts a literal or an IRI; a blank node is a + # type error (formal-semantics.md §4.2). + if not isinstance(term, (URIRef, Literal)): raise TypeError( - f"Str expects a Term (URIRef, Literal, BNode), got {type(term).__name__}" + f"Str expects a URI or Literal (SPARQL STR), got {type(term).__name__}" ) - # Check if already string-compatible - if isinstance(term, Literal): - if term.datatype == XSD.string: - return term # Already xsd:string, return as-is - elif term.language is not None: - return term # rdf:langString (datatype=None, language=xx), return as-is (compatible) - elif term.datatype is None and term.language is None: - # Plain literal without datatype or language - treat as string - return term - # Convert any other term to xsd:string - return Literal(str(term), datatype=XSD.string) + # The lexical form / codepoint representation as a simple literal + # (no datatype, no language tag) — as rdflib's SPARQL engine does. + return Literal(str(term)) def execute_json( self, arguments: dict, variable_stack: list = None diff --git a/src/web_algebra/operations/string/concat.py b/src/web_algebra/operations/string/concat.py index 586f4ed..607b976 100644 --- a/src/web_algebra/operations/string/concat.py +++ b/src/web_algebra/operations/string/concat.py @@ -8,12 +8,12 @@ class Concat(Operation, MCPTool): """ - Concatenates multiple string values into a single string. + Concatenates string literals, per SPARQL 1.1 CONCAT(). """ @classmethod def description(cls) -> str: - return "Concatenates multiple string values into a single string." + return "Concatenates multiple string values into a single string, per SPARQL's CONCAT() function: if all inputs carry the same language tag the result carries it too, otherwise the result is a simple (xsd:string) literal." @classmethod def inputSchema(cls) -> dict: @@ -33,15 +33,23 @@ def execute(self, inputs: List[Literal]) -> Literal: """Pure function: concatenate literals with RDFLib terms""" if not isinstance(inputs, list): raise TypeError(f"Concat.execute expects inputs to be list, got {type(inputs)}") - - # Convert all inputs to strings and concatenate - result_str = "" + for input_literal in inputs: if not isinstance(input_literal, Literal): raise TypeError(f"Concat.execute expects all inputs to be Literal, got {type(input_literal)}") - result_str += str(input_literal) - return Literal(result_str, datatype=XSD.string) + result_str = "".join(str(input_literal) for input_literal in inputs) + + # SPARQL 1.1 CONCAT() result kind (formal-semantics.md §4.2): all + # inputs typed xsd:string → xsd:string; all inputs carrying the same + # language tag → that tag; anything else → simple literal. + datatypes = {input_literal.datatype for input_literal in inputs} + languages = {input_literal.language for input_literal in inputs} + if inputs and datatypes == {XSD.string}: + return Literal(result_str, datatype=XSD.string) + if inputs and len(languages) == 1 and None not in languages: + return Literal(result_str, lang=next(iter(languages))) + return Literal(result_str) def execute_json(self, arguments: dict, variable_stack: list = None) -> Literal: """JSON execution: process arguments and call pure function""" diff --git a/src/web_algebra/operations/string/encode_for_uri.py b/src/web_algebra/operations/string/encode_for_uri.py index 2e9adb2..1a2bde7 100644 --- a/src/web_algebra/operations/string/encode_for_uri.py +++ b/src/web_algebra/operations/string/encode_for_uri.py @@ -44,7 +44,8 @@ def execute(self, input_str: Literal) -> Literal: encoded_value = quote(input_value, safe="") # No safe characters logging.info("Encoded URI: %s", encoded_value) - return Literal(encoded_value, datatype=XSD.string) + # simple literal per `simple literal ENCODE_FOR_URI(string literal)` + return Literal(encoded_value) def execute_json(self, arguments: dict, variable_stack: list = None) -> Literal: """JSON execution: process arguments and call pure function""" diff --git a/src/web_algebra/operations/string/replace.py b/src/web_algebra/operations/string/replace.py index a61b404..23ebd0e 100644 --- a/src/web_algebra/operations/string/replace.py +++ b/src/web_algebra/operations/string/replace.py @@ -1,4 +1,4 @@ -from typing import Any +from typing import Any, Optional import logging import re from rdflib import Literal @@ -10,14 +10,15 @@ class Replace(Operation, MCPTool): """ - Replaces occurrences of a specified pattern in an input string with a given replacement. - Aligns with SPARQL's REPLACE() function. + Regular-expression replacement, per SPARQL 1.1 REPLACE() / XPath + fn:replace: `string literal REPLACE(string literal arg, simple literal + pattern, simple literal replacement [, simple literal flags])`. """ @classmethod def description(cls) -> str: - return """Replaces occurrences of a specified pattern in an input string with a given replacement. This operation aligns with SPARQL's REPLACE() function, allowing for flexible string manipulation using regular expressions. - + return """Replaces occurrences of a regular-expression pattern in an input string, per SPARQL's REPLACE() / XPath fn:replace. The replacement string may reference capture groups as $1, $2, ...; flags `s`, `m`, `i`, `x`, `q` are supported. The result is a string literal of the same kind (datatype/language tag) as the input. + Note: this function should not be used to build URIs! That should be done using EncodeForURI()/ResolveURI(). """ @@ -35,66 +36,155 @@ def inputSchema(cls) -> dict: }, "pattern": { "type": "string", - "description": "The pattern to be replaced (regular expression)", + "description": "The regular expression to be replaced (XPath fn:replace syntax)", }, "replacement": { "type": "string", - "description": "The replacement value", + "description": "The replacement string; $1, $2, ... reference capture groups", + }, + "flags": { + "type": "string", + "description": "Optional XPath regex flags: any of s, m, i, x, q", }, }, "required": ["input", "pattern", "replacement"], } + # XPath flag → Python re flag (formal-semantics.md §4.2); `q` is handled + # separately (treat pattern and replacement as literal strings). + _FLAG_MAP = { + "i": re.IGNORECASE, + "s": re.DOTALL, + "m": re.MULTILINE, + "x": re.VERBOSE, + } + + @staticmethod + def _is_string_compatible(lit: Any) -> bool: + # SPARQL string literal: xsd:string, rdf:langString, or plain literal + return isinstance(lit, Literal) and ( + lit.datatype == XSD.string + or (lit.datatype is None and lit.language is not None) + or (lit.datatype is None and lit.language is None) + ) + + @staticmethod + def _is_simple(lit: Any) -> bool: + # SPARQL simple literal (xsd:string accepted per RDF 1.1) + return Operation.is_string_literal(lit) + + @staticmethod + def _translate_replacement(replacement: str) -> str: + """Translate an XPath fn:replace replacement string into Python + `re.sub` syntax: `$N` → `\\g`, `\\$` → `$`, `\\\\` → literal + backslash. Any other use of `\\` or `$` is an error (err:FORX0004). + """ + out = [] + i = 0 + n = len(replacement) + while i < n: + ch = replacement[i] + if ch == "\\": + if i + 1 < n and replacement[i + 1] == "\\": + out.append("\\\\") + i += 2 + continue + if i + 1 < n and replacement[i + 1] == "$": + out.append("$") + i += 2 + continue + raise ValueError( + "Replace: invalid escape in replacement string (XPath err:FORX0004)" + ) + if ch == "$": + j = i + 1 + while j < n and replacement[j].isdigit(): + j += 1 + if j == i + 1: + raise ValueError( + "Replace: '$' must be followed by a group number in the replacement string (XPath err:FORX0004)" + ) + out.append(f"\\g<{replacement[i + 1:j]}>") + i = j + continue + out.append(ch) + i += 1 + return "".join(out) + def execute( self, input_str: Literal, pattern: Literal, replacement: Literal, + flags: Optional[Literal] = None, ) -> Literal: """Pure function: replace pattern in string with RDFLib terms""" - - # Following SPARQL semantics: accept both xsd:string and rdf:langString (language-tagged literals) - def is_string_compatible(lit): - return isinstance(lit, Literal) and ( - lit.datatype == XSD.string # xsd:string - or ( - lit.datatype is None - and lit.language is not None - ) # rdf:langString - or ( - lit.datatype is None - and lit.language is None - ) # plain literal - ) - - if not is_string_compatible(input_str): - raise TypeError( - f"Replace operation expects input to be string-compatible Literal, got {type(input_str)} with datatype {getattr(input_str, 'datatype', None)}" - ) - if not is_string_compatible(pattern): - raise TypeError( - f"Replace operation expects pattern to be string-compatible Literal, got {type(pattern)} with datatype {getattr(pattern, 'datatype', None)}" - ) - if not is_string_compatible(replacement): + if not self._is_string_compatible(input_str): raise TypeError( - f"Replace operation expects replacement to be string-compatible Literal, got {type(replacement)} with datatype {getattr(replacement, 'datatype', None)}" + f"Replace expects input to be a string literal, got {type(input_str)} with datatype {getattr(input_str, 'datatype', None)}" ) + # Per the REPLACE signature, pattern/replacement/flags are simple + # literals — a language-tagged value is a type error. + for name, lit in (("pattern", pattern), ("replacement", replacement)): + if not self._is_simple(lit): + raise TypeError( + f"Replace expects {name} to be a simple literal, got {lit!r}" + ) + if flags is not None and not self._is_simple(flags): + raise TypeError(f"Replace expects flags to be a simple literal, got {flags!r}") input_value = str(input_str) pattern_value = str(pattern) replacement_value = str(replacement) + flags_value = str(flags) if flags is not None else "" + + re_flags = 0 + literal_mode = False + for flag_char in flags_value: + if flag_char == "q": + literal_mode = True + elif flag_char in self._FLAG_MAP: + re_flags |= self._FLAG_MAP[flag_char] + else: + raise ValueError( + f"Replace: invalid flag {flag_char!r} (XPath err:FORX0001)" + ) + + if literal_mode: + # q: pattern and replacement are taken literally + pattern_value = re.escape(pattern_value) + replacement_re = replacement_value.replace("\\", "\\\\") + else: + replacement_re = self._translate_replacement(replacement_value) + + try: + compiled = re.compile(pattern_value, re_flags) + except re.error as e: + raise ValueError(f"Replace: invalid regular expression: {e} (XPath err:FORX0002)") from None + if compiled.search(""): + raise ValueError( + "Replace: pattern matches a zero-length string (XPath err:FORX0003)" + ) logging.info( - "Resolving Replace arguments: input=%s, pattern=%s, replacement=%s", + "Resolving Replace arguments: input=%s, pattern=%s, replacement=%s, flags=%s", input_value, pattern_value, replacement_value, + flags_value, ) - formatted_string = re.sub(pattern_value, replacement_value, input_value) + try: + formatted_string = compiled.sub(replacement_re, input_value) + except re.error as e: + raise ValueError(f"Replace: invalid replacement string: {e} (XPath err:FORX0004)") from None logging.info("Formatted result: %s", formatted_string) - return Literal(formatted_string, datatype=XSD.string) + # SPARQL string-function convention: the result is a string literal + # of the same kind as the first argument. + if input_str.language is not None: + return Literal(formatted_string, lang=input_str.language) + return Literal(formatted_string, datatype=input_str.datatype) def execute_json( self, arguments: dict, variable_stack: list = None @@ -118,14 +208,28 @@ def execute_json( ) replacement_literal = self.to_string_literal(replacement_data) - return self.execute(input_literal, pattern_literal, replacement_literal) + flags_literal = None + if "flags" in arguments: + flags_data = Operation.process_json( + self.settings, arguments["flags"], self.context, variable_stack + ) + flags_literal = self.to_string_literal(flags_data) + + return self.execute( + input_literal, pattern_literal, replacement_literal, flags_literal + ) def mcp_run(self, arguments: dict, context: Any = None) -> Any: """MCP execution: plain args → plain results""" input_str = Literal(arguments["input"], datatype=XSD.string) pattern = Literal(arguments["pattern"], datatype=XSD.string) replacement = Literal(arguments["replacement"], datatype=XSD.string) + flags = ( + Literal(arguments["flags"], datatype=XSD.string) + if "flags" in arguments + else None + ) - result = self.execute(input_str, pattern, replacement) + result = self.execute(input_str, pattern, replacement, flags) return [types.TextContent(type="text", text=str(result))] diff --git a/src/web_algebra/operations/struuid.py b/src/web_algebra/operations/struuid.py index 36f4380..358133a 100644 --- a/src/web_algebra/operations/struuid.py +++ b/src/web_algebra/operations/struuid.py @@ -2,7 +2,6 @@ import uuid from typing import Any from rdflib import Literal -from rdflib.namespace import XSD from mcp import types from web_algebra.operation import Operation from web_algebra.mcp_tool import MCPTool @@ -27,7 +26,8 @@ def execute(self) -> Literal: generated_uuid = str(uuid.uuid4()) logging.info("Generated UUID: %s", generated_uuid) - return Literal(generated_uuid, datatype=XSD.string) + # simple literal per `simple literal STRUUID()` + return Literal(generated_uuid) def execute_json(self, arguments: dict, variable_stack: list = None) -> Literal: """JSON execution: process arguments and call pure function""" diff --git a/tests/SPEC_GAPS.md b/tests/SPEC_GAPS.md index 497dc63..f475b5b 100644 --- a/tests/SPEC_GAPS.md +++ b/tests/SPEC_GAPS.md @@ -24,6 +24,12 @@ live. - **Live-service behavior** — §3.7 pins transport failures to `urllib.error.HTTPError`/`URLError` propagating unwrapped, but content negotiation, redirects (beyond 308), timeouts, and retry policy remain unspecified. +- **XPath regex dialect coverage** — `Replace` compiles patterns with Python's `re`. + The common syntax is shared with XPath regular expressions, but XPath-only + constructs (`\p{...}` category escapes, `\i`/`\c`, character-class subtraction + `[a-z-[aeiou]]`) are not supported and surface as `ValueError` (invalid pattern). + This is an implementation gap against the normative fn:replace behavior, not + sanctioned spec behavior. ## Resolved in the 2026-07 spec revision @@ -31,16 +37,26 @@ Each item below is now normative in `formal-semantics.md` (section in parenthese and the corresponding tests are un-skipped. - **Catalog omissions**: `Concat` (§4.2) and `ExtractOntology` (§4.6) added. -- **Str result datatype** (§4.2): string-compatible literals pass through unchanged - (language tags preserved — documented divergence from SPARQL `STR()`); other Terms - → `xsd:string` of the lexical/IRI form. +- **W3C conformance rule** (§4.2 preamble): operations named after SPARQL 1.1 / + XPath functions follow those definitions *by normative reference*, signatures + included; simple literals are materialized as plain rdflib literals (no + datatype), exactly as rdflib's own SPARQL engine does. +- **Str** (§4.2): per `simple literal STR(literal ltrl)` / `simple literal STR(IRI + rsrc)` — lexical form / codepoint representation as a simple literal; language + tags are not carried over; BNode → `TypeError` (SPARQL type error). +- **Concat** (§4.2): per SPARQL `CONCAT()` result-kind rules — all `xsd:string` → + `xsd:string`; all same language tag → that tag; otherwise simple literal. +- **Replace** (§4.2): per SPARQL `REPLACE()` / `fn:replace` — optional `flags` + argument (`s m i x q`), `$N` capture-group references with `\$`/`\\` escapes, + `err:FORX000*` conditions → `ValueError`, result kind follows the first argument, + and `pattern`/`replacement`/`flags` must be simple literals. - **URI on BNode / invalid lexical form** (§4.2): BNode → `TypeError`; lexical forms are not validated against RFC 3986. - **EncodeForURI character set** (§4.2): percent-encode everything outside the - RFC 3986 unreserved set (`A–Z a–z 0–9 - . _ ~`), per XPath `fn:encode-for-uri`. -- **Replace pattern dialect** (§4.2): regular expression, Python `re` dialect - (documented divergence from SPARQL's XPath/XQuery regex). -- **STRUUID format** (§4.2): RFC 4122 version-4, lowercase hyphenated, `xsd:string`. + RFC 3986 unreserved set (`A–Z a–z 0–9 - . _ ~`), per XPath `fn:encode-for-uri`; + result is a simple literal. +- **STRUUID format** (§4.2): simple literal; RFC 4122 version-4, lowercase + hyphenated. - **Substitute** (§4.3): matches `?var` and `$var` at token boundaries; URI → ``, Literal → quoted with lang/datatype; BNode → `TypeError`; substitution is textual and documented as not parse-aware. diff --git a/tests/unit/test_concat.py b/tests/unit/test_concat.py new file mode 100644 index 0000000..b98d4ea --- /dev/null +++ b/tests/unit/test_concat.py @@ -0,0 +1,81 @@ +"""Spec: formal-semantics.md §4.2 "Concat — per SPARQL 1.1 CONCAT()": +string literal CONCAT(string literal ltrl1 ... string literal ltrln) +- All inputs typed xsd:string → xsd:string result. +- All inputs carrying the same language tag → result carries it too. +- All other cases (including no inputs) → simple literal. +""" + +from __future__ import annotations + +import pytest +from rdflib import Literal +from rdflib.namespace import XSD + +from web_algebra.operation import Operation + + +class TestConcatPure: + def test_concatenates_lexical_forms(self, settings): + op = Operation.get("Concat")(settings=settings) + result = op.execute([Literal("foo"), Literal("bar")]) + assert str(result) == "foobar" + + def test_all_xsd_string_yields_xsd_string(self, settings): + # §4.2: all inputs typed xsd:string → xsd:string + op = Operation.get("Concat")(settings=settings) + result = op.execute( + [Literal("foo", datatype=XSD.string), Literal("bar", datatype=XSD.string)] + ) + assert result.datatype == XSD.string + + def test_same_language_tag_is_carried(self, settings): + # §4.2: CONCAT("foo"@en, "bar"@en) → "foobar"@en + op = Operation.get("Concat")(settings=settings) + result = op.execute([Literal("foo", lang="en"), Literal("bar", lang="en")]) + assert result.language == "en" + assert str(result) == "foobar" + + def test_mixed_kinds_yield_simple_literal(self, settings): + # §4.2: in all other cases the result is a simple literal + op = Operation.get("Concat")(settings=settings) + result = op.execute([Literal("foo", lang="en"), Literal("bar")]) + assert result.datatype is None and result.language is None + mixed_langs = op.execute([Literal("foo", lang="en"), Literal("bar", lang="de")]) + assert mixed_langs.datatype is None and mixed_langs.language is None + + def test_empty_inputs_yield_empty_simple_literal(self, settings): + op = Operation.get("Concat")(settings=settings) + result = op.execute([]) + assert str(result) == "" + assert result.datatype is None and result.language is None + + def test_non_literal_input_raises_type_error(self, settings): + # §3.7 strict typing + op = Operation.get("Concat")(settings=settings) + with pytest.raises(TypeError): + op.execute([Literal("foo"), 42]) + with pytest.raises(TypeError): + op.execute("not-a-list") + + +class TestConcatJson: + def test_json_dispatch(self, settings): + # §4.2 JSON: inputs: array of string-compatible Literal forms. + # JSON string scalars coerce to xsd:string (§2.2), so the result is + # xsd:string per the all-xsd:string rule. + op = Operation.get("Concat")(settings=settings) + result = op.execute_json({"inputs": ["http://ex/", "x"]}) + assert str(result) == "http://ex/x" + assert result.datatype == XSD.string + + def test_nested_operations_in_inputs(self, settings): + op = Operation.get("Concat")(settings=settings) + result = op.execute_json( + { + "inputs": [ + {"@op": "Str", "args": {"input": {"@id": "http://ex/a"}}}, + "-b", + ] + } + ) + assert str(result) == "http://ex/a-b" diff --git a/tests/unit/test_encode_for_uri.py b/tests/unit/test_encode_for_uri.py index 3fb1f2d..a3175b8 100644 --- a/tests/unit/test_encode_for_uri.py +++ b/tests/unit/test_encode_for_uri.py @@ -37,6 +37,12 @@ def test_rfc3986_unreserved_set_passes_through(self, settings): result = op.execute(Literal("AZaz09-._~")) assert str(result) == "AZaz09-._~" + def test_result_is_simple_literal(self, settings): + # §4.2: simple literal per `simple literal ENCODE_FOR_URI(string literal)` + op = Operation.get("EncodeForURI")(settings=settings) + result = op.execute(Literal("hello world", lang="en")) + assert result.datatype is None and result.language is None + def test_reserved_characters_are_encoded(self, settings): # §4.2: reserved characters like / : * ' are encoded (UTF-8) op = Operation.get("EncodeForURI")(settings=settings) diff --git a/tests/unit/test_execute.py b/tests/unit/test_execute.py index f0a3479..93cf240 100644 --- a/tests/unit/test_execute.py +++ b/tests/unit/test_execute.py @@ -9,7 +9,6 @@ import pytest from rdflib import Literal -from rdflib.namespace import XSD from web_algebra.operation import Operation @@ -18,8 +17,8 @@ class TestExecutePure: def test_evaluates_operation_form(self, settings): op = Operation.get("Execute")(settings=settings) result = op.execute({"@op": "Str", "args": {"input": "hi"}}) - # §2.2: the scalar "hi" coerces to an xsd:string Literal - assert result == Literal("hi", datatype=XSD.string) + # §4.2: Str returns a simple literal + assert result == Literal("hi") def test_non_operation_form_raises_type_error(self, settings): # §4.1: the operand must be an operation-call form @@ -37,7 +36,7 @@ def test_json_dispatch(self, settings): result = op.execute_json( {"operation": {"@op": "Str", "args": {"input": "hi"}}} ) - assert result == Literal("hi", datatype=XSD.string) + assert result == Literal("hi") def test_operand_sees_current_environment(self, settings): # §4.1/§3.3: the quoted operand evaluates in the current variable diff --git a/tests/unit/test_for_each.py b/tests/unit/test_for_each.py index 90d3262..e6312ce 100644 --- a/tests/unit/test_for_each.py +++ b/tests/unit/test_for_each.py @@ -13,7 +13,6 @@ import pytest from rdflib import Literal -from rdflib.namespace import XSD from web_algebra.json_result import JSONResult from web_algebra.operation import Operation @@ -88,10 +87,8 @@ def test_sequence_results_stay_nested(self, settings): }, } ) - # §2.2: scalars coerce to xsd:string Literals - assert result == [ - [Literal("a", datatype=XSD.string), Literal("b", datatype=XSD.string)] - ] + # §4.2: Str returns simple literals + assert result == [[Literal("a"), Literal("b")]] def test_operation_array_yields_last_non_unit(self, settings): # §4.1: operation arrays evaluate in order within the iteration's @@ -109,7 +106,7 @@ def test_operation_array_yields_last_non_unit(self, settings): ], } ) - assert result == [Literal("a", datatype=XSD.string)] + assert result == [Literal("a")] def test_iteration_scope_does_not_leak(self, settings): # §3.4: each iteration runs in a fresh scope — bindings made inside diff --git a/tests/unit/test_replace.py b/tests/unit/test_replace.py index 8672555..38a1b75 100644 --- a/tests/unit/test_replace.py +++ b/tests/unit/test_replace.py @@ -1,29 +1,103 @@ -"""Spec: formal-semantics.md "Replace - Replace patterns in strings using regex" -Abstract: Literal × Literal × Literal → Literal -Python: def execute(self, input_str: rdflib.Literal, pattern: rdflib.Literal, - replacement: rdflib.Literal) -> rdflib.Literal -Plus Strict Type Checking property. +"""Spec: formal-semantics.md §4.2 "Replace — per SPARQL 1.1 REPLACE() / +XPath fn:replace": string literal REPLACE(string literal arg, simple literal +pattern, simple literal replacement [, simple literal flags]) +- Replacement string: $N = capture group, \\$ = literal dollar, \\\\ = literal + backslash; other uses of \\ or $ are errors. +- Flags: s, m, i, x, q; invalid flags/pattern/zero-length-matching pattern/ + invalid replacement → ValueError (XPath err:FORX000*). +- pattern/replacement/flags must be simple literals (language tag → TypeError). +- The result is a string literal of the same kind as arg. """ from __future__ import annotations import pytest from rdflib import Literal, URIRef +from rdflib.namespace import XSD from web_algebra.operation import Operation class TestReplacePure: def test_basic_replace(self, settings): - # Pattern that's identical as literal or regex — robust against the regex/literal ambiguity op = Operation.get("Replace")(settings=settings) result = op.execute(Literal("Hello World"), Literal("World"), Literal("Universe")) assert isinstance(result, Literal) assert str(result) == "Hello Universe" + def test_pattern_is_a_regular_expression(self, settings): + # §4.2: XPath fn:replace pattern semantics + op = Operation.get("Replace")(settings=settings) + result = op.execute(Literal("a1b2c3"), Literal("[0-9]"), Literal("#")) + assert str(result) == "a#b#c#" + + def test_group_reference_with_dollar(self, settings): + # §4.2: $N references capture group N — fn:replace example: + # replace("abracadabra", "a(.)", "a$1$1") = "abbraccaddabbra" + op = Operation.get("Replace")(settings=settings) + result = op.execute( + Literal("abracadabra"), Literal("a(.)"), Literal("a$1$1") + ) + assert str(result) == "abbraccaddabbra" + + def test_escaped_dollar_is_literal(self, settings): + # §4.2: \$ is a literal dollar in the replacement + op = Operation.get("Replace")(settings=settings) + result = op.execute(Literal("price"), Literal("price"), Literal("\\$5")) + assert str(result) == "$5" + + def test_bare_dollar_in_replacement_raises(self, settings): + # §4.2: any other use of $ is an error (err:FORX0004) + op = Operation.get("Replace")(settings=settings) + with pytest.raises(ValueError): + op.execute(Literal("abc"), Literal("b"), Literal("x$y")) + + def test_case_insensitive_flag(self, settings): + # §4.2: flags per fn:replace — i + op = Operation.get("Replace")(settings=settings) + result = op.execute(Literal("ABC"), Literal("b"), Literal("x"), Literal("i")) + assert str(result) == "AxC" + + def test_q_flag_treats_pattern_literally(self, settings): + # §4.2: flags per fn:replace — q + op = Operation.get("Replace")(settings=settings) + result = op.execute(Literal("a.b.c"), Literal("."), Literal("x"), Literal("q")) + assert str(result) == "axbxc" + + def test_invalid_flag_raises_value_error(self, settings): + # §3.7/§4.2: invalid flags → ValueError (err:FORX0001) + op = Operation.get("Replace")(settings=settings) + with pytest.raises(ValueError): + op.execute(Literal("abc"), Literal("b"), Literal("x"), Literal("z")) + + def test_zero_length_matching_pattern_raises(self, settings): + # §3.7/§4.2: pattern matching the zero-length string → ValueError + # (err:FORX0003) + op = Operation.get("Replace")(settings=settings) + with pytest.raises(ValueError): + op.execute(Literal("abc"), Literal("b?"), Literal("x")) + + def test_result_kind_follows_first_argument(self, settings): + # §4.2: the result is a string literal of the same kind as arg + op = Operation.get("Replace")(settings=settings) + typed = op.execute( + Literal("chat", datatype=XSD.string), Literal("ch"), Literal("h") + ) + assert typed.datatype == XSD.string + tagged = op.execute(Literal("chat", lang="en"), Literal("ch"), Literal("h")) + assert tagged.language == "en" + assert str(tagged) == "hat" + simple = op.execute(Literal("chat"), Literal("ch"), Literal("h")) + assert simple.datatype is None and simple.language is None + + def test_lang_tagged_pattern_raises_type_error(self, settings): + # §4.2: pattern must be a simple literal per the REPLACE signature + op = Operation.get("Replace")(settings=settings) + with pytest.raises(TypeError): + op.execute(Literal("chat"), Literal("ch", lang="en"), Literal("h")) + def test_uri_input_raises_type_error(self, settings): - # Strict Type Checking property: TypeError on mismatched input. - # Same as existing fixture tests/fixtures/negative/error-case-type-mismatch-uri-to-string.json + # §3.7 strict typing op = Operation.get("Replace")(settings=settings) with pytest.raises(TypeError): op.execute(URIRef("http://example.org/x"), Literal("x"), Literal("y")) @@ -38,16 +112,10 @@ def test_uri_replacement_raises(self, settings): with pytest.raises(TypeError): op.execute(Literal("Hello"), Literal("e"), URIRef("http://example.org/x")) - def test_pattern_is_a_regular_expression(self, settings): - # §4.2: the pattern is a regular expression (Python re dialect) - op = Operation.get("Replace")(settings=settings) - result = op.execute(Literal("a1b2c3"), Literal("[0-9]"), Literal("#")) - assert str(result) == "a#b#c#" - class TestReplaceJson: def test_basic_via_json(self, settings): - # JSON arg keys from existing fixture tests/fixtures/positive/complex-operation.json + # §4.2 JSON: input · pattern · replacement · flags (optional) op = Operation.get("Replace")(settings=settings) result = op.execute_json( {"input": "Hello World", "pattern": "World", "replacement": "Universe"} @@ -55,6 +123,13 @@ def test_basic_via_json(self, settings): assert isinstance(result, Literal) assert str(result) == "Hello Universe" + def test_flags_via_json(self, settings): + op = Operation.get("Replace")(settings=settings) + result = op.execute_json( + {"input": "ABC", "pattern": "b", "replacement": "x", "flags": "i"} + ) + assert str(result) == "AxC" + def test_uri_input_raises_via_json(self, settings): op = Operation.get("Replace")(settings=settings) with pytest.raises(TypeError): diff --git a/tests/unit/test_str.py b/tests/unit/test_str.py index 05e2306..0875a04 100644 --- a/tests/unit/test_str.py +++ b/tests/unit/test_str.py @@ -1,82 +1,74 @@ -"""Spec: formal-semantics.md §4.2 "Str — cast a Term to a string literal" -Abstract: Term → Literal -- String-compatible literals pass through unchanged (language tag preserved). -- Any other Term yields a Literal of its lexical/IRI form, datatype xsd:string. -- Non-Terms raise TypeError (§3.7 strict typing). +"""Spec: formal-semantics.md §4.2 "Str — the lexical form of a Term, per +SPARQL 1.1 STR()": simple literal STR(literal ltrl) / simple literal STR(IRI rsrc) +- Returns the lexical form / codepoint representation as a simple literal + (no datatype, no language tag). +- The language tag is not carried over. +- BNode raises TypeError (a SPARQL type error), as does any non-Term. """ from __future__ import annotations import pytest from rdflib import BNode, Literal, URIRef -from rdflib.namespace import XSD from web_algebra.operation import Operation +def _is_simple_literal(value) -> bool: + return ( + isinstance(value, Literal) + and value.datatype is None + and value.language is None + ) + + class TestStrPure: - def test_uri_returns_literal(self, settings): + def test_uri_yields_simple_literal_of_codepoints(self, settings): + # §4.2: simple literal STR(IRI rsrc) op = Operation.get("Str")(settings=settings) result = op.execute(URIRef("http://example.org/foo")) - assert isinstance(result, Literal) + assert _is_simple_literal(result) assert str(result) == "http://example.org/foo" - def test_literal_returns_literal(self, settings): + def test_plain_literal_yields_simple_literal(self, settings): op = Operation.get("Str")(settings=settings) result = op.execute(Literal("hello")) - assert isinstance(result, Literal) + assert _is_simple_literal(result) assert str(result) == "hello" - def test_bnode_returns_literal(self, settings): - op = Operation.get("Str")(settings=settings) - bn = BNode("b1") - result = op.execute(bn) - assert isinstance(result, Literal) - assert str(result) == str(bn) - - def test_non_term_raises_type_error(self, settings): - # Strict Type Checking property: "TypeError raised for mismatched input types" - op = Operation.get("Str")(settings=settings) - with pytest.raises(TypeError): - op.execute([1, 2, 3]) - - def test_uri_input_yields_xsd_string(self, settings): - # §4.2: a non-string-compatible Term yields xsd:string of its IRI form + def test_lang_tag_is_not_carried_over(self, settings): + # §4.2: as in SPARQL, STR("hallo"@de) → "hallo" (simple literal) op = Operation.get("Str")(settings=settings) - result = op.execute(URIRef("http://example.org/foo")) - assert result.datatype == XSD.string + result = op.execute(Literal("hallo", lang="de")) + assert _is_simple_literal(result) + assert str(result) == "hallo" - def test_plain_literal_passes_through_unchanged(self, settings): - # §4.2: string-compatible literals pass through unchanged + def test_typed_literal_yields_lexical_form(self, settings): + # §4.2: STR(42) → "42" (simple literal) op = Operation.get("Str")(settings=settings) - term = Literal("hello") - result = op.execute(term) - assert result == term - assert result.datatype is None + result = op.execute(Literal(42)) + assert _is_simple_literal(result) + assert str(result) == "42" - def test_lang_tagged_literal_preserves_tag(self, settings): - # §4.2: language tag is preserved on passthrough (documented - # divergence from SPARQL STR(), which drops it) + def test_bnode_raises_type_error(self, settings): + # §4.2: STR accepts a literal or an IRI; a blank node is a type error op = Operation.get("Str")(settings=settings) - term = Literal("hallo", lang="de") - result = op.execute(term) - assert result == term - assert result.language == "de" + with pytest.raises(TypeError): + op.execute(BNode("b1")) - def test_typed_literal_yields_xsd_string_of_lexical_form(self, settings): - # §4.2: any other Term → xsd:string of its lexical form + def test_non_term_raises_type_error(self, settings): + # §3.7 strict typing op = Operation.get("Str")(settings=settings) - result = op.execute(Literal(42)) - assert result.datatype == XSD.string - assert str(result) == "42" + with pytest.raises(TypeError): + op.execute([1, 2, 3]) class TestStrJson: def test_string_input_via_json(self, settings): - # JSON arg key derived from existing fixture tests/fixtures/positive/simple-operation.json + # §4.2 JSON: input: URI + Literal op = Operation.get("Str")(settings=settings) result = op.execute_json({"input": "hello"}) - assert isinstance(result, Literal) + assert _is_simple_literal(result) assert str(result) == "hello" def test_nested_uri_op(self, settings): @@ -84,5 +76,5 @@ def test_nested_uri_op(self, settings): result = op.execute_json( {"input": {"@op": "URI", "args": {"input": "http://example.org/x"}}} ) - assert isinstance(result, Literal) + assert _is_simple_literal(result) assert str(result) == "http://example.org/x" diff --git a/tests/unit/test_struuid.py b/tests/unit/test_struuid.py index 74da80f..ed52dce 100644 --- a/tests/unit/test_struuid.py +++ b/tests/unit/test_struuid.py @@ -23,14 +23,13 @@ def test_two_calls_differ(self, settings): assert str(a) != str(b) def test_uuid_format(self, settings): - # §4.2: RFC 4122 version-4 UUID, lowercase hyphenated, xsd:string + # §4.2: simple literal per `simple literal STRUUID()`, holding an + # RFC 4122 version-4 UUID in lowercase hyphenated form import re - from rdflib.namespace import XSD - op = Operation.get("STRUUID")(settings=settings) result = op.execute() - assert result.datatype == XSD.string + assert result.datatype is None and result.language is None assert re.fullmatch( r"[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}", str(result), From 166a8db38c108e44ab9197fd2e3ff8c412301b64 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Sun, 12 Jul 2026 17:45:54 +0300 Subject: [PATCH 04/11] Sync uv.lock with version 1.5.0 The 1.5.0 bump (510f4dd) updated pyproject.toml without regenerating the lockfile's own package entry. Co-Authored-By: Claude Fable 5 --- uv.lock | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/uv.lock b/uv.lock index 72cdf49..7624ac1 100644 --- a/uv.lock +++ b/uv.lock @@ -888,7 +888,7 @@ wheels = [ [[package]] name = "web-algebra" -version = "1.4.0" +version = "1.5.0" source = { editable = "." } dependencies = [ { name = "mcp", extra = ["cli"] }, From 291804bcbf00b02dd798e07f766385d9ece9a932 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Sun, 12 Jul 2026 17:47:59 +0300 Subject: [PATCH 05/11] Remove the document envelope Drop the optional {"@web-algebra": ..., "program": [...]} prolog: a document is simply a single form or a program array. Removes Operation.unwrap_document, the main.py unwrap step, the spec section and error-table row, the system-prompt mention, and the envelope tests. Co-Authored-By: Claude Fable 5 --- formal-semantics.md | 21 +-------------------- prompts/system.md | 1 - src/web_algebra/main.py | 3 +-- src/web_algebra/operation.py | 18 ------------------ tests/unit/test_document.py | 32 ++------------------------------ 5 files changed, 4 insertions(+), 71 deletions(-) diff --git a/formal-semantics.md b/formal-semantics.md index fdfa1c7..472e80b 100644 --- a/formal-semantics.md +++ b/formal-semantics.md @@ -52,29 +52,11 @@ Environment = list[dict[str, Any]] # the "variable stack" ### 2.1 Documents -A Web Algebra document is a JSON document in one of three shapes: +A Web Algebra document is a JSON document in one of two shapes: 1. **A single form** — most commonly an operation call object. 2. **A program** — a JSON array of forms, evaluated in order (§3.2, *sequence form*). -3. **An envelope** — a JSON object whose `@web-algebra` member identifies the - dialect version and whose `program` member holds a form or program: - -```json -{ - "@web-algebra": "1", - "name": "united-kingdom-cities", - "description": "Create a container and load UK city data into it", - "program": [ ... ] -} -``` - -The envelope is optional; the bare forms remain valid. `@web-algebra` is the -version of this specification the document targets (currently `"1"`). `name` -and `description` are optional and informative. Consumers ignore unknown -envelope members (forward compatibility). An envelope without a `program` -member is invalid (`ValueError`). The envelope is recognized at the document -top level only; it is not a form and cannot be nested. ### 2.2 Forms @@ -279,7 +261,6 @@ Failures raise Python exceptions per this table (normative): | Condition | Exception | |-----------|-----------| | unknown operation name in `@op` | `ValueError` | -| envelope without `program` | `ValueError` | | `null` form | `TypeError` | | missing required argument key | `KeyError` | | argument or operand of the wrong type (any layer) | `TypeError` | diff --git a/prompts/system.md b/prompts/system.md index 535d7b5..46104e5 100644 --- a/prompts/system.md +++ b/prompts/system.md @@ -13,7 +13,6 @@ Your output must be a **JSON-formatted structure** of operation calls, where **o - **ForEach supports executing multiple operations sequentially** when provided with a list of operations. Each operation in the list is executed for every row in the table before moving to the next row. - **Where an operation returns or expects RDF data, it is handled internally as an `rdflib.Graph`, but is represented as JSON-LD in the JSON structure.** - **SPARQL tabular data** (e.g., from `SELECT`) can be provided inline as a list of bindings, while **RDF Graph data** (e.g., from `GET`, `CONSTRUCT`, or merges) can be provided inline as JSON-LD objects. -- **A document may optionally be wrapped in an envelope** `{"@web-algebra": "1", "name": "...", "description": "...", "program": [...]}`; the bare operation object or array remains valid. ## Example JSON Output diff --git a/src/web_algebra/main.py b/src/web_algebra/main.py index 14b79f2..7cca750 100644 --- a/src/web_algebra/main.py +++ b/src/web_algebra/main.py @@ -65,8 +65,7 @@ def main(settings: BaseSettings, json_data: Optional[str]): with open(json_data) as json_file: json_input = json.load(json_file) - # Unwrap the optional document envelope, then execute the JSON input - json_input = Operation.unwrap_document(json_input) + # Execute the JSON input result = Operation.process_json(settings, json_input) # Serialize final result for output diff --git a/src/web_algebra/operation.py b/src/web_algebra/operation.py index 6498408..3c0e882 100644 --- a/src/web_algebra/operation.py +++ b/src/web_algebra/operation.py @@ -75,24 +75,6 @@ def list_operations(cls) -> List[Type["Operation"]]: def get(cls, name: str) -> Optional[Type["Operation"]]: return cls.registry.get(name) - @staticmethod - def unwrap_document(json_data: Any) -> Any: - """Unwrap an optional document envelope (formal-semantics.md §2.1). - - An envelope is a top-level object whose `@web-algebra` member names - the dialect version and whose `program` member holds the form(s) to - evaluate; other members (`name`, `description`, ...) are informative - and ignored. Bare documents pass through unchanged. The envelope is - recognized at the document top level only. - """ - if isinstance(json_data, dict) and "@web-algebra" in json_data: - if "program" not in json_data: - raise ValueError( - "Web Algebra envelope is missing its 'program' member" - ) - return json_data["program"] - return json_data - @classmethod def process_json( cls, diff --git a/tests/unit/test_document.py b/tests/unit/test_document.py index 107d286..fab27b1 100644 --- a/tests/unit/test_document.py +++ b/tests/unit/test_document.py @@ -1,7 +1,7 @@ """Spec: formal-semantics.md §2 (Document Model) and §3 (Evaluation Semantics). -Covers the envelope (§2.1), form discrimination and scalar coercion (§2.2), -the URI reference form, and sequence/variable scoping (§3.2, §3.4). +Covers form discrimination and scalar coercion (§2.2), the URI reference +form, and sequence/variable scoping (§3.2, §3.4). """ from __future__ import annotations @@ -13,34 +13,6 @@ from web_algebra.operation import Operation -class TestEnvelope: - def test_envelope_unwraps_program(self): - # §2.1: the `program` member holds the form(s) to evaluate - doc = {"@web-algebra": "1", "program": [{"@op": "STRUUID"}]} - assert Operation.unwrap_document(doc) == [{"@op": "STRUUID"}] - - def test_envelope_without_program_raises(self): - # §2.1/§3.7: an envelope without a `program` member is invalid - with pytest.raises(ValueError): - Operation.unwrap_document({"@web-algebra": "1"}) - - def test_informative_members_are_ignored(self): - # §2.1: `name`/`description` are informative; unknown members ignored - doc = { - "@web-algebra": "1", - "name": "x", - "description": "y", - "future-member": True, - "program": [], - } - assert Operation.unwrap_document(doc) == [] - - def test_bare_document_passes_through(self): - # §2.1: the envelope is optional; bare forms remain valid - bare = [{"@op": "STRUUID"}] - assert Operation.unwrap_document(bare) is bare - - class TestURIReferenceForm: def test_id_string_evaluates_to_uri(self, settings): # §2.2 rule 2: an object whose only member is `@id` evaluates to a URI From d7382dbd3076ac8e580d3b2bb8d75b8fecb39cad Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Sun, 12 Jul 2026 18:03:37 +0300 Subject: [PATCH 06/11] Add the formal evaluation definition and the XSLT focus (Position, Last) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit formal-semantics.md §3.8 now contains the formal definition the document's title promised: abstract syntax, semantic domains (Env, Focus, World), the big-step judgment ρ, φ ⊢ ⟨e, σ⟩ ⇓ ⟨v, ρ′, σ′⟩ with the full rule set (scalar/URI-ref/object/data/sequence/call plus the state-accessing forms Variable, Value, Current, Position, Last, ForEach, Execute), the operator interpretation δ_op with effect classes, and metatheory notes (termination by structural induction, determinism modulo declared non-determinism, why concurrent ForEach iterations are observationally sound). The prose of §§3.2–3.7 is now the restatement; §3.8 wins on conflict. The context (§3.5) generalizes to the focus — the triple (item, position, size), exactly XSLT's dynamic context. ForEach establishes it per iteration; new operations Position and Last expose it per XPath fn:position()/fn:last() as xsd:integer literals; Current and unprefixed Value lookups read the focus item. Outside any focus all three raise ValueError. Co-Authored-By: Claude Fable 5 --- README.md | 2 + formal-semantics.md | 199 ++++++++++++++++++++++--- prompts/system.md | 28 ++++ src/web_algebra/focus.py | 15 ++ src/web_algebra/operations/current.py | 14 +- src/web_algebra/operations/for_each.py | 14 +- src/web_algebra/operations/last.py | 34 +++++ src/web_algebra/operations/position.py | 35 +++++ src/web_algebra/operations/value.py | 7 +- tests/unit/test_current.py | 10 ++ tests/unit/test_last.py | 36 +++++ tests/unit/test_position.py | 47 ++++++ tests/unit/test_value.py | 10 +- 13 files changed, 420 insertions(+), 31 deletions(-) create mode 100644 src/web_algebra/focus.py create mode 100644 src/web_algebra/operations/last.py create mode 100644 src/web_algebra/operations/position.py create mode 100644 tests/unit/test_last.py create mode 100644 tests/unit/test_position.py diff --git a/README.md b/README.md index 2f0bf5b..2210318 100644 --- a/README.md +++ b/README.md @@ -71,6 +71,8 @@ The operations cover read-write Linked Data, SPARQL queries, URI manipulation, a - `Filter` - `Bindings` - `Current` + - `Position` + - `Last` - `Execute` - `Merge` - LinkedDataHub-specific diff --git a/formal-semantics.md b/formal-semantics.md index 472e80b..82c48f8 100644 --- a/formal-semantics.md +++ b/formal-semantics.md @@ -29,6 +29,8 @@ Position = integer ≥ 1 (XSLT-style 1-based index) Unit = no meaningful value (an operation executed for its effect) Context = the current iteration item (see §3.5); one of Binding + Term + Graph + JSON value +Focus = Context × Position × Position — the dynamic context of an + iteration: (item, position, size), exactly XSLT's focus (§3.5) Environment = stack of variable scopes; each scope maps names to values Operation = an unevaluated operation form (see quoting, §3.3) ``` @@ -190,8 +192,8 @@ Lisp sense, and their quoted operands are evaluated under a different regime | Operation | Operand | Evaluation regime | |-----------|---------|-------------------| -| `ForEach` | `operation` | once per iteration item, with that item as context | -| `Execute` | `operation` | once, in the current context and environment | +| `ForEach` | `operation` | once per iteration item, under the focus *(item, position, size)* | +| `Execute` | `operation` | once, under the current focus and environment | All other arguments of all operations are eagerly evaluated. An operation not in this table never sees an unevaluated form. @@ -217,16 +219,25 @@ name: `Variable` binds `name`, `Value` reads `$name`. Because the sigil decides the lookup domain, variable and context lookups never shadow each other. -### 3.5 Context - -The context is the current iteration item. It is established *only* by -`ForEach`, which evaluates its quoted `operation` once per item with that item -as context; nested `ForEach` shadows the outer context for the extent of its -own operand. Outside any iteration there is no context, and operations that -require one (`Current`, context-lookup `Value`) raise `ValueError`. - -- `Current` yields the context item itself. -- `Value` with an unprefixed name looks the name up *in* the context item: +### 3.5 Focus + +The **focus** is the dynamic context of an iteration — the triple +*(item, position, size)*, exactly XSLT's dynamic-context triple. It is +established *only* by `ForEach`, which evaluates its quoted `operation` once +per item with the focus *(item i, i, n)* where *n* is the number of items; +within any evaluation of the operand, 1 ≤ position ≤ size. A nested `ForEach` +shadows the outer focus for the extent of its own operand. Outside any +iteration there is no focus, and operations that require one raise +`ValueError`. + +The focus accessors, named after their XSLT/XPath counterparts: + +- `Current` yields the focus item itself (XSLT `current()`). +- `Position` yields the item's 1-based position as an `xsd:integer` Literal + (XPath `fn:position()`). +- `Last` yields the iteration size as an `xsd:integer` Literal (XPath + `fn:last()`). +- `Value` with an unprefixed name looks the name up *in* the focus item: - `Binding` (SPARQL row): the term bound to that variable name; - mapping (e.g. a JSON object item): the member value; - any other object: the attribute of that name; @@ -265,7 +276,7 @@ Failures raise Python exceptions per this table (normative): | missing required argument key | `KeyError` | | argument or operand of the wrong type (any layer) | `TypeError` | | unknown variable in `$name` lookup | `ValueError` | -| context lookup miss, or no context established | `ValueError` | +| focus-item lookup miss, or no focus established | `ValueError` | | `Filter` position < 1 or > length | `ValueError` | | regular-expression errors in `Replace` — invalid pattern or flags, zero-length-matching pattern, invalid replacement (XPath `err:FORX000*`) | `ValueError` | | unknown `type` in SPARQL JSON term form | `ValueError` | @@ -277,6 +288,133 @@ Type checking is strict: operations validate their inputs and raise `TypeError` kinds; the only implicit conversion anywhere is scalar coercion (§2.2) and string-compatibility (§4.2). +### 3.8 Formal definition + +This section is the definition; §§3.2–3.7 restate it in prose, and on any +disagreement this section wins. + +**Abstract syntax.** Forms `e` (concrete syntax per §2.2): + +``` +e ::= s scalar: string | integer | double | boolean + | {"@id": e} URI reference + | {k₁: e₁, …, kₙ: eₙ} generic object (no reserved keys) + | data RDF data form (JSON-LD object; may contain + operation-call holes) + | [e₁, …, eₙ] sequence + | op(k₁: e₁, …, kₙ: eₙ) operation call, op a catalog name; operands + marked ⟨quoted⟩ in the catalog are taken as + unevaluated forms +``` + +**Semantic domains.** + +``` +v ∈ Value §3.1 +ρ ∈ Env = Scope* stack of scopes; Scope = Name ⇀ Value +φ ∈ Focus⊥ = (Value × ℕ⁺ × ℕ⁺) + ⊥ (item, position, size), or absent +σ ∈ World external web state (graphs behind URIs, + endpoint contents); opaque +``` + +Each catalog operation `op` that is not treated by a rule below is a plain +operator with an interpretation + +``` +δ_op : Value* × World → (Value × World) + Err +``` + +*Pure* operations neither read nor write the World; *query* operations read +it; *update* operations read and write it; `STRUUID` and `SPARQLString` are +relations rather than functions (non-determinism, §3.6). + +**Judgments.** + +``` +ρ, φ ⊢ ⟨e, σ⟩ ⇓ ⟨v, ρ′, σ′⟩ e evaluates to v +ρ, φ ⊢ ⟨e, σ⟩ ⇓ err E e fails with E (per the §3.7 table) +``` + +The environment is threaded in *and out* because `Variable` writes into the +innermost scope; since binding only ever targets the innermost scope, popping +a scope restores the environment that surrounded it. Error propagation is +left-to-right: the first failing premise's error is the conclusion of the +rule (propagation rules are omitted below). + +**Rules.** + +``` +(SCALAR) ───────────────────────────────────── + ρ, φ ⊢ ⟨s, σ⟩ ⇓ ⟨coerce(s), ρ, σ⟩ coerce per §2.2 + (null: err TypeError) + + ρ, φ ⊢ ⟨e, σ⟩ ⇓ ⟨t, ρ′, σ′⟩ t ∈ Term +(URI-REF) ───────────────────────────────────── + ρ, φ ⊢ ⟨{"@id": e}, σ⟩ ⇓ ⟨uri(lex(t)), ρ′, σ′⟩ + + ρᵢ₋₁, φ ⊢ ⟨eᵢ, σᵢ₋₁⟩ ⇓ ⟨vᵢ, ρᵢ, σᵢ⟩ (i = 1…n, document order) +(OBJ) ───────────────────────────────────── + ρ₀, φ ⊢ ⟨{k₁:e₁,…,kₙ:eₙ}, σ₀⟩ ⇓ ⟨{k₁:v₁,…,kₙ:vₙ}, ρₙ, σₙ⟩ + +(DATA) as (OBJ), but only operation-call holes are evaluated (threaded + in document order); all other members are left untouched and the + value is the resulting JSON structure + + ρ₀ = ρ·∅ ρᵢ₋₁, φ ⊢ ⟨eᵢ, σᵢ₋₁⟩ ⇓ ⟨vᵢ, ρᵢ, σᵢ⟩ (i = 1…n) +(SEQ) ───────────────────────────────────── + ρ, φ ⊢ ⟨[e₁,…,eₙ], σ₀⟩ ⇓ ⟨[v₁,…,vₙ], pop(ρₙ), σₙ⟩ + + op has no quoted operands + ρᵢ₋₁, φ ⊢ ⟨eᵢ, σᵢ₋₁⟩ ⇓ ⟨vᵢ, ρᵢ, σᵢ⟩ (i = 1…n, document order) + δ_op(v₁,…,vₙ, σₙ) = (v, σ′) +(CALL) ───────────────────────────────────── + ρ₀, φ ⊢ ⟨op(k₁:e₁,…,kₙ:eₙ), σ₀⟩ ⇓ ⟨v, ρₙ, σ′⟩ + + ρ, φ ⊢ ⟨e, σ⟩ ⇓ ⟨v, ρ′, σ′⟩ +(VARIABLE) ───────────────────────────────────── + ρ, φ ⊢ ⟨Variable(name: x, value: e), σ⟩ + ⇓ ⟨unit, bind(ρ′, x, v), σ′⟩ + bind writes x ↦ v into the innermost scope (pushing one onto an + empty stack); rebinding overwrites + +(VALUE-VAR) ρ, φ ⊢ ⟨Value(name: $x), σ⟩ ⇓ ⟨lookup(ρ, x), ρ, σ⟩ + lookup searches scopes innermost→outermost; miss: err ValueError + +(VALUE-CTX) φ = (c, i, n) + ρ, φ ⊢ ⟨Value(name: x), σ⟩ ⇓ ⟨member(c, x), ρ, σ⟩ + member per §3.5 (Binding | mapping | attribute); + φ = ⊥ or miss: err ValueError + +(CURRENT) φ = (c, i, n) ⇒ ρ, φ ⊢ ⟨Current(), σ⟩ ⇓ ⟨c, ρ, σ⟩ +(POSITION) φ = (c, i, n) ⇒ ρ, φ ⊢ ⟨Position(), σ⟩ ⇓ ⟨int(i), ρ, σ⟩ +(LAST) φ = (c, i, n) ⇒ ρ, φ ⊢ ⟨Last(), σ⟩ ⇓ ⟨int(n), ρ, σ⟩ + each: φ = ⊥ ⇒ err ValueError; int(·) is an xsd:integer Literal + + ρ, φ ⊢ ⟨e_sel, σ⟩ ⇓ ⟨C, ρ′, σ₀⟩ items(C) = c₁ … cₙ + ρ′·∅, (cᵢ, i, n) ⊢ ⟨q, σᵢ₋₁⟩ ⇓ ⟨wᵢ, _, σᵢ⟩ (i = 1…n) +(FOREACH) ───────────────────────────────────── + ρ, φ ⊢ ⟨ForEach(select: e_sel, operation: q⟨quoted⟩), σ⟩ + ⇓ ⟨[wᵢ | wᵢ ≠ unit], ρ′, σₙ⟩ + items(Sequence) = its elements; items(Result) = its rows in + result order; other C: err TypeError. If q is an array + [q₁,…,q_m], the premise evaluates it as a sequence within the + iteration's scope and wᵢ is the last non-unit element value. + + q is an operation-call form ρ, φ ⊢ ⟨q, σ⟩ ⇓ ⟨v, ρ′, σ′⟩ +(EXECUTE) ───────────────────────────────────── + ρ, φ ⊢ ⟨Execute(operation: q⟨quoted⟩), σ⟩ ⇓ ⟨v, ρ′, σ′⟩ +``` + +**Metatheory.** Because forms are finite terms, there is no recursion, and +`ForEach` iterates a *computed, finite* sequence, every evaluation terminates +provided every δ_op does (structural induction on forms, with (FOREACH) +measured by the size of `items(C)`). Evaluation is deterministic up to the +declared non-deterministic operators and the World's own behavior. In +(FOREACH), each iteration's environment writes are confined to its fresh +scope and results are indexed by position, so evaluating iterations +concurrently is observationally equivalent for programs that do not rely on +cross-iteration effect ordering — the license granted in §3.6. + ## 4. Operation Catalog (normative) Catalog entry conventions: the *Abstract* signature is in the type language of @@ -287,7 +425,7 @@ expected value types after evaluation; ⟨quoted⟩ marks quoted operands (§3.3 ### 4.1 Control flow, variables, context **ForEach** — evaluate an operation once per item of a sequence or per row of -a SPARQL result; the item is the context (§3.5). +a SPARQL result; establishes the focus *(item, position, size)* (§3.5). ``` Abstract: (Sequence α + Result) × Operation⟨quoted⟩ → Sequence β Python: execute_json only (interpreter-level special form) @@ -295,7 +433,8 @@ JSON: select: Sequence α + Result · operation⟨quoted⟩: form or array o ``` - Iterates a `Sequence` item-by-item, a `Result` row-by-row in result order. Any other `select` value raises `TypeError`. -- Each iteration runs in a fresh variable scope with the item as context. +- Each iteration runs in a fresh variable scope under the focus + *(item i, i, n)*. - If `operation` is an array, its forms evaluate in order within the iteration's scope and the iteration's value is the *last* non-Unit value. - Iteration values that are Unit (`None`) are dropped from the output; @@ -331,22 +470,41 @@ JSON: name: String (plain JSON string, not a form) · value: any form in a `ForEach` operation array, Unit values do not become the iteration's value. -**Value** — read a variable (`$name`) or a context member (`name`). §3.4–3.5. +**Value** — read a variable (`$name`) or a focus-item member (`name`). §3.4–3.5. ``` Abstract: String → Any Python: def execute(self, name: str, context: Any, variable_stack: list) -> Any JSON: name: String (plain JSON string; `$` prefix selects variable lookup) ``` -**Current** — the context item itself (like XSLT `current()`). +**Current** — the focus item itself, per XSLT `current()`. ``` Abstract: () → Context Python: def execute(self, current_item: Any) -> Any JSON: (no arguments) ``` -- Raises `ValueError` when no context is established (§3.5). +- Raises `ValueError` when no focus is established (§3.5). + +**Position** — the 1-based position of the focus item, per XPath +`fn:position()`. +``` +Abstract: () → Literal +Python: def execute(self, focus: Focus) -> Literal +JSON: (no arguments) +``` +- An `xsd:integer` Literal; within a focus, 1 ≤ position ≤ size. Raises + `ValueError` when no focus is established (§3.5). + +**Last** — the size of the iterated sequence, per XPath `fn:last()`. +``` +Abstract: () → Literal +Python: def execute(self, focus: Focus) -> Literal +JSON: (no arguments) +``` +- An `xsd:integer` Literal. Raises `ValueError` when no focus is established + (§3.5). -**Execute** — evaluate a quoted operation form in the current context and +**Execute** — evaluate a quoted operation form under the current focus and environment. ``` Abstract: Operation⟨quoted⟩ → Any @@ -594,7 +752,8 @@ JSON: `endpoint: URI`. two concrete syntaxes of the same abstract algebra; operation names and abstract signatures are shared. Operations currently exclusive to one implementation (e.g. `Iterate` in REST-VKG; `PATCH`, `Values`, `Filter`, - `Bindings`, `URI` and the schema operations here) are slated for parity. + `Bindings`, `URI`, `Position`, `Last` and the schema operations here) are + slated for parity. - MCP exposure (`mcp_run`) is an interface adapter, not part of the algebra; its plain-JSON conversions are implementation detail. diff --git a/prompts/system.md b/prompts/system.md index 46104e5..4a3c58d 100644 --- a/prompts/system.md +++ b/prompts/system.md @@ -827,6 +827,34 @@ Result (example): } ``` +## Position() -> int + +Returns the 1-based position of the current iteration item, like XPath's `fn:position()`. Only meaningful inside `ForEach`, which establishes the focus (item, position, size). + +### Example JSON + +```json +{ + "@op": "Position" +} +``` + +Result (example): `2` (an `xsd:integer` literal) while processing the second row. + +## Last() -> int + +Returns the size of the sequence being iterated, like XPath's `fn:last()`. Only meaningful inside `ForEach`. Combine with `Position` for progress-style values, e.g. "item 2 of 10". + +### Example JSON + +```json +{ + "@op": "Last" +} +``` + +Result (example): `10` (an `xsd:integer` literal) while iterating ten rows. + ## ExtractClasses(endpoint: str) -> Graph Extracts OWL classes from an RDF dataset via SPARQL endpoint. diff --git a/src/web_algebra/focus.py b/src/web_algebra/focus.py new file mode 100644 index 0000000..cd76c38 --- /dev/null +++ b/src/web_algebra/focus.py @@ -0,0 +1,15 @@ +from dataclasses import dataclass +from typing import Any + + +@dataclass(frozen=True) +class Focus: + """The dynamic context of a ForEach iteration (formal-semantics.md §3.5): + the current item, its 1-based position, and the iteration size — exactly + XSLT's focus triple. Established only by ForEach; accessed by Current, + Position, Last and focus-item Value lookups. + """ + + item: Any + position: int + size: int diff --git a/src/web_algebra/operations/current.py b/src/web_algebra/operations/current.py index 9467200..e6938bf 100644 --- a/src/web_algebra/operations/current.py +++ b/src/web_algebra/operations/current.py @@ -1,5 +1,6 @@ from typing import Any from mcp import types +from web_algebra.focus import Focus from web_algebra.operation import Operation @@ -28,14 +29,19 @@ def execute(self, current_item: Any) -> Any: return current_item def execute_json(self, arguments: dict, variable_stack: list = None) -> Any: - """JSON execution: return current context item""" - # No iteration context established (formal-semantics.md §3.5) — the - # interpreter's default context is an empty dict. + """JSON execution: return the current focus item""" + # The focus is established by ForEach (formal-semantics.md §3.5); + # Current yields its item. + if isinstance(self.context, Focus): + return self.execute(self.context.item) + + # No focus established — the interpreter's default context is an + # empty dict. if self.context is None or ( isinstance(self.context, dict) and not self.context ): raise ValueError( - "Current requires an iteration context (only ForEach establishes one)" + "Current requires an iteration focus (only ForEach establishes one)" ) return self.execute(self.context) diff --git a/src/web_algebra/operations/for_each.py b/src/web_algebra/operations/for_each.py index bd1236b..2966b3b 100644 --- a/src/web_algebra/operations/for_each.py +++ b/src/web_algebra/operations/for_each.py @@ -1,6 +1,7 @@ from typing import Any, List, Union import logging from mcp import types +from web_algebra.focus import Focus from web_algebra.operation import Operation from rdflib.query import Result @@ -79,9 +80,14 @@ def execute_json(self, arguments: dict, variable_stack: list = None) -> List[Any ) results = [] - for item in items: + size = len(items) + for position, item in enumerate(items, start=1): logging.info("Processing item: %s", item) + # The focus (item, position, size) per formal-semantics.md §3.5, + # accessed by Current/Position/Last and focus-item Value lookups. + focus = Focus(item=item, position=position, size=size) + # Each iteration runs in a fresh variable scope # (formal-semantics.md §3.4): bindings made inside one iteration # do not leak into the next. @@ -89,7 +95,7 @@ def execute_json(self, arguments: dict, variable_stack: list = None) -> List[Any try: # Handle list of operations or single operation if isinstance(operation, list): - # Execute operations in sequence, with item as context; + # Execute operations in sequence under the focus; # the iteration's value is the last non-Unit result. last_result = None @@ -97,7 +103,7 @@ def execute_json(self, arguments: dict, variable_stack: list = None) -> List[Any result = Operation.process_json( self.settings, op, - context=item, + context=focus, variable_stack=variable_stack, ) if result is not None: @@ -111,7 +117,7 @@ def execute_json(self, arguments: dict, variable_stack: list = None) -> List[Any result = Operation.process_json( self.settings, operation, - context=item, + context=focus, variable_stack=variable_stack, ) # Only collect non-None results diff --git a/src/web_algebra/operations/last.py b/src/web_algebra/operations/last.py new file mode 100644 index 0000000..ee9e1a2 --- /dev/null +++ b/src/web_algebra/operations/last.py @@ -0,0 +1,34 @@ +from rdflib import Literal +from rdflib.namespace import XSD + +from web_algebra.focus import Focus +from web_algebra.operation import Operation + + +class Last(Operation): + """ + Returns the size of the iterated sequence, per XPath's fn:last(). + """ + + @classmethod + def description(cls) -> str: + return """Returns the size of the sequence being iterated, per XPath's fn:last(). + + Only meaningful inside ForEach, which establishes the focus + (item, position, size).""" + + @classmethod + def inputSchema(cls) -> dict: + return {"type": "object", "properties": {}, "additionalProperties": False} + + def execute(self, focus: Focus) -> Literal: + """Pure function: focus → iteration size as xsd:integer""" + if not isinstance(focus, Focus): + raise ValueError( + "Last requires an iteration focus (only ForEach establishes one)" + ) + return Literal(focus.size, datatype=XSD.integer) + + def execute_json(self, arguments: dict, variable_stack: list = None) -> Literal: + """JSON execution: read the size from the current focus""" + return self.execute(self.context) diff --git a/src/web_algebra/operations/position.py b/src/web_algebra/operations/position.py new file mode 100644 index 0000000..fb0da12 --- /dev/null +++ b/src/web_algebra/operations/position.py @@ -0,0 +1,35 @@ +from rdflib import Literal +from rdflib.namespace import XSD + +from web_algebra.focus import Focus +from web_algebra.operation import Operation + + +class Position(Operation): + """ + Returns the 1-based position of the current focus item, per XPath's + fn:position(). + """ + + @classmethod + def description(cls) -> str: + return """Returns the 1-based position of the current iteration item, per XPath's fn:position(). + + Only meaningful inside ForEach, which establishes the focus + (item, position, size); within a focus, 1 <= position <= size.""" + + @classmethod + def inputSchema(cls) -> dict: + return {"type": "object", "properties": {}, "additionalProperties": False} + + def execute(self, focus: Focus) -> Literal: + """Pure function: focus → 1-based position as xsd:integer""" + if not isinstance(focus, Focus): + raise ValueError( + "Position requires an iteration focus (only ForEach establishes one)" + ) + return Literal(focus.position, datatype=XSD.integer) + + def execute_json(self, arguments: dict, variable_stack: list = None) -> Literal: + """JSON execution: read the position from the current focus""" + return self.execute(self.context) diff --git a/src/web_algebra/operations/value.py b/src/web_algebra/operations/value.py index 75e2c84..e648b32 100644 --- a/src/web_algebra/operations/value.py +++ b/src/web_algebra/operations/value.py @@ -3,6 +3,7 @@ import logging from rdflib.query import ResultRow from mcp import types +from web_algebra.focus import Focus from web_algebra.operation import Operation @@ -39,8 +40,10 @@ def execute(self, name: str, context: Any, variable_stack: list) -> Any: ) return result else: - # Context lookup (formal-semantics.md §3.5): Binding → bound term, - # mapping → member value, other object → attribute. + # Focus-item lookup (formal-semantics.md §3.5): Binding → bound + # term, mapping → member value, other object → attribute. + if isinstance(context, Focus): + context = context.item if isinstance(context, ResultRow): # SPARQL result row - access by variable name try: diff --git a/tests/unit/test_current.py b/tests/unit/test_current.py index 5f672df..45ba523 100644 --- a/tests/unit/test_current.py +++ b/tests/unit/test_current.py @@ -34,3 +34,13 @@ def test_returns_context_value(self, settings): op = op_cls(settings=settings, context=ctx_value) result = op.execute_json({}) assert result == ctx_value + + def test_yields_the_focus_item(self, settings): + # §3.5: Current yields the focus item itself, not the focus triple + from web_algebra.focus import Focus + + op = Operation.get("Current")( + settings=settings, + context=Focus(item=Literal("the-item"), position=2, size=3), + ) + assert op.execute_json({}) == Literal("the-item") diff --git a/tests/unit/test_last.py b/tests/unit/test_last.py new file mode 100644 index 0000000..f43e6c9 --- /dev/null +++ b/tests/unit/test_last.py @@ -0,0 +1,36 @@ +"""Spec: formal-semantics.md §4.1 "Last — the size of the iterated sequence, +per XPath fn:last()" +Abstract: () → Literal +- An xsd:integer Literal (§3.5). +- Raises ValueError when no focus is established. +""" + +from __future__ import annotations + +import pytest +from rdflib.namespace import XSD + +from web_algebra.operation import Operation + + +class TestLastJson: + def test_last_is_the_iteration_size_in_every_iteration(self, settings): + op = Operation.get("ForEach")(settings=settings) + result = op.execute_json( + {"select": ["a", "b", "c"], "operation": {"@op": "Last"}} + ) + assert [int(v) for v in result] == [3, 3, 3] + assert all(v.datatype == XSD.integer for v in result) + + def test_empty_iteration_never_evaluates_last(self, settings): + # ForEach over the empty sequence never evaluates the operand, so + # Last (which would have no meaningful value) is never reached. + op = Operation.get("ForEach")(settings=settings) + result = op.execute_json({"select": [], "operation": {"@op": "Last"}}) + assert result == [] + + def test_no_focus_raises_value_error(self, settings): + # §3.5/§3.7: no focus established → ValueError + op = Operation.get("Last")(settings=settings) + with pytest.raises(ValueError): + op.execute_json({}) diff --git a/tests/unit/test_position.py b/tests/unit/test_position.py new file mode 100644 index 0000000..6e7f586 --- /dev/null +++ b/tests/unit/test_position.py @@ -0,0 +1,47 @@ +"""Spec: formal-semantics.md §4.1 "Position — the 1-based position of the +focus item, per XPath fn:position()" +Abstract: () → Literal +- An xsd:integer Literal; within a focus, 1 ≤ position ≤ size (§3.5). +- Raises ValueError when no focus is established. +""" + +from __future__ import annotations + +import pytest +from rdflib.namespace import XSD + +from web_algebra.operation import Operation + + +class TestPositionJson: + def test_positions_run_from_one_to_size(self, settings): + # §3.5: ForEach establishes the focus (item i, i, n) + op = Operation.get("ForEach")(settings=settings) + result = op.execute_json( + {"select": ["a", "b", "c"], "operation": {"@op": "Position"}} + ) + assert [int(v) for v in result] == [1, 2, 3] + assert all(v.datatype == XSD.integer for v in result) + + def test_nested_for_each_shadows_the_focus(self, settings): + # §3.5: a nested ForEach shadows the outer focus for its operand + op = Operation.get("ForEach")(settings=settings) + result = op.execute_json( + { + "select": [["x", "y"]], + "operation": { + "@op": "ForEach", + "args": { + "select": {"@op": "Current", "args": {}}, + "operation": {"@op": "Position"}, + }, + }, + } + ) + assert [[int(v) for v in inner] for inner in result] == [[1, 2]] + + def test_no_focus_raises_value_error(self, settings): + # §3.5/§3.7: no focus established → ValueError + op = Operation.get("Position")(settings=settings) + with pytest.raises(ValueError): + op.execute_json({}) diff --git a/tests/unit/test_value.py b/tests/unit/test_value.py index d642a4d..57dc526 100644 --- a/tests/unit/test_value.py +++ b/tests/unit/test_value.py @@ -30,11 +30,19 @@ def test_lookup_falls_back_to_outer_scope(self, settings): assert result == Literal("outer") def test_mapping_context_lookup(self, settings): - # §3.5: mapping context item → member value + # §3.5: mapping focus item → member value op = Operation.get("Value")(settings=settings) result = op.execute("city", {"city": Literal("Vilnius")}, []) assert result == Literal("Vilnius") + def test_lookup_unwraps_the_focus(self, settings): + # §3.5: the unprefixed lookup targets the focus *item* + from web_algebra.focus import Focus + + op = Operation.get("Value")(settings=settings) + focus = Focus(item={"city": Literal("Vilnius")}, position=1, size=1) + assert op.execute("city", focus, []) == Literal("Vilnius") + def test_attribute_context_lookup(self, settings): # §3.5: any other object → the attribute of that name class Item: From a5b4b4e95c1aadf86fa732d6e26bcb673456f549 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Sun, 12 Jul 2026 18:15:47 +0300 Subject: [PATCH 07/11] Close Value's focus-item lookup to Binding + mapping MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Drop the getattr fallback: "attribute of that name" was host-language reflection, not a defined path step — it had no possible Java translation and no focus-item shape that ForEach can produce needs it. The item shapes are now closed (Binding → bound term, mapping → member value; anything else raises ValueError), which makes Value fully well-defined and REST-VKG-portable. Also note in the catalog that Value has xsl:sequence semantics (value as-is, no string conversion) — xsl:value-of is expressible as Str(Value(...)). Co-Authored-By: Claude Fable 5 --- formal-semantics.md | 17 ++++++++++------- src/web_algebra/operations/value.py | 11 +++++------ tests/unit/test_process_json_jsonld.py | 8 ++++---- tests/unit/test_value.py | 18 +++++++++++------- 4 files changed, 30 insertions(+), 24 deletions(-) diff --git a/formal-semantics.md b/formal-semantics.md index 82c48f8..d485d47 100644 --- a/formal-semantics.md +++ b/formal-semantics.md @@ -237,11 +237,10 @@ The focus accessors, named after their XSLT/XPath counterparts: (XPath `fn:position()`). - `Last` yields the iteration size as an `xsd:integer` Literal (XPath `fn:last()`). -- `Value` with an unprefixed name looks the name up *in* the focus item: - - `Binding` (SPARQL row): the term bound to that variable name; - - mapping (e.g. a JSON object item): the member value; - - any other object: the attribute of that name; - - a miss, or an item supporting none of these, raises `ValueError`. +- `Value` with an unprefixed name looks the name up *in* the focus item. + The item shapes are **closed**: a `Binding` (SPARQL row) yields the term + bound to that variable name; a mapping (e.g. a JSON object item) yields + the member value. A miss, or any other item shape, raises `ValueError`. ### 3.6 Effects and ordering @@ -382,8 +381,8 @@ rule (propagation rules are omitted below). (VALUE-CTX) φ = (c, i, n) ρ, φ ⊢ ⟨Value(name: x), σ⟩ ⇓ ⟨member(c, x), ρ, σ⟩ - member per §3.5 (Binding | mapping | attribute); - φ = ⊥ or miss: err ValueError + member per §3.5, defined only for c ∈ Binding + mapping; + φ = ⊥, other item shapes, or miss: err ValueError (CURRENT) φ = (c, i, n) ⇒ ρ, φ ⊢ ⟨Current(), σ⟩ ⇓ ⟨c, ρ, σ⟩ (POSITION) φ = (c, i, n) ⇒ ρ, φ ⊢ ⟨Position(), σ⟩ ⇓ ⟨int(i), ρ, σ⟩ @@ -476,6 +475,10 @@ Abstract: String → Any Python: def execute(self, name: str, context: Any, variable_stack: list) -> Any JSON: name: String (plain JSON string; `$` prefix selects variable lookup) ``` +- Returns the value as-is, with no atomization or string conversion — the + semantics of `xsl:sequence`, not `xsl:value-of` (which is expressible as + `Str(Value(...))`). +- Focus-item lookup is defined for `Binding` and mapping items only (§3.5). **Current** — the focus item itself, per XSLT `current()`. ``` diff --git a/src/web_algebra/operations/value.py b/src/web_algebra/operations/value.py index e648b32..4babb52 100644 --- a/src/web_algebra/operations/value.py +++ b/src/web_algebra/operations/value.py @@ -40,8 +40,9 @@ def execute(self, name: str, context: Any, variable_stack: list) -> Any: ) return result else: - # Focus-item lookup (formal-semantics.md §3.5): Binding → bound - # term, mapping → member value, other object → attribute. + # Focus-item lookup (formal-semantics.md §3.5). The item shapes + # are closed: Binding → bound term, mapping → member value; + # anything else is an error. if isinstance(context, Focus): context = context.item if isinstance(context, ResultRow): @@ -55,11 +56,9 @@ def execute(self, name: str, context: Any, variable_stack: list) -> Any: return context[name] raise ValueError(f"Context member '{name}' not found in mapping") else: - # Other context types - if hasattr(context, name): - return getattr(context, name) raise ValueError( - f"Context variable '{name}' not found in {type(context)}" + f"Value cannot look up '{name}' in a {type(context).__name__} " + "focus item (expected a Binding or a mapping)" ) def execute_json(self, arguments: dict, variable_stack: list = None) -> Any: diff --git a/tests/unit/test_process_json_jsonld.py b/tests/unit/test_process_json_jsonld.py index 9fd8d14..e2fafed 100644 --- a/tests/unit/test_process_json_jsonld.py +++ b/tests/unit/test_process_json_jsonld.py @@ -18,7 +18,6 @@ from __future__ import annotations -from types import SimpleNamespace from rdflib import BNode, Graph, Literal, URIRef from rdflib.namespace import RDF @@ -159,15 +158,16 @@ def test_op_in_type_resolves_to_iri(self, settings): assert (DOC_URI, RDF.type, DOC_TYPE) in graph def test_variable_from_context_binding(self, settings): - # Bare name (no $) resolves from the context — the ForEach-row case from - # the original bug report. + # Bare name (no $) resolves from the focus item — the ForEach-row case + # from the original bug report. Item shapes are closed to Binding + + # mapping (formal-semantics.md §3.5), so the row stand-in is a mapping. json_data = { "@id": {"@op": "Value", "args": {"name": "doc"}}, str(FOAF_PRIMARY_TOPIC): { "@id": {"@op": "Value", "args": {"name": "topic"}} }, } - context = SimpleNamespace(doc=DOC_URI, topic=MESSAGE_URI) + context = {"doc": DOC_URI, "topic": MESSAGE_URI} result = Operation.process_json(settings, json_data, context, []) graph = Operation.to_graph(result) diff --git a/tests/unit/test_value.py b/tests/unit/test_value.py index 57dc526..e7f5ef1 100644 --- a/tests/unit/test_value.py +++ b/tests/unit/test_value.py @@ -1,10 +1,12 @@ """Spec: formal-semantics.md §4.1 "Value" with §3.4 (variable environment) -and §3.5 (context). +and §3.5 (focus). Abstract: String → Any - `$name` searches variable scopes innermost to outermost; miss → ValueError. -- Unprefixed `name` looks up in the context item: Binding → bound term, - mapping → member value, other object → attribute; miss → ValueError. +- Unprefixed `name` looks up in the focus item; the item shapes are closed: + Binding → bound term, mapping → member value; a miss or any other item + shape → ValueError. - The `$` sigil decides the lookup domain, so the two never shadow each other. +- Returns the value as-is (xsl:sequence semantics, no string conversion). """ from __future__ import annotations @@ -43,14 +45,16 @@ def test_lookup_unwraps_the_focus(self, settings): focus = Focus(item={"city": Literal("Vilnius")}, position=1, size=1) assert op.execute("city", focus, []) == Literal("Vilnius") - def test_attribute_context_lookup(self, settings): - # §3.5: any other object → the attribute of that name + def test_unsupported_item_shape_raises_value_error(self, settings): + # §3.5: the item shapes are closed (Binding + mapping) — anything + # else raises ValueError, even if the host object happens to carry + # an attribute of that name class Item: city = Literal("Kaunas") op = Operation.get("Value")(settings=settings) - result = op.execute("city", Item(), []) - assert result == Literal("Kaunas") + with pytest.raises(ValueError): + op.execute("city", Item(), []) def test_sigil_selects_lookup_domain(self, settings): # §3.4: `$name` reads the variable stack, plain `name` the context — From 9f7b3fb2e13048ace9befa9d651146087c85a1ff Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Sun, 12 Jul 2026 18:23:48 +0300 Subject: [PATCH 08/11] Close the remaining semantics gaps: RDF response contract, Result persistence, Value-domain precision MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Linked Data operations (§4.4) are now specified as RDF-specific and symmetric: they read and write RDF graphs; content negotiation is handled transparently by the implementation; a non-RDF response — unsupported media type, missing Content-Type, or a body that does not parse as the negotiated format — raises ValueError. §4.3 states the counterpart for the SPARQL operations. client.py conforms: missing Content-Type no longer crashes with AttributeError, and RDF-labelled bodies that fail to parse raise ValueError instead of leaking rdflib parser errors; new stub-opener tests pin the contract offline. Also: Result values declared materialized and re-iterable (§1.1); the Value domain's former JSON summand split into Object (generic-object results over Values) and Data (RDF data forms: Term holes + raw JSON) with their conversion boundaries stated (§3.1); scalar coercion row reworded to "number parsed as floating-point"; Position type/operation name overload noted in §1.1. Co-Authored-By: Claude Fable 5 --- formal-semantics.md | 41 +++++++++--- src/web_algebra/client.py | 31 +++++++-- tests/SPEC_GAPS.md | 16 ++++- tests/unit/test_client_responses.py | 97 +++++++++++++++++++++++++++++ 4 files changed, 170 insertions(+), 15 deletions(-) create mode 100644 tests/unit/test_client_responses.py diff --git a/formal-semantics.md b/formal-semantics.md index d485d47..5fd7822 100644 --- a/formal-semantics.md +++ b/formal-semantics.md @@ -22,10 +22,12 @@ BNode = blank node Term = URI + Literal + BNode Graph = RDF graph (set of triples) Result = SPARQL SELECT result: a variable list and an ordered sequence - of Bindings + of Bindings. Result values are materialized — they hold their + rows and may be iterated any number of times Binding = one solution row: a partial mapping from variable names to Terms Sequence α = ordered list of values of type α -Position = integer ≥ 1 (XSLT-style 1-based index) +Position = integer ≥ 1 (XSLT-style 1-based index; also the name of the + focus-accessor operation, §4.1 — context disambiguates) Unit = no meaningful value (an operation executed for its effect) Context = the current iteration item (see §3.5); one of Binding + Term + Graph + JSON value @@ -120,7 +122,7 @@ this is the *program* shape. |------|------| | string | `Literal` with datatype `xsd:string` | | integer | `Literal` with datatype `xsd:integer` | -| number with fraction | `Literal` with datatype `xsd:double` | +| number parsed as floating-point (e.g. `1.5`, `1.0`, `1e3`) | `Literal` with datatype `xsd:double` | | boolean | `Literal` with datatype `xsd:boolean` | A plain string is *always* a string literal, never a URI; URIs are written @@ -156,12 +158,23 @@ member values may themselves be computed by nested operation calls. Evaluation maps forms to values in the domain ``` -Value = Term + Graph + Result + Sequence Value + Binding + Unit + JSON +Value = Term + Graph + Result + Binding + Unit + Sequence Value + + Object + Data + +Object = JSON object whose member values are Values — the result of a + generic-object form (members are evaluated, so its scalars have + been coerced to Terms) +Data = JSON-LD structure whose evaluated hole positions hold Terms and + whose remaining content is raw, uncoerced JSON — the result of an + RDF data form ``` -(`JSON` covers RDF data forms and generic objects, which evaluate to JSON -structures with their holes filled; they become `Graph`s only at operation -boundaries.) +Note the two closures differ deliberately: a generic object's members are +forms and evaluate (§3.2), while an RDF data form's non-hole content is data +for a JSON-LD parser and must stay untouched. A `Data` value becomes a +`Graph` only at an operation boundary, parsed with that operation's base IRI +(§2.3); an `Object` value becomes a Term only where an operation's catalog +entry accepts the SPARQL JSON term form (§2.4). ### 3.2 Evaluation rules @@ -280,6 +293,7 @@ Failures raise Python exceptions per this table (normative): | regular-expression errors in `Replace` — invalid pattern or flags, zero-length-matching pattern, invalid replacement (XPath `err:FORX000*`) | `ValueError` | | unknown `type` in SPARQL JSON term form | `ValueError` | | blank node where SPARQL syntax forbids it (`Values` data) | `ValueError` | +| non-RDF response to a Linked Data or SPARQL operation — unsupported media type, missing `Content-Type`, or a body that does not parse as the negotiated format (§4.3–4.4) | `ValueError` | | HTTP/SPARQL transport failure | `urllib.error.HTTPError` / `URLError`, unwrapped | Type checking is strict: operations validate their inputs and raise `TypeError` @@ -623,6 +637,11 @@ JSON: base: URI · relative: string-compatible Literal ### 4.3 SPARQL operations +The query operations negotiate their response formats transparently (SPARQL +Results for `SELECT`, an RDF serialization for `CONSTRUCT`/`DESCRIBE`); a +response that does not parse as the negotiated format raises `ValueError` +(§3.7). + **SELECT** — execute a SPARQL SELECT query against an endpoint. *Query* effect. ``` Abstract: URI × Literal → Result @@ -685,6 +704,14 @@ JSON: question: Literal ### 4.4 Linked Data (HTTP) operations +The Linked Data operations are RDF-specific and **symmetric**: they read and +write RDF graphs. Content negotiation is handled transparently by the +implementation — RDF media types are requested and offered; the concrete +serializations on the wire are implementation detail and never visible in +the algebra. A response that is not an RDF representation — an unsupported +media type, a missing `Content-Type`, or a body that does not parse as its +declared RDF type — raises `ValueError` (§3.7). + `POST`, `PUT` and `PATCH` return a single-row `Result` with variables `status` (`xsd:integer` HTTP status) and `url` (the effective request URI). Transport failures propagate per §3.7. diff --git a/src/web_algebra/client.py b/src/web_algebra/client.py index 1a4e3d2..bdf9fa9 100644 --- a/src/web_algebra/client.py +++ b/src/web_algebra/client.py @@ -114,16 +114,27 @@ def get(self, url: str) -> Graph: # Read and decode the response data data = response.read().decode("utf-8") - content_type = response.headers.get("Content-Type").split(";")[0] + # Non-RDF responses are errors (formal-semantics.md §4.4): the + # Linked Data operations read and write RDF graphs only. + content_type_header = response.headers.get("Content-Type") + content_type = ( + content_type_header.split(";")[0].strip() if content_type_header else None + ) rdf_format = MEDIA_TYPES.get(content_type) if not rdf_format: raise ValueError( - f"Unsupported Content-Type: {content_type}. Supported types are: {', '.join(MEDIA_TYPES.keys())}" + f"Non-RDF response from {url}: Content-Type {content_type!r} is not " + f"an RDF media type (supported: {', '.join(MEDIA_TYPES.keys())})" ) # Parse the RDF data into an RDFLib Graph g = Graph() - g.parse(data=data, format=rdf_format, publicID=url) + try: + g.parse(data=data, format=rdf_format, publicID=url) + except Exception as e: + raise ValueError( + f"Non-RDF response from {url}: body does not parse as {content_type}: {e}" + ) from None return g def post(self, url: str, graph: Graph) -> HTTPResponse: @@ -384,11 +395,19 @@ def query(self, endpoint_url: str, query_string: str) -> dict: if accept == "application/n-triples": g = Graph() - # convert N-Triples to JSON-LD - g.parse(data=data.decode("utf-8"), format="nt") + # convert N-Triples to JSON-LD; a body that does not parse as the + # negotiated format is an error (formal-semantics.md §4.3) + try: + g.parse(data=data.decode("utf-8"), format="nt") + except Exception as e: + raise ValueError( + f"Non-RDF response from {endpoint_url}: body does not parse " + f"as N-Triples: {e}" + ) from None jsonld_str = g.serialize(format="json-ld") jsonld_data = json.loads(jsonld_str) return jsonld_data else: - # return SPARQL JSON results as a dict + # SPARQL JSON results as a dict; json.JSONDecodeError is a + # ValueError subclass, satisfying the §4.3 error contract return json.loads(data.decode("utf-8")) diff --git a/tests/SPEC_GAPS.md b/tests/SPEC_GAPS.md index f475b5b..32bb622 100644 --- a/tests/SPEC_GAPS.md +++ b/tests/SPEC_GAPS.md @@ -22,8 +22,9 @@ live. Literal); the operation is non-deterministic (LLM) and needs an OpenAI client, so even the type contract is only exercised in live runs. - **Live-service behavior** — §3.7 pins transport failures to - `urllib.error.HTTPError`/`URLError` propagating unwrapped, but content negotiation, - redirects (beyond 308), timeouts, and retry policy remain unspecified. + `urllib.error.HTTPError`/`URLError` propagating unwrapped, and §4.3–4.4 pin the + response contract (RDF-only, transparent conneg, non-RDF → `ValueError`); still + unspecified: timeouts, retry policy beyond 429, and redirect handling beyond 308. - **XPath regex dialect coverage** — `Replace` compiles patterns with Python's `re`. The common syntax is shared with XPath regular expressions, but XPath-only constructs (`\p{...}` category escapes, `\i`/`\c`, character-class subtraction @@ -91,3 +92,14 @@ and the corresponding tests are un-skipped. (Appendix A). - **Strict-typing divergences observed on first run** (Str, SELECT): resolved on the implementation side — both validate input types before any effect (§3.7). +- **Linked Data response contract** (§4.3–4.4): the HTTP operations are RDF-specific + and symmetric — they read and write RDF graphs; content negotiation is transparent + in the implementation; a non-RDF response (unsupported media type, missing + `Content-Type`, or a body that does not parse as the negotiated format) raises + `ValueError`. +- **Value-domain precision** (§3.1): the former `JSON` summand split into `Object` + (generic-object results, members are Values) and `Data` (RDF data forms, holes are + Terms, the rest raw JSON), with their conversion boundaries stated. +- **Result persistence** (§1.1): Result values are materialized and re-iterable. +- **Value focus-item lookup** (§3.5): closed to `Binding` + mapping; the former + host-reflection (attribute) fallback removed. diff --git a/tests/unit/test_client_responses.py b/tests/unit/test_client_responses.py new file mode 100644 index 0000000..ee087e0 --- /dev/null +++ b/tests/unit/test_client_responses.py @@ -0,0 +1,97 @@ +"""Implementation-level tests for the Linked Data response contract +(formal-semantics.md §4.3–4.4): the operations read and write RDF graphs; +content negotiation is transparent; a non-RDF response — unsupported media +type, missing Content-Type, or a body that does not parse as the negotiated +format — raises ValueError. + +These pin the contract at the client seam with a stubbed opener (no network). +""" + +from __future__ import annotations + +import pytest +from rdflib import URIRef + +from web_algebra.client import LinkedDataClient, SPARQLClient + + +class _FakeResponse: + def __init__(self, body: bytes, content_type: str | None): + self._body = body + self.headers = {} if content_type is None else {"Content-Type": content_type} + + def read(self) -> bytes: + return self._body + + +class _FakeOpener: + def __init__(self, response: _FakeResponse): + self._response = response + self.last_request = None + + def open(self, request): + self.last_request = request + return self._response + + +def _get_client(body: bytes, content_type: str | None) -> LinkedDataClient: + client = LinkedDataClient(verify_ssl=False) + client.opener = _FakeOpener(_FakeResponse(body, content_type)) + return client + + +class TestLinkedDataResponses: + def test_rdf_response_parses_to_graph(self): + client = _get_client( + b" .", "text/turtle" + ) + graph = client.get("http://ex/doc") + assert (URIRef("http://ex/s"), URIRef("http://ex/p"), URIRef("http://ex/o")) in graph + + def test_accept_header_offers_rdf_media_types_only(self): + # §4.4: content negotiation is transparent — RDF media types are + # requested + client = _get_client(b"", "text/turtle") + client.get("http://ex/doc") + accept = client.opener.last_request.get_header("Accept") + assert "text/turtle" in accept + assert "application/rdf+xml" in accept + + def test_non_rdf_media_type_raises_value_error(self): + # §4.4: unsupported media type → ValueError + client = _get_client(b"", "text/html") + with pytest.raises(ValueError): + client.get("http://ex/doc") + + def test_missing_content_type_raises_value_error(self): + # §4.4: missing Content-Type → ValueError + client = _get_client(b"anything", None) + with pytest.raises(ValueError): + client.get("http://ex/doc") + + def test_rdf_labelled_garbage_raises_value_error(self): + # §4.4: a body that does not parse as its declared RDF type → + # ValueError + client = _get_client(b"this is not turtle @@@", "text/turtle") + with pytest.raises(ValueError): + client.get("http://ex/doc") + + +class TestSPARQLResponses: + def test_construct_response_not_parsing_as_ntriples_raises(self): + # §4.3: a response that does not parse as the negotiated format → + # ValueError + client = SPARQLClient(verify_ssl=False) + client.opener = _FakeOpener(_FakeResponse(b"not n-triples @@@", None)) + with pytest.raises(ValueError): + client.query( + "http://ex/sparql", + "CONSTRUCT { } WHERE {}", + ) + + def test_select_response_not_parsing_as_json_raises(self): + # §4.3: json.JSONDecodeError is a ValueError subclass + client = SPARQLClient(verify_ssl=False) + client.opener = _FakeOpener(_FakeResponse(b"not json", None)) + with pytest.raises(ValueError): + client.query("http://ex/sparql", "SELECT * WHERE { ?s ?p ?o }") From 35989e7d1f3540fec3c7b5a128dce7b38c15277d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Sun, 12 Jul 2026 18:32:50 +0300 Subject: [PATCH 09/11] Port Iterate from REST-VKG: stateful iteration with parameter passing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The one operation the Java/XML Web Algebra had that the JSON side lacked, inspired by XSLT 3.0's xsl:iterate. JSON serialization: {"@op": "Iterate", "args": { "params": {name: form, ...}, eager, bound as loop variables "operation": , the loop body "next-iteration": {name: form, ...}, quoted; evaluated after each body in the iteration's environment (loop params + body bindings) and rebinds the parameters "break": {"name": ..., "equals"|"not-equals": ...}}} Without next-iteration exactly one iteration runs; break compares the named loop variable's lexical form after rebinding (missing variable compares as "", REST-VKG parity); the result is the sequence of iteration values (Units dropped); Iterate establishes no focus, so an enclosing ForEach focus stays visible — matching the Java implementation, which preserves the current binding across withVariable. The iteration count is capped at a normative 1000 (as in REST-VKG), which keeps the algebra terminating: §3.8 gains the (ITERATE) rule with a loop helper measured by CAP − k, and the metatheory note is updated. §5 records the Java implementation's fused merged-graph return and totalLimit as its specialization (Merge(Iterate(...)) expresses the fusion here). Spec §3.3/§3.6/§4.1/§3.8/§5, implementation, spec-derived tests (11 cases incl. the cap and cursor threading), README and system prompt updated. Co-Authored-By: Claude Fable 5 --- README.md | 1 + formal-semantics.md | 81 +++++++++-- prompts/system.md | 37 +++++ src/web_algebra/operations/iterate.py | 184 +++++++++++++++++++++++++ tests/unit/test_iterate.py | 190 ++++++++++++++++++++++++++ 5 files changed, 485 insertions(+), 8 deletions(-) create mode 100644 src/web_algebra/operations/iterate.py create mode 100644 tests/unit/test_iterate.py diff --git a/README.md b/README.md index 2210318..e733ce4 100644 --- a/README.md +++ b/README.md @@ -68,6 +68,7 @@ The operations cover read-write Linked Data, SPARQL queries, URI manipulation, a - `Value` - `Variable` - `ForEach` + - `Iterate` - `Filter` - `Bindings` - `Current` diff --git a/formal-semantics.md b/formal-semantics.md index 5fd7822..6cd72d8 100644 --- a/formal-semantics.md +++ b/formal-semantics.md @@ -206,6 +206,8 @@ Lisp sense, and their quoted operands are evaluated under a different regime | Operation | Operand | Evaluation regime | |-----------|---------|-------------------| | `ForEach` | `operation` | once per iteration item, under the focus *(item, position, size)* | +| `Iterate` | `operation` | once per loop iteration, in the loop's environment | +| `Iterate` | `next-iteration` member values | after each iteration's body, in the iteration's environment | | `Execute` | `operation` | once, under the current focus and environment | All other arguments of all operations are eagerly evaluated. An operation not @@ -273,6 +275,8 @@ Ordering guarantees: must not rely on effect ordering *across* iterations (within one iteration, sequence ordering applies). The Python implementation is currently sequential. +- `Iterate` is strictly sequential by definition: iteration *k+1*'s + parameters are computed from iteration *k*'s environment. There are no transactions: if evaluation fails midway, effects already performed are not rolled back. @@ -413,16 +417,37 @@ rule (propagation rules are omitted below). [q₁,…,q_m], the premise evaluates it as a sequence within the iteration's scope and wᵢ is the last non-unit element value. + ρ, φ ⊢ params member forms (document order) ⇓ P, ρ′, σ₀ (eager) + loop(ρ′·{P}, σ₀, 1) = ⟨[w₁ … w_m], σ′⟩ +(ITERATE) ───────────────────────────────────── + ρ, φ ⊢ ⟨Iterate(params, operation: q⟨quoted⟩, + next-iteration: N⟨quoted⟩, break: b), σ⟩ + ⇓ ⟨[wᵢ | wᵢ ≠ unit], ρ′, σ′⟩ + + where loop(ρ_it, σ, k) is defined by: + 1. ρ_it·∅, φ ⊢ ⟨q, σ⟩ ⇓ ⟨w, ρ_b, σ₁⟩ (array operand as in FOREACH) + 2. if N is absent or k = CAP: ⟨[w], σ₁⟩ + 3. ρ_b, φ ⊢ N's member forms in document order ⇓ v₁ … vⱼ, σ₂ + (each binding nᵢ ↦ vᵢ before the next evaluates) + 4. ρ_it ← ρ_it with each nᵢ ↦ vᵢ rebound in the loop scope; + the body scope is dropped + 5. if b holds of ρ_it: ⟨[w], σ₂⟩ + 6. else ⟨W, σ″⟩ = loop(ρ_it, σ₂, k+1); ⟨w·W, σ″⟩ + CAP = 1000 (normative). b holds iff the lexical form of the + named loop variable equals (resp. differs from) the eagerly + evaluated comparison value; a missing variable compares as "". + q is an operation-call form ρ, φ ⊢ ⟨q, σ⟩ ⇓ ⟨v, ρ′, σ′⟩ (EXECUTE) ───────────────────────────────────── ρ, φ ⊢ ⟨Execute(operation: q⟨quoted⟩), σ⟩ ⇓ ⟨v, ρ′, σ′⟩ ``` -**Metatheory.** Because forms are finite terms, there is no recursion, and -`ForEach` iterates a *computed, finite* sequence, every evaluation terminates -provided every δ_op does (structural induction on forms, with (FOREACH) -measured by the size of `items(C)`). Evaluation is deterministic up to the -declared non-deterministic operators and the World's own behavior. In +**Metatheory.** Because forms are finite terms, there is no recursion, +`ForEach` iterates a *computed, finite* sequence, and `Iterate` is bounded +by its normative iteration cap, every evaluation terminates provided every +δ_op does (structural induction on forms, with (FOREACH) measured by the +size of `items(C)` and (ITERATE) by `CAP − k`). Evaluation is deterministic +up to the declared non-deterministic operators and the World's own behavior. In (FOREACH), each iteration's environment writes are confined to its fresh scope and results are indexed by position, so evaluating iterations concurrently is observationally equivalent for programs that do not rely on @@ -454,6 +479,42 @@ JSON: select: Sequence α + Result · operation⟨quoted⟩: form or array o sequence-valued iteration results are kept nested (no flattening). Output length therefore equals input length minus Unit-valued iterations. +**Iterate** — stateful iteration with parameter passing between iterations, +inspired by XSLT 3.0's `xsl:iterate`; shared with the REST-VKG (XML) +serialization. +``` +Abstract: (Name ⇀ Value) × Operation⟨quoted⟩ + × Maybe (Name ⇀ Operation⟨quoted⟩) × Maybe Break → Sequence β +Python: execute_json only (interpreter-level special form) +JSON: params: Maybe object of name → form (eager) + · operation⟨quoted⟩: form or array of forms + · next-iteration⟨quoted⟩: Maybe object of name → form + · break: Maybe {name: String, equals: form} + or {name: String, not-equals: form} +``` +- `params` members are evaluated once, eagerly, in the enclosing environment + and bound as variables in a fresh loop scope (read via `$name`). +- Each iteration evaluates `operation` in a fresh scope inside the loop scope + (array operands as in `ForEach`: the last non-Unit value). Unit-valued + iterations are dropped from the output; the result is the sequence of + iteration values, in order. +- After the body, each `next-iteration` member is evaluated *in the + iteration's environment* — the loop parameters plus any bindings the body + made — in document order, each visible to the ones after it; the results + rebind the loop parameters for the next iteration. Without + `next-iteration`, exactly one iteration runs. +- `break` is tested after the parameters are rebound: the lexical form of + the named loop variable is compared with the eagerly evaluated `equals` + (or `not-equals`) value; a missing variable compares as the empty string. + Exactly one of `equals`/`not-equals` is required (`ValueError` otherwise). + This is the structured form of the XML serialization's + `test="$name = 'literal'"` / `!=` condition. +- The iteration count is bounded by a **normative cap of 1000**; reaching it + stops the loop (it is not an error), which keeps `Iterate` — and the + algebra — terminating. +- `Iterate` does not establish a focus; the enclosing focus, if any, remains + visible to the body. + **Filter** — positional selection from a sequence, XSLT-style. ``` Abstract: (Sequence α + Result) × Position → α @@ -781,9 +842,13 @@ JSON: `endpoint: URI`. - The JSON serialization here and the XML serialization used by REST-VKG are two concrete syntaxes of the same abstract algebra; operation names and abstract signatures are shared. Operations currently exclusive to one - implementation (e.g. `Iterate` in REST-VKG; `PATCH`, `Values`, `Filter`, - `Bindings`, `URI`, `Position`, `Last` and the schema operations here) are - slated for parity. + implementation (`PATCH`, `Values`, `Filter`, `Bindings`, `URI`, `Position`, + `Last` and the schema operations here) are slated for parity. +- `Iterate` is shared with REST-VKG, whose implementation returns the merged + graph of the iteration results (the same fused `Merge ∘ …` specialization + as its `ForEach`) and additionally honors a `totalLimit` parameter capping + the merged graph's distinct subjects; the JSON serialization returns the + iteration-value sequence, and `Merge(Iterate(…))` expresses the fusion. - MCP exposure (`mcp_run`) is an interface adapter, not part of the algebra; its plain-JSON conversions are implementation detail. diff --git a/prompts/system.md b/prompts/system.md index 4a3c58d..e9c0a35 100644 --- a/prompts/system.md +++ b/prompts/system.md @@ -827,6 +827,43 @@ Result (example): } ``` +## Iterate(params: Dict, operation: Union[Callable, List[Callable]], next-iteration: Dict, break: Dict) -> List + +Stateful iteration with parameter passing between iterations, inspired by XSLT 3.0's `xsl:iterate`. Use it for cursor- or URL-driven pagination, where the next request depends on the previous response. + +- `params`: initial parameters (name → value/operation), bound as variables and read via `{"@op": "Value", "args": {"name": "$name"}}`. +- `operation`: evaluated once per iteration; may be a list (executed in order, last non-null result is the iteration's value). +- `next-iteration` (optional): name → operation, evaluated after each iteration — it sees the loop parameters *and* any `Variable` bindings the body made — and rebinds the parameters. Without it, exactly one iteration runs. +- `break` (optional): `{"name": "param", "equals": "value"}` or `{"name": "param", "not-equals": "value"}`, tested after rebinding. + +Returns the list of iteration results. The iteration count is capped at 1000. + +### Example JSON + +```json +{ + "@op": "Iterate", + "args": { + "params": { + "url": {"@id": "https://api.example.com/items"} + }, + "operation": [ + { + "@op": "Variable", + "args": {"name": "page", "value": {"@op": "GET", "args": {"url": {"@op": "URI", "args": {"input": {"@op": "Value", "args": {"name": "$url"}}}}}}} + }, + {"@op": "Value", "args": {"name": "$page"}} + ], + "next-iteration": { + "url": {"@op": "Value", "args": {"name": "$nextPageUrl"}} + }, + "break": {"name": "url", "equals": ""} + } +} +``` + +Result: a list with one RDF graph per fetched page (merge them with `Merge` if a single graph is needed). + ## Position() -> int Returns the 1-based position of the current iteration item, like XPath's `fn:position()`. Only meaningful inside `ForEach`, which establishes the focus (item, position, size). diff --git a/src/web_algebra/operations/iterate.py b/src/web_algebra/operations/iterate.py new file mode 100644 index 0000000..8e1d10c --- /dev/null +++ b/src/web_algebra/operations/iterate.py @@ -0,0 +1,184 @@ +from typing import Any, Callable, ClassVar, List, Optional +import logging + +from web_algebra.operation import Operation + + +class Iterate(Operation): + """ + Stateful iteration with parameter passing between iterations, inspired by + XSLT 3.0's xsl:iterate. Shared with the REST-VKG (Java/XML) Web Algebra. + """ + + # Normative iteration cap (formal-semantics.md §4.1): reaching it stops + # the loop — it is not an error — and keeps the algebra terminating. + MAX_ITERATIONS: ClassVar[int] = 1000 + + @classmethod + def description(cls) -> str: + return """Stateful iteration with parameter passing between iterations, inspired by XSLT 3.0's xsl:iterate. + + `params` are evaluated once and bound as variables (read via $name). + Each iteration evaluates `operation`; afterwards the `next-iteration` + members are evaluated in the iteration's environment and rebind the + parameters. `break` compares a loop variable against a value and + stops the loop when it matches. Without `next-iteration`, exactly one + iteration runs. Returns the sequence of iteration values. Useful for + cursor- or URL-driven pagination.""" + + @classmethod + def inputSchema(cls) -> dict: + return { + "type": "object", + "properties": { + "params": { + "type": "object", + "description": "Initial parameters: name → form, evaluated once in the enclosing environment and bound as loop variables", + }, + "operation": { + "description": "Operation(s) evaluated each iteration (quoted)" + }, + "next-iteration": { + "type": "object", + "description": "name → form, evaluated after each iteration in the iteration's environment to rebind the loop parameters (quoted)", + }, + "break": { + "type": "object", + "description": "Loop exit test: {name, equals} or {name, not-equals}; the named loop variable's lexical form is compared with the value", + }, + }, + "required": ["operation"], + } + + def execute(self, *args) -> Any: + raise NotImplementedError( + "Iterate is an interpreter-level special form; use execute_json" + ) + + def execute_json(self, arguments: dict, variable_stack: list = None) -> List[Any]: + """JSON execution: run the parameterized loop (formal-semantics.md + §3.8 (ITERATE)).""" + if variable_stack is None: + variable_stack = [] + + params_arg = arguments.get("params", {}) + if not isinstance(params_arg, dict): + raise TypeError( + f"Iterate expects 'params' to be an object of name → form, got {type(params_arg)}" + ) + operation = arguments["operation"] # quoted + next_arg = arguments.get("next-iteration") # quoted + if next_arg is not None and not isinstance(next_arg, dict): + raise TypeError( + f"Iterate expects 'next-iteration' to be an object of name → form, got {type(next_arg)}" + ) + break_test = self._parse_break(arguments.get("break"), variable_stack) + + # params: eager, evaluated in the enclosing environment + params = { + name: Operation.process_json( + self.settings, form, self.context, variable_stack + ) + for name, form in params_arg.items() + } + + results: List[Any] = [] + # The loop scope holds the iteration parameters across iterations. + variable_stack.append(dict(params)) + try: + iteration = 0 + while iteration < self.MAX_ITERATIONS: + iteration += 1 + logging.info("Iterate: iteration %d", iteration) + + # Fresh body scope per iteration (§3.4); the enclosing focus, + # if any, remains visible (Iterate establishes none). + variable_stack.append({}) + try: + value = self._evaluate_body(operation, variable_stack) + if value is not None: + results.append(value) + + if next_arg is None: + # No next-iteration: exactly one iteration. + break + + # next-iteration members: evaluated in the iteration's + # environment (loop params + body bindings), in document + # order, each visible to the ones after it. + new_params = {} + for name, form in next_arg.items(): + new_value = Operation.process_json( + self.settings, form, self.context, variable_stack + ) + new_params[name] = new_value + self.set_variable(name, new_value, variable_stack) + finally: + variable_stack.pop() + + # Rebind the loop parameters for the next iteration. + variable_stack[-1].update(new_params) + + if break_test is not None and break_test(): + logging.info("Iterate: break condition met after %d iterations", iteration) + break + else: + logging.warning( + "Iterate: reached the iteration cap (%d)", self.MAX_ITERATIONS + ) + finally: + variable_stack.pop() + + return results + + def _evaluate_body(self, operation: Any, variable_stack: list) -> Any: + """Evaluate the quoted operation (form or array of forms); array + operands yield the last non-Unit value, as in ForEach.""" + if isinstance(operation, list): + last_result = None + for op in operation: + result = Operation.process_json( + self.settings, op, self.context, variable_stack + ) + if result is not None: + last_result = result + return last_result + return Operation.process_json( + self.settings, operation, self.context, variable_stack + ) + + def _parse_break( + self, break_arg: Any, variable_stack: list + ) -> Optional[Callable[[], bool]]: + """Build the break test: lexical-form (in)equality between a loop + variable and an eagerly evaluated comparison value. A missing + variable compares as the empty string (REST-VKG parity).""" + if break_arg is None: + return None + if not isinstance(break_arg, dict) or "name" not in break_arg: + raise TypeError( + "Iterate 'break' must be an object with 'name' and exactly one of 'equals'/'not-equals'" + ) + has_equals = "equals" in break_arg + has_not_equals = "not-equals" in break_arg + if has_equals == has_not_equals: + raise ValueError( + "Iterate 'break' requires exactly one of 'equals'/'not-equals'" + ) + name = break_arg["name"] + comparison_form = break_arg["equals" if has_equals else "not-equals"] + comparison = Operation.process_json( + self.settings, comparison_form, self.context, variable_stack + ) + comparison_lex = str(comparison) + + def test() -> bool: + try: + value = self.get_variable(name, variable_stack) + except ValueError: + value = "" + lexical = str(value) if value is not None else "" + matches = lexical == comparison_lex + return matches if has_equals else not matches + + return test diff --git a/tests/unit/test_iterate.py b/tests/unit/test_iterate.py new file mode 100644 index 0000000..171d80d --- /dev/null +++ b/tests/unit/test_iterate.py @@ -0,0 +1,190 @@ +"""Spec: formal-semantics.md §4.1 "Iterate — stateful iteration with +parameter passing between iterations" and §3.8 (ITERATE). +Abstract: (Name ⇀ Value) × Operation⟨quoted⟩ + × Maybe (Name ⇀ Operation⟨quoted⟩) × Maybe Break → Sequence β +- params bound as loop variables; without next-iteration exactly one + iteration runs; next-iteration members are evaluated in the iteration's + environment (loop params + body bindings) and rebind the parameters; + break compares a loop variable's lexical form after rebinding; the + iteration count is capped at 1000 (normative); Unit-valued iterations are + dropped; Iterate establishes no focus. +""" + +from __future__ import annotations + +import pytest +from rdflib import Literal + +from web_algebra.operation import Operation + + +def _str_of(name: str) -> dict: + return {"@op": "Str", "args": {"input": {"@op": "Value", "args": {"name": name}}}} + + +class TestIterateJson: + def test_single_iteration_without_next_iteration(self, settings): + # §4.1: without next-iteration, exactly one iteration runs; + # params are bound as variables and read via $name + op = Operation.get("Iterate")(settings=settings) + result = op.execute_json( + {"params": {"url": "https://ex/items"}, "operation": _str_of("$url")} + ) + assert result == [Literal("https://ex/items")] + + def test_next_iteration_rebinds_until_break(self, settings): + # §3.8 (ITERATE): body → next-iteration → rebind → break test + op = Operation.get("Iterate")(settings=settings) + result = op.execute_json( + { + "params": {"s": ""}, + "operation": _str_of("$s"), + "next-iteration": { + "s": { + "@op": "Concat", + "args": { + "inputs": [{"@op": "Value", "args": {"name": "$s"}}, "a"] + }, + } + }, + "break": {"name": "s", "equals": "aaa"}, + } + ) + assert [str(v) for v in result] == ["", "a", "aa"] + + def test_not_equals_break(self, settings): + # §4.1: break with not-equals stops as soon as the variable differs + op = Operation.get("Iterate")(settings=settings) + result = op.execute_json( + { + "params": {"s": "go"}, + "operation": _str_of("$s"), + "next-iteration": {"s": "stop"}, + "break": {"name": "s", "not-equals": "go"}, + } + ) + assert [str(v) for v in result] == ["go"] + + def test_body_bindings_visible_to_next_iteration(self, settings): + # §4.1: next-iteration members are evaluated in the iteration's + # environment — the loop parameters plus the body's bindings + op = Operation.get("Iterate")(settings=settings) + result = op.execute_json( + { + "params": {"url": "a"}, + "operation": [ + { + "@op": "Variable", + "args": { + "name": "page", + "value": { + "@op": "Concat", + "args": { + "inputs": [ + {"@op": "Value", "args": {"name": "$url"}}, + "!", + ] + }, + }, + }, + }, + _str_of("$page"), + ], + "next-iteration": {"url": {"@op": "Value", "args": {"name": "$page"}}}, + "break": {"name": "url", "equals": "a!!!"}, + } + ) + assert [str(v) for v in result] == ["a!", "a!!", "a!!!"] + + def test_unit_valued_iterations_are_dropped(self, settings): + # §4.1: Unit-valued iterations are dropped from the output + op = Operation.get("Iterate")(settings=settings) + result = op.execute_json( + { + "params": {"x": "v"}, + "operation": { + "@op": "Variable", + "args": {"name": "y", "value": "w"}, + }, + } + ) + assert result == [] + + def test_loop_scope_does_not_leak(self, settings): + # §3.4: the loop scope (params) and body bindings cease to exist + # after the Iterate + op = Operation.get("Iterate")(settings=settings) + stack: list = [{}] + op.execute_json( + {"params": {"url": "a"}, "operation": _str_of("$url")}, stack + ) + assert stack == [{}] + + def test_iteration_cap_stops_the_loop(self, settings): + # §4.1: the iteration count is bounded by a normative cap of 1000; + # reaching it stops the loop and is not an error + op = Operation.get("Iterate")(settings=settings) + result = op.execute_json( + { + "params": {"s": "x"}, + "operation": _str_of("$s"), + "next-iteration": {"s": {"@op": "Value", "args": {"name": "$s"}}}, + "break": {"name": "s", "equals": "never"}, + } + ) + assert len(result) == 1000 + + def test_enclosing_focus_remains_visible(self, settings): + # §4.1: Iterate establishes no focus; the enclosing one is visible + for_each = Operation.get("ForEach")(settings=settings) + result = for_each.execute_json( + { + "select": ["a"], + "operation": { + "@op": "Iterate", + "args": { + "operation": { + "@op": "Str", + "args": {"input": {"@op": "Current", "args": {}}}, + } + }, + }, + } + ) + assert result == [[Literal("a")]] + + def test_missing_operation_raises_key_error(self, settings): + # §3.7: missing required argument key → KeyError + op = Operation.get("Iterate")(settings=settings) + with pytest.raises(KeyError): + op.execute_json({"params": {"x": "v"}}) + + def test_break_requires_exactly_one_comparison(self, settings): + # §4.1: exactly one of equals/not-equals → ValueError otherwise + op = Operation.get("Iterate")(settings=settings) + with pytest.raises(ValueError): + op.execute_json( + { + "operation": _str_of("$s"), + "params": {"s": "x"}, + "break": {"name": "s"}, + } + ) + with pytest.raises(ValueError): + op.execute_json( + { + "operation": _str_of("$s"), + "params": {"s": "x"}, + "break": {"name": "s", "equals": "a", "not-equals": "b"}, + } + ) + + def test_malformed_arguments_raise_type_error(self, settings): + # §3.7: argument of the wrong type → TypeError + op = Operation.get("Iterate")(settings=settings) + with pytest.raises(TypeError): + op.execute_json({"operation": _str_of("$s"), "params": "not-an-object"}) + with pytest.raises(TypeError): + op.execute_json( + {"operation": _str_of("$s"), "break": "not-an-object"} + ) From 175e25cbfc298a6717cb957a6da4e2b96983ced5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Sun, 12 Jul 2026 22:47:05 +0300 Subject: [PATCH 10/11] Bump version to 2.0.0 Major release: the formal evaluation semantics, SPARQL/XPath conformance for the string operations, the XSLT focus (Position/Last), Iterate, the @id URI reference form, and the interpreter fixes in this branch are deliberately breaking (see PR #25 for the full list). Co-Authored-By: Claude Fable 5 --- pyproject.toml | 2 +- uv.lock | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index 35fd510..3a5fdd3 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "web-algebra" -version = "1.5.0" +version = "2.0.0" description = "Composable RDF operations in JSON" readme = "README.md" license = "Apache-2.0" diff --git a/uv.lock b/uv.lock index 7624ac1..a41e218 100644 --- a/uv.lock +++ b/uv.lock @@ -888,7 +888,7 @@ wheels = [ [[package]] name = "web-algebra" -version = "1.5.0" +version = "2.0.0" source = { editable = "." } dependencies = [ { name = "mcp", extra = ["cli"] }, From c34c0229f21a592ee1a1891c6df5432932a19811 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Sun, 12 Jul 2026 23:29:13 +0300 Subject: [PATCH 11/11] Python code health: exception taxonomy, HTTP-client mixin, honest contracts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Implements the remaining tractable Tier 3 items from architecture-evaluation.md (3.1 mutable defaults and the _serialize_for_json_context deletion already landed in the semantics work). Exception taxonomy (3.3): new exceptions.py with WebAlgebraError and the interpreter-level subclasses UnknownOperationError, InvalidFormError, VariableNotFoundError, NoFocusError. Each ALSO inherits the built-in the spec's error table (§3.7) mandates, so the normative contract and every existing pytest.raises(TypeError|ValueError) assertion still hold, while callers gain `except WebAlgebraError` to tell an ill-formed document from an unrelated bug. Applied at the dispatch/evaluation/variable/focus sites (operation.py, value, current, position, last); per-operation argument-type validation stays plain TypeError per §3.7. HTTP transport failures are deliberately NOT wrapped — §3.7 pins them unwrapped. §3.7 gains a sentence documenting the hierarchy. HTTP-client dedup (3.4): new ClientOperation mixin builds the cert-configured client once; GET/POST/PUT/PATCH (LinkedDataClient), SELECT/CONSTRUCT/DESCRIBE (SPARQLClient) and ldh-AddFile (FileClient) drop their duplicated model_post_init. SPARQLString keeps its own (OpenAI, different constructor). Honest contracts (3.5): removed nine dead mcp_run methods from operations that never claimed MCPTool (Bindings, Current, Execute, Filter, ForEach, Str, URI, Value, Variable) — the server only dispatches mcp_run on MCPTool instances, so those were unreachable. Establishes the invariant "mcp_run exists iff MCPTool is claimed." (If Str/URI should be MCP tools, that is a deliberate feature add: claim MCPTool + implement mcp_run.) ForEach/Iterate.execute now state plainly that they are interpreter-level special forms with no pure form (§4.1). Deferred to their own PR (large refactors coupled to the future parallel-ForEach work, not rushed onto the 2.0 branch): extracting the interpreter into an immutable ExecutionContext object (3.2), and generating inputSchema from pydantic argument models (3.5 tail). 265 passed (5 new exception tests), ruff clean. Co-Authored-By: Claude Fable 5 --- formal-semantics.md | 8 +++ src/web_algebra/client_operation.py | 27 +++++++++ src/web_algebra/exceptions.py | 45 +++++++++++++++ src/web_algebra/operation.py | 11 +++- src/web_algebra/operations/bindings.py | 12 +--- src/web_algebra/operations/current.py | 9 +-- src/web_algebra/operations/execute.py | 6 -- src/web_algebra/operations/filter.py | 19 ------- src/web_algebra/operations/for_each.py | 17 +++--- src/web_algebra/operations/iterate.py | 3 +- src/web_algebra/operations/last.py | 3 +- src/web_algebra/operations/linked_data/get.py | 10 +--- .../operations/linked_data/patch.py | 10 +--- .../operations/linked_data/post.py | 10 +--- src/web_algebra/operations/linked_data/put.py | 10 +--- .../operations/linkeddatahub/add_file.py | 13 ++--- src/web_algebra/operations/position.py | 3 +- .../operations/sparql/construct.py | 13 ++--- src/web_algebra/operations/sparql/describe.py | 13 ++--- src/web_algebra/operations/sparql/select.py | 13 ++--- src/web_algebra/operations/str.py | 13 ----- src/web_algebra/operations/uri.py | 13 ----- src/web_algebra/operations/value.py | 15 +++-- src/web_algebra/operations/variable.py | 5 -- tests/unit/test_exceptions.py | 56 +++++++++++++++++++ 25 files changed, 196 insertions(+), 161 deletions(-) create mode 100644 src/web_algebra/client_operation.py create mode 100644 src/web_algebra/exceptions.py create mode 100644 tests/unit/test_exceptions.py diff --git a/formal-semantics.md b/formal-semantics.md index 6cd72d8..ea7c493 100644 --- a/formal-semantics.md +++ b/formal-semantics.md @@ -305,6 +305,14 @@ Type checking is strict: operations validate their inputs and raise `TypeError` kinds; the only implicit conversion anywhere is scalar coercion (§2.2) and string-compatibility (§4.2). +Interpreter-level failures (unknown operation, invalid form, unresolved +variable, missing focus) are raised as `WebAlgebraError` subclasses that +*also* inherit the built-in class tabulated above, so the contract here holds +while a caller may `except WebAlgebraError` to distinguish an ill-formed +document from an unrelated bug. Per-operation argument-type validation stays +plain `TypeError`. Transport failures are the deliberate exception: they +propagate unwrapped as noted, never wrapped. + ### 3.8 Formal definition This section is the definition; §§3.2–3.7 restate it in prose, and on any diff --git a/src/web_algebra/client_operation.py b/src/web_algebra/client_operation.py new file mode 100644 index 0000000..d6a7f43 --- /dev/null +++ b/src/web_algebra/client_operation.py @@ -0,0 +1,27 @@ +from typing import Any, ClassVar, Type + +from web_algebra.client import LinkedDataClient + + +class ClientOperation: + """Mixin that builds an operation's HTTP client from the cert settings. + + Replaces the `model_post_init` boilerplate that the HTTP-backed operations + (GET/POST/PUT/PATCH, SELECT/CONSTRUCT/DESCRIBE, ldh-AddFile) each repeated + verbatim. Subclasses pick the client by overriding ``client_class`` + (``LinkedDataClient`` by default; ``SPARQLClient`` for the query ops, + ``FileClient`` for the multipart file op). + + All three client classes share the ``(cert_pem_path, cert_password, + verify_ssl)`` constructor, so a single builder covers them. ``verify_ssl`` + is off to match LinkedDataHub's self-signed development certificates. + """ + + client_class: ClassVar[Type] = LinkedDataClient + + def model_post_init(self, __context: Any) -> None: + self.client = self.client_class( + cert_pem_path=getattr(self.settings, "cert_pem_path", None), + cert_password=getattr(self.settings, "cert_password", None), + verify_ssl=False, + ) diff --git a/src/web_algebra/exceptions.py b/src/web_algebra/exceptions.py new file mode 100644 index 0000000..7b5fa16 --- /dev/null +++ b/src/web_algebra/exceptions.py @@ -0,0 +1,45 @@ +"""Web Algebra exception hierarchy. + +`WebAlgebraError` is the base for every failure the interpreter raises about a +Web Algebra *document* — an unknown operation, an unresolved variable, a +missing iteration focus, an ill-formed value. Each subclass also inherits the +built-in exception that the specification's error table +(`formal-semantics.md` §3.7) mandates, so the normative contract — and the +existing `pytest.raises(TypeError | ValueError | KeyError)` assertions — keep +holding, while callers gain `except WebAlgebraError` to tell an ill-formed +document apart from an unrelated bug. + +Scope: these classes cover the **interpreter / composition layer** (dispatch, +evaluation, variable and focus resolution). Individual operations validate +their own argument *types* with plain `TypeError` / `ValueError` per the +spec's Strict Type Checking property; those are not reclassified here. + +Two deliberate omissions: + +- There is no wrapper for HTTP/SPARQL transport failures. Spec §3.7 pins them + to `urllib.error.HTTPError` / `URLError` propagating **unwrapped**; wrapping + would contradict the normative contract. +- Per-operation argument type errors stay built-in `TypeError` for the same + reason (§3.7 names them `TypeError`), and to avoid churning ~170 leaf + validation sites whose meaning is already unambiguous. +""" + + +class WebAlgebraError(Exception): + """Base for errors the Web Algebra interpreter raises about a document.""" + + +class UnknownOperationError(WebAlgebraError, ValueError): + """An `@op` names an operation that is not registered (spec §3.7).""" + + +class InvalidFormError(WebAlgebraError, TypeError): + """A JSON value is not a valid Web Algebra form — e.g. `null` (spec §2.2, §3.7).""" + + +class VariableNotFoundError(WebAlgebraError, ValueError): + """A `$name` variable lookup or a focus-item member lookup found nothing (spec §3.7).""" + + +class NoFocusError(WebAlgebraError, ValueError): + """An operation that requires an iteration focus ran outside one (spec §3.5).""" diff --git a/src/web_algebra/operation.py b/src/web_algebra/operation.py index 3c0e882..c9f008d 100644 --- a/src/web_algebra/operation.py +++ b/src/web_algebra/operation.py @@ -8,6 +8,11 @@ from rdflib import URIRef, Literal, BNode, Graph from rdflib.namespace import XSD from rdflib.query import Result +from web_algebra.exceptions import ( + InvalidFormError, + UnknownOperationError, + VariableNotFoundError, +) # JSON-LD keyword set used to recognise a dict as RDF data (a JSON-LD @@ -95,7 +100,7 @@ def process_json( operation_cls = cls.get(op_name) if not operation_cls: - raise ValueError(f"Unknown operation: {op_name}") + raise UnknownOperationError(f"Unknown operation: {op_name}") operation = operation_cls(settings=settings, context=context) result = operation.execute_json(op_args, variable_stack) @@ -216,7 +221,7 @@ def get_variable(self, name: str, variable_stack: list) -> Any: for scope in reversed(variable_stack): if name in scope: return scope[name] - raise ValueError(f"Variable '{name}' not found") + raise VariableNotFoundError(f"Variable '{name}' not found") # Conversion helpers between different formats @staticmethod @@ -254,7 +259,7 @@ def json_to_rdflib(data) -> Node: """Convert JSON/binding objects to RDFLib terms""" if data is None: # formal-semantics.md §2.2: null is not a valid form. - raise TypeError("null is not a valid Web Algebra form") + raise InvalidFormError("null is not a valid Web Algebra form") if isinstance(data, dict) and "type" in data and "value" in data: # SPARQL binding object - values may have been processed to RDFLib terms type_str = str(data["type"]) # Convert potential Literal to string diff --git a/src/web_algebra/operations/bindings.py b/src/web_algebra/operations/bindings.py index 69451df..85686a1 100644 --- a/src/web_algebra/operations/bindings.py +++ b/src/web_algebra/operations/bindings.py @@ -1,5 +1,4 @@ -from typing import Any, List, Dict -from mcp import types +from typing import List, Dict from web_algebra.operation import Operation from rdflib.query import Result from rdflib.term import Node @@ -48,12 +47,3 @@ def execute_json( ) return self.execute(table_data) - - def mcp_run(self, arguments: dict, context: Any = None) -> Any: - """MCP execution: plain args → plain results""" - # For MCP, just return summary - return [ - types.TextContent( - type="text", text="Extracted bindings from SPARQL results" - ) - ] diff --git a/src/web_algebra/operations/current.py b/src/web_algebra/operations/current.py index e6938bf..ea2fca7 100644 --- a/src/web_algebra/operations/current.py +++ b/src/web_algebra/operations/current.py @@ -1,5 +1,5 @@ from typing import Any -from mcp import types +from web_algebra.exceptions import NoFocusError from web_algebra.focus import Focus from web_algebra.operation import Operation @@ -40,13 +40,8 @@ def execute_json(self, arguments: dict, variable_stack: list = None) -> Any: if self.context is None or ( isinstance(self.context, dict) and not self.context ): - raise ValueError( + raise NoFocusError( "Current requires an iteration focus (only ForEach establishes one)" ) return self.execute(self.context) - - def mcp_run(self, arguments: dict, context: Any = None) -> Any: - """MCP execution: plain args → plain results""" - # For MCP, we just return a placeholder since context handling is JSON-specific - return [types.TextContent(type="text", text="Current context accessed")] diff --git a/src/web_algebra/operations/execute.py b/src/web_algebra/operations/execute.py index b42ee7f..1a836c3 100644 --- a/src/web_algebra/operations/execute.py +++ b/src/web_algebra/operations/execute.py @@ -1,5 +1,4 @@ from typing import Any -from mcp import types from web_algebra.operation import Operation @@ -57,8 +56,3 @@ def execute_json(self, arguments: dict, variable_stack: list = None) -> Any: return Operation.process_json( self.settings, operation, self.context, variable_stack ) - - def mcp_run(self, arguments: dict, context: Any = None) -> Any: - """MCP execution: plain args → plain results""" - result = self.execute(arguments["operation"]) - return [types.TextContent(type="text", text=str(result))] diff --git a/src/web_algebra/operations/filter.py b/src/web_algebra/operations/filter.py index 3ab2f06..9cfe7c7 100644 --- a/src/web_algebra/operations/filter.py +++ b/src/web_algebra/operations/filter.py @@ -1,5 +1,4 @@ from typing import Any, Union -from mcp import types from web_algebra.operation import Operation @@ -100,21 +99,3 @@ def _apply_positional_filter(self, bindings: list, position: int) -> list: # Convert to 0-based index for Python list access and return as list return [bindings[position - 1]] - - def mcp_run(self, arguments: dict, context: Any = None) -> Any: - """MCP execution: plain args → plain results""" - # Convert plain args to RDFLib terms - input_json = arguments["input"] - from web_algebra.json_result import JSONResult - - input_result = JSONResult.from_json(input_json) - expression = arguments["expression"] - - result = self.execute(input_result, expression) - - # Return summary for MCP - return [ - types.TextContent( - type="text", text=f"Filtered to {len(result.bindings)} result(s)" - ) - ] diff --git a/src/web_algebra/operations/for_each.py b/src/web_algebra/operations/for_each.py index 2966b3b..0d345a2 100644 --- a/src/web_algebra/operations/for_each.py +++ b/src/web_algebra/operations/for_each.py @@ -1,6 +1,5 @@ from typing import Any, List, Union import logging -from mcp import types from web_algebra.focus import Focus from web_algebra.operation import Operation from rdflib.query import Result @@ -37,11 +36,15 @@ def inputSchema(cls) -> dict: def execute( self, select_data: Union[List[Any], Result], operation: Any ) -> List[Any]: - """Pure function: apply operation to each item in sequence or SPARQL results""" - # This is complex because we need to execute operations with context - # For now, this will be handled in execute_json + """Interpreter-level special form — no pure form (formal-semantics.md §4.1). + + `ForEach` evaluates a *quoted* operand once per item under a per-item + focus and variable scope; that requires the interpreter, so it has no + pure `execute()` and lives entirely in `execute_json`. + """ raise NotImplementedError( - "ForEach pure function needs operation execution context" + "ForEach is an interpreter-level special form (formal-semantics.md " + "§4.1); use execute_json" ) def execute_json(self, arguments: dict, variable_stack: list = None) -> List[Any]: @@ -127,7 +130,3 @@ def execute_json(self, arguments: dict, variable_stack: list = None) -> List[Any variable_stack.pop() return results - - def mcp_run(self, arguments: dict, context: Any = None) -> Any: - """MCP execution: plain args → plain results""" - return [types.TextContent(type="text", text="ForEach operation completed")] diff --git a/src/web_algebra/operations/iterate.py b/src/web_algebra/operations/iterate.py index 8e1d10c..c132468 100644 --- a/src/web_algebra/operations/iterate.py +++ b/src/web_algebra/operations/iterate.py @@ -52,7 +52,8 @@ def inputSchema(cls) -> dict: def execute(self, *args) -> Any: raise NotImplementedError( - "Iterate is an interpreter-level special form; use execute_json" + "Iterate is an interpreter-level special form (formal-semantics.md " + "§4.1); use execute_json" ) def execute_json(self, arguments: dict, variable_stack: list = None) -> List[Any]: diff --git a/src/web_algebra/operations/last.py b/src/web_algebra/operations/last.py index ee9e1a2..2d0f03a 100644 --- a/src/web_algebra/operations/last.py +++ b/src/web_algebra/operations/last.py @@ -1,6 +1,7 @@ from rdflib import Literal from rdflib.namespace import XSD +from web_algebra.exceptions import NoFocusError from web_algebra.focus import Focus from web_algebra.operation import Operation @@ -24,7 +25,7 @@ def inputSchema(cls) -> dict: def execute(self, focus: Focus) -> Literal: """Pure function: focus → iteration size as xsd:integer""" if not isinstance(focus, Focus): - raise ValueError( + raise NoFocusError( "Last requires an iteration focus (only ForEach establishes one)" ) return Literal(focus.size, datatype=XSD.integer) diff --git a/src/web_algebra/operations/linked_data/get.py b/src/web_algebra/operations/linked_data/get.py index a29ed5f..154afdd 100644 --- a/src/web_algebra/operations/linked_data/get.py +++ b/src/web_algebra/operations/linked_data/get.py @@ -3,23 +3,17 @@ from typing import Any from mcp import types from web_algebra.mcp_tool import MCPTool +from web_algebra.client_operation import ClientOperation from web_algebra.operation import Operation -from web_algebra.client import LinkedDataClient -class GET(Operation, MCPTool): +class GET(ClientOperation, Operation, MCPTool): """ Retrieves RDF data from a named graph using HTTP GET. The URL serves as both the resource identifier and the named graph address in systems with direct graph identification. Returns the RDF graph (describing the resource at that URL) as JSON-LD. """ - def model_post_init(self, __context: Any) -> None: - self.client = LinkedDataClient( - cert_pem_path=getattr(self.settings, "cert_pem_path", None), - cert_password=getattr(self.settings, "cert_password", None), - verify_ssl=False, # Optionally disable SSL verification - ) @classmethod def description(cls) -> str: diff --git a/src/web_algebra/operations/linked_data/patch.py b/src/web_algebra/operations/linked_data/patch.py index 8ae5583..b800f56 100644 --- a/src/web_algebra/operations/linked_data/patch.py +++ b/src/web_algebra/operations/linked_data/patch.py @@ -4,12 +4,12 @@ from rdflib.namespace import XSD from mcp import types from web_algebra.mcp_tool import MCPTool +from web_algebra.client_operation import ClientOperation from web_algebra.operation import Operation from rdflib.query import Result -from web_algebra.client import LinkedDataClient -class PATCH(Operation, MCPTool): +class PATCH(ClientOperation, Operation, MCPTool): """ Updates RDF data in a named graph using HTTP PATCH with SPARQL Update. The URL serves as both the resource identifier and the named graph address in systems with direct graph identification. @@ -20,12 +20,6 @@ class PATCH(Operation, MCPTool): Note: This operation does not return the updated graph, it only confirms the success of the operation. """ - def model_post_init(self, __context: Any) -> None: - self.client = LinkedDataClient( - cert_pem_path=getattr(self.settings, "cert_pem_path", None), - cert_password=getattr(self.settings, "cert_password", None), - verify_ssl=False, # Optionally disable SSL verification - ) @classmethod def description(cls) -> str: diff --git a/src/web_algebra/operations/linked_data/post.py b/src/web_algebra/operations/linked_data/post.py index 50b8f59..747f185 100644 --- a/src/web_algebra/operations/linked_data/post.py +++ b/src/web_algebra/operations/linked_data/post.py @@ -4,12 +4,12 @@ from rdflib.namespace import XSD from mcp import types from web_algebra.mcp_tool import MCPTool +from web_algebra.client_operation import ClientOperation from web_algebra.operation import Operation from rdflib.query import Result -from web_algebra.client import LinkedDataClient -class POST(Operation, MCPTool): +class POST(ClientOperation, Operation, MCPTool): """ Creates or appends RDF data to a named graph using HTTP POST. The URL serves as both the resource identifier and the named graph address in systems with direct graph identification. @@ -18,12 +18,6 @@ class POST(Operation, MCPTool): Note: This operation does not return the updated graph, it only confirms the success of the operation. """ - def model_post_init(self, __context: Any) -> None: - self.client = LinkedDataClient( - cert_pem_path=getattr(self.settings, "cert_pem_path", None), - cert_password=getattr(self.settings, "cert_password", None), - verify_ssl=False, # Optionally disable SSL verification - ) @classmethod def description(cls) -> str: diff --git a/src/web_algebra/operations/linked_data/put.py b/src/web_algebra/operations/linked_data/put.py index e016e97..132fa7b 100644 --- a/src/web_algebra/operations/linked_data/put.py +++ b/src/web_algebra/operations/linked_data/put.py @@ -4,12 +4,12 @@ from rdflib.namespace import XSD from mcp import types from web_algebra.mcp_tool import MCPTool +from web_algebra.client_operation import ClientOperation from web_algebra.operation import Operation from rdflib.query import Result -from web_algebra.client import LinkedDataClient -class PUT(Operation, MCPTool): +class PUT(ClientOperation, Operation, MCPTool): """ Replaces RDF data in a named graph using HTTP PUT. The URL serves as both the resource identifier and the named graph address in systems with direct graph identification. @@ -20,12 +20,6 @@ class PUT(Operation, MCPTool): Note: This operation does not return the updated graph, it only confirms the success of the operation. """ - def model_post_init(self, __context: Any) -> None: - self.client = LinkedDataClient( - cert_pem_path=getattr(self.settings, "cert_pem_path", None), - cert_password=getattr(self.settings, "cert_password", None), - verify_ssl=False, # Optionally disable SSL verification - ) @classmethod def description(cls) -> str: diff --git a/src/web_algebra/operations/linkeddatahub/add_file.py b/src/web_algebra/operations/linkeddatahub/add_file.py index 75ae753..2f188c3 100644 --- a/src/web_algebra/operations/linkeddatahub/add_file.py +++ b/src/web_algebra/operations/linkeddatahub/add_file.py @@ -1,4 +1,4 @@ -from typing import Any, Optional +from typing import Any, Optional, ClassVar, Type import logging import mimetypes import urllib.parse @@ -12,10 +12,11 @@ from web_algebra.client import FileClient from web_algebra.json_result import JSONResult from web_algebra.mcp_tool import MCPTool +from web_algebra.client_operation import ClientOperation from web_algebra.operation import Operation -class AddFile(Operation, MCPTool): +class AddFile(ClientOperation, Operation, MCPTool): """RDF/POST a file to a LinkedDataHub document, returning the minted upload URI. The file's RDF description (`nfo:FileDataObject` + filename + MIME type + @@ -29,12 +30,8 @@ class AddFile(Operation, MCPTool): `FileClient` instance instead of inheriting `LinkedDataClient` plumbing. """ - def model_post_init(self, __context: Any) -> None: - self.client = FileClient( - cert_pem_path=getattr(self.settings, "cert_pem_path", None), - cert_password=getattr(self.settings, "cert_password", None), - verify_ssl=False, - ) + client_class: ClassVar[Type] = FileClient + @classmethod def name(cls): diff --git a/src/web_algebra/operations/position.py b/src/web_algebra/operations/position.py index fb0da12..3d00d74 100644 --- a/src/web_algebra/operations/position.py +++ b/src/web_algebra/operations/position.py @@ -1,6 +1,7 @@ from rdflib import Literal from rdflib.namespace import XSD +from web_algebra.exceptions import NoFocusError from web_algebra.focus import Focus from web_algebra.operation import Operation @@ -25,7 +26,7 @@ def inputSchema(cls) -> dict: def execute(self, focus: Focus) -> Literal: """Pure function: focus → 1-based position as xsd:integer""" if not isinstance(focus, Focus): - raise ValueError( + raise NoFocusError( "Position requires an iteration focus (only ForEach establishes one)" ) return Literal(focus.position, datatype=XSD.integer) diff --git a/src/web_algebra/operations/sparql/construct.py b/src/web_algebra/operations/sparql/construct.py index dc344cb..501cc72 100644 --- a/src/web_algebra/operations/sparql/construct.py +++ b/src/web_algebra/operations/sparql/construct.py @@ -1,24 +1,21 @@ import logging -from typing import Any +from typing import Any, ClassVar, Type from rdflib import URIRef, Literal, Graph from rdflib.namespace import XSD from mcp import types from web_algebra.mcp_tool import MCPTool +from web_algebra.client_operation import ClientOperation from web_algebra.operation import Operation from web_algebra.client import SPARQLClient -class CONSTRUCT(Operation, MCPTool): +class CONSTRUCT(ClientOperation, Operation, MCPTool): """ Executes a SPARQL CONSTRUCT query against a specified endpoint. """ - def model_post_init(self, __context: Any) -> None: - self.client = SPARQLClient( - cert_pem_path=getattr(self.settings, "cert_pem_path", None), - cert_password=getattr(self.settings, "cert_password", None), - verify_ssl=False, # Optionally disable SSL verification - ) + client_class: ClassVar[Type] = SPARQLClient + @classmethod def description(cls) -> str: diff --git a/src/web_algebra/operations/sparql/describe.py b/src/web_algebra/operations/sparql/describe.py index d43f745..b8395ca 100644 --- a/src/web_algebra/operations/sparql/describe.py +++ b/src/web_algebra/operations/sparql/describe.py @@ -1,24 +1,21 @@ import logging -from typing import Any +from typing import Any, ClassVar, Type from rdflib import URIRef, Literal, Graph from rdflib.namespace import XSD from mcp import types from web_algebra.mcp_tool import MCPTool +from web_algebra.client_operation import ClientOperation from web_algebra.operation import Operation from web_algebra.client import SPARQLClient -class DESCRIBE(Operation, MCPTool): +class DESCRIBE(ClientOperation, Operation, MCPTool): """ Executes a SPARQL DESCRIBE query against a specified endpoint and returns a JSON-LD response. """ - def model_post_init(self, __context: Any) -> None: - self.client = SPARQLClient( - cert_pem_path=getattr(self.settings, "cert_pem_path", None), - cert_password=getattr(self.settings, "cert_password", None), - verify_ssl=False, # Optionally disable SSL verification - ) + client_class: ClassVar[Type] = SPARQLClient + @classmethod def description(cls) -> str: diff --git a/src/web_algebra/operations/sparql/select.py b/src/web_algebra/operations/sparql/select.py index 45e3bff..28579b4 100644 --- a/src/web_algebra/operations/sparql/select.py +++ b/src/web_algebra/operations/sparql/select.py @@ -1,25 +1,22 @@ -from typing import Any +from typing import Any, ClassVar, Type import logging from rdflib import URIRef, Literal from rdflib.namespace import XSD from rdflib.query import Result from mcp import types from web_algebra.mcp_tool import MCPTool +from web_algebra.client_operation import ClientOperation from web_algebra.operation import Operation from web_algebra.client import SPARQLClient -class SELECT(Operation, MCPTool): +class SELECT(ClientOperation, Operation, MCPTool): """ Executes SPARQL SELECT queries against endpoints """ - def model_post_init(self, __context: Any) -> None: - self.client = SPARQLClient( - cert_pem_path=getattr(self.settings, "cert_pem_path", None), - cert_password=getattr(self.settings, "cert_password", None), - verify_ssl=False, - ) + client_class: ClassVar[Type] = SPARQLClient + @classmethod def description(cls) -> str: diff --git a/src/web_algebra/operations/str.py b/src/web_algebra/operations/str.py index 9863251..69d0f79 100644 --- a/src/web_algebra/operations/str.py +++ b/src/web_algebra/operations/str.py @@ -1,7 +1,5 @@ -from typing import Any from rdflib.term import Node from rdflib import Literal, URIRef -from mcp import types from web_algebra.operation import Operation @@ -55,14 +53,3 @@ def execute_json( # Call pure function return self.execute(input_data) - - def mcp_run(self, arguments: dict, context: Any = None) -> Any: - """MCP execution: plain args → plain results""" - # Convert plain input to RDFLib term - rdflib_term = Operation.plain_to_rdflib(arguments["input"]) - - # Call pure function - result = self.execute(rdflib_term) - - # Convert result to plain string for MCP - return [types.TextContent(type="text", text=str(result))] diff --git a/src/web_algebra/operations/uri.py b/src/web_algebra/operations/uri.py index 3f90f51..32b9787 100644 --- a/src/web_algebra/operations/uri.py +++ b/src/web_algebra/operations/uri.py @@ -1,7 +1,5 @@ -from typing import Any from rdflib import BNode, URIRef from rdflib.term import Node -from mcp import types from web_algebra.operation import Operation @@ -51,14 +49,3 @@ def execute_json(self, arguments: dict, variable_stack: list = None) -> URIRef: # Call pure function return self.execute(input_data) - - def mcp_run(self, arguments: dict, context: Any = None) -> Any: - """MCP execution: plain args → plain results""" - # Convert plain input to RDFLib term - rdflib_term = Operation.plain_to_rdflib(arguments["input"]) - - # Call pure function - result = self.execute(rdflib_term) - - # Convert result to plain string for MCP - return [types.TextContent(type="text", text=str(result))] diff --git a/src/web_algebra/operations/value.py b/src/web_algebra/operations/value.py index 4babb52..85f774e 100644 --- a/src/web_algebra/operations/value.py +++ b/src/web_algebra/operations/value.py @@ -2,7 +2,7 @@ from typing import Any import logging from rdflib.query import ResultRow -from mcp import types +from web_algebra.exceptions import VariableNotFoundError from web_algebra.focus import Focus from web_algebra.operation import Operation @@ -50,11 +50,15 @@ def execute(self, name: str, context: Any, variable_stack: list) -> Any: try: return context[name] # Already RDFLib term except KeyError: - raise ValueError(f"Variable '{name}' not found in ResultRow") + raise VariableNotFoundError( + f"Variable '{name}' not found in ResultRow" + ) elif isinstance(context, Mapping): if name in context: return context[name] - raise ValueError(f"Context member '{name}' not found in mapping") + raise VariableNotFoundError( + f"Context member '{name}' not found in mapping" + ) else: raise ValueError( f"Value cannot look up '{name}' in a {type(context).__name__} " @@ -69,8 +73,3 @@ def execute_json(self, arguments: dict, variable_stack: list = None) -> Any: logging.info("Resolving Value variable: %s", var_name) return self.execute(var_name, self.context, variable_stack) - - def mcp_run(self, arguments: dict, context: Any = None) -> Any: - """MCP execution: plain args → plain results""" - # For MCP, we don't have variable stack or context, so just return the name - return [types.TextContent(type="text", text=arguments["name"])] diff --git a/src/web_algebra/operations/variable.py b/src/web_algebra/operations/variable.py index 891f422..9afb42d 100644 --- a/src/web_algebra/operations/variable.py +++ b/src/web_algebra/operations/variable.py @@ -1,5 +1,4 @@ from typing import Any -from mcp import types from web_algebra.operation import Operation @@ -49,7 +48,3 @@ def execute_json(self, arguments: dict, variable_stack: list = None) -> None: # Call pure function to store the variable return self.execute(name, value, variable_stack) - - def mcp_run(self, arguments: dict, context: Any = None) -> Any: - """MCP execution: plain args → confirmation""" - return [types.TextContent(type="text", text="Variable set successfully")] diff --git a/tests/unit/test_exceptions.py b/tests/unit/test_exceptions.py new file mode 100644 index 0000000..5040dc6 --- /dev/null +++ b/tests/unit/test_exceptions.py @@ -0,0 +1,56 @@ +"""The interpreter's exception taxonomy (src/web_algebra/exceptions.py). + +Each interpreter-level error is a `WebAlgebraError` *and* the built-in the +spec's error table (formal-semantics.md §3.7) mandates — so callers may +`except WebAlgebraError` while the normative built-in contract still holds. +""" + +from __future__ import annotations + +import pytest + +from web_algebra.exceptions import ( + InvalidFormError, + NoFocusError, + UnknownOperationError, + VariableNotFoundError, + WebAlgebraError, +) +from web_algebra.operation import Operation + + +class TestTaxonomyIsBackwardCompatible: + def test_unknown_operation_is_web_algebra_error_and_value_error(self, settings): + # §3.7: unknown `@op` → ValueError + with pytest.raises(UnknownOperationError) as exc: + Operation.process_json(settings, {"@op": "NoSuchOperation"}) + assert isinstance(exc.value, WebAlgebraError) + assert isinstance(exc.value, ValueError) + + def test_null_form_is_web_algebra_error_and_type_error(self, settings): + # §3.7: null form → TypeError + with pytest.raises(InvalidFormError) as exc: + Operation.process_json(settings, None) + assert isinstance(exc.value, WebAlgebraError) + assert isinstance(exc.value, TypeError) + + def test_missing_variable_is_web_algebra_error_and_value_error(self, settings): + # §3.7: unknown variable in `$name` lookup → ValueError + op = Operation.get("Value")(settings=settings) + with pytest.raises(VariableNotFoundError) as exc: + op.execute("$missing", {}, []) + assert isinstance(exc.value, WebAlgebraError) + assert isinstance(exc.value, ValueError) + + def test_no_focus_is_web_algebra_error_and_value_error(self, settings): + # §3.5/§3.7: Current outside an iteration focus → ValueError + op = Operation.get("Current")(settings=settings) + with pytest.raises(NoFocusError) as exc: + op.execute_json({}) + assert isinstance(exc.value, WebAlgebraError) + assert isinstance(exc.value, ValueError) + + def test_web_algebra_error_catches_the_family(self, settings): + # a caller can classify "ill-formed document" with one except clause + with pytest.raises(WebAlgebraError): + Operation.process_json(settings, {"@op": "NoSuchOperation"})