Formal evaluation semantics, SPARQL/XPath conformance, XSLT focus, Iterate#25
Open
namedgraph wants to merge 11 commits into
Open
Formal evaluation semantics, SPARQL/XPath conformance, XSLT focus, Iterate#25namedgraph wants to merge 11 commits into
namedgraph wants to merge 11 commits into
Conversation
…plan Evaluates the composed-operations premise, the formal semantics, the JSON DSL, the Python codebase and the rdflib data model; diffs the operation set and execution semantics against the Java/XML sibling (REST-VKG) and lays out a three-tier plan: spec unification, two-way operation parity, and Python code health. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…d tests
Spec (formal-semantics.md rewrite):
- Add evaluation semantics: eager depth-first evaluation, quoted operands
(ForEach.operation, Execute.operation), sequence forms with fresh variable
scopes, context rules, effect classes and ordering guarantees, and a
normative error taxonomy.
- Add the document model: optional envelope ({"@web-algebra": "1", "program":
[...]}), the URI reference form ({"@id": ...}), normative JSON-LD
discrimination keys, scalar coercion table, base-IRI rule.
- Rebuild the catalog with per-operation JSON argument tables; add missing
Concat and ExtractOntology entries; fix Unit vs bottom on Variable; pin
every decision tracked in tests/SPEC_GAPS.md (Str datatype, EncodeForURI
charset, Replace dialect, STRUUID format, Substitute and Values rules,
Merge set-union, Bindings order, Filter positional signature, ForEach
output shape, Value/Current context rules, Execute narrative, Extract*
endpoint role). ldh-* moves to an informative appendix.
Implementation alignment:
- Fix mutable default arguments across the interpreter and all operations
(variable_stack/context leak between documents in one process, e.g. the
MCP server).
- Sequence forms and ForEach iterations now push/pop a variable scope
(replacing copy() semantics that leaked writes when an outer scope existed).
- Implement the URI reference form and Operation.unwrap_document; unwrap the
envelope in main.py.
- Execute threads the variable environment (was silently dropped); Current
errors without an iteration context; Value supports mapping context items;
URI and Substitute reject BNodes (Substitute previously emitted invalid
"_: N" syntax); Filter raises TypeError for non-integer expressions;
boolean scalars coerce to xsd:boolean (bool-before-int); null forms raise
TypeError; remove dead _serialize_for_json_context.
Tests: 42 UNCLEAR(spec) skips un-skipped and authored from the new spec;
new test_document.py covers the envelope, URI reference form, coercions and
scoping. 221 passed, 8 skipped (was 145/50).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The spec no longer documents divergences from W3C functions; operations named after SPARQL/XPath functions follow those definitions by normative reference, signatures included. Simple literals are materialized as plain rdflib literals (no datatype) exactly as rdflib's own SPARQL engine does. - Str: per `simple literal STR(literal ltrl)` / `STR(IRI rsrc)` — returns the lexical form / codepoint representation as a simple literal; language tags are no longer carried over; BNode is a type error. - Replace: per REPLACE()/fn:replace — adds the optional flags argument (s m i x q), XPath replacement syntax ($N group references, \$ and \\ escapes), err:FORX000* conditions as ValueError (invalid flags/pattern/ replacement, zero-length-matching pattern), result kind follows the first argument, and pattern/replacement/flags must be simple literals. - Concat: per CONCAT() result-kind rules — all xsd:string inputs yield xsd:string, a shared language tag is carried, anything else yields a simple literal. - EncodeForURI, STRUUID: return simple literals per their signatures. - SELECT/CONSTRUCT/DESCRIBE accept simple-literal queries via a new Operation.is_string_literal predicate (RDF 1.1 equivalence), so Str output composes into query arguments. tests/SPEC_GAPS.md records the one honest implementation gap: XPath-only regex constructs unsupported by Python re surface as ValueError. New test_concat.py; Str/Replace suites rewritten against the W3C behavior. 238 passed, 8 skipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 1.5.0 bump (510f4dd) updated pyproject.toml without regenerating the lockfile's own package entry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Drop the optional {"@web-algebra": ..., "program": [...]} prolog: a
document is simply a single form or a program array. Removes
Operation.unwrap_document, the main.py unwrap step, the spec section and
error-table row, the system-prompt mention, and the envelope tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
formal-semantics.md §3.8 now contains the formal definition the document's title promised: abstract syntax, semantic domains (Env, Focus, World), the big-step judgment ρ, φ ⊢ ⟨e, σ⟩ ⇓ ⟨v, ρ′, σ′⟩ with the full rule set (scalar/URI-ref/object/data/sequence/call plus the state-accessing forms Variable, Value, Current, Position, Last, ForEach, Execute), the operator interpretation δ_op with effect classes, and metatheory notes (termination by structural induction, determinism modulo declared non-determinism, why concurrent ForEach iterations are observationally sound). The prose of §§3.2–3.7 is now the restatement; §3.8 wins on conflict. The context (§3.5) generalizes to the focus — the triple (item, position, size), exactly XSLT's dynamic context. ForEach establishes it per iteration; new operations Position and Last expose it per XPath fn:position()/fn:last() as xsd:integer literals; Current and unprefixed Value lookups read the focus item. Outside any focus all three raise ValueError. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Drop the getattr fallback: "attribute of that name" was host-language reflection, not a defined path step — it had no possible Java translation and no focus-item shape that ForEach can produce needs it. The item shapes are now closed (Binding → bound term, mapping → member value; anything else raises ValueError), which makes Value fully well-defined and REST-VKG-portable. Also note in the catalog that Value has xsl:sequence semantics (value as-is, no string conversion) — xsl:value-of is expressible as Str(Value(...)). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…sistence, Value-domain precision Linked Data operations (§4.4) are now specified as RDF-specific and symmetric: they read and write RDF graphs; content negotiation is handled transparently by the implementation; a non-RDF response — unsupported media type, missing Content-Type, or a body that does not parse as the negotiated format — raises ValueError. §4.3 states the counterpart for the SPARQL operations. client.py conforms: missing Content-Type no longer crashes with AttributeError, and RDF-labelled bodies that fail to parse raise ValueError instead of leaking rdflib parser errors; new stub-opener tests pin the contract offline. Also: Result values declared materialized and re-iterable (§1.1); the Value domain's former JSON summand split into Object (generic-object results over Values) and Data (RDF data forms: Term holes + raw JSON) with their conversion boundaries stated (§3.1); scalar coercion row reworded to "number parsed as floating-point"; Position type/operation name overload noted in §1.1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The one operation the Java/XML Web Algebra had that the JSON side lacked,
inspired by XSLT 3.0's xsl:iterate. JSON serialization:
{"@op": "Iterate", "args": {
"params": {name: form, ...}, eager, bound as loop variables
"operation": <quoted form or array>, the loop body
"next-iteration": {name: form, ...}, quoted; evaluated after each body
in the iteration's environment
(loop params + body bindings) and
rebinds the parameters
"break": {"name": ..., "equals"|"not-equals": ...}}}
Without next-iteration exactly one iteration runs; break compares the
named loop variable's lexical form after rebinding (missing variable
compares as "", REST-VKG parity); the result is the sequence of iteration
values (Units dropped); Iterate establishes no focus, so an enclosing
ForEach focus stays visible — matching the Java implementation, which
preserves the current binding across withVariable.
The iteration count is capped at a normative 1000 (as in REST-VKG), which
keeps the algebra terminating: §3.8 gains the (ITERATE) rule with a loop
helper measured by CAP − k, and the metatheory note is updated. §5 records
the Java implementation's fused merged-graph return and totalLimit as its
specialization (Merge(Iterate(...)) expresses the fusion here).
Spec §3.3/§3.6/§4.1/§3.8/§5, implementation, spec-derived tests (11 cases
incl. the cap and cursor threading), README and system prompt updated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Major release: the formal evaluation semantics, SPARQL/XPath conformance for the string operations, the XSLT focus (Position/Last), Iterate, the @id URI reference form, and the interpreter fixes in this branch are deliberately breaking (see PR #25 for the full list). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tracts Implements the remaining tractable Tier 3 items from architecture-evaluation.md (3.1 mutable defaults and the _serialize_for_json_context deletion already landed in the semantics work). Exception taxonomy (3.3): new exceptions.py with WebAlgebraError and the interpreter-level subclasses UnknownOperationError, InvalidFormError, VariableNotFoundError, NoFocusError. Each ALSO inherits the built-in the spec's error table (§3.7) mandates, so the normative contract and every existing pytest.raises(TypeError|ValueError) assertion still hold, while callers gain `except WebAlgebraError` to tell an ill-formed document from an unrelated bug. Applied at the dispatch/evaluation/variable/focus sites (operation.py, value, current, position, last); per-operation argument-type validation stays plain TypeError per §3.7. HTTP transport failures are deliberately NOT wrapped — §3.7 pins them unwrapped. §3.7 gains a sentence documenting the hierarchy. HTTP-client dedup (3.4): new ClientOperation mixin builds the cert-configured client once; GET/POST/PUT/PATCH (LinkedDataClient), SELECT/CONSTRUCT/DESCRIBE (SPARQLClient) and ldh-AddFile (FileClient) drop their duplicated model_post_init. SPARQLString keeps its own (OpenAI, different constructor). Honest contracts (3.5): removed nine dead mcp_run methods from operations that never claimed MCPTool (Bindings, Current, Execute, Filter, ForEach, Str, URI, Value, Variable) — the server only dispatches mcp_run on MCPTool instances, so those were unreachable. Establishes the invariant "mcp_run exists iff MCPTool is claimed." (If Str/URI should be MCP tools, that is a deliberate feature add: claim MCPTool + implement mcp_run.) ForEach/Iterate.execute now state plainly that they are interpreter-level special forms with no pure form (§4.1). Deferred to their own PR (large refactors coupled to the future parallel-ForEach work, not rushed onto the 2.0 branch): extracting the interpreter into an immutable ExecutionContext object (3.2), and generating inputSchema from pydantic argument models (3.5 tail). 265 passed (5 new exception tests), ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Turns
formal-semantics.mdfrom a type catalog into an actual semantics, and aligns the implementation and test suite with it. Follow-up to the architecture evaluation (architecture-evaluation.md, first commit).ρ, φ ⊢ ⟨e, σ⟩ ⇓ ⟨v, ρ′, σ′⟩with the full rule set, operator interpretation with effect classes, and metatheory (termination, determinism, ForEach-concurrency soundness). §3.8 is the definition; the §§3.2–3.7 prose restates it.{"@id": …}URI reference form, scalar coercion table, normative JSON-LD reserved-key list, base-IRI rule.tests/SPEC_GAPS.md(42 formerlyUNCLEAR(spec)tests un-skipped).Str,Concat,Replace,EncodeForURI,STRUUIDnow follow their W3C signatures exactly (simple literals materialized as plain rdflib literals, as rdflib's own SPARQL engine does).Replacegains theflagsargument and XPath$Nreplacement syntax witherr:FORX*conditions.ForEachestablishes (item, position, size); newPositionandLastoperations perfn:position()/fn:last();Value's focus-item lookup closed to Binding + mapping.Iterateported from REST-VKG (XSLT 3.0xsl:iterate-inspired):params/ quotedoperation/ quotednext-iteration/ structuredbreak, normative 1000-iteration cap (keeps the algebra terminating). The operation catalog is now a strict superset of the Java/XML implementation's.ValueError.bool→xsd:integercoercion bug,Substitute's invalid_: NBNode serialization,Executedropping the variable environment, scope-leaking list evaluation.Deliberate, conformance-driven; a major version bump is warranted:
Str: language tags are no longer preserved (STR("hallo"@de)→"hallo", per SPARQL);Str(BNode)now raisesTypeError; results are simple literals (no datatype), notxsd:string.EncodeForURI,STRUUID: return simple literals instead ofxsd:string(datatype-sensitive comparisons change).Concat: result kind now follows SPARQL CONCAT rules (was alwaysxsd:string); shared language tags are carried.Replace: replacement syntax is now XPath ($1for groups; bare$/\are errors — previously literal); zero-length-matching patterns and invalid flags now raiseValueError; language-taggedpattern/replacementnowTypeError; result kind follows the first argument.URI(BNode)andSubstitutewith a BNode binding: nowTypeError(previously returned garbage).Value: attribute lookup on arbitrary objects removed — focus-item shapes are closed to Binding + mapping (ValueErrorotherwise).Current(and context-Value) outside ForEach: nowValueError(previously returned the default empty context).xsd:boolean(previouslyxsd:integervia a bool-before-int bug);nullis now aTypeError(previously the literal"None").{"@id": "..."}objects: an object whose only member is@idnow evaluates to a URI (the new URI reference form) instead of passing through as JSON-LD data.ForEachcontext: operations now receive aFocusobject rather than the bare item — third-partyOperationsubclasses that readself.contextdirectly must unwrap it (all in-tree operations updated).TypeError(wasNotImplementedError).ValueError(previouslyAttributeError/raw rdflib parser errors leaked).Test plan
uv run pytest: 260 passed, 8 skipped (live-service/LLM-dependent), 20 deselected (network/sparql/ldh markers) — up from 145/50 onmain.uv run ruff check src tests: clean.Position/Last,Iterate(incl. the 1000-cap and cursor threading),Concat.🤖 Generated with Claude Code