Formal evaluation semantics, SPARQL/XPath conformance, XSLT focus, Iterate

README.md

@@ -68,9 +68,12 @@ The operations cover read-write Linked Data, SPARQL queries, URI manipulation, a
   - `Value`
   - `Variable`
   - `ForEach`
+  - `Iterate`
   - `Filter`
   - `Bindings`
   - `Current`
+  - `Position`
+  - `Last`
   - `Execute`
   - `Merge`
 - LinkedDataHub-specific

prompts/system.md

@@ -9,6 +9,7 @@ Your output must be a **JSON-formatted structure** of operation calls, where **o
 - **Operations must be represented as JSON objects**. Each operation corresponds to a function call with a specific signature.
 - **Operations may be nested inside arguments** to indicate dependencies.
 - **A result can be used directly as an argument in another operation** instead of requiring explicit intermediate variables.
+- **URIs are written with the URI reference form** `{"@id": "https://..."}` — an object whose only member is `@id`. A plain JSON string is **always a string literal, never a URI**. To produce a URI from a computed value, use `URI` or `ResolveURI`.
 - **ForEach supports executing multiple operations sequentially** when provided with a list of operations. Each operation in the list is executed for every row in the table before moving to the next row.
 - **Where an operation returns or expects RDF data, it is handled internally as an `rdflib.Graph`, but is represented as JSON-LD in the JSON structure.**
 - **SPARQL tabular data** (e.g., from `SELECT`) can be provided inline as a list of bindings, while **RDF Graph data** (e.g., from `GET`, `CONSTRUCT`, or merges) can be provided inline as JSON-LD objects.
@@ -28,7 +29,9 @@ would produce this JSON output:
     "select": {
       "@op": "SELECT",
       "args": {
-        "endpoint": "https://dbpedia.org/sparql",
+        "endpoint": {
+          "@id": "https://dbpedia.org/sparql"
+        },
         "query": {
           "@op": "SPARQLString",
           "args": {
@@ -43,7 +46,9 @@ would produce this JSON output:
         "url": {
           "@op": "ResolveURI",
           "args": {
-            "base": "http://localhost/denmark/",
+            "base": {
+              "@id": "http://localhost/denmark/"
+            },
             "relative": {
               "@op": "Value",
               "args": {
@@ -56,7 +61,7 @@ would produce this JSON output:
           "@op": "GET",
           "args": {
             "url": {
-              "@op": "Str",
+              "@op": "URI",
               "args": {
                 "input": {
                   "@op": "Value",
@@ -822,6 +827,71 @@ Result (example):
 }
 ```
 
+## Iterate(params: Dict, operation: Union[Callable, List[Callable]], next-iteration: Dict, break: Dict) -> List
+
+Stateful iteration with parameter passing between iterations, inspired by XSLT 3.0's `xsl:iterate`. Use it for cursor- or URL-driven pagination, where the next request depends on the previous response.
+
+- `params`: initial parameters (name → value/operation), bound as variables and read via `{"@op": "Value", "args": {"name": "$name"}}`.
+- `operation`: evaluated once per iteration; may be a list (executed in order, last non-null result is the iteration's value).
+- `next-iteration` (optional): name → operation, evaluated after each iteration — it sees the loop parameters *and* any `Variable` bindings the body made — and rebinds the parameters. Without it, exactly one iteration runs.
+- `break` (optional): `{"name": "param", "equals": "value"}` or `{"name": "param", "not-equals": "value"}`, tested after rebinding.
+
+Returns the list of iteration results. The iteration count is capped at 1000.
+
+### Example JSON
+
+```json
+{
+  "@op": "Iterate",
+  "args": {
+    "params": {
+      "url": {"@id": "https://api.example.com/items"}
+    },
+    "operation": [
+      {
+        "@op": "Variable",
+        "args": {"name": "page", "value": {"@op": "GET", "args": {"url": {"@op": "URI", "args": {"input": {"@op": "Value", "args": {"name": "$url"}}}}}}}
+      },
+      {"@op": "Value", "args": {"name": "$page"}}
+    ],
+    "next-iteration": {
+      "url": {"@op": "Value", "args": {"name": "$nextPageUrl"}}
+    },
+    "break": {"name": "url", "equals": ""}
+  }
+}
+```
+
+Result: a list with one RDF graph per fetched page (merge them with `Merge` if a single graph is needed).
+
+## Position() -> int
+
+Returns the 1-based position of the current iteration item, like XPath's `fn:position()`. Only meaningful inside `ForEach`, which establishes the focus (item, position, size).
+
+### Example JSON
+
+```json
+{
+  "@op": "Position"
+}
+```
+
+Result (example): `2` (an `xsd:integer` literal) while processing the second row.
+
+## Last() -> int
+
+Returns the size of the sequence being iterated, like XPath's `fn:last()`. Only meaningful inside `ForEach`. Combine with `Position` for progress-style values, e.g. "item 2 of 10".
+
+### Example JSON
+
+```json
+{
+  "@op": "Last"
+}
+```
+
+Result (example): `10` (an `xsd:integer` literal) while iterating ten rows.
+
 ## ExtractClasses(endpoint: str) -> Graph
 
 Extracts OWL classes from an RDF dataset via SPARQL endpoint.

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "web-algebra"
-version = "1.5.0"
+version = "2.0.0"
 description = "Composable RDF operations in JSON"
 readme = "README.md"
 license = "Apache-2.0"

src/web_algebra/client.py

@@ -114,16 +114,27 @@ def get(self, url: str) -> Graph:
 
         # Read and decode the response data
         data = response.read().decode("utf-8")
-        content_type = response.headers.get("Content-Type").split(";")[0]
+        # Non-RDF responses are errors (formal-semantics.md §4.4): the
+        # Linked Data operations read and write RDF graphs only.
+        content_type_header = response.headers.get("Content-Type")
+        content_type = (
+            content_type_header.split(";")[0].strip() if content_type_header else None
+        )
         rdf_format = MEDIA_TYPES.get(content_type)
         if not rdf_format:
             raise ValueError(
-                f"Unsupported Content-Type: {content_type}. Supported types are: {', '.join(MEDIA_TYPES.keys())}"
+                f"Non-RDF response from {url}: Content-Type {content_type!r} is not "
+                f"an RDF media type (supported: {', '.join(MEDIA_TYPES.keys())})"
             )
 
         # Parse the RDF data into an RDFLib Graph
         g = Graph()
-        g.parse(data=data, format=rdf_format, publicID=url)
+        try:
+            g.parse(data=data, format=rdf_format, publicID=url)
+        except Exception as e:
+            raise ValueError(
+                f"Non-RDF response from {url}: body does not parse as {content_type}: {e}"
+            ) from None
         return g
 
     def post(self, url: str, graph: Graph) -> HTTPResponse:
@@ -384,11 +395,19 @@ def query(self, endpoint_url: str, query_string: str) -> dict:
 
         if accept == "application/n-triples":
             g = Graph()
-            # convert N-Triples to JSON-LD
-            g.parse(data=data.decode("utf-8"), format="nt")
+            # convert N-Triples to JSON-LD; a body that does not parse as the
+            # negotiated format is an error (formal-semantics.md §4.3)
+            try:
+                g.parse(data=data.decode("utf-8"), format="nt")
+            except Exception as e:
+                raise ValueError(
+                    f"Non-RDF response from {endpoint_url}: body does not parse "
+                    f"as N-Triples: {e}"
+                ) from None
             jsonld_str = g.serialize(format="json-ld")
             jsonld_data = json.loads(jsonld_str)
             return jsonld_data
         else:
-            # return SPARQL JSON results as a dict
+            # SPARQL JSON results as a dict; json.JSONDecodeError is a
+            # ValueError subclass, satisfying the §4.3 error contract
             return json.loads(data.decode("utf-8"))

src/web_algebra/client_operation.py

@@ -0,0 +1,27 @@
+from typing import Any, ClassVar, Type
+
+from web_algebra.client import LinkedDataClient
+
+
+class ClientOperation:
+    """Mixin that builds an operation's HTTP client from the cert settings.
+
+    Replaces the `model_post_init` boilerplate that the HTTP-backed operations
+    (GET/POST/PUT/PATCH, SELECT/CONSTRUCT/DESCRIBE, ldh-AddFile) each repeated
+    verbatim. Subclasses pick the client by overriding ``client_class``
+    (``LinkedDataClient`` by default; ``SPARQLClient`` for the query ops,
+    ``FileClient`` for the multipart file op).
+
+    All three client classes share the ``(cert_pem_path, cert_password,
+    verify_ssl)`` constructor, so a single builder covers them. ``verify_ssl``
+    is off to match LinkedDataHub's self-signed development certificates.
+    """
+
+    client_class: ClassVar[Type] = LinkedDataClient
+
+    def model_post_init(self, __context: Any) -> None:
+        self.client = self.client_class(
+            cert_pem_path=getattr(self.settings, "cert_pem_path", None),
+            cert_password=getattr(self.settings, "cert_password", None),
+            verify_ssl=False,
+        )

src/web_algebra/exceptions.py

@@ -0,0 +1,45 @@
+"""Web Algebra exception hierarchy.
+
+`WebAlgebraError` is the base for every failure the interpreter raises about a
+Web Algebra *document* — an unknown operation, an unresolved variable, a
+missing iteration focus, an ill-formed value. Each subclass also inherits the
+built-in exception that the specification's error table
+(`formal-semantics.md` §3.7) mandates, so the normative contract — and the
+existing `pytest.raises(TypeError | ValueError | KeyError)` assertions — keep
+holding, while callers gain `except WebAlgebraError` to tell an ill-formed
+document apart from an unrelated bug.
+
+Scope: these classes cover the **interpreter / composition layer** (dispatch,
+evaluation, variable and focus resolution). Individual operations validate
+their own argument *types* with plain `TypeError` / `ValueError` per the
+spec's Strict Type Checking property; those are not reclassified here.
+
+Two deliberate omissions:
+
+- There is no wrapper for HTTP/SPARQL transport failures. Spec §3.7 pins them
+  to `urllib.error.HTTPError` / `URLError` propagating **unwrapped**; wrapping
+  would contradict the normative contract.
+- Per-operation argument type errors stay built-in `TypeError` for the same
+  reason (§3.7 names them `TypeError`), and to avoid churning ~170 leaf
+  validation sites whose meaning is already unambiguous.
+"""
+
+
+class WebAlgebraError(Exception):
+    """Base for errors the Web Algebra interpreter raises about a document."""
+
+
+class UnknownOperationError(WebAlgebraError, ValueError):
+    """An `@op` names an operation that is not registered (spec §3.7)."""
+
+
+class InvalidFormError(WebAlgebraError, TypeError):
+    """A JSON value is not a valid Web Algebra form — e.g. `null` (spec §2.2, §3.7)."""
+
+
+class VariableNotFoundError(WebAlgebraError, ValueError):
+    """A `$name` variable lookup or a focus-item member lookup found nothing (spec §3.7)."""
+
+
+class NoFocusError(WebAlgebraError, ValueError):
+    """An operation that requires an iteration focus ran outside one (spec §3.5)."""

src/web_algebra/focus.py

@@ -0,0 +1,15 @@
+from dataclasses import dataclass
+from typing import Any
+
+
+@dataclass(frozen=True)
+class Focus:
+    """The dynamic context of a ForEach iteration (formal-semantics.md §3.5):
+    the current item, its 1-based position, and the iteration size — exactly
+    XSLT's focus triple. Established only by ForEach; accessed by Current,
+    Position, Last and focus-item Value lookups.
+    """
+
+    item: Any
+    position: int
+    size: int