Add OpenResty guestbook example: Shen web app on nginx+LuaJIT

.gitignore

@@ -5,3 +5,4 @@ build/shen-bundle.lua
 .fasl-test/
 luacov.stats.out
 luacov.report.out
+examples/openresty/logs/

README.md

@@ -257,11 +257,20 @@ exhaustively at the top of [`lua_interop.lua`](lua_interop.lua).
 
 ## Examples
 
+The flagship is **[`examples/openresty/`](examples/openresty/)** — a complete
+guestbook web app whose validation rules are written once in Shen and run on
+*both* ends: as a typechecked core on the server (shen-lua inside OpenResty) and
+as a [Ratatoskr](https://github.com/pyrex41/ratatoskr)-shaken,
+[ShenScript](https://github.com/pyrex41/ShenScript)-compiled module in the
+browser. One `rules.shen`, two runtimes, no client/server drift. See its
+[README](examples/openresty/README.md) for the walkthrough.
+
 | | |
 |---|---|
 | [`examples/hello_embed.lua`](examples/hello_embed.lua) | the smallest useful embedding: boot, define a typed function, call it both ways (~25 lines) |
 | [`examples/family.shen`](examples/family.shen) | Shen Prolog in twenty lines: facts, rules, queries via `bin/shen` |
 | [`examples/config_check.lua`](examples/config_check.lua) | the showcase: Shen datatypes + rules as a **typed validation layer** for nested Lua config tables — the typechecker rejects buggy rules at load time ([walkthrough](examples/README.md)) |
+| [`examples/openresty/`](examples/openresty/) | a **complete web app in Shen on OpenResty** (nginx + LuaJIT): typed request validators + a Shen router behind a JSON API, with a front end that runs the **same** typed rules in the browser — Ratatoskr-shaken and ShenScript-compiled to a ~140 KB module. One `rules.shen`, validated client- and server-side. Runs standalone (`luajit examples/openresty/selftest.lua`) or under `openresty` ([README](examples/openresty/README.md)) |
 
 ## Certification / Testing
 

examples/README.md

@@ -1,13 +1,14 @@
 # Examples
 
-Three examples, smallest first (all run with plain `luajit`/`bin/shen`,
-no external dependencies, no network):
+Four examples, smallest first (the first three run with plain
+`luajit`/`bin/shen`, no external dependencies, no network):
 
 | | |
 |---|---|
 | [`hello_embed.lua`](hello_embed.lua) | the smallest useful embedding: boot, define a typed Shen function, call it from Lua, pass lists both ways. `luajit examples/hello_embed.lua` |
 | [`family.shen`](family.shen) | Shen Prolog in twenty lines — facts, rules, yes/no and binding queries. `bin/shen examples/family.shen` |
 | [`config_check.lua`](config_check.lua) | the showcase, walked through below. `luajit examples/config_check.lua` |
+| [`openresty/`](openresty/) | a complete web app — typed Shen validators + a Shen router on OpenResty (nginx + LuaJIT), with a front end that runs the **same** rules in the browser (Ratatoskr-shaken, ShenScript-compiled). Runs standalone via `luajit examples/openresty/selftest.lua`; see [its README](openresty/README.md) to serve it. |
 
 ---
 

examples/openresty/README.md

@@ -0,0 +1,194 @@
+# A web app in Shen, on OpenResty
+
+A small but complete web app whose **server-side logic is written in Shen** and
+runs on [OpenResty](https://openresty.org) (nginx + LuaJIT). It's a guestbook:
+a JSON API plus a one-file front end. The point isn't the guestbook — it's the
+shape of a real Shen web app and the handful of rules that make it work.
+
+OpenResty is nginx embedding **LuaJIT 2.1**, which is exactly shen-lua's primary
+host, so the backend can literally be Shen with a thin Lua glue layer. The front
+end runs Shen too: the browser validates with a build of the **same**
+`rules.shen`, compiled to JavaScript by
+[ShenScript](https://github.com/pyrex41/ShenScript) and tree-shaken by
+[Ratatoskr](https://github.com/pyrex41/ratatoskr), so the field rules are
+checked client-side AND server-side from one source of truth.
+
+```
+examples/openresty/
+  rules.shen      the TYPED core — field rules, proved sound at load time;
+                  loaded by the server AND shaken into the browser build
+  app.shen        the router — dispatch + storage orchestration (untyped shell)
+  app.lua         the glue — boots Shen, marshals JSON <-> Shen, the handler
+  nginx.conf      OpenResty config: boot once per worker, serve rules.shen
+  public/         the front end (imports the shaken validator module)
+  public/vendor/shen-rules.client.js  the shaken+compiled client validator
+                  (~140 KB, generated from rules.shen — see "Front end" below)
+  scripts/        build-client.sh + helpers that regenerate that module
+  selftest.lua    drives the whole app under plain luajit, no nginx needed
+  json_shim.lua   a tiny JSON codec used only off-nginx (OpenResty has cjson)
+```
+
+## Try it without nginx
+
+The Shen code and all of its routing/validation logic run under plain LuaJIT:
+
+```sh
+luajit examples/openresty/selftest.lua
+```
+
+```
+== guestbook API (in-memory store) ==
+GET empty list               -> 200  {"messages":[]}
+POST valid                   -> 201  {"ok":true}
+GET list (2 rows)            -> 200  {"messages":[{"name":"ada",...}]}
+POST missing name            -> 400  {"errors":["name: is required"]}
+POST blank message           -> 400  {"errors":["message: must be 1..280 characters"]}
+POST not an object           -> 400  {"errors":["body: must be a JSON object"]}
+unknown route -> 404         -> 404  {"error":"not found"}
+OK — all cases passed
+```
+
+## Run it under OpenResty
+
+The browser validator (`public/vendor/shen-rules.client.js`) is committed, so
+there is nothing to vendor — just run it. With `openresty` on your PATH, from
+the repo root:
+
+```sh
+mkdir -p examples/openresty/logs
+openresty -p "$PWD/examples/openresty" -c nginx.conf
+```
+
+Then open <http://127.0.0.1:8080/> and sign the guestbook. The API is also
+reachable directly:
+
+```sh
+curl -s localhost:8080/api/messages
+curl -s localhost:8080/api/messages -d '{"name":"ada","message":"hi"}'
+curl -s localhost:8080/api/messages -d '{"message":"no name"}'   # -> 400 + typed errors
+```
+
+(`-p` sets the nginx prefix to this directory, so `logs/` and `public/` resolve
+here; `init_by_lua` derives the repo root from the prefix to put shen-lua on
+`package.path`.)
+
+## How it fits together
+
+```
+browser ─ validate-message in a shaken build of rules.shen ─ instant feedback
+   │  HTTP (only client-valid requests)
+   ▼
+nginx location /api/  ──> app.lua: handle()
+                              │  JSON  -> Shen `val`
+                              ▼
+                      (route Method Path Body)   app.shen   [untyped shell]
+                              │  └─ (validate-message Body)  rules.shen [typed core]
+                              │  └─ (lua.call "host.store_*") -> lua_shared_dict
+                              ▼
+                          [Status BodyVal]
+                              │  Shen `val` -> JSON
+                              ▼
+                          HTTP response
+
+The browser runs validate-message from a tree-shaken build of rules.shen before
+posting; the server re-runs rules.shen as the authoritative check. One source,
+both ends.
+```
+
+### Boot once per worker — the one rule that matters
+
+Booting the Shen kernel (and typechecking `rules.shen`) costs real time —
+tens of ms warm from the bytecode cache, ~1 s cold. `nginx.conf` does it in
+`init_worker_by_lua`, so it happens **once per worker, never per request**. A
+warm worker handles requests with no per-call boot cost.
+
+### Typed core, untyped shell
+
+- **`rules.shen` loads under `(tc +)`.** Its field rules are checked by
+  Shen's sequent-calculus typechecker *at load time* — a type error in a
+  validator aborts startup, before the server ever takes a request (the same
+  guarantee as [`examples/config_rules.shen`](../config_rules.shen)). It is
+  pure, portable Shen (`cn`/`str`/`tlstr` only, no host bridges), which is
+  exactly why the browser can load the same file.
+- **`app.shen` loads under `(tc -)`.** Routing and storage are effectful (they
+  touch nginx and a shared dict), so this half is untyped on purpose. It still
+  calls the typed validators directly — both files load into one environment.
+
+### Crossing the Lua boundary
+
+`app.lua` marshals a cjson-decoded request body into the tagged `val` shape
+(`"x"` → `[s "x"]`, `8080` → `[n 8080]`, objects → `[obj [[k v] ...]]`) that the
+Shen rules pattern-match, and marshals the `[Status BodyVal]` result back to
+JSON. Storage is reached the other way: the Shen router calls
+`(lua.call "host.store_add" ...)` against plain Lua functions backed by a
+`lua_shared_dict`. See [`lua_interop.lua`](../../lua_interop.lua) for the
+marshaling rules. (The browser does the same marshaling in JS — see
+`public/index.html`, where two form strings become a `val` for the in-browser
+`validate-message`.)
+
+## Things to know before building on this
+
+- **Never use Shen's native file I/O for DB/network calls** — it's blocking and
+  would stall nginx's event loop. Reach the outside world through OpenResty's
+  non-blocking cosocket libraries (`lua-resty-mysql`, `lua-resty-http`, …)
+  called from Shen via `lua.call`, exactly as storage is here.
+- **CPU-bound Shen blocks the worker.** A heavy Prolog query or typecheck holds
+  the worker until it returns. Fine for routing/validation; mind anything long.
+- **Kernel state is per worker.** Globals don't cross workers — use
+  `lua_shared_dict`, Redis, or a DB for shared state (storage here is a shared
+  dict, so it's visible to every worker).
+- **The bytecode cache** (`.shen-kernel-cache.bin`) is written in the worker's
+  cwd (the nginx prefix) on first boot; it's gitignored.
+
+## The front end: Shen in the browser, tree-shaken
+
+`public/index.html` runs Shen **in the browser**. It imports
+`public/vendor/shen-rules.client.js`, calls `createValidator()`, and uses the
+result to check the form *in the browser* (an invalid entry never reaches the
+network); only a client-valid entry is POSTed, where the server re-runs
+`rules.shen` as the authoritative check.
+
+That client module is not the whole ShenScript kernel — it is a **tree-shaken
+build of `rules.shen`**:
+
+1. [Ratatoskr](https://github.com/pyrex41/ratatoskr), a Shen tree-shaker, walks
+   the kernel call graph and emits only the ~100 kernel functions these rules
+   can reach. Because the rules never touch `eval`/`read`/`tc`, the reader, the
+   macro expander, the typechecker and `eval` itself all fall away
+   (`needs-eval=false` in the manifest).
+2. [ShenScript](https://github.com/pyrex41/ShenScript) compiles that slice to
+   JavaScript ahead of time, and `scripts/build-client.mjs` wraps it as a
+   self-contained ES module that exports `createValidator()`.
+
+The result is ~140 KB and inits in tens of milliseconds, versus ~660 KB and
+~2.3 s for the full ShenScript kernel bundle. It embeds ShenScript's pure
+`runtime.js`/`overrides.js`, so it needs no ShenScript checkout or npm install
+at runtime — just the committed file.
+
+This is the payoff of shen-lua (server) and ShenScript (browser) being two
+ports of the **same language**: the field rules live in one typed `.shen` file,
+proved sound at server startup and *generated into* the client build — one type
+system, no client/server drift. The only browser-only code is the marshaling
+glue ([`scripts/client.glue.shen`](scripts/client.glue.shen), four lines that
+turn two form strings into the tagged `val` the rules match); everything about
+*what counts as valid* lives in `rules.shen`.
+
+### Regenerating the client module
+
+The committed `shen-rules.client.js` is generated; rerun the build whenever
+`rules.shen` changes so client and server stay in lockstep:
+
+```sh
+examples/openresty/scripts/build-client.sh
+```
+
+It needs sibling checkouts of [Ratatoskr](https://github.com/pyrex41/ratatoskr)
+(the `ratatoskr` binary) and [ShenScript](https://github.com/pyrex41/ShenScript),
+plus `luajit` (the shake host) and Node 20+. Override locations with
+`$RATATOSKR` and `$SHENSCRIPT_DIR`. The script concatenates `rules.shen` +
+`scripts/client.glue.shen`, shakes the slice, and compiles it to the module.
+
+The running page is self-documenting too: a "What this demonstrates" panel with
+a browser→server flow diagram, links to the live `/rules.shen`, and an
+expandable view of the rule source — so anyone opening the example sees the
+rules and the architecture without reading the code.

examples/openresty/app.lua

@@ -0,0 +1,142 @@
+-- examples/openresty/app.lua — the glue between OpenResty and shen-lua.
+--
+-- Loaded once per nginx worker (from init_worker_by_lua in nginx.conf): it
+-- boots the Shen kernel, registers the typed Lua bridges, loads the typed
+-- core (validate.shen) and the router (app.shen), and exposes M.handle() as
+-- the content handler. Per request it marshals nginx data into Shen, calls
+-- (route Method Path Body), and turns the [Status BodyVal] result into JSON.
+--
+-- The expensive work (kernel boot ~ tens of ms warm, ~1 s cold) happens at
+-- module-load time, i.e. ONCE per worker — never per request. This is the
+-- single most important rule for running shen-lua under nginx.
+
+-- Resolve this file's own directory so the .shen loads are absolute and don't
+-- depend on the nginx worker's cwd.
+local APP_DIR = debug.getinfo(1, "S").source:match("^@(.*)[/\\][^/\\]+$") or "."
+
+local shen = require("shen")
+local IO   = require("lua_interop")   -- the marshaling API (Lua array <-> Shen list)
+local P    = shen.prims               -- F-table, load, ...
+
+-- cjson ships with OpenResty. Off-nginx (e.g. selftest.lua under plain
+-- luajit) it may be absent, so fall back to a tiny self-contained JSON shim
+-- with the same surface we use: decode, encode, and the empty_array sentinel.
+local cjson
+do
+  local ok, m = pcall(require, "cjson.safe")
+  if not ok then ok, m = pcall(require, "cjson") end
+  cjson = ok and m or assert(loadfile(APP_DIR .. "/json_shim.lua"))()
+end
+
+-- A pluggable store so this module is testable off-nginx (selftest.lua injects
+-- an in-memory table; under nginx M.use_store wires it to a lua_shared_dict).
+local store = {
+  add  = function(_, _) return 0 end,
+  list = function() return {} end,
+}
+local function set_store(s) store = s end
+
+-- ---- host services the Shen code calls (Shen -> Lua) ------------------------
+-- `host` is a global so the dotted paths below ("host.store_add", ...) resolve
+-- through lua.call / lua.function, the same convention as examples/config_check.
+host = {
+  -- storage, delegated to whatever store is installed
+  store_add  = function(name, message) return store.add(name, message) end,
+  store_list = function() return store.list() end,
+}
+
+shen.boot{quiet = true}
+
+-- Load the typed core under (tc +) and the router under (tc -). rules.shen is
+-- the SAME file the browser loads via ShenScript (see public/index.html) — one
+-- source of truth for the field rules. It needs no Lua bridges: it is pure,
+-- portable Shen (cn/str/tlstr only), which is exactly why it runs unchanged on
+-- both ports. A type error in it aborts startup, before the first request.
+shen.eval("(tc +)")
+P.F["load"](APP_DIR .. "/rules.shen")
+shen.eval("(tc -)")
+P.F["load"](APP_DIR .. "/app.shen")
+
+local route = IO.fn("route")   -- marshals args in and the result back out
+
+-- ---- marshaling: cjson value <-> Shen `val` ---------------------------------
+local sym = IO.sym
+
+-- Lua (cjson-decoded) -> Shen `val`. Tags are interned SYMBOLS so the Shen
+-- patterns [s X]/[n X]/... match; strings stay strings (never auto-interned).
+local function to_val(v)
+  local t = type(v)
+  if t == "string"  then return { sym("s"), v } end
+  if t == "number"  then return { sym("n"), v } end
+  if t == "boolean" then return { sym("b"), v } end
+  if t == "table" then
+    if v[1] ~= nil or next(v) == nil then        -- array (or empty) -> [arr ...]
+      local a = {}
+      for i, e in ipairs(v) do a[i] = to_val(e) end
+      return { sym("arr"), a }
+    end
+    local es, i = {}, 0                            -- object -> [obj [[k v] ...]]
+    for k, val in pairs(v) do
+      if type(k) == "string" then i = i + 1; es[i] = { k, to_val(val) } end
+    end
+    return { sym("obj"), es }
+  end
+  return { sym("s"), tostring(v) }                 -- null / unsupported -> string
+end
+
+-- Shen `val` (already marshaled by IO.fn to nested Lua arrays, tags as
+-- strings) -> a plain Lua value ready for cjson.encode.
+local function from_val(v)
+  local tag, payload = v[1], v[2]
+  if tag == "s" or tag == "n" or tag == "b" then return payload end
+  if tag == "arr" then
+    if #payload == 0 then return cjson.empty_array end
+    local out = {}
+    for i, e in ipairs(payload) do out[i] = from_val(e) end
+    return out
+  end
+  if tag == "obj" then
+    local o = {}
+    for _, pair in ipairs(payload) do o[pair[1]] = from_val(pair[2]) end
+    return o
+  end
+  return nil
+end
+
+-- ---- the request handler ----------------------------------------------------
+-- Pure given (method, path, decoded-body); shared with selftest.lua.
+local function dispatch(method, path, decoded_body)
+  local body = decoded_body and to_val(decoded_body) or nil
+  local resp = route(method, path, body)           -- { Status, BodyVal }
+  return resp[1], from_val(resp[2])
+end
+
+local M = { dispatch = dispatch, to_val = to_val, from_val = from_val,
+            use_store = set_store, json = cjson }
+
+-- content_by_lua entry point.
+function M.handle()
+  local method = ngx.req.get_method()
+  local path   = ngx.var.uri
+  local decoded
+  if method == "POST" or method == "PUT" then
+    ngx.req.read_body()
+    local raw = ngx.req.get_body_data()
+    if raw and raw ~= "" then
+      local d, err = cjson.decode(raw)
+      if d == nil then
+        ngx.status = 400
+        ngx.header.content_type = "application/json"
+        ngx.say(cjson.encode({ errors = { "invalid JSON: " .. tostring(err) } }))
+        return
+      end
+      decoded = d
+    end
+  end
+  local status, body = dispatch(method, path, decoded)
+  ngx.status = status
+  ngx.header.content_type = "application/json"
+  ngx.say(cjson.encode(body))
+end
+
+return M