Add Rayforce

rayforce/.gitignore

@@ -0,0 +1,2 @@
+# Splayed table written by ./load (one file per column plus .sym).
+hits/

rayforce/README.md

@@ -0,0 +1,144 @@
+# Rayforce
+
+[Rayforce](https://github.com/RayforceDB/rayforce) is a zero-dependency
+columnar analytics and graph engine written in pure C (MIT). Columnar
+scans and graph traversals share one operation DAG, which is optimized
+and then executed as morsel-driven bytecode over 1024-row batches.
+Queries are written in **Rayfall**, Rayforce's Lisp-like query language,
+so `queries.sql` holds 43 Rayfall expressions (one per line) rather than
+SQL.
+
+## Install
+
+`./install` builds from source with `make release`. Rayforce publishes an
+amd64 `.deb` and prebuilt tarballs, but ClickBench also runs on aarch64
+machines and the engine is dependency-free C that builds with a plain
+`make` on both, so building is the only setup that covers every machine
+in the matrix. `RAYFORCE_VERSION=vX.Y.Z ./install` pins a release;
+unset, it takes the latest GitHub release.
+
+## Data layout
+
+`./load` streams `hits.csv` into a splayed on-disk table under `./hits`
+with `.csv.splayed` — one file per column plus the `.sym` dictionary,
+parsed in parallel from an mmap of the CSV without materializing the
+table in memory. The column names come from `create.rfl`; passing an
+explicit name vector also tells the reader the input has no header row,
+which is the shape of the published `hits.csv`.
+
+Types follow `../clickhouse/create.sql`: `BIGINT` → `I64`, `INTEGER` →
+`I32`, `SMALLINT` → `I16`, `TIMESTAMP` → `TIMESTAMP`, `Date` → `DATE`.
+Rayforce's CSV reader parses the `YYYY-MM-DD` and `YYYY-MM-DD HH:MM:SS`
+forms in the file into its native date/timestamp types, so
+`EventDate >= '2013-07-01'`, `extract(minute FROM EventTime)` and
+`DATE_TRUNC('minute', EventTime)` all work on native values.
+
+### Why every text column is SYM
+
+Rayforce has two text column types: `SYM` (dictionary-encoded, one
+integer index per row into a global intern table) and `STR`
+(variable-length, 12 bytes inline or a per-vector byte pool). `STR` is
+the type its docs recommend for high-cardinality text such as URLs, but
+it does not reach ClickBench scale in the current release:
+
+* **The pool offset is a `uint32_t`**, so a column's pool is capped at
+  4 GiB (`src/io/csv.c` bails out above it). At 100M rows the `URL`,
+  `Title`, `Referer` and `OriginalURL` columns hold roughly 9, 11, 8 and
+  5 GB of bytes, so none of them fit in one `STR` column.
+* **Opening a `STR` column validates every element** (bounds plus a
+  4-byte prefix compare against the pool, `col_validate_str_region`),
+  and the cost is superlinear: on a subset of this dataset a splayed
+  table with five `STR` columns opened in 0.4 s at 2M rows but 199 s at
+  9.6M rows. The same table with dictionary-encoded text opened in
+  17 s at 9.6M rows.
+
+Loading all TEXT / VARCHAR / CHAR columns as `SYM` sidesteps both: it
+has no 4 GiB limit, and the columns become narrow integer vectors (the
+9.6M-row subset is 4.4 GB with every text column dictionary-encoded,
+versus 7.5 GB with those five columns as `STR`).
+Dictionary encoding is what makes the dataset loadable at all here; it
+also makes `GROUP BY URL` an integer group-by, while the string
+operations in Q28 and Q29 pay an extra indirection per row to resolve
+symbol ids back to bytes. A third reason not to go back to `STR` for
+now: `take:` — i.e. every `LIMIT` — currently returns empty strings for
+pool-backed values
+([RayforceDB/rayforce#404](https://github.com/RayforceDB/rayforce/issues/404)).
+
+## Server mode
+
+Rayforce is embeddable and normally invoked as a CLI, but this entry runs
+it as an IPC server (`./start` → `rayforce -p 5000 server.rfl`) because
+opening the table is eager: it validates every column file and loads the
+symbol dictionary before the first query. Paying that once per server
+start instead of once per query process keeps it out of the reported
+numbers and out of the run's wall clock. The listening socket only
+accepts connections after `server.rfl` finishes, so `./check` — a
+one-expression IPC round trip — is a genuine readiness probe.
+
+`./query` sends the query text; the server evaluates
+`(timeit (set rf-result <query>))`, which returns the elapsed
+milliseconds from a nanosecond clock, and a second untimed round trip
+pulls the result back so it can be printed. Timing therefore covers
+server-side query execution only, the same convention as the other
+embedded engines here (e.g. DuckDB's `.timer`).
+
+### Cold runs
+
+`BENCH_RESTARTABLE=yes` is required, not optional: with the server left
+running, `drop_caches` cannot evict pages that a live process still has
+mapped, and a "cold" query measured 0.029 s — exactly its warm time —
+versus 0.44 s after a real restart. Restarting between queries makes the
+cold number honest, at the price of re-reading the table on every start
+(hence `BENCH_CHECK_TIMEOUT=1800`). Note the flip side: because the open
+is eager, on a machine whose RAM comfortably exceeds the dataset the
+first query still runs against a fully resident table.
+
+## Query adaptations
+
+The queries are direct Rayfall translations of the ClickBench SQL. All 43
+were checked against `clickhouse-local` on a 2M-row subset of the same
+CSV; the only differences are which of several equally-ranked rows a
+`LIMIT` over ties returns, plus the projection differences noted below.
+Points worth knowing:
+
+* **`COUNT(*)`** is `(count hits)`; `COUNT(*)` over a filter is
+  `(count (select {...}))`.
+* **Ungrouped `COUNT(DISTINCT c)`** (Q5, Q6) is a whole-column reducer,
+  `(count (distinct (at hits 'c)))`, and not a `select` projection: the
+  projection form returns the row count instead of the distinct count
+  ([RayforceDB/rayforce#405](https://github.com/RayforceDB/rayforce/issues/405)).
+  Per-group `COUNT(DISTINCT c)` (Q9-Q12, Q14, Q23) is written inline as
+  `(count (distinct UserID))` and lowers to Rayforce's grouped
+  count-distinct kernels.
+* **`HAVING`** (Q28, Q29) has no clause form: the group-by runs as an
+  inner `select` and the outer one filters on the aggregate.
+* **`LIMIT n OFFSET m`** (Q39-Q43) is `take: [m n]`.
+* **`extract(minute FROM EventTime)`** (Q19) is `(minute EventTime)`;
+  **`DATE_TRUNC('minute', EventTime)`** (Q43) is
+  `(xbar EventTime 60000000000)`, i.e. truncation to whole minutes of
+  the nanosecond timestamp.
+* **`CASE WHEN ... END`** (Q40) is the row-wise `(if cond then else)`,
+  which the query compiler lowers to the DAG's ternary select.
+* **`ORDER BY` a column that is not selected** (Q25, Q27): sorting
+  happens after projection, so those queries project `EventTime`
+  alongside `SearchPhrase`. Same rows, same order, one extra column in
+  the printed output.
+* **`GROUP BY 1, URL`** (Q35): a constant is not accepted as a group
+  key, and grouping by `(1, URL)` is the same partition as grouping by
+  `URL`, so the constant is projected by an outer `select` instead.
+* **Derived group keys** (Q36) go in the `by:` dict as
+  `ip1: (- ClientIP 1)`, which the optimizer turns into a synthetic
+  column rather than a materialized one. Rayforce emits the aggregate
+  before the derived keys, so the result has the same rows as the SQL
+  with the columns in a different order.
+* **`REGEXP_REPLACE`** (Q29): Rayforce has no regex engine, so the
+  host extraction `^https?://(?:www\.)?([^/]+)/.*$` is spelled out with
+  the string builtins — `str-find` for `://`, `www.` and the first `/`,
+  `substr` to cut, and `if` to fall back to the whole `Referer` when the pattern
+  does not match (no `http`/`https` scheme at offset 0, or no `/` after
+  the host). On this dataset it reproduces `REGEXP_REPLACE` exactly,
+  including the fallback rows: the group keys, counts, `MIN(Referer)`
+  and average lengths all match ClickHouse.
+* **`MIN(URL)` / `MIN(Title)`** (Q22, Q23) work directly on `SYM`
+  columns and return the lexicographic minimum, not the minimum
+  dictionary index.

rayforce/benchmark.sh

@@ -0,0 +1,11 @@
+#!/bin/bash
+export BENCH_DOWNLOAD_SCRIPT="download-hits-csv"
+# Rayforce runs as an IPC server here (./start), so the stop/drop_caches/start
+# cold cycle is meaningful and the concurrent-QPS test hits a shared process.
+export BENCH_RESTARTABLE=yes
+export BENCH_DURABLE=yes
+# Opening the splayed table validates every column file and loads the symbol
+# dictionary; at 100M rows that takes minutes, and it happens on every restart,
+# so the readiness probe needs a much longer window than the 300s default.
+export BENCH_CHECK_TIMEOUT="${BENCH_CHECK_TIMEOUT:-1800}"
+exec ../lib/benchmark-common.sh

rayforce/check

@@ -0,0 +1,4 @@
+#!/bin/bash
+set -e
+export RAYFORCE_PORT="${RAYFORCE_PORT:-5000}"
+rayforce check.rfl >/dev/null

rayforce/check.rfl

@@ -0,0 +1,8 @@
+; Readiness probe: connect over IPC and evaluate a trivial expression.
+; The server binds its socket before running server.rfl but only accepts
+; connections once that script has finished opening the table, so a successful
+; round trip means "table open, ready to query".
+(set h (.ipc.open (format "127.0.0.1:%" (.os.getenv "RAYFORCE_PORT")) 2000))
+(set ok (.ipc.send h "(+ 1 1)"))
+(.ipc.close h)
+(if (== ok 2) (exit 0) (exit 1))

rayforce/create.rfl

@@ -0,0 +1,42 @@
+; ClickBench `hits` schema for Rayforce.
+;
+; `load` feeds these two vectors to `.csv.splayed`, which streams
+; hits.csv straight into the on-disk splayed table without a header row
+; (an explicit name vector implies headerless input).
+;
+; Types follow ../clickhouse/create.sql: BIGINT -> I64, INTEGER -> I32,
+; SMALLINT -> I16, TIMESTAMP -> TIMESTAMP, Date -> DATE.  Every TEXT /
+; VARCHAR / CHAR column is loaded as SYM (dictionary-encoded) rather
+; than STR -- see README.md "Why every text column is SYM".
+
+(set hits-names [
+  WatchID JavaEnable Title GoodEvent EventTime EventDate CounterID ClientIP
+  RegionID UserID CounterClass OS UserAgent URL Referer IsRefresh
+  RefererCategoryID RefererRegionID URLCategoryID URLRegionID
+  ResolutionWidth ResolutionHeight ResolutionDepth FlashMajor FlashMinor
+  FlashMinor2 NetMajor NetMinor UserAgentMajor UserAgentMinor CookieEnable
+  JavascriptEnable IsMobile MobilePhone MobilePhoneModel Params IPNetworkID
+  TraficSourceID SearchEngineID SearchPhrase AdvEngineID IsArtifical
+  WindowClientWidth WindowClientHeight ClientTimeZone ClientEventTime
+  SilverlightVersion1 SilverlightVersion2 SilverlightVersion3
+  SilverlightVersion4 PageCharset CodeVersion IsLink IsDownload IsNotBounce
+  FUniqID OriginalURL HID IsOldCounter IsEvent IsParameter DontCountHits
+  WithHash HitColor LocalEventTime Age Sex Income Interests Robotness
+  RemoteIP WindowName OpenerName HistoryLength BrowserLanguage
+  BrowserCountry SocialNetwork SocialAction HTTPError SendTiming DNSTiming
+  ConnectTiming ResponseStartTiming ResponseEndTiming FetchTiming
+  SocialSourceNetworkID SocialSourcePage ParamPrice ParamOrderID
+  ParamCurrency ParamCurrencyID OpenstatServiceName OpenstatCampaignID
+  OpenstatAdID OpenstatSourceID UTMSource UTMMedium UTMCampaign UTMContent
+  UTMTerm FromTag HasGCLID RefererHash URLHash CLID
+])
+
+(set hits-types [
+  I64 I16 SYM I16 TIMESTAMP DATE I32 I32 I32 I64 I16 I16 I16 SYM SYM I16
+  I16 I32 I16 I32 I16 I16 I16 I16 I16 SYM I16 I16 I16 SYM I16 I16 I16 I16
+  SYM SYM I32 I16 I16 SYM I16 I16 I16 I16 I16 TIMESTAMP I16 I16 I32 I16 SYM
+  I32 I16 I16 I16 I64 SYM I32 I16 I16 I16 I16 I16 SYM TIMESTAMP I16 I16 I16
+  I16 I16 I32 I32 I32 I16 SYM SYM SYM SYM I16 I32 I32 I32 I32 I32 I32 I16
+  SYM I64 SYM SYM I16 SYM SYM SYM SYM SYM SYM SYM SYM SYM SYM I16 I64 I64
+  I32
+])

rayforce/data-size

@@ -0,0 +1,5 @@
+#!/bin/bash
+set -e
+
+# Splayed table: one file per column plus the `.sym` dictionary, all under ./hits.
+du -sb hits | cut -f1

rayforce/install

@@ -0,0 +1,36 @@
+#!/bin/bash
+# Builds Rayforce from source.
+#
+# Rayforce publishes an amd64 .deb and prebuilt tarballs, but ClickBench also
+# runs on aarch64 machines (c8g.*), and the engine is zero-dependency C that
+# builds with a plain `make release` on both. Building from source is therefore
+# the only setup that works everywhere, and it is what upstream documents first.
+#
+# RAYFORCE_VERSION pins the release tag; unset means "latest GitHub release"
+# (falling back to a known-good tag when the unauthenticated API call is rate
+# limited).
+set -e
+
+RAYFORCE_FALLBACK_VERSION=v2.5.14
+
+if ! command -v rayforce >/dev/null 2>&1; then
+    sudo apt-get update
+    sudo apt-get install -y build-essential git curl
+
+    version="${RAYFORCE_VERSION:-}"
+    if [ -z "$version" ]; then
+        version=$(curl -sS --max-time 30 \
+            https://api.github.com/repos/RayforceDB/rayforce/releases/latest \
+            | grep -o '"tag_name": *"[^"]*"' | head -n1 | cut -d'"' -f4 || true)
+        [ -z "$version" ] && version="$RAYFORCE_FALLBACK_VERSION"
+    fi
+    echo "Building Rayforce $version"
+
+    rm -rf "$HOME/rayforce-src"
+    git clone --depth 1 --branch "$version" \
+        https://github.com/RayforceDB/rayforce "$HOME/rayforce-src"
+    make -C "$HOME/rayforce-src" release -j"$(nproc)"
+    sudo install -m 755 "$HOME/rayforce-src/rayforce" /usr/local/bin/rayforce
+fi
+
+rayforce --help >/dev/null

rayforce/load

@@ -0,0 +1,15 @@
+#!/bin/bash
+# Loads hits.csv into the splayed table ./hits.
+#
+# The load runs in its own rayforce process, not through the server: the server
+# started by the driver before ./load has no table open yet and picks it up at
+# the next restart, which happens before the first query.
+set -e
+
+rm -rf hits
+
+rayforce load.rfl
+
+# The CSV is no longer needed once the columns are on disk.
+rm -f hits.csv
+sync

rayforce/load.rfl

@@ -0,0 +1,9 @@
+; Stream hits.csv into a splayed on-disk table under ./hits.
+;
+; `.csv.splayed` mmaps the CSV, parses chunks in parallel and writes one column
+; file per field plus the `.sym` dictionary, without materialising the whole
+; table in memory. The name vector implies the input has no header row, which is
+; the shape of the published hits.csv.
+(load "create.rfl")
+(.csv.splayed hits-names hits-types "hits.csv" "hits")
+(exit 0)