From ba55951cd0a0ae6d671563d225e3a46411fd5550 Mon Sep 17 00:00:00 2001 From: plx Date: Fri, 17 Jul 2026 16:30:41 -0600 Subject: [PATCH 1/3] Post-vtable-refactor: docs + relocate plx_diag_prefix into the engine Follow-up to #2 (the PlxSurface.parse_body vtable refactor). Docs: - ARCHITECTURE.md: rewrite "The transpiler" to describe the engine + per-dialect parse_body vtable split (plx_engine.h, plx_parse_brace.c); refresh the file list and dialect list to the current 9 dialects; add the missing per-dialect chapter links. - CHANGELOG.md: add an Unreleased entry for the refactor + this follow-up. Code: - Move the dialect-neutral plx_diag_prefix() helper out of plx_dialect_ruby.c and back into the engine (plx_transpile.c), where its other callers (plx_dialect_python.c, plx_parse_brace.c) already reach it. The refactor had parked this shared engine helper in a dialect front end. Pure relocation, no functional change. Verified: clean rebuild (zero warnings) and 13/13 installcheck pass. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 16 +++++++++ doc/ARCHITECTURE.md | 75 ++++++++++++++++++++++++++++++------------ src/plx_dialect_ruby.c | 28 ---------------- src/plx_transpile.c | 28 ++++++++++++++++ 4 files changed, 98 insertions(+), 49 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ecb6281..d0896e9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,22 @@ All notable changes to plx are recorded here. The format follows [Keep a Changelog](https://keepachangelog.com/), and plx uses the extension version in `plx.control` (currently `1.0`). +## [Unreleased] + +### Changed + +- Internal refactor of the transpiler behind a `PlxSurface.parse_body` vtable + (#2). The single ~11k-line `src/plx_transpile.c` is split into a + dialect-neutral engine (declared in the new `src/plx_engine.h`) plus per-dialect + front ends in `src/plx_dialect_*.c` and the shared `src/plx_parse_brace.c`; + the hardcoded `block_style` dispatch is replaced by a per-dialect `parse_body` + function pointer. No functional change — generated plpgsql is byte-identical + and all regression tests pass. Adding a dialect no longer touches shared code. +- Follow-up to the above: relocated the dialect-neutral `plx_diag_prefix()` + helper out of `src/plx_dialect_ruby.c` and back into the engine + (`src/plx_transpile.c`), where its other callers (Python and the brace + front end) already live. No functional change. + ## [1.3.1] - 2026-07-16 Code-only patch release (no catalog changes) carrying the memory-safety and diff --git a/doc/ARCHITECTURE.md b/doc/ARCHITECTURE.md index 2ba8d36..7d80805 100644 --- a/doc/ARCHITECTURE.md +++ b/doc/ARCHITECTURE.md @@ -1,8 +1,9 @@ # plx Architecture -plx lets a function body be written in a Ruby, PHP, JavaScript, or Python -dialect and executed by the standard plpgsql interpreter. This document -describes how that works in the extension as built. +plx lets a function body be written in a Ruby, PHP, JavaScript, TypeScript, +Python, Go, COBOL, Oracle PL/SQL, or Transact-SQL dialect and executed by the +standard plpgsql interpreter. This document describes how that works in the +extension as built. ## The core idea @@ -85,16 +86,36 @@ It is dialect-pluggable through a `PlxSurface` (in `plx_int.h`) that each dialec supplies. The surface describes what varies between languages: - the keyword table, mapping each dialect's spellings to canonical keywords; -- the block style: keyword-delimited (`end`), brace-delimited (`{ }`), or - indentation (INDENT/DEDENT); +- the block style, used as a lexer hint (indentation-tokenized vs. + newline-tokenized); - comment syntax, the variable sigil (for example `$`), the string-concatenation operator, and how string interpolation is written (`#{}`, `$var` and `{$e}`, - `${}` template literals, or f-strings). - -The shared code (`plx_transpile.c`) is dialect-neutral: the lexer, the three -block parsers, the expression rewriter, DECLARE-hoisting and type inference, the -statement lowering, and the intrinsics (`query`, `fetch_one`, `perform`, -`execute`, `return_query`, cursors, and so on) are all driven by the surface. + `${}` template literals, or f-strings); +- `parse_body`, the front-end entry point (see below), and + `self_contained_block`, a flag for dialects that emit their own + `DECLARE`/`BEGIN`/`END` (PL/SQL). + +### Engine and front ends + +The transpiler is split into a dialect-neutral **engine** and per-dialect +**front ends**, selected through the `parse_body` function pointer on the +surface (a vtable method). `plx_transpile()` just calls +`cx->surf->parse_body(cx)` and then runs one assemble tail — there is no +per-dialect branching in the driver. + +- The **engine** lives in `plx_transpile.c` and is declared to the front ends + through `plx_engine.h`: the shared byte lexer (`plx_lex`), the expression + rewriter (`plx_rewrite_expr`), the leaf-statement emitter and intrinsics + (`query`, `fetch_one`, `perform`, `execute`, `return_query`, cursors, and so + on), the symbol table, string/interpolation decoding, and the final + DECLARE-hoisting + assemble. It contains no dialect-specific code. +- Each **front end** owns its dialect's tokenizer, parser, and statement + lowering, and implements `parse_body` by transforming `cx->body` into + `cx->out`. The text-family dialects (`php`/`js`/`ts`) share a brace parser in + `plx_parse_brace.c`; Ruby (keyword-`end`) and Python (indentation) parse on + top of the shared lexer inside their own translation units; and the + standalone dialects (COBOL, PL/SQL, T-SQL, Go) run their own tokenizer and + emitter, calling back into the engine only for shared services. ## Files @@ -102,19 +123,29 @@ statement lowering, and the intrinsics (`query`, `fetch_one`, `perform`, plx.control, plx--1.0.sql extension control and install SQL src/plx.h public ABI (PlxDialect, PlxFuncMeta) src/plx_int.h internal ABI (PlxSurface, canonical keywords) +src/plx_engine.h engine interface: PlxCtx, tokens, symtab, and the + plx_* entry points the front ends call src/plx_core.c PL handler binding, registry, generic validator and inline handler -src/plx_transpile.c the shared transpiler -src/plx_dialect_ruby.c the plxruby surface and trampolines -src/plx_dialect_php.c the plxphp surface and trampolines -src/plx_dialect_js.c the plxjs surface and trampolines -src/plx_dialect_python.c the plxpython3 surface and trampolines +src/plx_transpile.c the dialect-neutral engine + plx_transpile() driver +src/plx_strbuild.c string-builder intrinsic helpers +src/plx_parse_brace.c shared brace front end (php/js/ts) + TS preprocess +src/plx_dialect_ruby.c the plxruby surface + Ruby front end +src/plx_dialect_php.c the plxphp surface (parse_body -> brace front end) +src/plx_dialect_js.c the plxjs surface (parse_body -> brace front end) +src/plx_dialect_ts.c the plxts surface (parse_body -> brace front end) +src/plx_dialect_python.c the plxpython3 surface + Python front end +src/plx_dialect_go.c the plxgo surface + Go front end +src/plx_dialect_cobol.c the plxcobol surface + COBOL front end +src/plx_dialect_plsql.c the plxplsql surface + PL/SQL front end +src/plx_dialect_tsql.c the plxtsql surface + T-SQL front end ``` -Everything links into a single `plx.so`. A dialect is a `PlxSurface` plus three -small trampolines (validator, inline handler, and the shared call-handler -binding), registered in `_PG_init`. Adding a dialect is a new surface and a few -`CREATE LANGUAGE` lines; it does not touch the shared transpiler. +Everything links into a single `plx.so`. A dialect is a `PlxSurface` (including +its `parse_body` front end) plus small trampolines (validator, inline handler, +and the shared call-handler binding), registered in `_PG_init`. Adding a dialect +is a new `plx_dialect_X.c` — a surface with its `parse_body`, plus a few +`CREATE LANGUAGE` lines — and does not touch the shared engine. ## Trust @@ -132,4 +163,6 @@ fuzzed (see `test/fuzz.py`). - [TRANSPILER.md](TRANSPILER.md): the original transpiler design specification. - [PARITY.md](PARITY.md): the plpgsql construct parity matrix. - The per-dialect chapters: [plxruby](plxruby.md), [plxphp](plxphp.md), - [plxjs](plxjs.md), [plxpython3](plxpython3.md). + [plxjs](plxjs.md), [plxts](plxts.md), [plxpython3](plxpython3.md), + [plxgo](plxgo.md), [plxcobol](plxcobol.md), [plxplsql](plxplsql.md), + [plxtsql](plxtsql.md). diff --git a/src/plx_dialect_ruby.c b/src/plx_dialect_ruby.c index e5431b7..9e8bc25 100644 --- a/src/plx_dialect_ruby.c +++ b/src/plx_dialect_ruby.c @@ -409,34 +409,6 @@ parse_iter(Ctx *cx, int a, int do_pos, int e, int ind) appendStringInfoString(o, "END LOOP;\n"); } -/* GET STACKED DIAGNOSTICS lines for the fields used in a handler (see diag_mask) */ -char * -plx_diag_prefix(int mask, int ind) -{ - static const struct { int bit; const char *fld; const char *item; } t[] = { - {PLX_DIAG_DETAIL, "detail", "PG_EXCEPTION_DETAIL"}, - {PLX_DIAG_HINT, "hint", "PG_EXCEPTION_HINT"}, - {PLX_DIAG_CONSTRAINT, "constraint", "CONSTRAINT_NAME"}, - {PLX_DIAG_COLUMN, "column", "COLUMN_NAME"}, - {PLX_DIAG_TABLE, "table", "TABLE_NAME"}, - {PLX_DIAG_SCHEMA, "schema", "SCHEMA_NAME"}, - {PLX_DIAG_DATATYPE, "datatype", "PG_DATATYPE_NAME"}, - }; - StringInfoData p; - int i; - - if (!mask) - return pstrdup(""); - initStringInfo(&p); - for (i = 0; i < (int) (sizeof(t) / sizeof(t[0])); i++) - if (mask & t[i].bit) - { - appendStringInfoSpaces(&p, ind * 2); - appendStringInfo(&p, "GET STACKED DIAGNOSTICS __plx_%s = %s;\n", t[i].fld, t[i].item); - } - return p.data; -} - /* begin [body] [rescue [Class] [=> e] handler]* [ensure body] end */ static void parse_begin(Ctx *cx, int ind) diff --git a/src/plx_transpile.c b/src/plx_transpile.c index f981721..061174f 100644 --- a/src/plx_transpile.c +++ b/src/plx_transpile.c @@ -1591,6 +1591,34 @@ plx_parse_args(Ctx *cx, int a, int *as, int *ae, int maxargs, int *after) return n; } +/* GET STACKED DIAGNOSTICS lines for the fields used in a handler (see diag_mask) */ +char * +plx_diag_prefix(int mask, int ind) +{ + static const struct { int bit; const char *fld; const char *item; } t[] = { + {PLX_DIAG_DETAIL, "detail", "PG_EXCEPTION_DETAIL"}, + {PLX_DIAG_HINT, "hint", "PG_EXCEPTION_HINT"}, + {PLX_DIAG_CONSTRAINT, "constraint", "CONSTRAINT_NAME"}, + {PLX_DIAG_COLUMN, "column", "COLUMN_NAME"}, + {PLX_DIAG_TABLE, "table", "TABLE_NAME"}, + {PLX_DIAG_SCHEMA, "schema", "SCHEMA_NAME"}, + {PLX_DIAG_DATATYPE, "datatype", "PG_DATATYPE_NAME"}, + }; + StringInfoData p; + int i; + + if (!mask) + return pstrdup(""); + initStringInfo(&p); + for (i = 0; i < (int) (sizeof(t) / sizeof(t[0])); i++) + if (mask & t[i].bit) + { + appendStringInfoSpaces(&p, ind * 2); + appendStringInfo(&p, "GET STACKED DIAGNOSTICS __plx_%s = %s;\n", t[i].fld, t[i].item); + } + return p.data; +} + /* Emit a Ruby string token as raw SQL text (interp #{e} -> rewritten expr, * inline; no surrounding quotes). Used for static query/perform/fetch SQL. */ void From 320cb57ad55f974bceb9f19ce68bc6cf1eb51dd6 Mon Sep 17 00:00:00 2001 From: plx Date: Fri, 17 Jul 2026 16:40:20 -0600 Subject: [PATCH 2/3] =?UTF-8?q?Post-vtable-refactor:=20tighten=20linkage?= =?UTF-8?q?=20+=20drop=20dead=20code=20(audit=20#2=E2=80=93#4)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Follow-on cleanup from the post-merge audit of #2, in the same spirit as the plx_diag_prefix relocation: - Linkage (#2): the six single-dialect front ends (ruby/python/cobol/plsql/ tsql/go plx_*_parse_body) are only ever referenced by their own surface's .parse_body in their own translation unit, so make them static and drop their prototypes from plx_engine.h. Only plx_brace_parse_body is genuinely shared (php/js/ts) and keeps its prototype. - Dead code (#3): drop the write-only Ctx.nt field (set once in plx_lex, never read; the standalone tokenizers track their own token counts on their own structs). - Comment (#4): COBOL's surface carries no keyword table (.kws = NULL); fix the header comment that claimed a table was "kept for consistency". No functional change. Clean rebuild (zero warnings), 13/13 installcheck pass. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 5 +++++ src/plx_dialect_cobol.c | 6 +++--- src/plx_dialect_go.c | 2 +- src/plx_dialect_plsql.c | 2 +- src/plx_dialect_python.c | 2 +- src/plx_dialect_ruby.c | 2 +- src/plx_dialect_tsql.c | 2 +- src/plx_engine.h | 14 +++++--------- src/plx_transpile.c | 1 - 9 files changed, 18 insertions(+), 18 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d0896e9..da9baec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,11 @@ version in `plx.control` (currently `1.0`). helper out of `src/plx_dialect_ruby.c` and back into the engine (`src/plx_transpile.c`), where its other callers (Python and the brace front end) already live. No functional change. +- Tightened linkage and removed dead code left by the refactor: the six + single-dialect `plx_*_parse_body` front ends are now `static` (only the + shared `plx_brace_parse_body` keeps a prototype in `plx_engine.h`); dropped + the write-only `Ctx.nt` field; and corrected a stale COBOL comment about a + keyword table the surface does not carry. No functional change. ## [1.3.1] - 2026-07-16 diff --git a/src/plx_dialect_cobol.c b/src/plx_dialect_cobol.c index 5fac9db..33ffa49 100644 --- a/src/plx_dialect_cobol.c +++ b/src/plx_dialect_cobol.c @@ -5,8 +5,8 @@ * it does not fit the shared byte lexer's expression model (hyphenated words, * period sentence terminators, PICTURE clauses), so the COBOL front end (below) * uses its own tokenizer and recursive-descent parser and emits plpgsql - * directly; the dialect-neutral engine lives in plx_transpile.c. The keyword - * table is unused by the COBOL path but kept for consistency. + * directly; the dialect-neutral engine lives in plx_transpile.c. The COBOL + * front end does not use the shared keyword table (its surface carries none). */ #include "postgres.h" @@ -2184,7 +2184,7 @@ cob_block(Cb *cb, int ind) cb->cx->depth--; } -void +static void plx_cobol_parse_body(Ctx *cx) { Cb cb; diff --git a/src/plx_dialect_go.c b/src/plx_dialect_go.c index 6e79226..5c2688f 100644 --- a/src/plx_dialect_go.c +++ b/src/plx_dialect_go.c @@ -1913,7 +1913,7 @@ go_block(Go *g, int ind) g->pos++; } -void +static void plx_go_parse_body(Ctx *cx) { Go g; diff --git a/src/plx_dialect_plsql.c b/src/plx_dialect_plsql.c index 70881d3..560fb84 100644 --- a/src/plx_dialect_plsql.c +++ b/src/plx_dialect_plsql.c @@ -270,7 +270,7 @@ pl_emit_raw(Pl *pl, int a, int b, StringInfo out) (int) (pl->t[b - 1].s + pl->t[b - 1].len - pl->t[a].s)); } -void +static void plx_plsql_parse_body(Ctx *cx) { Pl pl; diff --git a/src/plx_dialect_python.c b/src/plx_dialect_python.c index a0d07fe..242b331 100644 --- a/src/plx_dialect_python.c +++ b/src/plx_dialect_python.c @@ -566,7 +566,7 @@ parse_py_program(Ctx *cx) -void +static void plx_python_parse_body(Ctx *cx) { plx_lex(cx); diff --git a/src/plx_dialect_ruby.c b/src/plx_dialect_ruby.c index 9e8bc25..918a950 100644 --- a/src/plx_dialect_ruby.c +++ b/src/plx_dialect_ruby.c @@ -735,7 +735,7 @@ parse_stmt_inner(Ctx *cx, int ind, bool toplevel) } -void +static void plx_ruby_parse_body(Ctx *cx) { plx_lex(cx); diff --git a/src/plx_dialect_tsql.c b/src/plx_dialect_tsql.c index a41691a..fd03afd 100644 --- a/src/plx_dialect_tsql.c +++ b/src/plx_dialect_tsql.c @@ -1488,7 +1488,7 @@ tq_block(Tq *tq, int ind) tq_stmt(tq, ind); } -void +static void plx_tsql_parse_body(Ctx *cx) { Tq tq; diff --git a/src/plx_engine.h b/src/plx_engine.h index 095b77a..9f949a6 100644 --- a/src/plx_engine.h +++ b/src/plx_engine.h @@ -84,7 +84,7 @@ typedef struct PlxCtx { const char *body; Tok *t; - int nt, pos; + int pos; const PlxFuncMeta *meta; const PlxSurface *surf; StringInfoData out; /* emitted BEGIN..END body */ @@ -137,14 +137,10 @@ PlxLocal2 *plx_local_find(Ctx *cx, const char *name, int len); bool plx_is_param(Ctx *cx, const char *name, int len); /* ---------------------------------------------- dialect front ends (vtable) */ -/* Each is wired into a PlxSurface.parse_body in the owning plx_dialect_*.c / - * plx_parse_brace.c. They transform cx->body into cx->out. */ -void plx_ruby_parse_body(Ctx *cx); +/* Each dialect wires a parse_body into its PlxSurface (see plx_dialect_*.c); + * the function transforms cx->body into cx->out. Only the brace front end is + * shared across translation units (php / js / ts) and so needs a prototype + * here; every other parse_body is static to its own dialect file. */ void plx_brace_parse_body(Ctx *cx); /* php / js / ts (ts sets ts_types) */ -void plx_python_parse_body(Ctx *cx); -void plx_cobol_parse_body(Ctx *cx); -void plx_plsql_parse_body(Ctx *cx); -void plx_tsql_parse_body(Ctx *cx); -void plx_go_parse_body(Ctx *cx); #endif /* PLX_ENGINE_H */ diff --git a/src/plx_transpile.c b/src/plx_transpile.c index 061174f..0ec1eb4 100644 --- a/src/plx_transpile.c +++ b/src/plx_transpile.c @@ -507,7 +507,6 @@ plx_lex(Ctx *cx) t[n].kind = T_EOF; t[n].line = line; t[n].s = tokstart; n++; } cx->t = t; - cx->nt = n; cx->pos = 0; #undef PUSH } From 94f79652f5fee64c78e1aff8fe59cda865414d31 Mon Sep 17 00:00:00 2001 From: plx Date: Fri, 17 Jul 2026 16:41:36 -0600 Subject: [PATCH 3/3] Docs: remove em-dashes to match plx doc style The existing docs use no em-dashes; the follow-up edits introduced four. Replace with a semicolon / parentheses per the house style. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 2 +- doc/ARCHITECTURE.md | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index da9baec..157e0f7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,7 +13,7 @@ version in `plx.control` (currently `1.0`). dialect-neutral engine (declared in the new `src/plx_engine.h`) plus per-dialect front ends in `src/plx_dialect_*.c` and the shared `src/plx_parse_brace.c`; the hardcoded `block_style` dispatch is replaced by a per-dialect `parse_body` - function pointer. No functional change — generated plpgsql is byte-identical + function pointer. No functional change; generated plpgsql is byte-identical and all regression tests pass. Adding a dialect no longer touches shared code. - Follow-up to the above: relocated the dialect-neutral `plx_diag_prefix()` helper out of `src/plx_dialect_ruby.c` and back into the engine diff --git a/doc/ARCHITECTURE.md b/doc/ARCHITECTURE.md index 7d80805..b0244ba 100644 --- a/doc/ARCHITECTURE.md +++ b/doc/ARCHITECTURE.md @@ -100,7 +100,7 @@ supplies. The surface describes what varies between languages: The transpiler is split into a dialect-neutral **engine** and per-dialect **front ends**, selected through the `parse_body` function pointer on the surface (a vtable method). `plx_transpile()` just calls -`cx->surf->parse_body(cx)` and then runs one assemble tail — there is no +`cx->surf->parse_body(cx)` and then runs one assemble tail; there is no per-dialect branching in the driver. - The **engine** lives in `plx_transpile.c` and is declared to the front ends @@ -144,8 +144,8 @@ src/plx_dialect_tsql.c the plxtsql surface + T-SQL front end Everything links into a single `plx.so`. A dialect is a `PlxSurface` (including its `parse_body` front end) plus small trampolines (validator, inline handler, and the shared call-handler binding), registered in `_PG_init`. Adding a dialect -is a new `plx_dialect_X.c` — a surface with its `parse_body`, plus a few -`CREATE LANGUAGE` lines — and does not touch the shared engine. +is a new `plx_dialect_X.c` (a surface with its `parse_body`, plus a few +`CREATE LANGUAGE` lines), and does not touch the shared engine. ## Trust