Skip to content

HF-249 bullet 3: generate built-in functions docs from the HF API (single source of truth)#1699

Open
marcin-kordas-hoc wants to merge 21 commits into
feature/hf-249-function-metadata-apifrom
feature/hf-249-docs-single-source
Open

HF-249 bullet 3: generate built-in functions docs from the HF API (single source of truth)#1699
marcin-kordas-hoc wants to merge 21 commits into
feature/hf-249-function-metadata-apifrom
feature/hf-249-docs-single-source

Conversation

@marcin-kordas-hoc

@marcin-kordas-hoc marcin-kordas-hoc commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator

HF-249 bullet 3 — the docs functions table is generated from HyperFormula (single source of truth)

Third in-scope bullet of HF-249: the built-in functions table in docs/guide/built-in-functions.md is generated from the public metadata API (getAvailableFunctions / getFunctionDetails), so function data lives in one place — the catalogue in src/interpreter/functionMetadata/.

What's here

  • Pure render core in src/interpreter/functionMetadata/ (browser-safe, not exported from the package index): formatFunctionSyntax, renderBuiltinFunctionsTable, spliceFunctionsTable. Deterministic (pinned Intl.Collator, LF, byte-idempotent).
  • Dev-only generator scripts/generate-builtin-functions-doc.ts (excluded from the npm package via tsconfig include:["src"]): sources data only from the API and splices the table between the AUTOGENERATED:FUNCTIONS markers; hand-written prose and footnote defs stay outside.
  • Generated at build, not gated. npm run docs:generate-functions runs as the first step of docs:build and docs:dev, so the committed table is a build product that can never drift. The old docs:functions:check CI drift-gate and its npm alias are removed.
  • getFunctionDetails returns plain text (stripDocMarkup): <br>, links, and footnote refs are stripped from the details API, while the generated table keeps the raw markdown so its links stay clickable. The Tier-1 getAvailableFunctions list entries stay raw — they are the generator's source.
  • OFFSET and VERSION are surfaced via the metadata API (they're callable in formulas, so the Formula Builder must know them). Static getFunctionDetails is now protected-aware so it agrees with getAvailableFunctions. OFFSET's parameter names are snake_case, matching the Function metadata API: getAvailableFunctions / getFunctionDetails (HF-249) #1692 catalogue convention.
  • The one-time docs → catalogue migration script is deleted (the catalogue is now the source); its dangling provenance comments are softened.
  • Per-function anchors (<a id="canonicalName">) on every row. The table documents 421 functions.
  • CHANGELOG + DEV_DOCS updated.

Decisions to confirm (defaults taken — happy to flip)

  1. Build hook = npm step (docs:generate-functions prepended to docs:build/docs:dev) rather than a VuePress plugin — simpler and deterministic.
  2. Committed snapshot: the generated table is committed and regenerated idempotently (stays present for VuePress nav + GitHub browsing) rather than gitignoring the region.
  3. Drift gate dropped: the table self-heals every build, so the CI docs:functions:check step is removed as redundant.
  4. Markup contract = API strips / docs keep raw: getFunctionDetails.shortDescription is plain text; getAvailableFunctions entries stay raw markdown because they're the generator's source. This is the one field that differs across the two API tiers — flag if you'd prefer uniform stripping with the generator reading raw from the catalogue instead.

Verification

  • npm run docs:build green; the table is present and correct in the built site (421 functions incl. OFFSET/VERSION, snake_case syntax). tsc --noEmit + lint clean.
  • Metadata unit specs green (54 tests) against this branch — paired hyperformula-tests branch feature/hf-249-docs-single-source, rebased onto the updated API tests (snake_case fixtures; OFFSET/VERSION now assert full details rather than undefined).

Base: feature/hf-249-function-metadata-api (ships with the HF-249 API in #1692).


Note

Medium Risk
Changes public metadata API surface (protected VERSION/OFFSET now listable) and replaces a large generated doc snapshot; runtime formula behavior is largely unchanged.

Overview
The built-in functions guide is no longer hand-edited in bulk: the region between AUTOGENERATED:FUNCTIONS markers in built-in-functions.md is produced from getAvailableFunctions / getFunctionDetails, with formatFunctionSyntax, renderBuiltinFunctionsTable, and spliceFunctionsTable in functionMetadata/. A dev script npm run docs:generate-functions refreshes the file; --check fails on byte drift or function-ID set changes. docs:dev and docs:build run generation first so the committed table stays aligned.

The old docs → catalogue migration script is removed; catalogue edits in src/interpreter/functionMetadata/ are now the source. VERSION and OFFSET are included in the metadata API and the generated table via PROTECTED_FUNCTION_METADATA and catalogue entries, even though they are not ordinary registered plugins. getFunctionDetails returns plain-text descriptions (stripDocMarkup); the guide table keeps richer markup for links and footnotes.

Minor doc tweaks: removed the standalone “Function metadata” section from the guide (covered by the API elsewhere), per-row anchors for deep links, and updated CHANGELOG / DEV_DOCS contributor notes.

Reviewed by Cursor Bugbot for commit 490c6eb. Bugbot is set up for automated code reviews on this repo. Configure here.

@qunabu

qunabu commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown

Performance comparison of head (490c6eb) vs base (40b6468)

                                     testName |   base |   head | change
------------------------------------------------------------------------
                                      Sheet A |    509 | 499.77 | -1.81%
                                      Sheet B | 161.74 | 164.38 | +1.63%
                                      Sheet T | 143.75 | 143.13 | -0.43%
                                Column ranges | 525.91 | 530.77 | +0.92%
Sheet A:  change value, add/remove row/column |  16.95 |  16.89 | -0.35%
 Sheet B: change value, add/remove row/column | 141.36 | 138.49 | -2.03%
                   Column ranges - add column | 160.12 | 160.59 | +0.29%
                Column ranges - without batch | 490.94 | 492.39 | +0.30%
                        Column ranges - batch |  126.9 | 129.32 | +1.91%

@marcin-kordas-hoc

Copy link
Copy Markdown
Collaborator Author

Paired tests PR: handsontable/hyperformula-tests#19

@marcin-kordas-hoc marcin-kordas-hoc requested a review from sequba July 1, 2026 21:54
marcin-kordas-hoc added a commit that referenced this pull request Jul 15, 2026
A traceability audit (run during HF-300) found migrated shortDescriptions that
either misstate behavior or leak markup into the plain-text metadata API. Per
Kuba's call (Slack #hyperformula-dev, 2026-07-15) they belong on the parent
HF-249 branch, since the catalogue is becoming the single source of truth
(docs/guide generated from it at build):

- XNPV: was verbatim identical to NPV; now conveys its non-periodic/date-based nature.
- MIRR: "modified internal value" -> "modified internal rate of return".
- SPLIT/TEXT: dropped <br> + markdown (rendered literally through the API); TEXT
  now also mentions the stringifyCurrency option.
- SECH: "given angle (in radians)" -> "given value" (the argument is a plain number).
- BASE: "positive integer" -> "non-negative integer" (0 is accepted by the impl).
- typos: MROUND "neares"/"multiplicity", Fischer->Fisher, densitity->density,
  skeweness->skewness (x2).

Catalogue-only; docs/guide regenerates from these once #1699 lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
marcin-kordas-hoc and others added 21 commits July 15, 2026 11:35
…API (HF-249)

getAvailableFunctions/getFunctionDetails excluded the protected functions
OFFSET and VERSION because they have no plugin for resolveFunctionMetadata
to read arity from. A user can still call both from a formula, so they must
be documented too: getListableFunctionIds (static and instance) now include
the protected ids, and resolveFunctionMetadata resolves them from a new
authored PROTECTED_FUNCTION_METADATA map together with their catalogue
FunctionDoc entries in the lookup-and-reference and information categories.
…venance comments (HF-249)

Mirror buildAvailableFunctions' canonicalName tiebreaker in the built-in functions
table renderer so equal localizedNames sort deterministically regardless of input
order. Also soften the information/lookup-and-reference category file comments,
which claimed full ownership by scripts/hf249-migrate-function-docs.ts even though
they now carry hand-authored protected-function entries (VERSION, OFFSET).
…1692 refactor (HF-249)

Rebasing #1699 onto the refactored #1692 combined that branch's protected-aware
resolveFunctionMetadata (which surfaces VERSION/OFFSET) with #1692's rewritten
static getFunctionDetails, which bails at `plugin === undefined` before reaching
the protected path. Result: getAvailableFunctions listed VERSION/OFFSET but
getFunctionDetails returned undefined for them, so the docs generator threw
"No details for listed function VERSION".

Handle the protected ids explicitly in the plugin-less branch, mirroring
getAvailableFunctions, so the list and the details API agree again. Protected ids
without a catalogue doc still resolve to undefined, so both stay consistent there.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…I drift gate (HF-249)

Run docs:generate-functions as the first step of docs:build and docs:dev so the
built-in-functions table is a build product of the catalogue, not a hand-kept
artifact. With the table regenerated on every build it can never drift, so remove
the redundant docs:functions:check CI step and its npm alias, and update DEV_DOCS
to describe the build-time regeneration instead of a CI staleness check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tion (HF-249)

The catalogue short descriptions carry presentational markdown (<br>, links,
footnote refs) that the guide table renders. Add stripDocMarkup and apply it when
building Tier-2 details, so getFunctionDetails returns plain prose. The Tier-1 list
entry keeps the raw markdown, so the generated table's links stay clickable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…g references (HF-249)

The catalogue is now the source of truth and the guide is generated from it, so the
one-time, reverse-direction migration script (guide -> catalogue) is a footgun. Delete
it and soften the ~16 provenance comments that named it so they no longer point at a
removed file, while keeping the note about SUM/SUMIF carrying hand-authored examples.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ue convention (HF-249)

OFFSET was added to the catalogue on this branch before #1692 standardized every
parameter name to snake_case, so after the rebase it was the lone PascalCase entry
(Reference/Rows/Columns/Height/Width) among snake_case siblings. Align it, so
getFunctionDetails('OFFSET') and the generated table read consistently.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…HF-249)

Reflects the rebased catalogue: snake_case parameter names (#1692 refactor, incl.
OFFSET), the new VSTACK/HSTACK entries (#1698), and the protected VERSION/OFFSET
functions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@marcin-kordas-hoc marcin-kordas-hoc force-pushed the feature/hf-249-docs-single-source branch from 4b5ef17 to 490c6eb Compare July 15, 2026 17:32

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 490c6eb. Configure here.

Comment thread package.json
"docs:code-examples:generate-all-js": "bash docs/code-examples-generator.sh --generateAll",
"docs:code-examples:format-all-ts": "bash docs/code-examples-generator.sh --formatAllTsExamples",
"snippets:extract": "node script/extract-doc-snippets.js",
"docs:generate-functions": "npm run tsnode scripts/generate-builtin-functions-doc.ts",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Drift check npm script missing

Medium Severity

The PR adds a --check mode on scripts/generate-builtin-functions-doc.ts for CI drift detection, but package.json only defines docs:generate-functions (write path). There is no docs:functions:check script to run --check, so the documented drift gate is easy to omit from CI unless callers hard-code the tsnode invocation.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 490c6eb. Configure here.

Comment thread src/HyperFormula.ts
// a user can call them from a formula: resolve them here from the authored catalogue doc and structural
// metadata instead of from a plugin. Functions without a catalogue entry stay unlisted.
if (FunctionRegistry.functionIsProtected(functionId) && FUNCTION_DOCS[functionId] !== undefined) {
return {doc: FUNCTION_DOCS[functionId], metadata: PROTECTED_FUNCTION_METADATA[functionId]}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Undefined protected metadata crashes

Low Severity

For protected function ids, resolveFunctionMetadata can return a catalogue doc with metadata set to PROTECTED_FUNCTION_METADATA[functionId] when that key is missing. buildFunctionDetails then reads metadata.repeatLastArgs without optional chaining on metadata, which throws when metadata is undefined.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 490c6eb. Configure here.

@codecov

codecov Bot commented Jul 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.45%. Comparing base (40b6468) to head (490c6eb).

Additional details and impacted files

Impacted file tree graph

@@                           Coverage Diff                            @@
##           feature/hf-249-function-metadata-api    #1699      +/-   ##
========================================================================
+ Coverage                                 96.43%   96.45%   +0.01%     
========================================================================
  Files                                       192      195       +3     
  Lines                                     15589    15648      +59     
  Branches                                   3447     3458      +11     
========================================================================
+ Hits                                      15034    15094      +60     
+ Misses                                      555      554       -1     
Files with missing lines Coverage Δ
src/HyperFormula.ts 99.75% <100.00%> (+<0.01%) ⬆️
src/interpreter/FunctionRegistry.ts 100.00% <100.00%> (+0.56%) ⬆️
...eter/functionMetadata/buildFunctionDescriptions.ts 100.00% <100.00%> (ø)
.../functionMetadata/categories/array-manipulation.ts 100.00% <ø> (ø)
...nterpreter/functionMetadata/categories/database.ts 100.00% <ø> (ø)
...reter/functionMetadata/categories/date-and-time.ts 100.00% <ø> (ø)
...rpreter/functionMetadata/categories/engineering.ts 100.00% <ø> (ø)
...terpreter/functionMetadata/categories/financial.ts 100.00% <ø> (ø)
...rpreter/functionMetadata/categories/information.ts 100.00% <ø> (ø)
...interpreter/functionMetadata/categories/logical.ts 100.00% <ø> (ø)
... and 10 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants