Add benchmark CLI

CONTRIBUTING.md

@@ -161,8 +161,50 @@ To get the project running locally:
    ```
 
 5. **Linting and formatting**:
+
    ```bash
    pnpm lint:all        # Run linting across workspace and framework packages
    pnpm format          # Format code with Prettier
    pnpm type-check:all  # Run type checking across workspace and framework packages
    ```
+
+### Running Benchmarks Locally
+
+Use the benchmark CLI to run the configured benchmarks for one or more
+frameworks. Framework names select both their `starter-*` and `app-*` packages;
+package names select only that package.
+
+```bash
+# See every framework, package, and configured benchmark
+pnpm benchmark --list
+
+# Run one benchmark for multiple framework apps
+pnpm benchmark astro next --measurement ssrLoad
+
+# Benchmark a specific framework version
+pnpm benchmark next --version 15.5.9 --measurement ssrLoad
+
+# Select exact packages and reduce the browser benchmark to three runs
+pnpm benchmark app-astro app-sveltekit \
+  --measurement clientSideRendered \
+  --runs 3
+
+# Preview the installs, builds, and benchmark commands without executing them
+pnpm benchmark --all --measurement ssrRequestThroughput --dry-run
+```
+
+The CLI installs each selected package with its own lockfile and builds runtime
+apps before benchmarking them. Pass `--skip-install` to reuse existing
+`node_modules` directories. Browser benchmarks require Chrome or Chromium; set
+`CHROME_PATH` when it is not installed in a standard location.
+
+Install and build benchmarks default to their run frequency from
+`.github/frameworks.json`. Use `--runs` to override it. Benchmark results are
+written to the selected package's existing `ci-stats.json`,
+`install-stats.json`, or `build-stats.json`, so expect the working tree to
+change.
+
+`--version` can be used when the selection resolves to one framework. The CLI
+temporarily updates that framework dependency in every selected package, uses a
+non-frozen install for the benchmark, and restores the original manifests,
+lockfiles, and installed dependencies afterward.

package.json

@@ -8,6 +8,7 @@
     "node": ">=24.0.0"
   },
   "scripts": {
+    "benchmark": "pnpm --filter @framework-tracker/stats-generator benchmark",
     "dev:docs": "pnpm --filter @framework-tracker/docs dev",
     "build:docs": "pnpm --filter @framework-tracker/docs build",
     "preview:docs": "pnpm --filter @framework-tracker/docs preview",

packages/stats-generator/package.json

@@ -4,6 +4,7 @@
   "private": true,
   "type": "module",
   "scripts": {
+    "benchmark": "node src/run-benchmarks.ts",
     "collect": "node src/collect-stats.ts",
     "run:ssr-request-throughput": "node src/run-ssr-request-throughput-benchmark.ts",
     "run:ssr-load": "node src/run-ssr-load-benchmark.ts",

packages/stats-generator/src/clientSideRendered/index.ts

@@ -117,12 +117,17 @@ async function spawnServer(
     })
   })
 
-  await Promise.race([
-    waitForServer(
-      `http://${CLIENT_SIDE_RENDERED_HOST}:${CLIENT_SIDE_RENDERED_PORT}/client-side-rendered`,
-    ),
-    exitPromise,
-  ])
+  try {
+    await Promise.race([
+      waitForServer(
+        `http://${CLIENT_SIDE_RENDERED_HOST}:${CLIENT_SIDE_RENDERED_PORT}/client-side-rendered`,
+      ),
+      exitPromise,
+    ])
+  } catch (error) {
+    if (!exited) proc.kill('SIGTERM')
+    throw error
+  }
 
   return () => {
     if (!exited) proc.kill('SIGTERM')