Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .changeset/adr-0057-lifecycle-followups.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
'@objectstack/spec': minor
'@objectstack/cli': minor
'@objectstack/service-job': minor
'@objectstack/service-messaging': minor
'@objectstack/service-automation': patch
'@objectstack/platform-objects': patch
---

ADR-0057 data-lifecycle follow-ups (#2834): the per-plugin retention sweepers are retired, telemetry separation goes live in dev, and the lifecycle contract reaches the Studio.

- **BREAKING (ships as minor per the launch-window convention)**: `JobRunRetention` / `NotificationRetention` and the `retentionDays` / `retentionSweepMs` options on `JobServicePlugin` / `MessagingServicePlugin` are removed. The platform LifecycleService enforces the same windows from the `lifecycle` declarations (`sys_job_run` 30d, notification pipeline 90d); tune them at runtime via the `lifecycle` settings namespace (`retention_overrides`, tenant-scoped).
- **Fix**: `sys_automation_run` no longer declares a blanket 30d lifecycle retention — that table interleaves live SUSPENDED runs (an approval may stay paused for months) with terminal history, and a blanket age reap could strand in-flight approvals. Bounding stays with the automation store's terminal-only sweep.
- **CLI**: `objectstack dev` now provisions a dedicated `telemetry` datasource (`<primary>.telemetry.db`) for file-backed SQLite primaries, so lifecycle-classed system data stops sharing the business dev DB (`OS_TELEMETRY_DB=0` opts out; `OS_TELEMETRY_DB=<path>` opts in anywhere). New `os db clean` runs the one-time `VACUUM` that lets legacy files adopt `auto_vacuum=INCREMENTAL` and reports reclaimed bytes.
- **Studio**: the object metadata form exposes the `lifecycle` block (class + retention/TTL/rotation/archive/reclaim); metadata-forms i18n bundles regenerated with curated zh-CN translations.
25 changes: 25 additions & 0 deletions content/docs/getting-started/cli.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ predicate/schema/binding mistakes that fail silently at runtime), and `os dev --
| `os init [name]` | | Initialize a new ObjectStack project in the current directory |
| `os dev [package]` | | Start development mode with hot reload |
| `os serve [config]` | | Start the ObjectStack server with plugin auto-detection |
| `os db clean` | | Reclaim SQLite free space with a one-time `VACUUM` (ADR-0057) |

#### `os init`

Expand Down Expand Up @@ -134,6 +135,13 @@ os dev --database file:./data/test.db --auth-secret $(openssl rand -hex 32)
| `--no-compile` | — | Skip the auto-compile step (errors if no artifact present) |
| `-v, --verbose` | — | Verbose output |

With a file-backed SQLite database, dev also provisions a sibling
`<db>.telemetry.<ext>` file registered as the `telemetry` datasource —
lifecycle-classed system data (activity streams, job runs, notifications,
audit) lands there instead of the business DB (ADR-0057). Opt out with
`OS_TELEMETRY_DB=0`, or point it elsewhere (any mode, including `serve`)
with `OS_TELEMETRY_DB=<path>`.

#### `os serve`

Starts the ObjectStack server with automatic plugin discovery:
Expand Down Expand Up @@ -195,6 +203,23 @@ export default defineStack({
});
```

#### `os db clean`

Reclaims SQLite free space with a one-time `VACUUM` (ADR-0057 §3.4). The
platform reclaims space incrementally (`auto_vacuum=INCREMENTAL`), but that
setting only takes effect on a fresh database — files created before it
stay pinned at their high-water mark until one full `VACUUM` rebuilds them.
Non-destructive: every row survives; free pages return to the OS. Cleans
the telemetry sibling too when one exists.

```bash
os db clean # default: the per-project dev DB
os db clean --database file:./data/app.db # explicit target
```

**Options:**
- `-d, --database <url>` — SQLite database URL/path (defaults to `$OS_DATABASE_URL`, then the per-project dev DB)

#### Console UI

Launch the development server with the Console UI:
Expand Down
9 changes: 9 additions & 0 deletions docs/launch-readiness.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,15 @@ fix or acceptance.**
`AutomationServicePluginOptions.maxLogSize` (default unchanged at 1000,
`DEFAULT_MAX_EXECUTION_LOG_SIZE`); +2 tests. Durable `sys_automation_run`-style
persistence is deferred to the HA fast-follow (roadmap), not a GA blocker.
- **Superseded (ADR-0057, #2786/#2834):** the plugin-local sweepers above were
the stop-gap. Retention is now a *declarative platform primitive*: objects
carry a `lifecycle` block (`sys_job_run` 30d, notification pipeline 90d) and
the ObjectQL-registered **LifecycleService** is the ONE sweeper —
`JobRunRetention` / `NotificationRetention` and their `retentionDays`
options were removed. Windows are tuned via the `lifecycle` settings
namespace (`retention_overrides`, tenant-scoped). `sys_automation_run`
deliberately keeps its OWN terminal-only sweep (suspended runs are live
resumable state; the declarative contract has no status predicate).
- **Owner:** _______ · Verify ✅ (confirmed real @ `main`; scope corrected) · Sign-off ☐ · Notes: `sys_job_run` retention is the one true fix; messaging default-flipped; automation already bounded (now tunable). Awaiting human sign-off.

### P1-3 — Graceful shutdown (mostly a false positive; one real drain bug fixed)
Expand Down
103 changes: 103 additions & 0 deletions packages/cli/src/commands/db/clean.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

import { Command, Flags } from '@oclif/core';
import { statSync, existsSync } from 'node:fs';
import chalk from 'chalk';
import { printError } from '../../utils/format.js';
import { resolveDefaultDevDbUrl } from '../dev.js';
import { resolveTelemetryDbPath } from '../../utils/telemetry-datasource.js';

/**
* `os db clean` — one-time SQLite space reclamation (ADR-0057 §3.4).
*
* The platform LifecycleService reclaims space incrementally
* (`auto_vacuum=INCREMENTAL` + `PRAGMA incremental_vacuum` after sweeps) —
* but `auto_vacuum` only changes the page layout of a FRESH database, so a
* file created before the ADR-0057 default stays pinned at its high-water
* mark until one full `VACUUM` rebuilds it. This command runs exactly that:
* set the pragma, `VACUUM`, report the reclaimed bytes. Non-destructive —
* every row survives; only free pages are returned to the OS.
*/
export default class DbClean extends Command {
static override description =
'Reclaim SQLite free space (one-time VACUUM) so legacy files adopt auto_vacuum=INCREMENTAL — ADR-0057 §3.4';

static override examples = [
'$ os db clean',
'$ os db clean --database file:./.objectstack/data/dev.db',
];

static override flags = {
database: Flags.string({
char: 'd',
description: 'SQLite database URL/path (defaults to $OS_DATABASE_URL, then the per-project dev DB)',
env: 'OS_DATABASE_URL',
}),
};

async run(): Promise<void> {
const { flags } = await this.parse(DbClean);

const raw =
flags.database?.trim() ||
resolveDefaultDevDbUrl({ env: process.env, cwd: process.cwd() }) ||
'';
const primary = raw.replace(/^file:/i, '').replace(/^sqlite:/i, '');

if (!primary || primary === ':memory:' || primary.startsWith(':')) {
printError('No file-backed SQLite database to clean (in-memory databases reclaim nothing).');
this.exit(1);
return;
}
if (!/\.(db|sqlite3|sqlite)$/i.test(primary) && !existsSync(primary)) {
printError(`Not a SQLite database path: ${primary}`);
this.exit(1);
return;
}

// Clean the telemetry sibling too when one exists (ADR-0057 §3.6).
const telemetrySibling = resolveTelemetryDbPath({ primaryPath: primary, env: process.env, dev: true });
const targets = [primary, telemetrySibling].filter(
(p): p is string => !!p && existsSync(p),
);
if (targets.length === 0) {
printError(`Database file not found: ${primary}`);
this.exit(1);
return;
}

const { resolveSqliteDriver } = await import('@objectstack/service-datasource');

let failed = false;
for (const file of targets) {
const before = statSync(file).size;
try {
const resolved = await resolveSqliteDriver({
filename: file,
dev: true, // allow the wasm step-down; memory fallback is rejected below
warn: (m) => console.warn(chalk.yellow(m)),
});
if (resolved.engine === 'memory') {
throw new Error('no SQLite engine available (native and wasm both failed to load)');
}
// Order matters: the pragma must be set BEFORE the VACUUM so the
// rebuilt file carries auto_vacuum=INCREMENTAL from here on.
await resolved.driver.execute('PRAGMA auto_vacuum = INCREMENTAL');
await resolved.driver.execute('VACUUM');
await resolved.driver.disconnect();

const after = statSync(file).size;
const saved = before - after;
const fmt = (n: number) => `${(n / 1024 / 1024).toFixed(2)} MB`;
console.log(
`${chalk.green('✓')} ${file}: ${fmt(before)} → ${fmt(after)}` +
(saved > 0 ? chalk.dim(` (reclaimed ${fmt(saved)})`) : chalk.dim(' (already compact)')),
);
} catch (error: any) {
failed = true;
printError(`VACUUM failed for ${file}: ${error?.message ?? error}`);
}
}
if (failed) this.exit(1);
}
}
32 changes: 32 additions & 0 deletions packages/cli/src/commands/serve.ts
Original file line number Diff line number Diff line change
Expand Up @@ -664,6 +664,38 @@ export default class Serve extends Command {
trackPlugin(resolved.engine === 'memory' ? 'MemoryDriver' : resolved.engine === 'sqlite-wasm' ? 'SqliteWasmDriver' : 'SqlDriver');
resolvedDriverLabel = resolved.label;
resolvedDatabaseUrl = resolved.engine === 'memory' ? '(in-memory)' : (databaseUrl ?? ':memory:');

// ADR-0057 §3.6 (#2834 ②): provision the dedicated `telemetry`
// datasource — a sibling SQLite file the engine routes every
// telemetry/event/audit-classed object to, so platform-generated
// growth can never again bloat the business DB. Dev default-on
// for file-backed primaries; `OS_TELEMETRY_DB=0` opts out,
// `OS_TELEMETRY_DB=<path>` opts in anywhere (incl. serve).
if (resolved.engine !== 'memory') {
const { resolveTelemetryDbPath } = await import('../utils/telemetry-datasource.js');
const telemetryPath = resolveTelemetryDbPath({ primaryPath: filePath, env: process.env, dev: isDev });
if (telemetryPath) {
try {
const telemetry = await resolveSqliteDriver({
filename: telemetryPath,
dev: isDev,
autoMigrate: isDev ? 'safe' : undefined,
warn: (m) => console.warn(chalk.yellow(m)),
});
if (telemetry.engine !== 'memory') {
// The engine keys datasources by driver name — the
// lifecycle router looks this exact name up.
Object.defineProperty(telemetry.driver, 'name', { value: 'telemetry' });
await kernel.use(new DriverPlugin(telemetry.driver, { datasourceName: 'telemetry', registerAsDefault: false }));
trackPlugin('TelemetryDatasource');
console.log(chalk.dim(` telemetry datasource: ${telemetryPath} (lifecycle-classed system data; OS_TELEMETRY_DB=0 to disable)`));
}
} catch {
// Best-effort: a failed telemetry provision must never block
// boot — objects simply stay on the primary datasource.
}
}
}
} else if (driverType === 'sqlite-wasm' || driverType === 'wasm-sqlite' || driverType === 'wasm') {
const { SqliteWasmDriver } = await import('@objectstack/driver-sqlite-wasm');
const filePath = (databaseUrl ?? ':memory:').replace(/^file:/, '').replace(/^wasm-sqlite:\/\//, '').replace(/^sqlite:/, '');
Expand Down
39 changes: 39 additions & 0 deletions packages/cli/src/utils/telemetry-datasource.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

import { describe, it, expect } from 'vitest';
import { resolveTelemetryDbPath } from './telemetry-datasource.js';

describe('resolveTelemetryDbPath (ADR-0057 §3.6)', () => {
it('derives a sibling file next to a file-backed dev primary', () => {
expect(resolveTelemetryDbPath({ primaryPath: '/p/.objectstack/data/dev.db', env: {}, dev: true }))
.toBe('/p/.objectstack/data/dev.telemetry.db');
expect(resolveTelemetryDbPath({ primaryPath: './app.sqlite', env: {}, dev: true }))
.toBe('./app.telemetry.sqlite');
expect(resolveTelemetryDbPath({ primaryPath: 'data/store', env: {}, dev: true }))
.toBe('data/store.telemetry.db');
});

it('never derives one for in-memory primaries', () => {
expect(resolveTelemetryDbPath({ primaryPath: ':memory:', env: {}, dev: true })).toBeUndefined();
expect(resolveTelemetryDbPath({ primaryPath: '', env: {}, dev: true })).toBeUndefined();
});

it('is opt-in outside dev', () => {
expect(resolveTelemetryDbPath({ primaryPath: '/srv/prod.db', env: {}, dev: false })).toBeUndefined();
expect(
resolveTelemetryDbPath({ primaryPath: '/srv/prod.db', env: { OS_TELEMETRY_DB: '/srv/prod.telemetry.db' }, dev: false }),
).toBe('/srv/prod.telemetry.db');
});

it('OS_TELEMETRY_DB=0|false|off opts out entirely, even in dev', () => {
for (const off of ['0', 'false', 'off', 'OFF']) {
expect(resolveTelemetryDbPath({ primaryPath: '/p/dev.db', env: { OS_TELEMETRY_DB: off }, dev: true })).toBeUndefined();
}
});

it('an explicit OS_TELEMETRY_DB path wins over derivation and strips url prefixes', () => {
expect(
resolveTelemetryDbPath({ primaryPath: '/p/dev.db', env: { OS_TELEMETRY_DB: 'file:/tmp/t.db' }, dev: true }),
).toBe('/tmp/t.db');
});
});
44 changes: 44 additions & 0 deletions packages/cli/src/utils/telemetry-datasource.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* ADR-0057 §3.6 (#2834 ②) — where the dedicated telemetry datasource lives.
*
* When a datasource named `telemetry` is registered, the engine routes every
* `telemetry`/`event`/`audit`-classed object to it, so platform-generated
* growth can never again bloat the business DB. This helper decides whether
* (and where) the CLI should provision that second SQLite file:
*
* - `OS_TELEMETRY_DB=0|false|off` → never (explicit opt-out)
* - `OS_TELEMETRY_DB=<path>` → always, at that path (dev AND serve)
* - dev mode + file-backed primary → default ON, `<primary>.telemetry.<ext>`
* (`dev.db` → `dev.telemetry.db`, same directory)
* - everything else (prod serve, `:memory:`, non-sqlite) → off
*
* Production stays opt-in: a second file appearing next to a prod database
* is a deployment-topology change an operator should choose, not inherit.
*/
export function resolveTelemetryDbPath(opts: {
/** Primary sqlite file path (already stripped of `file:`/`sqlite:`). */
primaryPath: string;
env: Record<string, string | undefined>;
dev: boolean;
}): string | undefined {
const raw = opts.env.OS_TELEMETRY_DB?.trim();
if (raw) {
const lowered = raw.toLowerCase();
if (lowered === '0' || lowered === 'false' || lowered === 'off') return undefined;
return raw.replace(/^file:/i, '').replace(/^sqlite:/i, '');
}

if (!opts.dev) return undefined;

const primary = opts.primaryPath.trim();
// Only a real on-disk primary gets a sibling: separating one `:memory:`
// store into another has no reclamation value.
if (!primary || primary === ':memory:' || primary.startsWith(':')) return undefined;

if (/\.(db|sqlite3|sqlite)$/i.test(primary)) {
return primary.replace(/\.(db|sqlite3|sqlite)$/i, '.telemetry.$1');
}
return `${primary}.telemetry.db`;
}
Loading