Skip to content

Commit 962bc48

Browse files
d-csclaude
d-cs
and
claude
authored
feat(run-ops): automatically migrate the dedicated run-ops database (#4150)
## What Adds the ability to **automatically migrate the dedicated run-ops database** (the NEW DB in the run-ops split), matching how every other database in the system is migrated. Follow-up to the run-ops split activation. ## Changes - **Migrate runner** — new `internal-packages/run-ops-database/scripts/migrate.mjs`, exposed as `db:migrate:deploy` / `db:migrate:status`. Connects via `RUN_OPS_DATABASE_URL` (the same var the app uses) and expands `${VAR}` refs like Prisma's dotenv. - **Self-host** — `docker/scripts/entrypoint.sh` runs the run-ops migration on boot when the DB is configured, gated by `SKIP_RUN_OPS_MIGRATIONS`. Single-DB installs never set the URL, so it's a clean no-op. - **Single env-var family** — the run-ops DB is now addressed by one canonical `RUN_OPS_*` family, connect path and migrations resolving the identical URL: - `RUN_OPS_DATABASE_URL` (writer) — replaces `TASK_RUN_DATABASE_URL` - `RUN_OPS_LEGACY_DATABASE_URL` — replaces `TASK_RUN_LEGACY_DATABASE_URL` - `RUN_OPS_DATABASE_READ_REPLICA_URL` — replaces `TASK_RUN_DATABASE_READ_REPLICA_URL` - the old `TASK_RUN_*` aliases, the `??` coalesce, the `runOpsNewDatabaseUrl` indirection, and the migrate-only `directUrl` are all removed (consumers read `env.RUN_OPS_DATABASE_URL` directly). `directUrl` was dropped because it was only ever used by `prisma migrate` (never the app runtime) to bypass a pooler for advisory locks — premature here since the run-ops connection isn't wired to the app yet. If a pooler is later introduced for the app, a direct URL can be reintroduced then. ## Safety - **Pure rename** — nothing deployed sets any `TASK_RUN_*` var (the split isn't activated anywhere yet; `.env.example`, docker-compose, and cloud already use `RUN_OPS_*`), so there is no config migration. - **Single-DB / self-host** — no new required env var; entrypoint and migrate are no-ops when `RUN_OPS_DATABASE_URL` is unset. - **Cloud** — runs migrations as pre-deploy ECS tasks (companion cloud PR), calling these same `db:migrate:deploy` / `db:migrate:status` commands. ## Verification - Live migration against a fresh scratch DB with only `RUN_OPS_DATABASE_URL` set: both migrations applied, no `P1012`/`P1013`; `${VAR}` expansion, idempotent re-run, `status`, and no-op skip all pass. - Schema parity 4/4; `typecheck --filter webapp` 18/18; affected split/replication tests 34/34. ## Scope This delivers automatic migrations only. Enabling the app to *use* the new DB (setting `RUN_OPS_DATABASE_URL` + `RUN_OPS_SPLIT_ENABLED` on the service) is a separate activation step. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent c9f427e commit 962bc48

14 files changed

Lines changed: 122 additions & 58 deletions

File tree

‎.env.example‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,6 @@ DIRECT_URL=${DATABASE_URL}
1010
# Dedicated run-ops database (@internal/run-ops-database). Only needed to run prisma commands
1111
# against it or to enable the run-ops split; start it with `docker compose --profile runops up`.
1212
RUN_OPS_DATABASE_URL=postgresql://postgres:postgres@localhost:5434/postgres?schema=public
13-
RUN_OPS_DATABASE_DIRECT_URL=${RUN_OPS_DATABASE_URL}
1413
REMIX_APP_PORT=3030
1514
# Dev-only: stream the webapp's logs over a local telnet/TCP socket (nc localhost 6767). Uncomment to enable.
1615
# WEBAPP_TELNET_LOGS_PORT=6767

‎.server-changes/run-ops-auto-migrate.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
area: webapp
3+
type: feature
4+
---
5+
6+
Automatically migrate the dedicated run-ops database on deploy (entrypoint + `@internal/run-ops-database` deploy/status scripts), and standardize the run-ops DB connection on a single `RUN_OPS_DATABASE_URL` family (dropping the `TASK_RUN_DATABASE_URL` aliases) so migrations always target the DB the app connects to.

‎apps/webapp/app/db.server.ts‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -237,16 +237,16 @@ export function selectRunOpsTopology(
237237
// nothing new. The builders apply the SAME wrapper pair the control-plane
238238
// singletons use (captureInfrastructureErrors(tagDatasource(role, raw))).
239239
const runOpsTopology: RunOpsTopology = singleton("runOpsTopology", () => {
240-
const newUrl = env.TASK_RUN_DATABASE_URL;
240+
const newUrl = env.RUN_OPS_DATABASE_URL;
241241
// Gate on the opt-in flag too: the distinct-DB sentinel only runs when the flag is on.
242-
const splitEnabled = env.RUN_OPS_SPLIT_ENABLED && !!newUrl && !!env.TASK_RUN_LEGACY_DATABASE_URL;
242+
const splitEnabled = env.RUN_OPS_SPLIT_ENABLED && !!newUrl && !!env.RUN_OPS_LEGACY_DATABASE_URL;
243243

244244
return selectRunOpsTopology(
245245
{
246246
splitEnabled,
247-
legacyUrl: env.TASK_RUN_LEGACY_DATABASE_URL,
247+
legacyUrl: env.RUN_OPS_LEGACY_DATABASE_URL,
248248
newUrl,
249-
newReplicaUrl: env.TASK_RUN_DATABASE_READ_REPLICA_URL,
249+
newReplicaUrl: env.RUN_OPS_DATABASE_READ_REPLICA_URL,
250250
},
251251
{
252252
controlPlane: { writer: prisma, replica: $replica },
@@ -278,8 +278,8 @@ export const runOpsSplitReadEnabled: boolean = computeRunOpsSplitReadEnabled({
278278
newReplica: runOpsNewReplicaClient,
279279
controlPlaneWriter: prisma,
280280
controlPlaneReplica: $replica,
281-
hasNewUrl: !!env.TASK_RUN_DATABASE_URL,
282-
hasLegacyUrl: !!env.TASK_RUN_LEGACY_DATABASE_URL,
281+
hasNewUrl: !!env.RUN_OPS_DATABASE_URL,
282+
hasLegacyUrl: !!env.RUN_OPS_LEGACY_DATABASE_URL,
283283
});
284284

285285
// Boot-time interlock: if the flag is on but the distinct-DB sentinel does not

‎apps/webapp/app/env.server.ts‎

Lines changed: 9 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -132,35 +132,24 @@ const EnvironmentSchema = z
132132
// Explicit positive opt-in. Split behavior is unreachable unless this is true
133133
// AND the distinct-DB sentinel confirms the two URLs are physically distinct DBs.
134134
RUN_OPS_SPLIT_ENABLED: BoolEnv.default(false),
135-
// Datasource URL for the dedicated run-ops Prisma schema (migrations/generation).
136-
// The webapp runtime pool is driven by TASK_RUN_DATABASE_URL, not this var.
135+
// Canonical connection URL for the dedicated NEW run-ops DB — drives the runtime pool, the split
136+
// decision, replication, and migrations. Optional so single-DB installs never set it.
137137
RUN_OPS_DATABASE_URL: z
138138
.string()
139139
.refine(isValidDatabaseUrl, "RUN_OPS_DATABASE_URL is invalid")
140140
.optional(),
141-
// The NEW dedicated run-ops DB writer. Optional so single-DB installs never set it.
142-
TASK_RUN_DATABASE_URL: z
143-
.string()
144-
.refine(isValidDatabaseUrl, "TASK_RUN_DATABASE_URL is invalid")
145-
.optional(),
146-
// The NEW run-ops DB unpooled/direct endpoint (Prisma migrate/introspection;
147-
// connection poolers break advisory locks). Consumed by the migrations.
148-
TASK_RUN_DATABASE_DIRECT_URL: z
149-
.string()
150-
.refine(isValidDatabaseUrl, "TASK_RUN_DATABASE_DIRECT_URL is invalid")
151-
.optional(),
152141
// The LEGACY run-ops DB (the control-plane DB during the transition). When unset, legacy
153142
// run-ops reuses the existing DATABASE_URL (legacy run-ops == control-plane DB initially).
154-
TASK_RUN_LEGACY_DATABASE_URL: z
143+
RUN_OPS_LEGACY_DATABASE_URL: z
155144
.string()
156-
.refine(isValidDatabaseUrl, "TASK_RUN_LEGACY_DATABASE_URL is invalid")
145+
.refine(isValidDatabaseUrl, "RUN_OPS_LEGACY_DATABASE_URL is invalid")
157146
.optional(),
158147
// The NEW dedicated run-ops DB read replica. Optional; self-host never sets it.
159148
// Refined (unlike the unrefined control-plane DATABASE_READ_REPLICA_URL) so a malformed run-ops
160149
// replica URL fails boot loudly rather than silently degrading — do not align it down to the CP shape.
161-
TASK_RUN_DATABASE_READ_REPLICA_URL: z
150+
RUN_OPS_DATABASE_READ_REPLICA_URL: z
162151
.string()
163-
.refine(isValidDatabaseUrl, "TASK_RUN_DATABASE_READ_REPLICA_URL is invalid")
152+
.refine(isValidDatabaseUrl, "RUN_OPS_DATABASE_READ_REPLICA_URL is invalid")
164153
.optional(),
165154
// --- Control-plane datasource repoint. Additive-only. ---
166155
// Optional control-plane DB. Unset (self-host/single-DB) -> getClient()/getReplicaClient() fall back to
@@ -1724,9 +1713,9 @@ const EnvironmentSchema = z
17241713

17251714
// --- Run-ops DB split — second replication source (the NEW dedicated run-ops DB). ---
17261715
// Cloud-only; only consulted when isSplitEnabled() is true. Self-host never sets these.
1727-
// The NEW source's connection URL is TASK_RUN_DATABASE_URL; these add
1728-
// the NEW source's replication slot/publication and an explicit per-source enable so it can be
1729-
// brought up independently of the legacy source during the transition.
1716+
// The NEW source's connection URL is RUN_OPS_DATABASE_URL; these add the NEW source's replication
1717+
// slot/publication and an explicit per-source enable so it can be brought up independently of the
1718+
// legacy source during the transition.
17301719
RUN_REPLICATION_NEW_SLOT_NAME: z.string().default("task_runs_to_clickhouse_v2"),
17311720
RUN_REPLICATION_NEW_PUBLICATION_NAME: z
17321721
.string()

‎apps/webapp/app/services/runsReplicationInstance.server.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -172,7 +172,7 @@ function initializeRunsReplicationInstance() {
172172
const sources = buildReplicationSources({
173173
splitEnabled,
174174
legacyUrl: DATABASE_URL,
175-
newUrl: env.RUN_OPS_DATABASE_URL ?? env.TASK_RUN_DATABASE_URL,
175+
newUrl: env.RUN_OPS_DATABASE_URL,
176176
newSourceOverride: env.RUN_REPLICATION_NEW_ENABLED === "disabled" ? false : undefined,
177177
legacySlotName: env.RUN_REPLICATION_SLOT_NAME,
178178
legacyPublicationName: env.RUN_REPLICATION_PUBLICATION_NAME,

‎apps/webapp/app/v3/runOpsMigration/controlPlaneResolver.server.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -449,7 +449,7 @@ export class ControlPlaneResolver {
449449
// run-ops topology factory uses); the async isSplitEnabled() distinct-DB sentinel is enforced
450450
// at boot elsewhere and is never awaited on a resolver hot path.
451451
const SPLIT_ENABLED =
452-
env.RUN_OPS_SPLIT_ENABLED && !!env.TASK_RUN_DATABASE_URL && !!env.TASK_RUN_LEGACY_DATABASE_URL;
452+
env.RUN_OPS_SPLIT_ENABLED && !!env.RUN_OPS_DATABASE_URL && !!env.RUN_OPS_LEGACY_DATABASE_URL;
453453

454454
export const controlPlaneResolver = new ControlPlaneResolver({
455455
controlPlanePrimary: prisma,

‎apps/webapp/app/v3/runOpsMigration/splitMode.server.ts‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ export async function computeSplitEnabled(
3030
// Both URLs are required to even consider a split.
3131
if (!config.legacyUrl || !config.newUrl) {
3232
deps.logger?.warn(
33-
"RUN_OPS_SPLIT_ENABLED is on but TASK_RUN_LEGACY_DATABASE_URL / TASK_RUN_DATABASE_URL are not both set; staying single-DB."
33+
"RUN_OPS_SPLIT_ENABLED is on but RUN_OPS_LEGACY_DATABASE_URL / RUN_OPS_DATABASE_URL are not both set; staying single-DB."
3434
);
3535
return false;
3636
}
@@ -70,8 +70,8 @@ export function isSplitEnabled(): Promise<boolean> {
7070
cached = computeSplitEnabled(
7171
{
7272
flagEnabled: env.RUN_OPS_SPLIT_ENABLED,
73-
legacyUrl: env.TASK_RUN_LEGACY_DATABASE_URL,
74-
newUrl: env.TASK_RUN_DATABASE_URL,
73+
legacyUrl: env.RUN_OPS_LEGACY_DATABASE_URL,
74+
newUrl: env.RUN_OPS_DATABASE_URL,
7575
},
7676
{ logger }
7777
);

‎apps/webapp/app/v3/runStore.server.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ export function buildRunStore(deps: BuildRunStoreDeps): RunStore {
7676
// RUN_OPS_SPLIT_ENABLED. Reads must fan out across both DBs so a run that lives on the new
7777
// DB stays visible even with the flag off (matches the db.server topology factory). The flag
7878
// governs write/mint residency + migration via isSplitEnabled(), not read visibility.
79-
const ROUTING_ENABLED = !!env.TASK_RUN_DATABASE_URL && !!env.TASK_RUN_LEGACY_DATABASE_URL;
79+
const ROUTING_ENABLED = !!env.RUN_OPS_DATABASE_URL && !!env.RUN_OPS_LEGACY_DATABASE_URL;
8080

8181
// Resolve the run-ops handles, tolerating contexts where they are absent — tests that mock
8282
// ~/db.server minimally omit them, and accessing a missing export under vi.mock throws. A

‎apps/webapp/test/runsReplicationInstance.test.ts‎

Lines changed: 2 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -126,35 +126,19 @@ describe("buildReplicationSources (pure)", () => {
126126
expect(sources[0].id).toBe("legacy");
127127
});
128128

129-
it("RUN_OPS_DATABASE_URL takes precedence: new source pgConnectionUrl === RUN_OPS_DATABASE_URL when both are supplied", () => {
129+
it("new source pgConnectionUrl === the provided RUN_OPS_DATABASE_URL", () => {
130130
const runOpsUrl = "postgres://run-ops-dedicated";
131-
const taskRunUrl = "postgres://task-run-legacy-alias";
132131

133132
const sources = buildReplicationSources({
134133
...baseArgs,
135134
splitEnabled: true,
136-
// Simulates env.RUN_OPS_DATABASE_URL ?? env.TASK_RUN_DATABASE_URL with RUN_OPS set
137-
newUrl: runOpsUrl ?? taskRunUrl,
135+
newUrl: runOpsUrl,
138136
});
139137

140138
expect(sources).toHaveLength(2);
141139
expect(sources[1]!.id).toBe("new");
142140
expect(sources[1]!.pgConnectionUrl).toBe(runOpsUrl);
143141
});
144-
145-
it("falls back to TASK_RUN_DATABASE_URL when RUN_OPS_DATABASE_URL is absent", () => {
146-
const taskRunUrl = "postgres://task-run-legacy-alias";
147-
148-
const sources = buildReplicationSources({
149-
...baseArgs,
150-
splitEnabled: true,
151-
// Simulates env.RUN_OPS_DATABASE_URL ?? env.TASK_RUN_DATABASE_URL with RUN_OPS unset
152-
newUrl: undefined ?? taskRunUrl,
153-
});
154-
155-
expect(sources).toHaveLength(2);
156-
expect(sources[1]!.pgConnectionUrl).toBe(taskRunUrl);
157-
});
158142
});
159143

160144
describe("assertReplicationCoversSplit (boot gate-coupling)", () => {

‎docker/scripts/entrypoint.sh‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,20 @@ else
1313
echo "SKIP_POSTGRES_MIGRATIONS=1, skipping Postgres migrations."
1414
fi
1515

16+
# Run-ops split: migrate the dedicated NEW run-ops database only when it is configured. Single-DB
17+
# installs never set the URL, so this is a no-op there.
18+
if [ -n "$RUN_OPS_DATABASE_URL" ]; then
19+
if [ "$SKIP_RUN_OPS_MIGRATIONS" != "1" ]; then
20+
echo "Running run-ops migrations"
21+
pnpm --filter @internal/run-ops-database db:migrate:deploy
22+
echo "Run-ops migrations done"
23+
else
24+
echo "SKIP_RUN_OPS_MIGRATIONS=1, skipping run-ops migrations."
25+
fi
26+
else
27+
echo "RUN_OPS_DATABASE_URL not set, skipping run-ops migrations."
28+
fi
29+
1630
if [ "$SKIP_DASHBOARD_AGENT_MIGRATIONS" != "1" ]; then
1731
echo "Running dashboard agent migrations"
1832
pnpm --filter @internal/dashboard-agent-db db:migrate:deploy

0 commit comments

Comments
 (0)