Skip to content

fix(dev): make 'make dev-up' work — readable temp compose files + HAKeeper bootstrap#24974

Open
fengttt wants to merge 3 commits into
matrixorigin:mainfrom
fengttt:fix/dev-up-multi-cn
Open

fix(dev): make 'make dev-up' work — readable temp compose files + HAKeeper bootstrap#24974
fengttt wants to merge 3 commits into
matrixorigin:mainfrom
fengttt:fix/dev-up-multi-cn

Conversation

@fengttt

@fengttt fengttt commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

What

Two fixes that let the etc/docker-multi-cn-local-disk stack (make dev-up) actually start a working multi-CN cluster. The first failure masked the second, so both are needed.

1. Temp compose fragments written to /tmp (start.sh)

start.sh generated the default-limits and extra-mounts compose fragments under /tmp. A snap-confined Docker daemon cannot read /tmp, so docker compose failed before any container started:

open /tmp/docker-compose.default-limits.<pid>.yml: no such file or directory

Now the fragments are written in the project directory (under $HOME, which the daemon can access) using relative paths, so docker compose keeps it as the project directory. Also fixes a latent cleanup bug: the two per-block trap ... EXIT handlers clobbered each other (the second overwrote the first), leaking the default-limits file whenever extra mounts were used — both temp files are now tracked in one list and removed by a single trap.

2. HAKeeper never bootstrapped (generate-config.sh)

The generated log.toml had no [logservice] section. Each node is started separately with -cfg log.toml (not -launch), so nothing ever bootstrapped HAKeeper and the log store panicked on first start:

panic: shard not bootstrapped

The launch stacks avoid this only because -launch builds the bootstrap config programmatically; no checked-in toml sets bootstrap-cluster. (This was invisible until fix #1, since compose always died at the /tmp step before a container ran.)

generate-config.sh now emits a [logservice] section that gives mo-log a UUID and bootstraps a single-replica HAKeeper (1 log shard, 1 tn shard, 1 replica, init member 131072:<uuid>), matching pkg/embed/cluster.go and logservice.DefaultConfig. Addresses advertise the mo-log hostname while listening on 0.0.0.0, and gossip-allow-self-as-seed is set because Fill() (unlike DefaultConfig) does not default it — required for a lone seed node. The *.toml files are gitignored (generated), so the fix lives in the generator.

Verification

From a clean state (make dev-clean && make dev-up), validated through the generated config path (tomls removed → regenerated by generate-config.sh → fresh bring-up):

  • all 5 containers up, mo-log reports healthy, zero shard not bootstrapped panics
  • create / insert / select sum() round-trips through the proxy on port 6001

🤖 Generated with Claude Code

fengttt and others added 2 commits June 13, 2026 11:03
A snap-confined Docker daemon cannot read files under /tmp, so 'make dev-up'
failed with 'open /tmp/docker-compose.default-limits.<pid>.yml: no such file or
directory' before any container started. Write the generated default-limits and
extra-mounts compose fragments in the project directory (under $HOME, which the
daemon can access) using relative paths so docker compose keeps it as the
project directory.

Also fixes a latent cleanup bug: the two per-block 'trap ... EXIT' handlers
clobbered each other (the second overwrote the first), leaking the
default-limits file whenever extra mounts were used. Both temp files are now
tracked in one list and removed by a single trap.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The generated log.toml had no [logservice] section, so when each node was
started separately with '-cfg log.toml' nothing ever bootstrapped HAKeeper and
the log store panicked with 'shard not bootstrapped' on first start. (The launch
stacks avoid this only because '-launch' builds the bootstrap config
programmatically; no checked-in toml sets bootstrap-cluster. This was masked
until now because compose previously died reading the /tmp limits file before
any container started.)

generate-config.sh now emits a [logservice] section that gives mo-log a UUID and
bootstraps a single-replica HAKeeper (1 log shard, 1 tn shard, 1 replica, init
member 131072:<uuid>), matching pkg/embed/cluster.go and logservice.DefaultConfig.
Addresses advertise the mo-log hostname while listening on 0.0.0.0, and
gossip-allow-self-as-seed is set because Fill() (unlike DefaultConfig) does not
default it — required for a lone seed node. The toml files themselves are
gitignored (generated), so the fix lives in the generator.

Verified from a clean state: make dev-clean && make dev-up brings up all 5
containers, mo-log reports healthy with no panic, and create/insert/select
round-trips through the proxy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@fengttt fengttt requested a review from XuPeng-SH as a code owner June 13, 2026 18:12
@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes the local multi-CN Docker Compose dev stack (etc/docker-multi-cn-local-disk, invoked via make dev-up) so it can reliably start a functional cluster by (1) generating compose override fragments in a Docker-readable location and cleaning them up correctly, and (2) ensuring the logservice boots HAKeeper when started via -cfg log.toml (not -launch).

Changes:

  • Write temporary docker-compose fragments next to docker-compose.yml (instead of /tmp) and clean them up via a single trap to avoid leaking temp files.
  • Extend generate-config.sh to emit a [logservice] section (including BootstrapConfig) in log.toml so a single-node HAKeeper is bootstrapped for this dev stack.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
etc/docker-multi-cn-local-disk/start.sh Writes compose fragment temp files in the project directory and centralizes cleanup so docker compose works with snap-confined Docker.
etc/docker-multi-cn-local-disk/generate-config.sh Adds logservice + bootstrap configuration to generated log.toml so HAKeeper is initialized and log store doesn’t panic on first start.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/S Denotes a PR that changes [10,99] lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants