Skip to content

fix(start): suggest recovery for exec format and backup migration failures #5937

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rohanpatel2002 wants to merge 1 commit into from
+124 −3
Closed

fix(start): suggest recovery for exec format and backup migration failures #5937

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions apps/cli-go/internal/db/start/start.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,11 +213,20 @@ func WaitForHealthyService(ctx context.Context, timeout time.Duration, started .
policy := NewBackoffPolicy(ctx, timeout)
err := backoff.Retry(probe, policy)
if err != nil && !errors.Is(err, context.Canceled) {
// Print container logs for easier debugging
// Print container logs for easier debugging and suggest recovery steps.
var logBuf strings.Builder
for _, containerId := range started {
fmt.Fprintln(os.Stderr, containerId, "container logs:")
if err := utils.DockerStreamLogsOnce(context.Background(), containerId, os.Stderr, os.Stderr); err != nil {
fmt.Fprintln(os.Stderr, err)
w := io.MultiWriter(os.Stderr, &logBuf)
if logErr := utils.DockerStreamLogsOnce(context.Background(), containerId, w, w); logErr != nil {
fmt.Fprintln(os.Stderr, logErr)
}
}
if suggestion := SuggestFromUnhealthyLogs(logBuf.String()); len(suggestion) > 0 {

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Scope destructive recovery hints to stack startup

When WaitForHealthyService is used for temporary containers outside supabase start (for example the db diff shadow database in apps/cli-go/internal/db/diff/shadow.go), this unconditional log scan can surface suggestions that tell users to run supabase stop --no-backup. In those contexts the local stack volume is not the failed resource, and following the hint can delete local development data unnecessarily; only add these recovery steps for local-stack startup checks or tailor them per caller.

Useful? React with 👍 / 👎.

if len(utils.CmdSuggestion) > 0 {
utils.CmdSuggestion += "\n\n" + suggestion
} else {
utils.CmdSuggestion = suggestion
}
}
}
Expand Down
57 changes: 57 additions & 0 deletions apps/cli-go/internal/db/start/suggest_unhealthy.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
package start

import (
"fmt"
"runtime"
"strings"

"github.com/supabase/cli/internal/utils"
)

// SuggestFromUnhealthyLogs returns a CmdSuggestion for common local-stack
// startup failures seen after image upgrades or wrong-arch pulls
// (see https://github.com/supabase/supabase/issues/48224).
func SuggestFromUnhealthyLogs(logs string) string {
lower := strings.ToLower(logs)
var parts []string

if strings.Contains(lower, "exec format error") {
arch := runtime.GOARCH
parts = append(parts, fmt.Sprintf(
"A container failed with \"exec format error\" (wrong CPU architecture image).\n"+
"Try removing the mismatched images and restarting for linux/%s:\n"+
" %s\n"+
" docker image prune -f\n"+

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove tagged images for exec-format recovery

When the exec-format failure is caused by a tagged local image with the wrong architecture, this recovery leaves it in place: Docker documents that docker image prune removes dangling images by default and only removes other unused images with -a (https://docs.docker.com/reference/cli/docker/image/prune/), while DockerResolveImageIfNotCached returns any locally inspectable tag before reaching the new platform-pinned pull. After supabase stop --no-backup && docker image prune -f, supabase start can reuse the same tagged wrong-arch image and hit exec format error again; the hint should remove the affected Supabase image tag(s) or force a platform-aware repull.

Useful? React with 👍 / 👎.

" %s",
arch,
utils.Aqua("supabase stop --no-backup"),
utils.Aqua("supabase start"),
))
}

if strings.Contains(lower, "migrations_name_key") ||
(strings.Contains(lower, "migration failed") && strings.Contains(lower, "duplicate key")) {
parts = append(parts, fmt.Sprintf(
"Storage migrations failed against an existing database volume (often after upgrading images while restoring a backup).\n"+
"Reset local data and start fresh:\n"+
" %s\n"+
" %s",
utils.Aqua("supabase stop --no-backup"),
utils.Aqua("supabase start"),
))
}

if strings.Contains(lower, "err_invalid_package_config") ||
strings.Contains(lower, "invalid package config") {
parts = append(parts, fmt.Sprintf(
"Studio image looks corrupted or incomplete. Remove it and re-pull:\n"+
" %s\n"+
" docker image rm -f $(docker images -q supabase/studio) 2>/dev/null\n"+

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove mirrored Studio images too

For the default registry path, Studio is cached as public.ecr.aws/supabase/studio (and the fallback can be ghcr.io/supabase/studio), but this command only asks Docker for the exact repository supabase/studio; Docker's image-list docs state the [REPOSITORY[:TAG]] argument must be an exact match (https://docs.docker.com/reference/cli/docker/image/ls/). In the corrupted default/fallback image scenario the command produces no IDs, removes nothing, and the next supabase start reuses the broken Studio image, so the hint should match the configured/mirrored repository names instead of only Docker Hub.

Useful? React with 👍 / 👎.

" %s",
utils.Aqua("supabase stop --no-backup"),
utils.Aqua("supabase start"),
))
}

return strings.Join(parts, "\n\n")
}
51 changes: 51 additions & 0 deletions apps/cli-go/internal/db/start/suggest_unhealthy_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
package start

import (
"runtime"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)

func TestSuggestFromUnhealthyLogs(t *testing.T) {
t.Run("suggests recovery for exec format error", func(t *testing.T) {
suggestion := SuggestFromUnhealthyLogs("exec /mailpit: exec format error\n")
require.NotEmpty(t, suggestion)
assert.Contains(t, suggestion, "exec format error")
assert.Contains(t, suggestion, "supabase stop --no-backup")
assert.Contains(t, suggestion, "supabase start")
assert.Contains(t, suggestion, "linux/"+runtime.GOARCH)
})

t.Run("suggests recovery for storage migrations_name_key", func(t *testing.T) {
logs := `Migration failed. Reason: duplicate key value violates unique constraint "migrations_name_key"`
suggestion := SuggestFromUnhealthyLogs(logs)
require.NotEmpty(t, suggestion)
assert.Contains(t, suggestion, "Storage migrations failed")
assert.Contains(t, suggestion, "supabase stop --no-backup")
assert.Contains(t, suggestion, "supabase start")
})

t.Run("suggests recovery for studio invalid package config", func(t *testing.T) {
logs := "Error: Invalid package config /app/apps/studio/node_modules/next/package.json.\n code: 'ERR_INVALID_PACKAGE_CONFIG'"
suggestion := SuggestFromUnhealthyLogs(logs)
require.NotEmpty(t, suggestion)
assert.Contains(t, suggestion, "Studio")
assert.Contains(t, suggestion, "supabase stop --no-backup")
assert.Contains(t, suggestion, "supabase start")
})

t.Run("combines multiple failure suggestions", func(t *testing.T) {
logs := "exec /mailpit: exec format error\nmigrations_name_key\nERR_INVALID_PACKAGE_CONFIG"
suggestion := SuggestFromUnhealthyLogs(logs)
require.NotEmpty(t, suggestion)
assert.Contains(t, suggestion, "exec format error")
assert.Contains(t, suggestion, "Storage migrations failed")
assert.Contains(t, suggestion, "Studio")
})

t.Run("returns empty for unrelated logs", func(t *testing.T) {
assert.Empty(t, SuggestFromUnhealthyLogs("listening on :5432\nready\n"))
})
}
4 changes: 4 additions & 0 deletions apps/cli-go/internal/utils/docker.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (
"log"
"os"
"regexp"
"runtime"
"strings"
"sync"
"time"
Expand Down Expand Up @@ -289,6 +290,9 @@ func registryFromImage(imageTag string) string {
func DockerImagePull(ctx context.Context, imageTag string, w io.Writer) error {
out, err := Docker.ImagePull(ctx, imageTag, image.PullOptions{
RegistryAuth: GetRegistryAuthForImage(imageTag),
// Prefer the host architecture so services like mailpit do not start
// with a wrong-arch binary (exec format error). See supabase/supabase#48224.
Platform: "linux/" + runtime.GOARCH,

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use the Docker daemon architecture

When the CLI talks to a Docker daemon with a different architecture than the CLI binary, such as an amd64 CLI using a remote arm64 Docker context/DOCKER_HOST or Rosetta, this forces pulls to linux/<runtime.GOARCH>. Docker's pull option explicitly sets the platform (https://docs.docker.com/reference/cli/docker/image/pull/), so the daemon can receive an image for the client architecture rather than the daemon architecture and then fail to run it without emulation; derive the platform from the daemon or make the override conditional.

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Pin the compose pre-pull path too

In supabase start, the first pull attempt is pullImagesUsingCompose (apps/cli-go/internal/start/start.go:284), whose RetryClient.ImagePull forwards Docker's pull options without setting a platform; only images that compose skipped reach this new platform-pinned DockerImagePull. If the normal compose pre-pull is the path that fetches a wrong-arch tag, ensureImagesCached then sees the tag locally and never executes this pinned pull, so the Mailpit exec-format failure can still happen on an empty cache unless the compose pull path is pinned or disabled as well.

Useful? React with 👍 / 👎.

})
if err != nil {
return errors.Errorf("failed to pull docker image: %w", err)
Expand Down
Loading