Skip to content

install-latest.sh: corrupt tarball download produces opaque tar error on intermittent CDN failures #381

@robobryce

Description

@robobryce

What

bin/install-latest.sh uses curl -sL "${DOWNLOAD_URL}" to fetch the release tarball. With -s (silent) and no -f (fail on HTTP errors), curl quietly writes whatever GitHub's CDN returns — including HTML error pages, partial content, or 404 bodies — into ${TMP_DIR}/brev.tar.gz. The next line, tar -xzf "${TMP_DIR}/brev.tar.gz", then fails with:

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

…which leaves the user with no signal that the actual problem was an HTTP-layer failure on the download.

When it happens

Intermittently on EC2 instances during fresh provisioning. We hit it ~3 times across ~16 deploys today on AWS m8i-flex.2xlarge instances pulling https://github.com/brevdev/brev-cli/releases/download/v0.6.323/brev-cli_0.6.323_linux_amd64.tar.gz. The same URL fetched cleanly on retry (and from non-EC2 hosts in the same window). Looks like GitHub's CDN occasionally rate-limits / 5xx's anonymous downloads from EC2 IP ranges; the CLI installer treats every response as success.

Reproduction

$ # On a fresh EC2 instance, occasionally:
$ curl -fsSL https://raw.githubusercontent.com/brevdev/brev-cli/main/bin/install-latest.sh | bash
[bootstrap] Installing / updating Brev CLI via official installer...
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

Fix

Two small changes on the download line in install-latest.sh:

  1. Switch curl -sL to curl -fsSL. -f makes curl exit non-zero on HTTP 4xx/5xx instead of writing the error body to the output file.
  2. Add a sanity check on the downloaded file before invoking tar, e.g. file "${TMP_DIR}/brev.tar.gz" | grep -q "gzip compressed" — emit a clearer "download corrupted; retry" message if it isn't actually a gzip blob.

Optionally, retry on transient failures (for i in 1 2 3; do curl -fsSL ... && break; sleep 5; done) so a single CDN blip doesn't kill an automated install.

Why this matters

Bootstrap automation (e.g. bootstrap-manager.bash in our deployment, or anyone curling install-latest.sh as part of a cloud-init / Brev startup script) sees an opaque tar error rather than the actual HTTP failure, so retry logic that triggers on "brev install failed" doesn't know whether to retry the download or treat it as a real failure.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions