feat: add Docker-based e2e install test#55
Open
omargallob wants to merge 3 commits into
Open
Conversation
Add a Go-based e2e test that validates the full devops-starter install flow inside a clean Ubuntu 24.04 Docker container: 1. devops-starter setup --non-interactive 2. devops-starter install --yes 3. Verify every registered tool: binary on PATH + version match Components: - tests/e2e/cmd/verify/main.go: Go verifier binary that imports the registry and state packages directly (no shell parsing needed) - tests/e2e/Dockerfile: Ubuntu 24.04 with curl/git/unzip, non-root user - .github/workflows/e2e.yml: Weekly cron (Sundays 3am UTC) + manual dispatch - Makefile test-e2e target for local testing The verifier supports E2E_GROUPS env var for filtering (e.g., E2E_GROUPS=utilities for fast iteration). It uses existing version probes from internal/state for accurate detection and reports a pass/fail/skip summary table. Local testing: make test-e2e # all groups E2E_GROUPS=utilities make test-e2e # single group
- Replace 'go build' with 'bazel build --config=linux_amd64' in Makefile - Single bazel build command for both targets in CI workflow - Add //tests:__subpackages__ visibility to internal/registry and internal/config BUILD.bazel files
Tests cover: - groupMatchesFilter: exact match, multi-group, no match, trimming - statusIcon: all status values - printResults: exit codes (all pass, one fail, skipped, warns) - verifyTool: binary not found, binary exists (no probe), InstallName override - verifyGhExtension: gh not installed, extension found in list
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a Go-based end-to-end test that validates the full devops-starter install flow inside a clean Ubuntu 24.04 Docker container.
What it tests
devops-starter setup --non-interactive— config creationdevops-starter install --yes— full tool installation (all enabled groups)GetInstallName())state.DetectVersionAtPath)gh extension listArchitecture
tests/e2e/cmd/verify/main.gotests/e2e/Dockerfile.github/workflows/e2e.ymlmake test-e2eThe verifier imports
internal/registry,internal/state, andpkg/tooldefdirectly — no JSON parsing or shell scripting needed. It's always in sync with the tool catalog.Workflow trigger
mainworkflow_dispatchfor on-demand runsLocal testing
Features
E2E_GROUPSenv var for filtering groups during testinggh extension list)Local test results
Validated locally with
E2E_GROUPS=utilities:Files
tests/e2e/cmd/verify/main.go— Go verifier binary (230 lines)tests/e2e/cmd/verify/BUILD.bazel— Bazel target (Gazelle-generated)tests/e2e/Dockerfile— Ubuntu 24.04 base image.github/workflows/e2e.yml— Weekly CI workflowMakefile— Addedtest-e2etarget.gitignore— Ignore cross-compiled binaries intests/e2e/