Add OpenBao engine support to safe CLI and CI matrix#7
Open
itsouvalas wants to merge 11 commits into
Open
Conversation
Adds an Engine interface with vaultEngine and baoEngine implementations plus selectLocalEngine() which picks the preferred available binary on PATH. Order: prefer bao (OpenBao), fall back to vault (HashiCorp Vault). An optional preference string pins a specific engine or reports unknown engines cleanly. Test coverage includes auto-detect (bao-only, vault-only, both, neither) and preference-override paths. No production code yet consumes selectLocalEngine — subsequent commits wire it into safe local.
safe local previously discarded the root token returned by v.Init(1, 1) and regenerated one via /sys/generate-root/attempt. OpenBao disables that endpoint, so the flow failed with 'unsupported operation' on every fresh bao server. Capture Init's token directly, and only fall back to NewRootToken when it is empty — i.e. when --file points at an existing vault whose unseal key the user supplied but whose original root token is unknown. On vault this saves one round trip. On bao it unblocks --memory and fresh --file backends. Re-targeting an existing bao file backend still requires --engine vault because generate-root is unavailable.
Extends the 'safe local' help block with the new --engine flag in the Usage line and a paragraph explaining the auto-detect order (prefer bao over vault when both are installed). Adds a '### local' section to README covering --memory vs --file usage, the --engine flag, and the OpenBao caveat that the --file backend can only be used with a fresh directory — re-targeting an existing one needs the original root token, which requires --engine vault because OpenBao disables the legacy generate-root API.
Adds --engine {vault,bao} to 'safe local' and routes server spawn
through Engine.Binary() instead of hard-coded 'vault'. The
selection respects the flag when set, otherwise auto-detects
per selectLocalEngine.
Also drops the pre-0.8.0 Vault storage-key version probe.
The 'backend' HCL key was renamed to 'storage' in Vault 0.8
(August 2017); any Vault recent enough to run safe uses
'storage' unconditionally, and OpenBao has never supported the
legacy key. The storage key is now hard-coded, removing one
exec.Command('vault', 'version') round trip and the
goto-based fallback.
The hard-coded 'Vault is not installed' error message is
replaced by the engine-aware messages produced inside
selectLocalEngine (names both candidate binaries when neither
is found; names the specific binary when a preference is
unreachable).
`^time.Duration(0) >> 1` on a signed int64 uses arithmetic right shift, so `^(-1) >> 1 == -1`, not MaxInt64 as intended. With maxDuration==-1ns, `maxDuration/time.Hour==0`, so the overflow guard `v > 0/hoursPerYear` rejected every non-zero year count and surfaced as `duration overflow: 10 years is too large` against the default 10y CA TTL. Swap in `^uint64(0) >> 1` (unsigned logical shift) to get the intended MaxInt64. Add util_test.go covering happy paths, the overflow detection still kicking in at the right scale, and unrecognised duration strings. Introduced alongside the G115 gosec overflow guards in 83f7830. Invisible until the ci/scripts/tests round-trip suite was reconnected via make test-release.
ci/scripts/tests now selects between HashiCorp Vault and OpenBao
via the ENGINE env var (default vault). The download URL,
archive format, binary name, and OS/arch aliasing vary per
engine; the suite itself runs unchanged.
Helper functions live in ci/scripts/lib/engine.sh:
engine_url, engine_archive_ext, engine_binary_name,
engine_local_binary, engine_arch_for, engine_os_for,
engine_startup_mode.
Unit tests in ci/scripts/t/engine_test.sh exercise every
supported {engine,os,arch} combination plus the error paths
(unknown engine / arch / os). Pure bash, no network, <1s.
restart_engine_server dispatches on engine_startup_mode:
- vault -> _start_engine_dev (existing `server -dev` flow)
- bao -> _start_engine_config (writes HCL config with
disable_unauthed_rekey_endpoints=false so /sys/rekey/*
stays reachable, then init + unseal + pre-mount secret/
via the HTTP API)
Also drops the old arch=amd64 hardcode in favour of a
uname -m case that handles x86_64/amd64/aarch64/arm64. The
vault path's behaviour is preserved identically when ENGINE is
unset or vault.
Three test expectations in ci/scripts/tests referenced the pre-cleanup uppercase-first error strings (`!! Failed ...`, `!! No file ...`, `!! Key submission ...`). Safe's user-facing error output was normalised to lowercase-first during an earlier gosec/staticcheck pass (same cleanup wave as ed78ec7/83f7830). These assertions have been silently wrong since the round-trip suite was disconnected from make test, and surfaced as soon as make test-release wired them back up. Update the expected strings to match current output; no safe code change.
The make test target was rewritten in commit ed78ec7 into a plain `go test` invocation, which silently ignored the TEST_PATH / SAFE_PATH / VAULT_VERSIONS variables that ci/scripts/test-release-against-vault has been passing via CI. The ~2000-line round-trip suite in ci/scripts/tests has not actually run in Concourse since that commit landed in 2025. Add a distinct test-release target that invokes the parameterized round-trip script directly, honoring ENGINE / VERSIONS / SAFE_PATH / TEST_PATH. `make test` (Go unit tests) is left untouched so nothing that depends on it regresses. Also add test-release-engine as a convenience for running the fast bash-level helper unit tests without touching the live engine.
Split the Concourse task entry point into two scripts: - test-release-against-engine: canonical wrapper. Accepts ENGINE env (default vault), picks VAULT_VERSIONS or OPENBAO_VERSIONS per engine, and invokes `make test-release` with the right env. - test-release-against-vault: trivial back-compat shim that `exec`s test-release-against-engine with ENGINE=vault. Preserves the existing task.yml reference until ci/tasks/test.yml flips in the follow-up commit. Also fixes a pre-existing bug where the wrapper validated VAULT_VERSION (singular) but forwarded VAULT_VERSIONS (plural); task.yml always passes plural. New wrapper validates per engine on the plural name.
Adds a github-release resource tracking the latest OpenBao 2.5.x patch (tag_filter v?(2.5.[^v].*), mirrors the existing Vault resources) and a test-openbao-2.5 job that runs the round-trip suite against it with ENGINE=bao and OPENBAO_VERSIONS loaded from the resource's version file. Appends test-openbao-2.5 to the build-safe-releases group via spruce (( append )), so the job shows up alongside the five existing test-vault-* jobs without touching pipeline/base.yml (which is marked DO NOT MODIFY). Flips ci/tasks/test.yml from the back-compat vault shim to the generalized test-release-against-engine wrapper. Vault jobs continue to work unchanged (ENGINE defaults to vault in the wrapper); the OpenBao job passes ENGINE=bao via params. Validated locally via spruce merge + docker-compose Concourse: fly set-pipeline accepts the merged yaml, fly check-resource resolves github-openbao-2.5 to v2.5.3, test-vault-1.13 to v1.13.13. Full end-to-end task run of test-openbao-2.5 (via fly execute on a linux worker) completes clean — 1724 assertions passing, 0 failing.
Add a Testing section covering:
- make test (Go unit tests)
- make test-release ENGINE={vault,bao} VERSIONS=...
with multi-version and SAFE_DISABLE_GPG_TESTS examples
- make test-release-engine (fast helper lib unit tests)
- Concourse matrix description and release-page links
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
safe localgains an--engine vault|baoflag with an engine-abstractionlayer; auto-detects whichever binary is on
$PATHwhen unset (prefersbao). Vault<0.8 compatibility code removed along the way.ci/scripts/testsround-trip suite (~2000 assertions) parameterised onENGINE. Handles the Vault↔OpenBao archive-naming divergence(
vault_…_linux_amd64.zipvsbao_…_Linux_x86_64.tar.gz) via a smallbash helper library with its own unit tests.
test-openbao-2.5Concourse job tracking the latest 2.5.x patch,sibling to the existing
test-vault-1.9…test-vault-1.13jobs. RunsOpenBao in non-dev mode with
disable_unauthed_rekey_endpoints = "false"so
/sys/rekey/*stays reachable (bao defaults it off since v2.5.0).make test-releasetarget; the post-ed78ec7cleanupredefined
make testto plaingo test, silently disconnecting theround-trip script from CI.
duration():^time.Duration(0) >> 1yielded-1not
MaxInt64, so the overflow guard rejected every non-zero yearcount (the default 10y CA TTL included).
(normalised in the earlier gosec/staticcheck cleanup wave).
Test plan
make test— go unit tests passmake test-release-engine— 29/29 bash helper unit tests greenmake test-release ENGINE=vault VERSIONS=1.13.13— 1694/1694 greenSAFE_DISABLE_GPG_TESTS=1 make test-release ENGINE=bao VERSIONS=2.5.3— 1694/1694 greenfly executeof the bao task on a linux/amd64 Concourse worker — 1724/1724 greenfly set-pipelineon a docker-compose Concourse resolves thenew
github-openbao-2.5resource tov2.5.3; existinggithub-vault-1.13continues to resolve tov1.13.13.