Skip to content

test(smoke): harden rc dual-stack catalog checks#715

Merged
bussyjd merged 2 commits into
integration/v0.13.0-rc0from
codex/rc0-smoke-flow13-flow11-qa
Jul 7, 2026
Merged

test(smoke): harden rc dual-stack catalog checks#715
bussyjd merged 2 commits into
integration/v0.13.0-rc0from
codex/rc0-smoke-flow13-flow11-qa

Conversation

@bussyjd

@bussyjd bussyjd commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Summary

  • fix the flow-13 catalog assertion to parse the current /api/services.json envelope while retaining compatibility with the old bare-array shape
  • stop marking confirmed ERC-8004 registration receipt lookup misses as SKIP in flow-11 and the matching live-OBOL flow path
  • document the QA gaps from the RC smoke: runner vs inference endpoint separation, no flow mutation in QA worktrees, Hermes bedag/raw dependency failures, and kube-state-metrics registry stalls

Why

The RC smoke failed flow-13 because the helper iterated the current catalog envelope as if it were a list. The product was serving the right schema; the smoke assertion was stale. Flow-11 completed its payment path but surfaced as SKIP because receipt lookup was unavailable even though the ServiceOffer status already reflected registration.

Hermes still uses the third-party bedag/raw chart to render generated Kubernetes resources. When that chart fetch times out, Hermes install can fail before payment or agent logic is exercised; the follow-up direction is to replace that external chart dependency with a first-party/local embedded chart or first-party Hermes chart.

The current RC rerun also reproed a separate flow-02 dependency stall: kubelet remained on Pulling for registry.k8s.io/kube-state-metrics/kube-state-metrics:v2.18.0 for the full flow poll window. That is captured as a QA preflight/dependency note rather than hidden by a flow change.

Validation

Local:

  • bash -n flows/*.sh
  • git diff --check
  • go test ./internal/hermes -count=1
  • local parser sanity check against both catalog envelope and legacy list payloads

Remote release-smoke attempt:

  • commit under test: d875552 (code fix before the docs-only QA preflight commit)
  • LLM preflight passed with the explicit configured model
  • flow-01 passed
  • flow-02 logged FAIL at step 7 after 900s: monitoring-kube-state-metrics remained ContainerCreating while pulling registry.k8s.io/kube-state-metrics/kube-state-metrics:v2.18.0
  • flow-02 still proved frontend, eRPC, Base Sepolia routing, serviceoffer-controller, and the x402-buyer Prometheus scrape before the run was stopped at the repeated monitoring check

Release-smoke status

This PR addresses flow-13's catalog parser failure and the flow-11 SKIP accounting issue from the RC smoke report. It does not claim the full RC smoke is green; flow-02 kube-state-metrics image pull/bootstrap remains an environment/dependency blocker that must be cleared before a full rerun can validate the remaining flows.

@bussyjd bussyjd merged commit a29c826 into integration/v0.13.0-rc0 Jul 7, 2026
@bussyjd

bussyjd commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

Merged into integration/v0.13.0-rc0 at a29c826 and reran targeted flow13 from that tip.

Result: the previous catalog-envelope failure is fixed. Bob's agent pod reached /api/services.json and logged:

HTTP 200 alice-obol-inference ... OBOL permit2
PASS: [44] Agent pod found alice-obol-inference (OBOL) in service catalog

The full flow still failed later at step 45 because no PurchaseRequest was created after the retry. That is a separate agent-buy blocker, not the catalog parsing failure fixed by this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant