Skip to content

feat(snapshots): in place resume & fs-only snapshots - #3483

Draft
bchalios wants to merge 55 commits into
mainfrom
feat/in-place-resume
Draft

feat(snapshots): in place resume & fs-only snapshots#3483
bchalios wants to merge 55 commits into
mainfrom
feat/in-place-resume

Conversation

@bchalios

Copy link
Copy Markdown
Contributor

TBD

ben-fornefeld and others added 30 commits July 27, 2026 23:11
…ructor (#3423)

Makes `packages/auth` usable by a service that authenticates a route
**before** the caller is a user — signup. Today no verifier fits that,
so the closest one silently skips OIDC discovery, the issuer cross-check
and the `iss` claim.

## Verifiers, now one axis

| | keys from | establishes |
|---|---|---|
| `JWKSVerifier` | issuer's JWKS path | claims |
| `OIDCVerifier` | OIDC discovery | what the token asserts |
| `LinkedOIDCVerifier` | OIDC discovery | + the internal user |

Each adds to the one before. Picking a lower rung is a choice about what
you need, never a weaker check — discovery and issuer validation are
identical across both OIDC levels.

`LinkedOIDCVerifier` embeds `OIDCVerifier`, so linking is additive: the
unlinked type has no `Verify` to call, rather than one that fails at
runtime.

## Also

- **`NewAuthenticator(AuthenticatorConfig[T])`** — build an
authenticator for a scheme this package doesn't name. Every existing
constructor is already a thin literal over `commonAuthenticator`, but
they fix the scheme name and context key, and the type is `internal`.
Named constructors unchanged.
- **`iss`/`sub` extraction** moved into `VerifyIdentity`; the resolving
path is built on it, so those claims are read in one place.

## Breaking

| before | after |
|---|---|
| `ProviderVerifier` | `LinkedOIDCVerifier` |
| `OIDCVerifier` (single-issuer) | `OIDCIssuerVerifier` |
| `AdminVerifier` | `JWKSVerifier` |

Aliases removed rather than deprecated — only belt used them, and it
pins by commit. The `OIDCVerifier` reuse can't break silently: the new
one takes a `ProviderConfig` where the old took a `JWTConfig`, so
callers get a compile error.

`NewAdminJWTAuthenticator` keeps its name — `AdminJWTAuth` is a real
security scheme.

## Verification

Behaviour unchanged for every current caller; `dashboard-api` verifies
exactly as before. New tests cover the identity verifier (reports claims
without a lookup, rejects a missing subject, rejects a discovery issuer
mismatch), the authenticator constructor (scheme, prefix stripping, 401
on missing header, optional context setter), and that nil verifiers deny
rather than panic — the last one panics without its guard, since
promotion through an embedded value dereferences before the inner nil
check.

No `docs/ARCHITECTURE.md` change: no service, port, protocol, data
store, flow or topology affected.
A filesystem-only pause must quiesce the guest rootfs before
snapshotting to close the sync->pause write race. Today only envd >=
0.6.6 (native /fsfreeze) does a real FIFREEZE; older guests fall back to
a plain `sync`, which does not block writes and can capture a torn ext4
journal.

For those guests, when the fsfreeze-via-exec flag is on, run `fsfreeze
-f /` through the envd process API instead — the same FIFREEZE, no
native endpoint needed, running entirely inside the guest (no host
attack surface). Falls back to `sync` per-pause if the guest lacks the
fsfreeze binary or the freeze fails. On the pause-failure rollback the
rootfs is thawed via the same API (bounded, best-effort).

Gated behind featureflags.FsFreezeViaExecFlag (off by default).

Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Makes filesystem-only pauses distinguishable in metrics, so the
[Filesystem-only Snapshots
dashboard](https://github.com/e2b-dev/monitoring) can stop relying on
"(all pauses)" context panels.

## Change
- **New histogram `orchestrator.sandbox.pause.duration`** — recorded in
the `Pause` handler with attributes `fs_only` (filesystem-only vs
memory) and `success`. Gives fs-only pause **call-count** (count),
**error rate** (`success="false"`), and **e2e latency** (quantiles) —
none of which were separable before, since the gRPC Pause RPC metric
carries no fs-only label (fs-only-ness was only a span attribute).
- **`fs_only` attribute on `orchestrator.snapshot.upload.failed`** —
threaded through `snapshotResult` so the async upload can label failures
fs-only vs memory.

## Why
The orchestrator recorded fs-only-ness only as a *span* attribute
(`fs-only-snapshot`), not a metric label, so pause e2e latency / errors
/ upload failures could not be scoped to fs-only pauses. The dashboard
worked around this with clearly-labeled "(all pauses)" panels; these
labels let those panels filter `fs_only="true"` and become real.

## Notes
- `Pause` now uses named returns so the deferred metric can capture
success.
- No change to the resume side — fs-only resume is already
distinguishable via `envd_init.duration{start_type="reboot"}`.
- Follow-up: a small monitoring-repo PR will switch the dashboard's
pause/upload panels to `fs_only="true"` once this deploys.

Verified: `go build`/`vet`/`golangci-lint` clean, `go test
./pkg/server/` passes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…dates (#3417)

Bumps the go_modules group with 1 update in the /packages/api directory:
[github.com/getkin/kin-openapi](https://github.com/getkin/kin-openapi).
Bumps the go_modules group with 1 update in the /packages/auth
directory:
[github.com/getkin/kin-openapi](https://github.com/getkin/kin-openapi).
Bumps the go_modules group with 1 update in the /packages/dashboard-api
directory:
[github.com/getkin/kin-openapi](https://github.com/getkin/kin-openapi).
Bumps the go_modules group with 1 update in the /packages/db directory:
[github.com/google/cel-go](https://github.com/google/cel-go).
Bumps the go_modules group with 1 update in the /packages/envd
directory:
[github.com/getkin/kin-openapi](https://github.com/getkin/kin-openapi).
Bumps the go_modules group with 1 update in the /packages/orchestrator
directory:
[github.com/getkin/kin-openapi](https://github.com/getkin/kin-openapi).
Bumps the go_modules group with 1 update in the /tests/integration
directory:
[github.com/getkin/kin-openapi](https://github.com/getkin/kin-openapi).

Updates `github.com/getkin/kin-openapi` from 0.139.0 to 0.144.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/getkin/kin-openapi/releases">github.com/getkin/kin-openapi's
releases</a>.</em></p>
<blockquote>
<h2>v0.144.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3filter: prefer non-empty value for repeated scalar query
params by <a
href="https://github.com/sonnemusk"><code>@​sonnemusk</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1231">getkin/kin-openapi#1231</a></li>
<li>some fixes to please my moulinette by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1232">getkin/kin-openapi#1232</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/sonnemusk"><code>@​sonnemusk</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1231">getkin/kin-openapi#1231</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/getkin/kin-openapi/compare/v0.143.0...v0.144.0">https://github.com/getkin/kin-openapi/compare/v0.143.0...v0.144.0</a></p>
<h2>v0.143.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3: remove StringMap and its pre-OriginTree origin-stripping
machinery by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1219">getkin/kin-openapi#1219</a></li>
<li>openapi3: huge test suite: make it more parallel by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1221">getkin/kin-openapi#1221</a></li>
<li>openapi31: support 3.1+ reference metadata overrides by <a
href="https://github.com/AlexanderWangY"><code>@​AlexanderWangY</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1215">getkin/kin-openapi#1215</a></li>
<li>docs: move the StringMap changelog entry to v0.143.0 by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1222">getkin/kin-openapi#1222</a></li>
<li>openapi3: add T.WalkParameters to visit every parameter in a
document by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1224">getkin/kin-openapi#1224</a></li>
<li>openapi3filter: reuse deepObject bracket regex by <a
href="https://github.com/matiasinsaurralde"><code>@​matiasinsaurralde</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1225">getkin/kin-openapi#1225</a></li>
<li>openapi3filter: skip Split allocation for single-value array query
params by <a
href="https://github.com/matiasinsaurralde"><code>@​matiasinsaurralde</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1226">getkin/kin-openapi#1226</a></li>
<li>openapi3: preserve origin for a $ref to a schema under an arbitrary
top-level key by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1227">getkin/kin-openapi#1227</a></li>
<li>openapi3: stable codes for validation errors by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1223">getkin/kin-openapi#1223</a></li>
<li>openapi3filter: skip schema checks for empty allowEmptyValue strings
by <a
href="https://github.com/snowyukitty"><code>@​snowyukitty</code></a> in
<a
href="https://redirect.github.com/getkin/kin-openapi/pull/1228">getkin/kin-openapi#1228</a></li>
<li>openapi3gen: inline embedded struct with options-only JSON tag by <a
href="https://github.com/mvanhorn"><code>@​mvanhorn</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1229">getkin/kin-openapi#1229</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/matiasinsaurralde"><code>@​matiasinsaurralde</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1225">getkin/kin-openapi#1225</a></li>
<li><a
href="https://github.com/snowyukitty"><code>@​snowyukitty</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1228">getkin/kin-openapi#1228</a></li>
<li><a href="https://github.com/mvanhorn"><code>@​mvanhorn</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1229">getkin/kin-openapi#1229</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/getkin/kin-openapi/compare/v0.142.0...v0.143.0">https://github.com/getkin/kin-openapi/compare/v0.142.0...v0.143.0</a></p>
<h2>v0.142.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3: fix test that relied on now non-resolving domain by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1220">getkin/kin-openapi#1220</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/getkin/kin-openapi/compare/v0.141.0...v0.142.0">https://github.com/getkin/kin-openapi/compare/v0.141.0...v0.142.0</a></p>
<h2>v0.141.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3filter: honor comma-separated and wildcard
encoding.contentType by <a
href="https://github.com/utah-KT"><code>@​utah-KT</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1201">getkin/kin-openapi#1201</a></li>
<li>openapi3: typed context wrappers for combinator and tag validation
by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1199">getkin/kin-openapi#1199</a></li>
<li>openapi3: enforce unique required entries and unique tag names in
Validate() by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1203">getkin/kin-openapi#1203</a></li>
<li>openapi3gen: Add an option to customize generated field names
(properties) for structs by <a
href="https://github.com/d1vbyz3r0"><code>@​d1vbyz3r0</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1204">getkin/kin-openapi#1204</a></li>
<li>openapi3: add T.WalkSchemas to visit every schema in a document by
<a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1206">getkin/kin-openapi#1206</a></li>
<li>openapi3: record block end on Origin.Key (EndLine/EndColumn) by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1207">getkin/kin-openapi#1207</a></li>
<li>openapi3: record scalar-valued map keys for origin location lookup
by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1208">getkin/kin-openapi#1208</a></li>
<li>docs: Go Report Card has been sunset by <a
href="https://github.com/alexandear"><code>@​alexandear</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1210">getkin/kin-openapi#1210</a></li>
<li>openapi3: support OAS 3.2 itemSchema by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1213">getkin/kin-openapi#1213</a></li>
<li>openapi3: make operation responses optional for 3.1+ by <a
href="https://github.com/AlexanderWangY"><code>@​AlexanderWangY</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1214">getkin/kin-openapi#1214</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/getkin/kin-openapi/commit/f0407d53b0730280266f454b755010e7eeb985da"><code>f0407d5</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/c9476701743509b5c9ed05858554eacf30672398"><code>c947670</code></a>
some fixes to please my moulinette (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1232">#1232</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/68ac2affa325514d7d6e731204d6a1edf6bdff64"><code>68ac2af</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/3e5d7f4d5a0c335d8085de5c0e552efe61e37e03"><code>3e5d7f4</code></a>
openapi3filter: prefer non-empty value for repeated scalar query params
(<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1231">#1231</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/30e292371a89ca6d1f91a9553e0aa518b49f838d"><code>30e2923</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/c11a4812ba7be3ee3d81453be0c397ee84c238d5"><code>c11a481</code></a>
openapi3gen: inline embedded struct with options-only JSON tag (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1229">#1229</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/0fb1f0e28a8f4e5f802bc8a4c93a0e02277e3fac"><code>0fb1f0e</code></a>
openapi3filter: skip schema checks for empty allowEmptyValue strings (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1228">#1228</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/a872c570b1c6eba1720edbb34cd4a0f53b55f3b8"><code>a872c57</code></a>
openapi3: stable codes for validation errors (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1223">#1223</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/d2f115e64a8302aeb7db45e88f898cbfb7a0cfb1"><code>d2f115e</code></a>
openapi3: preserve origin for a $ref to a schema under an arbitrary
top-level...</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/49523e0e533482a5af1ee14b658c2a9ff63e8450"><code>49523e0</code></a>
openapi3filter: skip Split allocation for single-value array query
params (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1">#1</a>...</li>
<li>Additional commits viewable in <a
href="https://github.com/getkin/kin-openapi/compare/v0.139.0...v0.144.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `github.com/getkin/kin-openapi` from 0.139.0 to 0.144.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/getkin/kin-openapi/releases">github.com/getkin/kin-openapi's
releases</a>.</em></p>
<blockquote>
<h2>v0.144.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3filter: prefer non-empty value for repeated scalar query
params by <a
href="https://github.com/sonnemusk"><code>@​sonnemusk</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1231">getkin/kin-openapi#1231</a></li>
<li>some fixes to please my moulinette by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1232">getkin/kin-openapi#1232</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/sonnemusk"><code>@​sonnemusk</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1231">getkin/kin-openapi#1231</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/getkin/kin-openapi/compare/v0.143.0...v0.144.0">https://github.com/getkin/kin-openapi/compare/v0.143.0...v0.144.0</a></p>
<h2>v0.143.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3: remove StringMap and its pre-OriginTree origin-stripping
machinery by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1219">getkin/kin-openapi#1219</a></li>
<li>openapi3: huge test suite: make it more parallel by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1221">getkin/kin-openapi#1221</a></li>
<li>openapi31: support 3.1+ reference metadata overrides by <a
href="https://github.com/AlexanderWangY"><code>@​AlexanderWangY</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1215">getkin/kin-openapi#1215</a></li>
<li>docs: move the StringMap changelog entry to v0.143.0 by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1222">getkin/kin-openapi#1222</a></li>
<li>openapi3: add T.WalkParameters to visit every parameter in a
document by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1224">getkin/kin-openapi#1224</a></li>
<li>openapi3filter: reuse deepObject bracket regex by <a
href="https://github.com/matiasinsaurralde"><code>@​matiasinsaurralde</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1225">getkin/kin-openapi#1225</a></li>
<li>openapi3filter: skip Split allocation for single-value array query
params by <a
href="https://github.com/matiasinsaurralde"><code>@​matiasinsaurralde</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1226">getkin/kin-openapi#1226</a></li>
<li>openapi3: preserve origin for a $ref to a schema under an arbitrary
top-level key by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1227">getkin/kin-openapi#1227</a></li>
<li>openapi3: stable codes for validation errors by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1223">getkin/kin-openapi#1223</a></li>
<li>openapi3filter: skip schema checks for empty allowEmptyValue strings
by <a
href="https://github.com/snowyukitty"><code>@​snowyukitty</code></a> in
<a
href="https://redirect.github.com/getkin/kin-openapi/pull/1228">getkin/kin-openapi#1228</a></li>
<li>openapi3gen: inline embedded struct with options-only JSON tag by <a
href="https://github.com/mvanhorn"><code>@​mvanhorn</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1229">getkin/kin-openapi#1229</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/matiasinsaurralde"><code>@​matiasinsaurralde</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1225">getkin/kin-openapi#1225</a></li>
<li><a
href="https://github.com/snowyukitty"><code>@​snowyukitty</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1228">getkin/kin-openapi#1228</a></li>
<li><a href="https://github.com/mvanhorn"><code>@​mvanhorn</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1229">getkin/kin-openapi#1229</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/getkin/kin-openapi/compare/v0.142.0...v0.143.0">https://github.com/getkin/kin-openapi/compare/v0.142.0...v0.143.0</a></p>
<h2>v0.142.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3: fix test that relied on now non-resolving domain by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1220">getkin/kin-openapi#1220</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/getkin/kin-openapi/compare/v0.141.0...v0.142.0">https://github.com/getkin/kin-openapi/compare/v0.141.0...v0.142.0</a></p>
<h2>v0.141.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3filter: honor comma-separated and wildcard
encoding.contentType by <a
href="https://github.com/utah-KT"><code>@​utah-KT</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1201">getkin/kin-openapi#1201</a></li>
<li>openapi3: typed context wrappers for combinator and tag validation
by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1199">getkin/kin-openapi#1199</a></li>
<li>openapi3: enforce unique required entries and unique tag names in
Validate() by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1203">getkin/kin-openapi#1203</a></li>
<li>openapi3gen: Add an option to customize generated field names
(properties) for structs by <a
href="https://github.com/d1vbyz3r0"><code>@​d1vbyz3r0</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1204">getkin/kin-openapi#1204</a></li>
<li>openapi3: add T.WalkSchemas to visit every schema in a document by
<a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1206">getkin/kin-openapi#1206</a></li>
<li>openapi3: record block end on Origin.Key (EndLine/EndColumn) by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1207">getkin/kin-openapi#1207</a></li>
<li>openapi3: record scalar-valued map keys for origin location lookup
by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1208">getkin/kin-openapi#1208</a></li>
<li>docs: Go Report Card has been sunset by <a
href="https://github.com/alexandear"><code>@​alexandear</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1210">getkin/kin-openapi#1210</a></li>
<li>openapi3: support OAS 3.2 itemSchema by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1213">getkin/kin-openapi#1213</a></li>
<li>openapi3: make operation responses optional for 3.1+ by <a
href="https://github.com/AlexanderWangY"><code>@​AlexanderWangY</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1214">getkin/kin-openapi#1214</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/getkin/kin-openapi/commit/f0407d53b0730280266f454b755010e7eeb985da"><code>f0407d5</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/c9476701743509b5c9ed05858554eacf30672398"><code>c947670</code></a>
some fixes to please my moulinette (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1232">#1232</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/68ac2affa325514d7d6e731204d6a1edf6bdff64"><code>68ac2af</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/3e5d7f4d5a0c335d8085de5c0e552efe61e37e03"><code>3e5d7f4</code></a>
openapi3filter: prefer non-empty value for repeated scalar query params
(<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1231">#1231</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/30e292371a89ca6d1f91a9553e0aa518b49f838d"><code>30e2923</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/c11a4812ba7be3ee3d81453be0c397ee84c238d5"><code>c11a481</code></a>
openapi3gen: inline embedded struct with options-only JSON tag (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1229">#1229</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/0fb1f0e28a8f4e5f802bc8a4c93a0e02277e3fac"><code>0fb1f0e</code></a>
openapi3filter: skip schema checks for empty allowEmptyValue strings (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1228">#1228</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/a872c570b1c6eba1720edbb34cd4a0f53b55f3b8"><code>a872c57</code></a>
openapi3: stable codes for validation errors (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1223">#1223</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/d2f115e64a8302aeb7db45e88f898cbfb7a0cfb1"><code>d2f115e</code></a>
openapi3: preserve origin for a $ref to a schema under an arbitrary
top-level...</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/49523e0e533482a5af1ee14b658c2a9ff63e8450"><code>49523e0</code></a>
openapi3filter: skip Split allocation for single-value array query
params (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1">#1</a>...</li>
<li>Additional commits viewable in <a
href="https://github.com/getkin/kin-openapi/compare/v0.139.0...v0.144.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `github.com/getkin/kin-openapi` from 0.139.0 to 0.144.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/getkin/kin-openapi/releases">github.com/getkin/kin-openapi's
releases</a>.</em></p>
<blockquote>
<h2>v0.144.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3filter: prefer non-empty value for repeated scalar query
params by <a
href="https://github.com/sonnemusk"><code>@​sonnemusk</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1231">getkin/kin-openapi#1231</a></li>
<li>some fixes to please my moulinette by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1232">getkin/kin-openapi#1232</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/sonnemusk"><code>@​sonnemusk</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1231">getkin/kin-openapi#1231</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/getkin/kin-openapi/compare/v0.143.0...v0.144.0">https://github.com/getkin/kin-openapi/compare/v0.143.0...v0.144.0</a></p>
<h2>v0.143.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3: remove StringMap and its pre-OriginTree origin-stripping
machinery by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1219">getkin/kin-openapi#1219</a></li>
<li>openapi3: huge test suite: make it more parallel by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1221">getkin/kin-openapi#1221</a></li>
<li>openapi31: support 3.1+ reference metadata overrides by <a
href="https://github.com/AlexanderWangY"><code>@​AlexanderWangY</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1215">getkin/kin-openapi#1215</a></li>
<li>docs: move the StringMap changelog entry to v0.143.0 by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1222">getkin/kin-openapi#1222</a></li>
<li>openapi3: add T.WalkParameters to visit every parameter in a
document by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1224">getkin/kin-openapi#1224</a></li>
<li>openapi3filter: reuse deepObject bracket regex by <a
href="https://github.com/matiasinsaurralde"><code>@​matiasinsaurralde</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1225">getkin/kin-openapi#1225</a></li>
<li>openapi3filter: skip Split allocation for single-value array query
params by <a
href="https://github.com/matiasinsaurralde"><code>@​matiasinsaurralde</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1226">getkin/kin-openapi#1226</a></li>
<li>openapi3: preserve origin for a $ref to a schema under an arbitrary
top-level key by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1227">getkin/kin-openapi#1227</a></li>
<li>openapi3: stable codes for validation errors by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1223">getkin/kin-openapi#1223</a></li>
<li>openapi3filter: skip schema checks for empty allowEmptyValue strings
by <a
href="https://github.com/snowyukitty"><code>@​snowyukitty</code></a> in
<a
href="https://redirect.github.com/getkin/kin-openapi/pull/1228">getkin/kin-openapi#1228</a></li>
<li>openapi3gen: inline embedded struct with options-only JSON tag by <a
href="https://github.com/mvanhorn"><code>@​mvanhorn</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1229">getkin/kin-openapi#1229</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/matiasinsaurralde"><code>@​matiasinsaurralde</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1225">getkin/kin-openapi#1225</a></li>
<li><a
href="https://github.com/snowyukitty"><code>@​snowyukitty</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1228">getkin/kin-openapi#1228</a></li>
<li><a href="https://github.com/mvanhorn"><code>@​mvanhorn</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1229">getkin/kin-openapi#1229</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/getkin/kin-openapi/compare/v0.142.0...v0.143.0">https://github.com/getkin/kin-openapi/compare/v0.142.0...v0.143.0</a></p>
<h2>v0.142.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3: fix test that relied on now non-resolving domain by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1220">getkin/kin-openapi#1220</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/getkin/kin-openapi/compare/v0.141.0...v0.142.0">https://github.com/getkin/kin-openapi/compare/v0.141.0...v0.142.0</a></p>
<h2>v0.141.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3filter: honor comma-separated and wildcard
encoding.contentType by <a
href="https://github.com/utah-KT"><code>@​utah-KT</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1201">getkin/kin-openapi#1201</a></li>
<li>openapi3: typed context wrappers for combinator and tag validation
by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1199">getkin/kin-openapi#1199</a></li>
<li>openapi3: enforce unique required entries and unique tag names in
Validate() by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1203">getkin/kin-openapi#1203</a></li>
<li>openapi3gen: Add an option to customize generated field names
(properties) for structs by <a
href="https://github.com/d1vbyz3r0"><code>@​d1vbyz3r0</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1204">getkin/kin-openapi#1204</a></li>
<li>openapi3: add T.WalkSchemas to visit every schema in a document by
<a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1206">getkin/kin-openapi#1206</a></li>
<li>openapi3: record block end on Origin.Key (EndLine/EndColumn) by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1207">getkin/kin-openapi#1207</a></li>
<li>openapi3: record scalar-valued map keys for origin location lookup
by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1208">getkin/kin-openapi#1208</a></li>
<li>docs: Go Report Card has been sunset by <a
href="https://github.com/alexandear"><code>@​alexandear</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1210">getkin/kin-openapi#1210</a></li>
<li>openapi3: support OAS 3.2 itemSchema by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1213">getkin/kin-openapi#1213</a></li>
<li>openapi3: make operation responses optional for 3.1+ by <a
href="https://github.com/AlexanderWangY"><code>@​AlexanderWangY</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1214">getkin/kin-openapi#1214</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/getkin/kin-openapi/commit/f0407d53b0730280266f454b755010e7eeb985da"><code>f0407d5</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/c9476701743509b5c9ed05858554eacf30672398"><code>c947670</code></a>
some fixes to please my moulinette (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1232">#1232</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/68ac2affa325514d7d6e731204d6a1edf6bdff64"><code>68ac2af</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/3e5d7f4d5a0c335d8085de5c0e552efe61e37e03"><code>3e5d7f4</code></a>
openapi3filter: prefer non-empty value for repeated scalar query params
(<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1231">#1231</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/30e292371a89ca6d1f91a9553e0aa518b49f838d"><code>30e2923</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/c11a4812ba7be3ee3d81453be0c397ee84c238d5"><code>c11a481</code></a>
openapi3gen: inline embedded struct with options-only JSON tag (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1229">#1229</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/0fb1f0e28a8f4e5f802bc8a4c93a0e02277e3fac"><code>0fb1f0e</code></a>
openapi3filter: skip schema checks for empty allowEmptyValue strings (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1228">#1228</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/a872c570b1c6eba1720edbb34cd4a0f53b55f3b8"><code>a872c57</code></a>
openapi3: stable codes for validation errors (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1223">#1223</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/d2f115e64a8302aeb7db45e88f898cbfb7a0cfb1"><code>d2f115e</code></a>
openapi3: preserve origin for a $ref to a schema under an arbitrary
top-level...</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/49523e0e533482a5af1ee14b658c2a9ff63e8450"><code>49523e0</code></a>
openapi3filter: skip Split allocation for single-value array query
params (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1">#1</a>...</li>
<li>Additional commits viewable in <a
href="https://github.com/getkin/kin-openapi/compare/v0.139.0...v0.144.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `github.com/google/cel-go` from 0.25.0 to 0.29.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/google/cel-go/releases">github.com/google/cel-go's
releases</a>.</em></p>
<blockquote>
<h2>Release v0.29.0</h2>
<h2>What's Changed</h2>
<h3>New Features</h3>
<ul>
<li><strong>Add JSON encoder extension</strong> — a new <code>ext</code>
library for JSON encoding by <a
href="https://github.com/rohan-patnaik"><code>@​rohan-patnaik</code></a>
in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1340">cel-expr/cel-go#1340</a></li>
<li><strong>Upstream <code>network.IP</code>/<code>CIDR</code>
support</strong> from Kubernetes by <a
href="https://github.com/tdesrosi"><code>@​tdesrosi</code></a> in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1238">cel-expr/cel-go#1238</a></li>
<li><strong>Managed execution frame with async call foundations</strong>
by <a
href="https://github.com/TristonianJones"><code>@​TristonianJones</code></a>
in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1316">cel-expr/cel-go#1316</a></li>
<li><strong>Execution frame integration with updated
<code>InterpretableV2</code></strong> by <a
href="https://github.com/TristonianJones"><code>@​TristonianJones</code></a>
in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1344">cel-expr/cel-go#1344</a></li>
<li><strong>Add conformance test runner for CEL Policies</strong> by <a
href="https://github.com/l46kok"><code>@​l46kok</code></a> in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1319">cel-expr/cel-go#1319</a></li>
</ul>
<h3>Bug Fixes</h3>
<ul>
<li><strong>Fix unintentional unwrapping of optionals</strong> at any
level of depth by <a
href="https://github.com/TristonianJones"><code>@​TristonianJones</code></a>
in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1315">cel-expr/cel-go#1315</a></li>
<li><strong>Fix <code>has()</code> unknown propagation</strong> during
partial evaluation by <a
href="https://github.com/0xASTRA"><code>@​0xASTRA</code></a> in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1324">cel-expr/cel-go#1324</a></li>
<li><strong>Enforce expression size limit before source
construction</strong> by <a
href="https://github.com/thesmartshadow"><code>@​thesmartshadow</code></a>
in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1302">cel-expr/cel-go#1302</a></li>
<li><strong>Policy variable scoping fixes</strong> by <a
href="https://github.com/jnthntatum"><code>@​jnthntatum</code></a> in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1321">cel-expr/cel-go#1321</a></li>
<li><strong>Emit valid CEL for doubles in scientific notation</strong>
in the parser/unparser by <a
href="https://github.com/moraneus"><code>@​moraneus</code></a> in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1326">cel-expr/cel-go#1326</a></li>
<li><strong>Allow aliases for simple names</strong> in containers by <a
href="https://github.com/rohan-patnaik"><code>@​rohan-patnaik</code></a>
in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1327">cel-expr/cel-go#1327</a></li>
<li><strong>Reject out-of-range minutes</strong> in timezone offset
parsing by <a href="https://github.com/alhudz"><code>@​alhudz</code></a>
in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1336">cel-expr/cel-go#1336</a></li>
<li><strong>Validate offset on empty-string path</strong> in
<code>indexOf</code> and <code>lastIndexOf</code> by <a
href="https://github.com/alhudz"><code>@​alhudz</code></a> in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1335">cel-expr/cel-go#1335</a></li>
<li><strong>Guard <code>int32</code>/<code>uint32</code> map key
narrowing</strong> in <code>qualifyInternal</code> by <a
href="https://github.com/alhudz"><code>@​alhudz</code></a> in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1337">cel-expr/cel-go#1337</a></li>
<li><strong>Add max size check to <code>genRange()</code></strong> in
<code>ext/lists</code> to prevent OOM by <a
href="https://github.com/marwan9696"><code>@​marwan9696</code></a> in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1310">cel-expr/cel-go#1310</a></li>
</ul>
<h3>Cost Tracking</h3>
<ul>
<li><strong>Ensure receiver and global matches cost estimates
agree</strong> by <a
href="https://github.com/TristonianJones"><code>@​TristonianJones</code></a>
in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1350">cel-expr/cel-go#1350</a></li>
<li><strong><code>startsWith</code>/<code>endsWith</code> runtime cost
agreement</strong> with checked cost by <a
href="https://github.com/TristonianJones"><code>@​TristonianJones</code></a>
in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1351">cel-expr/cel-go#1351</a></li>
<li><strong>Avoid repeated construction of cost tracker</strong> by <a
href="https://github.com/pohly"><code>@​pohly</code></a> in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1357">cel-expr/cel-go#1357</a></li>
</ul>
<h3>Testing &amp; Tooling</h3>
<ul>
<li><strong>Test Runner usability improvements</strong> by <a
href="https://github.com/jnthntatum"><code>@​jnthntatum</code></a> in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1320">cel-expr/cel-go#1320</a></li>
<li><strong>Minor updates to test coverage and code quality</strong> by
<a
href="https://github.com/TristonianJones"><code>@​TristonianJones</code></a>
in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1318">cel-expr/cel-go#1318</a></li>
<li><strong>Remove duplicate policy tests</strong> now covered by policy
conformance tests by <a
href="https://github.com/l46kok"><code>@​l46kok</code></a> in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1332">cel-expr/cel-go#1332</a></li>
<li><strong>Minor Google3 import fixes</strong> by <a
href="https://github.com/l46kok"><code>@​l46kok</code></a> in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1331">cel-expr/cel-go#1331</a></li>
</ul>
<h3>Documentation</h3>
<ul>
<li><strong>Update documentation for nested first-match rules</strong>
by <a href="https://github.com/jnthntatum"><code>@​jnthntatum</code></a>
in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1323">cel-expr/cel-go#1323</a></li>
<li><strong>Add migration warning to README</strong> by <a
href="https://github.com/dmitriplotnikov"><code>@​dmitriplotnikov</code></a>
in <a
href="https://redirect.github.com/cel-expr/cel-go/pull/1330">cel-expr/cel-go#1330</a></li>
</ul>
<h2>Release v0.28.1</h2>
<h2>What's Changed</h2>
<ul>
<li>fix: deleted <code>IntToDuration</code> overload by <a
href="https://github.com/alexsnaps"><code>@​alexsnaps</code></a> in <a
href="https://redirect.github.com/google/cel-go/pull/1300">google/cel-go#1300</a></li>
<li>Support shorthand types in env yaml and REPL by <a
href="https://github.com/jnthntatum"><code>@​jnthntatum</code></a> in <a
href="https://redirect.github.com/google/cel-go/pull/1301">google/cel-go#1301</a></li>
<li>Preserve operation interrupted in ContextEval error by <a
href="https://github.com/dims"><code>@​dims</code></a> in <a
href="https://redirect.github.com/google/cel-go/pull/1303">google/cel-go#1303</a></li>
<li>Use shorthand type specifiers in env yaml files by <a
href="https://github.com/jnthntatum"><code>@​jnthntatum</code></a> in <a
href="https://redirect.github.com/google/cel-go/pull/1305">google/cel-go#1305</a></li>
<li>Cost estimation and tracking for strings extension by <a
href="https://github.com/TristonianJones"><code>@​TristonianJones</code></a>
in <a
href="https://redirect.github.com/google/cel-go/pull/1307">google/cel-go#1307</a></li>
<li>Update repl type string for doubles by <a
href="https://github.com/TristonianJones"><code>@​TristonianJones</code></a>
in <a
href="https://redirect.github.com/google/cel-go/pull/1308">google/cel-go#1308</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/alexsnaps"><code>@​alexsnaps</code></a>
made their first contribution in <a
href="https://redirect.github.com/google/cel-go/pull/1300">google/cel-go#1300</a></li>
<li><a href="https://github.com/dims"><code>@​dims</code></a> made their
first contribution in <a
href="https://redirect.github.com/google/cel-go/pull/1303">google/cel-go#1303</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/cel-expr/cel-go/commit/fa16799e423d709cfb1a20c10c1db7dc6e640e55"><code>fa16799</code></a>
avoid repeated construction of cost tracker (<a
href="https://redirect.github.com/google/cel-go/issues/1357">#1357</a>)</li>
<li><a
href="https://github.com/cel-expr/cel-go/commit/ea3d5c059bf6a1739d4a2ff30f925a8adffdb4ea"><code>ea3d5c0</code></a>
feat(ext): add json encoder (<a
href="https://redirect.github.com/google/cel-go/issues/1340">#1340</a>)</li>
<li><a
href="https://github.com/cel-expr/cel-go/commit/a4d0d643deeea6408654de2ec9944895d23f36a5"><code>a4d0d64</code></a>
startsWith / endsWith runtime cost agreement with checked cost (<a
href="https://redirect.github.com/google/cel-go/issues/1351">#1351</a>)</li>
<li><a
href="https://github.com/cel-expr/cel-go/commit/d4efa7716addf9d7f0e080311359b9711c6f1222"><code>d4efa77</code></a>
Ensure receiver and global matches cost estimates agree (<a
href="https://redirect.github.com/google/cel-go/issues/1350">#1350</a>)</li>
<li><a
href="https://github.com/cel-expr/cel-go/commit/13cff33f367085ae4ce20ddd472cbc41bd477682"><code>13cff33</code></a>
ext/lists: add max size check to genRange() to prevent OOM (<a
href="https://redirect.github.com/google/cel-go/issues/1310">#1310</a>)</li>
<li><a
href="https://github.com/cel-expr/cel-go/commit/f0ffa7e32eef67fad74d72b1fe0f422f0c10d65c"><code>f0ffa7e</code></a>
Execution frame integration with updated IntepretableV2 (<a
href="https://redirect.github.com/google/cel-go/issues/1344">#1344</a>)</li>
<li><a
href="https://github.com/cel-expr/cel-go/commit/f1ec2f6fe571afa8a226933c1bc51078b4f2d985"><code>f1ec2f6</code></a>
guard int32/uint32 map key narrowing in qualifyInternal (<a
href="https://redirect.github.com/google/cel-go/issues/1337">#1337</a>)</li>
<li><a
href="https://github.com/cel-expr/cel-go/commit/258e7c89f4c3facbc8be8f7c6bc3f932684283c3"><code>258e7c8</code></a>
Managed execution frame with async call foundations (<a
href="https://redirect.github.com/google/cel-go/issues/1316">#1316</a>)</li>
<li><a
href="https://github.com/cel-expr/cel-go/commit/14f6746a286f198c2a976749331b7bf4aa2d4bd2"><code>14f6746</code></a>
validate offset on empty-string path in indexOf and lastIndexOf (<a
href="https://redirect.github.com/google/cel-go/issues/1335">#1335</a>)</li>
<li><a
href="https://github.com/cel-expr/cel-go/commit/783267d0a5f2f94396e4ec90a109127bf9754591"><code>783267d</code></a>
reject out-of-range minutes in timezone offset parsing (<a
href="https://redirect.github.com/google/cel-go/issues/1336">#1336</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/google/cel-go/compare/v0.25.0...v0.29.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `github.com/getkin/kin-openapi` from 0.139.0 to 0.144.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/getkin/kin-openapi/releases">github.com/getkin/kin-openapi's
releases</a>.</em></p>
<blockquote>
<h2>v0.144.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3filter: prefer non-empty value for repeated scalar query
params by <a
href="https://github.com/sonnemusk"><code>@​sonnemusk</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1231">getkin/kin-openapi#1231</a></li>
<li>some fixes to please my moulinette by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1232">getkin/kin-openapi#1232</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/sonnemusk"><code>@​sonnemusk</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1231">getkin/kin-openapi#1231</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/getkin/kin-openapi/compare/v0.143.0...v0.144.0">https://github.com/getkin/kin-openapi/compare/v0.143.0...v0.144.0</a></p>
<h2>v0.143.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3: remove StringMap and its pre-OriginTree origin-stripping
machinery by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1219">getkin/kin-openapi#1219</a></li>
<li>openapi3: huge test suite: make it more parallel by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1221">getkin/kin-openapi#1221</a></li>
<li>openapi31: support 3.1+ reference metadata overrides by <a
href="https://github.com/AlexanderWangY"><code>@​AlexanderWangY</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1215">getkin/kin-openapi#1215</a></li>
<li>docs: move the StringMap changelog entry to v0.143.0 by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1222">getkin/kin-openapi#1222</a></li>
<li>openapi3: add T.WalkParameters to visit every parameter in a
document by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1224">getkin/kin-openapi#1224</a></li>
<li>openapi3filter: reuse deepObject bracket regex by <a
href="https://github.com/matiasinsaurralde"><code>@​matiasinsaurralde</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1225">getkin/kin-openapi#1225</a></li>
<li>openapi3filter: skip Split allocation for single-value array query
params by <a
href="https://github.com/matiasinsaurralde"><code>@​matiasinsaurralde</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1226">getkin/kin-openapi#1226</a></li>
<li>openapi3: preserve origin for a $ref to a schema under an arbitrary
top-level key by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1227">getkin/kin-openapi#1227</a></li>
<li>openapi3: stable codes for validation errors by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1223">getkin/kin-openapi#1223</a></li>
<li>openapi3filter: skip schema checks for empty allowEmptyValue strings
by <a
href="https://github.com/snowyukitty"><code>@​snowyukitty</code></a> in
<a
href="https://redirect.github.com/getkin/kin-openapi/pull/1228">getkin/kin-openapi#1228</a></li>
<li>openapi3gen: inline embedded struct with options-only JSON tag by <a
href="https://github.com/mvanhorn"><code>@​mvanhorn</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1229">getkin/kin-openapi#1229</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/matiasinsaurralde"><code>@​matiasinsaurralde</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1225">getkin/kin-openapi#1225</a></li>
<li><a
href="https://github.com/snowyukitty"><code>@​snowyukitty</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1228">getkin/kin-openapi#1228</a></li>
<li><a href="https://github.com/mvanhorn"><code>@​mvanhorn</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1229">getkin/kin-openapi#1229</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/getkin/kin-openapi/compare/v0.142.0...v0.143.0">https://github.com/getkin/kin-openapi/compare/v0.142.0...v0.143.0</a></p>
<h2>v0.142.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3: fix test that relied on now non-resolving domain by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1220">getkin/kin-openapi#1220</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/getkin/kin-openapi/compare/v0.141.0...v0.142.0">https://github.com/getkin/kin-openapi/compare/v0.141.0...v0.142.0</a></p>
<h2>v0.141.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3filter: honor comma-separated and wildcard
encoding.contentType by <a
href="https://github.com/utah-KT"><code>@​utah-KT</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1201">getkin/kin-openapi#1201</a></li>
<li>openapi3: typed context wrappers for combinator and tag validation
by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1199">getkin/kin-openapi#1199</a></li>
<li>openapi3: enforce unique required entries and unique tag names in
Validate() by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1203">getkin/kin-openapi#1203</a></li>
<li>openapi3gen: Add an option to customize generated field names
(properties) for structs by <a
href="https://github.com/d1vbyz3r0"><code>@​d1vbyz3r0</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1204">getkin/kin-openapi#1204</a></li>
<li>openapi3: add T.WalkSchemas to visit every schema in a document by
<a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1206">getkin/kin-openapi#1206</a></li>
<li>openapi3: record block end on Origin.Key (EndLine/EndColumn) by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1207">getkin/kin-openapi#1207</a></li>
<li>openapi3: record scalar-valued map keys for origin location lookup
by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1208">getkin/kin-openapi#1208</a></li>
<li>docs: Go Report Card has been sunset by <a
href="https://github.com/alexandear"><code>@​alexandear</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1210">getkin/kin-openapi#1210</a></li>
<li>openapi3: support OAS 3.2 itemSchema by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1213">getkin/kin-openapi#1213</a></li>
<li>openapi3: make operation responses optional for 3.1+ by <a
href="https://github.com/AlexanderWangY"><code>@​AlexanderWangY</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1214">getkin/kin-openapi#1214</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/getkin/kin-openapi/commit/f0407d53b0730280266f454b755010e7eeb985da"><code>f0407d5</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/c9476701743509b5c9ed05858554eacf30672398"><code>c947670</code></a>
some fixes to please my moulinette (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1232">#1232</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/68ac2affa325514d7d6e731204d6a1edf6bdff64"><code>68ac2af</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/3e5d7f4d5a0c335d8085de5c0e552efe61e37e03"><code>3e5d7f4</code></a>
openapi3filter: prefer non-empty value for repeated scalar query params
(<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1231">#1231</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/30e292371a89ca6d1f91a9553e0aa518b49f838d"><code>30e2923</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/c11a4812ba7be3ee3d81453be0c397ee84c238d5"><code>c11a481</code></a>
openapi3gen: inline embedded struct with options-only JSON tag (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1229">#1229</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/0fb1f0e28a8f4e5f802bc8a4c93a0e02277e3fac"><code>0fb1f0e</code></a>
openapi3filter: skip schema checks for empty allowEmptyValue strings (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1228">#1228</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/a872c570b1c6eba1720edbb34cd4a0f53b55f3b8"><code>a872c57</code></a>
openapi3: stable codes for validation errors (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1223">#1223</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/d2f115e64a8302aeb7db45e88f898cbfb7a0cfb1"><code>d2f115e</code></a>
openapi3: preserve origin for a $ref to a schema under an arbitrary
top-level...</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/49523e0e533482a5af1ee14b658c2a9ff63e8450"><code>49523e0</code></a>
openapi3filter: skip Split allocation for single-value array query
params (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1">#1</a>...</li>
<li>Additional commits viewable in <a
href="https://github.com/getkin/kin-openapi/compare/v0.139.0...v0.144.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `github.com/getkin/kin-openapi` from 0.137.0 to 0.144.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/getkin/kin-openapi/releases">github.com/getkin/kin-openapi's
releases</a>.</em></p>
<blockquote>
<h2>v0.144.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3filter: prefer non-empty value for repeated scalar query
params by <a
href="https://github.com/sonnemusk"><code>@​sonnemusk</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1231">getkin/kin-openapi#1231</a></li>
<li>some fixes to please my moulinette by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1232">getkin/kin-openapi#1232</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/sonnemusk"><code>@​sonnemusk</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1231">getkin/kin-openapi#1231</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/getkin/kin-openapi/compare/v0.143.0...v0.144.0">https://github.com/getkin/kin-openapi/compare/v0.143.0...v0.144.0</a></p>
<h2>v0.143.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3: remove StringMap and its pre-OriginTree origin-stripping
machinery by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1219">getkin/kin-openapi#1219</a></li>
<li>openapi3: huge test suite: make it more parallel by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1221">getkin/kin-openapi#1221</a></li>
<li>openapi31: support 3.1+ reference metadata overrides by <a
href="https://github.com/AlexanderWangY"><code>@​AlexanderWangY</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1215">getkin/kin-openapi#1215</a></li>
<li>docs: move the StringMap changelog entry to v0.143.0 by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1222">getkin/kin-openapi#1222</a></li>
<li>openapi3: add T.WalkParameters to visit every parameter in a
document by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1224">getkin/kin-openapi#1224</a></li>
<li>openapi3filter: reuse deepObject bracket regex by <a
href="https://github.com/matiasinsaurralde"><code>@​matiasinsaurralde</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1225">getkin/kin-openapi#1225</a></li>
<li>openapi3filter: skip Split allocation for single-value array query
params by <a
href="https://github.com/matiasinsaurralde"><code>@​matiasinsaurralde</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1226">getkin/kin-openapi#1226</a></li>
<li>openapi3: preserve origin for a $ref to a schema under an arbitrary
top-level key by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1227">getkin/kin-openapi#1227</a></li>
<li>openapi3: stable codes for validation errors by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1223">getkin/kin-openapi#1223</a></li>
<li>openapi3filter: skip schema checks for empty allowEmptyValue strings
by <a
href="https://github.com/snowyukitty"><code>@​snowyukitty</code></a> in
<a
href="https://redirect.github.com/getkin/kin-openapi/pull/1228">getkin/kin-openapi#1228</a></li>
<li>openapi3gen: inline embedded struct with options-only JSON tag by <a
href="https://github.com/mvanhorn"><code>@​mvanhorn</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1229">getkin/kin-openapi#1229</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/matiasinsaurralde"><code>@​matiasinsaurralde</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1225">getkin/kin-openapi#1225</a></li>
<li><a
href="https://github.com/snowyukitty"><code>@​snowyukitty</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1228">getkin/kin-openapi#1228</a></li>
<li><a href="https://github.com/mvanhorn"><code>@​mvanhorn</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1229">getkin/kin-openapi#1229</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/getkin/kin-openapi/compare/v0.142.0...v0.143.0">https://github.com/getkin/kin-openapi/compare/v0.142.0...v0.143.0</a></p>
<h2>v0.142.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3: fix test that relied on now non-resolving domain by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1220">getkin/kin-openapi#1220</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/getkin/kin-openapi/compare/v0.141.0...v0.142.0">https://github.com/getkin/kin-openapi/compare/v0.141.0...v0.142.0</a></p>
<h2>v0.141.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3filter: honor comma-separated and wildcard
encoding.contentType by <a
href="https://github.com/utah-KT"><code>@​utah-KT</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1201">getkin/kin-openapi#1201</a></li>
<li>openapi3: typed context wrappers for combinator and tag validation
by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1199">getkin/kin-openapi#1199</a></li>
<li>openapi3: enforce unique required entries and unique tag names in
Validate() by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1203">getkin/kin-openapi#1203</a></li>
<li>openapi3gen: Add an option to customize generated field names
(properties) for structs by <a
href="https://github.com/d1vbyz3r0"><code>@​d1vbyz3r0</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1204">getkin/kin-openapi#1204</a></li>
<li>openapi3: add T.WalkSchemas to visit every schema in a document by
<a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1206">getkin/kin-openapi#1206</a></li>
<li>openapi3: record block end on Origin.Key (EndLine/EndColumn) by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1207">getkin/kin-openapi#1207</a></li>
<li>openapi3: record scalar-valued map keys for origin location lookup
by <a
href="https://github.com/reuvenharrison"><code>@​reuvenharrison</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1208">getkin/kin-openapi#1208</a></li>
<li>docs: Go Report Card has been sunset by <a
href="https://github.com/alexandear"><code>@​alexandear</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1210">getkin/kin-openapi#1210</a></li>
<li>openapi3: support OAS 3.2 itemSchema by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1213">getkin/kin-openapi#1213</a></li>
<li>openapi3: make operation responses optional for 3.1+ by <a
href="https://github.com/AlexanderWangY"><code>@​AlexanderWangY</code></a>
in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1214">getkin/kin-openapi#1214</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/getkin/kin-openapi/commit/f0407d53b0730280266f454b755010e7eeb985da"><code>f0407d5</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/c9476701743509b5c9ed05858554eacf30672398"><code>c947670</code></a>
some fixes to please my moulinette (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1232">#1232</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/68ac2affa325514d7d6e731204d6a1edf6bdff64"><code>68ac2af</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/3e5d7f4d5a0c335d8085de5c0e552efe61e37e03"><code>3e5d7f4</code></a>
openapi3filter: prefer non-empty value for repeated scalar query params
(<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1231">#1231</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/30e292371a89ca6d1f91a9553e0aa518b49f838d"><code>30e2923</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/c11a4812ba7be3ee3d81453be0c397ee84c238d5"><code>c11a481</code></a>
openapi3gen: inline embedded struct with options-only JSON tag (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1229">#1229</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/0fb1f0e28a8f4e5f802bc8a4c93a0e02277e3fac"><code>0fb1f0e</code></a>
openapi3filter: skip schema checks for empty allowEmptyValue strings (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1228">#1228</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/a872c570b1c6eba1720edbb34cd4a0f53b55f3b8"><code>a872c57</code></a>
openapi3: stable codes for validation errors (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1223">#1223</a>)</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/d2f115e64a8302aeb7db45e88f898cbfb7a0cfb1"><code>d2f115e</code></a>
openapi3: preserve origin for a $ref to a schema under an arbitrary
top-level...</li>
<li><a
href="https://github.com/getkin/kin-openapi/commit/49523e0e533482a5af1ee14b658c2a9ff63e8450"><code>49523e0</code></a>
openapi3filter: skip Split allocation for single-value array query
params (<a
href="https://redirect.github.com/getkin/kin-openapi/issues/1">#1</a>...</li>
<li>Additional commits viewable in <a
href="https://github.com/getkin/kin-openapi/compare/v0.139.0...v0.144.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `github.com/getkin/kin-openapi` from 0.139.0 to 0.144.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/getkin/kin-openapi/releases">github.com/getkin/kin-openapi's
releases</a>.</em></p>
<blockquote>
<h2>v0.144.0</h2>
<h2>What's Changed</h2>
<ul>
<li>openapi3filter: prefer non-empty value for repeated scalar query
params by <a
href="https://github.com/sonnemusk"><code>@​sonnemusk</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1231">getkin/kin-openapi#1231</a></li>
<li>some fixes to please my moulinette by <a
href="https://github.com/fenollp"><code>@​fenollp</code></a> in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1232">getkin/kin-openapi#1232</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/sonnemusk"><code>@​sonnemusk</code></a>
made their first contribution in <a
href="https://redirect.github.com/getkin/kin-openapi/pull/1231">getkin/kin-openapi#1231</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/getkin/kin-…
…_type (#3427)

Completes the `/v1/management` contract against its one caller, `belt`'s
`workspace-api`. Every operation it invokes now has a definition here.
Two did not line up.

## 1. The missing route

`POST /v1/management/projects/{teamID}/members/batch` — the bulk path
behind group and directory fan-outs, where the per-member routes cost
one request each.

The caller already sends this. Its client is hand-written precisely
because there was nothing here to generate from, so the schema is taken
from the bytes it puts on the wire rather than designed fresh:

```json
[{"user_id": "…", "present": true}, {"user_id": "…", "present": false}]
```

Presence is **stated** rather than implied by inclusion, so one request
carries both additions and removals and entries converge in any order.
`maxItems: 1024` matches the chunk size the caller already uses. Stubbed
`501` like its five siblings — serving it is separate work.

## 2. The mismatch that was breaking provisioning

`AdminControlPlaneProjectType` enumerated `development | staging |
production`, from the scaffolding in #3314 — written before any caller
existed.

The caller sends tier names (`base_v1`, `pro_v1`) and gates on the
generated `Valid()` *before opening a connection*:

```go
projectType := managementapi.AdminControlPlaneProjectType(project.ProjectType)
if !projectType.Valid() {
    return fmt.Errorf("%q: %w", project.ProjectType, ErrInvalidProjectType)
}
```

So **every project upsert failed client-side and no request ever arrived
here.** Signup's default type is `base_v1`, so this was every project.

The enum is now gone. Nothing on this side reads the value — there is no
column for it, and limits arrive in full and explicitly through
`upsertProjectLimits`. Enumerating a field we only record would make
adding a tier a cross-repo change to a value we never interpret, so the
caller keeps its vocabulary and this keeps a `string` with `minLength:
1`.

## Tests

The batch caller is hand-written, so nothing but a test keeps the two
sides in step:

- The batch request decodes from the literal JSON that caller emits.
- An upsert decodes with real tier names, which fails if someone
reintroduces a closed set guessing at the caller's vocabulary.
- `/members/batch` is not swallowed by the `/members/{userId}` parameter
beside it — driven through the real router, asserting which handler ran.

The first is mutation-verified: renaming the property in the spec breaks
the build.

## Coordination

**This needs a companion `belt` change before the spec syncs.**
`Valid()` does not exist on a plain string, which is one compile error
at `controlplanes/management.go:76`. Verified by regenerating belt's
client against this spec:

```
internal/controlplanes/management.go:76:18: projectType.Valid undefined
```

Loud and single-line, which is the right failure mode — but it should
land alongside.

## Verification

Builds, vets, tests and `golangci-lint` clean in `dashboard-api`.
Contract diff confirms 1:1 coverage: seven caller operations, seven
definitions, no orphans either direction.

No `ARCHITECTURE.md` change — it describes the `/v1/management` surface
and its auth without enumerating operations, and neither is altered.
Follow-up to #3425. Adds per-step snapshot latency histograms so a pause
can be decomposed into where the time actually goes.

## Change
- **`orchestrator.sandbox.snapshot.process_rootfs.duration`**
{`fs_only`, `success`} — rootfs export+diff, recorded in
`pauseProcessRootfs` (runs for both pause kinds;
`pauseOpts.filesystemSnapshot` is threaded in for the label).
- **`orchestrator.sandbox.snapshot.process_memory.duration`**
{`success`} — memory export+diff, recorded in `pauseProcessMemory`.
Memory pauses only (fs-only skips memory processing), so no `fs_only`
label. Times the synchronous export; the async header-dedup goroutine
correctly outlives the span/timer as before.

## Why
`process-memory`/`process-rootfs` existed only as **trace spans**, so
pause latency couldn't be decomposed in metrics/dashboards. With these +
`pause.duration` (#3425), a filesystem-only pause decomposes as:
```
pause.duration{fs_only="true"}                 e2e
  ├── guest_sync.duration                       quiesce (sync/freeze)
  └── process_rootfs.duration{fs_only="true"}   rootfs export/diff
```
and memory pauses additionally get `process_memory.duration`.

## Notes
- Pure additive instrumentation; no pause/snapshot behavior changed.
- Follow-up: a monitoring-repo panel stacking these per-step durations
(held until this deploys — the series have no data before then).

Verified: `go build`/`vet`/`golangci-lint` clean on changed files, `go
test ./pkg/sandbox/ -run TestGuestPrepareFsForPause` passes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
# feat(orch): live-upgrade envd inside a running sandbox at resume

## Why

envd is the in-VM daemon every sandbox depends on, but its version is
baked into
the snapshot at build time. A long-lived paused sandbox therefore
resumes on
whatever envd it was built with, potentially many releases behind —
there is no
way to ship an envd fix or behavioral change to the existing fleet of
snapshots
short of rebuilding every template. We want to roll a new envd to
running
sandboxes the same way we roll Firecracker versions: gated by a
LaunchDarkly
flag, ramped by cohort, and without disturbing the customer's workload.

The hard part is doing it without a visible interruption: the swap has
to keep
arbitrary customer processes — and their stdio/PTYs, timeouts,
filesystem
watchers, and recently-observed exit codes — alive across the binary
change.

## What

Two halves: an in-guest handover mechanism in envd, and an
orchestrator-side
resume-time trigger driven by a flag. They are independent — the envd
half is
inert until something calls `POST /upgrade`.

### In-guest handover (envd)

A new authenticated `POST /upgrade` endpoint performs a **same-PID
`syscall.Exec`** into a new envd binary and re-adopts the running
workload, so
customer processes never notice the daemon changed underneath them.

- **Delivery + trigger in one call.** The new binary is streamed in the
request
body, written inside the guest (default `/usr/bin/envd.next`), then
swapped.
  This uses the token-authenticated `/upgrade` path rather than the
unauthenticated `/files` copy, which a live post-`/init` sandbox
rejects.
- **The swap.** envd freezes the workload cgroups, serializes the
process table
to a tmpfs handover blob, relocates the carried fds and `syscall.Exec`s
under
`ForkLock`. On success it never responds — the process image is
replaced. The
freeze lock is **held across the whole handover** (freeze → serialize →
execve,
via `FreezeHold`), not just the freeze sweep, so a concurrent `/init` or
`/unfreeze` thaw can't slip in and unfreeze the workload
mid-serialization; on
`execve` it evaporates with the process image, and on any failure it's
released
before the thaw. The port scanner is left running: holding `ForkLock`
across
  the fd relocation already serializes it against the scanner's fd/socat
  creation, so there's no CLOEXEC race to quiesce.
- **Guarded fd relocation.** The carried fds are `dup3`'d to fixed high
numbers
  (so the fresh runtime's early startup can't have grabbed them) under
`ForkLock`, and each target is checked free first — a collision aborts
the
  upgrade (closing what it already relocated) instead of `dup3` silently
  clobbering a live fd, leaving the old envd running.
- **What the handover carries** (tmpfs `/run/e2b/envd-handover.json`, so
it
never touches the rootfs diff): per-process pid / tag / cgroup / config,
the
stdio + PTY fds (dup'd with CLOEXEC cleared so they survive execve), the
terminal-event **retention cache** (exit codes of processes that ended
just
before the swap), the filesystem **watchers**, and each process's
**remaining
  timeout**.
- **The incoming side** re-adopts each process from its inherited fds,
resumes
streaming its output, and re-arms its timeout — recording a fresh
deadline so a
*further* chained upgrade re-carries the remaining timeout. It restores
the
retention cache (skipping any PID already re-adopted as live, so a
reused PID
resolves to the live process, not a stale exit) and re-arms the
filesystem
watchers **before** thawing the workload, so the workload stays frozen
until
the watches are live and no event is dropped in the gap. Re-adopted
processes
are reaped via **pidfd + a pid-specific `wait4`** — never `wait4(-1)`,
which
would steal `os/exec`'s post-upgrade children — and the reaper retains a
terminal event **synchronously, before** closing its event channel, so a
late
reconnect always finds the exit in the cache rather than racing the
retain.
- **Failure handling — the sandbox is never left worse off.** On an
execve
failure the outgoing envd closes the fds it relocated (CLOEXEC-cleared —
else
they'd leak into the still-running old envd and its future children),
thaws the
workload it froze, and keeps running the old binary. The incoming resume
path always thaws via a top-level `defer`
(success, error, or panic) and is recover-wrapped, so a malformed blob
can't
crash envd into a systemd restart that orphans the workload. Degraded at
  worst, never hung or dead.
- **One shared freeze lock.** The freeze/unfreeze sweep and its
serializing lock
are factored into a single `cgroups.WorkloadFreezer` shared by the HTTP
API
(`/freeze`, `/unfreeze`, the `/init` deferred thaw) and the upgrade, so
the
upgrade's freeze can no longer interleave with the resume thaw and
strand the
  workload frozen.

envd is bumped to **0.6.11**, the first upgrade-aware version.

### Resume-time trigger + gate (orchestrator + shared)

On resume (`Create` with `snapshot=true`, and the checkpoint-resume
path),
`maybeUpgradeEnvd` decides against the **live running envd version**,
delivers,
and confirms — best-effort, recover-wrapped, and bounded so it can never
disrupt
resume.

- **Live version, not built-with.** envd advertises its running version
via an
`X-Envd-Version` response header on `/init`; the orchestrator captures
it off
the resume-path `/init` it already makes (no extra round-trip) and keys
the
  decision, gate, `from_version`, and success check on it. The template
  built-with never changes across a live upgrade, so keying on it would
  re-trigger the handover on every resume.
- **Flag-driven target resolution.** `ResolveEnvdUpgrade` (shared, the
resume
analog of `ResolveFirecrackerVersion`) reads the `envd-upgrade-target`
flag and
returns the target binary's local path + version, or `""` when the
target is
not strictly newer than the live version — so an already-upgraded
sandbox
(live == target) is a true no-op, skipped with no delivery or handover,
and a
  downgrade is refused. Target binaries live on the node-local read-only
  `/fc-envd` gcsfuse mount.
- **Version gate.** The *running* envd must already speak `/upgrade`, so
the
trigger skips a live version below `MinEnvdVersionForUpgrade` (0.6.11)
and
  counts it (`gated{reason=old_envd}`).
- **Delivery + confirmation.** `Sandbox.CallEnvdUpgrade` streams the
binary over
the authenticated `/upgrade` body to a fixed guest path; envd reading
the body
then exec'ing without replying (transport drop) is the expected path.
After
`WaitForEnvd` (which re-reads the version off the new envd's `/init`),
success
is confirmed by the running version actually equalling the target — a
transport
quirk can't mislabel the outcome. Only unambiguous never-reached-envd
errors
  (connection refused / dial) are treated as delivery failures.

Flags / knobs:

| Flag | Values | Default | Meaning |
|------|--------|---------|---------|
| `envd-upgrade-target` (LD string) | `off` \| `promoted` \| `<git-sha>`
| `off` | `off`: no upgrade. `promoted`: track node-local
`HOST_ENVD_PATH`, upgrade when strictly newer than the live version.
`<sha>`: pin `/fc-envd/envd.<sha>`. |
| `ENVD_UPGRADE_TARGET` (env fallback) | same | `off` | Overrides the
flag fallback where there is no LD (dev), mirroring
`DEFAULT_FIRECRACKER_VERSION`. |
| `MinEnvdVersionForUpgrade` (const) | — | `0.6.11` | Below this the
running envd lacks `/upgrade`; trigger skips. |

### Rollout metrics

envd has no metrics pipeline, so its side reports via Loki summary logs
(`handover_resumed{procs,retained}`, `watchers_rearmed{...}`). The
orchestrator
side adds:

- `orchestrator.envd.upgrade.attempts{result,from_version,to_version}` —
`result ∈ {success, delivery_failed, not_ready, version_mismatch,
panic}`;
`from_version` is the **live** running version. `success` is recorded
only when
the running version actually flips to `to_version` (not inferred from
the
transport outcome). `success/total` is the rollout success rate; the
common
per-resume no-op (flag off / already on target) resolves to `""` and is
  deliberately **not** counted.
- `orchestrator.envd.upgrade.duration{result}` — wall-time of delivery +
trigger
  + `WaitForEnvd`, i.e. the overhead added to the resume.
- `orchestrator.envd.upgrade.gated{reason=old_envd}` — targeted resumes
skipped
  by the version gate (watch during a ramp).
- an `envd.upgraded` bool label on
`orchestrator.sandbox.create.duration`, and an
  `envd-upgrade` Tempo child span.

The post-upgrade readiness re-check re-runs `/init` (to re-read the
running
version) but is guarded to record the per-start resume KPIs — the
envd-init
duration histogram + call counter and `StartedAt` — **once per start**,
so a
successful upgrade doesn't double-count the init or push `StartedAt`
(and thus
`execution_time`) to a later, wrong timestamp.

```promql
# rollout success rate
sum(rate(orchestrator_envd_upgrade_attempts_total{result="success"}[5m]))
  / sum(rate(orchestrator_envd_upgrade_attempts_total[5m]))
```

## Validation

- **Unit tests (root-free, in CI):** a `Connect` after a gap-exit
returns the
retained Start+End by pid/tag (unknown → NotFound); a live PID is never
served a
stale cached exit and a reused PID keeps its successor; a stale
retention timer
can't evict a newer entry; a re-adopted process whose `pidfd_open` fails
still
emits a terminal event (no orphan); a watcher exported then imported
keeps its
  id, delivers post-handover events, and preserves pending events;
`ResumeFromHandover` always thaws on a malformed blob and on no blob;
`Upgrade`
refuses a target other than the fixed path; the resolver is table-driven
over
off/unset/promoted/sha × newer/same/older/missing (downgrade refused)
without a
LaunchDarkly client; `/init` advertises `X-Envd-Version`; the
delivery-error
  classifier flags only never-reached-envd; the three metrics have
  description+unit entries and construct.
- **Dev cluster, end-to-end (exact branch binaries; re-validated
2026-07-23 on
the final code):** this branch's envd was rebuilt at 0.6.11 (baked into
the
template) and 0.6.12 (staged as the `promoted` target), and this
branch's
orchestrator (`0.2.0-c21705aeb`) deployed to the dev client nodes.
Validated
against a live workload — a stdout-writing + tmpfs-appending background
counter,
so a broken fd-carry would SIGPIPE-kill it and line growth proves it
kept
  running:
- **upgrade fires + workload survives:** flag → target ⇒ resume
auto-upgrades
0.6.11→0.6.12; the running envd becomes `/usr/bin/envd.next
--resume-handover`
and the counter process **keeps the same PID and keeps writing across
the
swap** (7→20 lines) — exercising stdio fd inheritance and pidfd reaping
end
      to end. The orchestrator confirms by ground truth:

`envd_upgrade_attempts_total{result="success",from_version="0.6.11",to_version="0.6.12"}
= 1`.
- **flag off ⇒ no upgrade:** plain resume, workload preserved on 0.6.11,
no
      `self-upgrade` in the journal.
- **gap-exit retention:** a process that exits during the handover gap
has
its exit code (42) recovered by a late `Connect` via the carried
retention
      cache.
  The rollout metrics are emitted on the same path
(`attempts{result,from,to}`, `duration`, `gated`,
`create.duration{envd_upgraded}`),
observed on dev Prometheus with a per-upgrade `duration` of ≈ 345–440
ms.
- **Note:** the 2026-07-23 re-run above exercised the **final** binaries
(all the
review-driven changes: live-version decision, version-confirmed success,
the
shared freeze lock held across the handover, the freeze/serialize/exec
path, and
the once-per-start resume-KPI guard on the post-upgrade readiness
re-check). The
remaining review changes that only manifest on failure/edge paths —
reaper
retaining before closing its channel, watcher re-arm before thaw,
chained-upgrade
deadline carry, exec-fail fd cleanup, and the PID-reuse / retain-timer /
reaper-orphan / duplicate-forwarder fixes — are covered by the unit
tests above;
  they don't change the success path the e2e validated.

## Key decisions

- **Same-PID `syscall.Exec`, not restart.** A fresh process would be
reparented
by systemd (`Type=simple`), orphaning the workload; keeping the PID
means envd
  stays the workload's parent and the inherited fds/PTYs stay valid.
- **pidfd + pid-specific `wait4`, never `wait4(-1)`.** A blanket reaper
would
steal children that `os/exec` spawns after the upgrade; targeted reaping
keeps
  both the re-adopted and the new processes correct.
- **Trigger at resume, gated + best-effort.** Resume is the natural
quiescent
window and gives free cohort ramping (the resume-site LD context already
carries envd-version/team/template). The whole path is recover-wrapped,
  version-gated, and bounded, so a failed or slow upgrade degrades to
`upgraded=false` and the sandbox resumes on the old envd. *Caveat:* the
trigger
is currently synchronous on the resume-critical path, so on
pathologically slow
resumes (large cold-fault working sets) it can add up to its timeout
budget to
  the tail; moving it off the critical path is a candidate follow-up.
- **Decide and confirm on the live version, read for free off `/init`.**
The
orchestrator can't cheaply know the running envd version, but envd can
report
it — so envd advertises `pkg.Version` via an `X-Envd-Version` header on
`/init`
(which the resume path already calls) and the orchestrator keys the
decision,
`from_version`, idempotency, and the success check on it. Re-resumes of
an
upgraded sandbox become true no-ops and success is
ground-truth-accurate, with
no new endpoint and no extra round-trip. Version comparison still relies
on
`packages/envd/pkg/version.go` being bumped per behavioral change (repo
rule);
  the resolver refuses any target that isn't strictly newer.
- **Fixed exec target.** envd writes/execs the delivered binary only at
a fixed
path and refuses any other, so an authenticated-but-malformed `/upgrade`
can't
  turn the same-PID exec into arbitrary code execution.
- **`getVersion` injected into the shared resolver** so
`packages/shared` gains
  no dependency on the orchestrator's envd build package.
- **Handover blob on tmpfs**, additive-fields-only JSON, so it never
enters the
rootfs diff and an outgoing/incoming envd built at different versions
stay
  compatible.
- **Rollout is off by default** (`envd-upgrade-target=off`, env fallback
`off`);
  dev has no LaunchDarkly so the feature is inert there unless
  `ENVD_UPGRADE_TARGET` is set.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Nikita Kalyazin <nikita.kalyazin@e2b.dev>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…3429)

`team_limits` derives a team's limits by joining `tiers` to `addons`.
`addons` is written **only by billing** and read by nothing in this repo
except that view — so the sandbox-creation read path depends on a table
this side doesn't own, and there is nowhere for the owner to push an
effective limit to.

This adds that destination.

## What it does

`public.project_limits`, and every view column becomes
`COALESCE(override, existing expression)`.

Nothing writes the table yet, so the view returns exactly what it
returned before. Landing it alone is the point: it edits the object
gating every sandbox creation while nothing depends on the edit.

| Choice | Why |
|---|---|
| All nine columns `NOT NULL` | A team is overridden completely or not
at all — no half-overridden row to reason about |
| `bigint` throughout | Matches what the view already yields; narrower
would silently clamp a pushed value |
| `CHECK >= 0`, where `tiers` uses `> 0` | A push target: rejecting a
value the caller considers valid turns a product decision into a retry
loop that never drains. Negatives stay rejected |
| FK to `teams`, `ON DELETE CASCADE` | Intra-schema, so it survives the
billing detachment |

## The type change is a fix

Generated `TeamLimit` widens from `int32` to `int64` on six columns.

Those columns are `bigint`. sqlc previously couldn't infer the type of
the view's arithmetic and defaulted to `int32`; it resolves through
`COALESCE` now. The old type was a latent truncation, not a deliberate
width.

One caller feeds an `int32` response schema and now narrows explicitly
at that boundary, rather than relying on codegen to do it invisibly.
Same values.

## Tests

- **Inertness** — `team_limits` matches the tier+addon arithmetic it
replaced, for teams with and without addons. Asserted against the
arithmetic rather than fixed numbers, so it stays honest if tier
defaults move.
- **Override wins** — a populated row beats tier and addon on every
column. Without this, a `COALESCE` pointed the wrong way would still
pass the first test.
- **Cascade** — deleting a team removes its overrides.

The first two are mutation-verified: dropping an addon term from one
column, and reversing a `COALESCE`, each fail the intended test.

Down migration restores the prior view and drops the table.
`ARCHITECTURE.md` gains `project_limits` in the data stores table,
alongside `tiers`.
Resolve the base image's distro from its `/etc/os-release` ID and drive
provisioning from a declared per-family profile — Debian/Ubuntu (apt),
the RHEL family (dnf/microdnf/yum), Arch (pacman), and Alpine on OpenRC
(apk) — instead of probing for a package manager. Unsupported or
identity-less images are rejected with a clear build-log error.

Once merged, the biggest change is adduser -> useradd and introduction
of /usr/local/bin/e2b-seed-certs instead of long one-liner for envd.

Supersedes #3381 (reopened from a clean branch, no bot-thread churn).
NixOS support follows in a stacked PR #3412.
The first `/v1/management` handler to leave 501, now that
`project_limits` (#3429) gives it somewhere to write.

## What it does

Stores a project's effective limits exactly as sent. This side does **no
arithmetic** — the caller owns plans and add-ons and has already
resolved them. That's the point of the push, and it's what lets
`team_limits` prefer this row over the tier it would otherwise compute
from.

## Contract: two missing dimensions

`tiers` gained `default_free_disk_size_mb` and `max_disk_size_mb`, and
the view exposes both — but `AdminControlPlaneProjectLimits` carried
only seven fields, so an override could never have set the other two.

Both added, **required** like the rest: a partial override has no
meaning against a table whose columns are all `NOT NULL`.

This is a breaking contract change for the caller, and deliberately so —
belt's `exhaustruct` rule is scoped to these generated types precisely
to make a new field a build failure rather than a silently-zero payload.

## Errors are distinguished, not collapsed

| Failure | Status | Why |
|---|---|---|
| Foreign key violation | **404** | The team is the row's only FK, so it
means the project is unknown here — not that the payload was bad |
| Check violation | **400** | The caller sent a combination the column
set refuses. Currently free disk above the ceiling, which the request
schema can't express |
| Anything else | 500 | Ours |

Adds `dberrors.IsCheckViolation` alongside the existing helpers.

## Cache invalidation, and a bug it uncovered

The team cache carries limits, so **without invalidating it the write is
invisible until the entry expires.**

Review found `InvalidateTeamCache` was itself incomplete: it cleared
`team-<id>` and the API-key hashes but not the per-member entries
`ValidateAuthProviderTeam` populates — which hold a full `Team`
including limits, and gate sandbox creation on the browser-session path.
It now sweeps all three key kinds via a new `GetTeamMemberIDs`.

That was **already a bug** for the other caller:
`admin_kill_team_sandboxes` invalidates after blocking a team and left
the same entries stale.

The handler logs rather than fails: the row is already committed, and
returning an error would invite a retry that can't improve on a stale
cache.

## Tests

Driven through the handler, asserting through `team_limits` rather than
the table — so the two are proven wired together rather than assumed.
Idempotency is covered because the caller retries.

The 404, the 400 and the cache invalidation are each mutation-verified:
removing the FK mapping, the check mapping, or the invalidation call
fails exactly the intended test.

## Verification

`packages/db` and `dashboard-api` build, vet, test and lint clean.
Codegen idempotent.

## Also: finishing the management rename

#3361 moved these routes from `/admin/v1` to `/v1/management` but
touched only the path keys. The surface has carried three names for one
concept ever since:

| Layer | Was | Now |
|---|---|---|
| Paths | `/v1/management/*` | unchanged |
| Schemas | `AdminControlPlane*` | **`Management*`** |
| operationIds | `upsertProject` | **`managementUpsertProject`** |
| Handler methods | `UpsertProject` | **`ManagementUpsertProject`** |
| Tag | `workspace-admin` | **`control-plane-management`** |
| Handler files | one `workspace_admin.go` | **one `management_*.go` per
operation** |
| Scheme | `AdminJWTAuth` | unchanged |

`AdminJWTAuth` stays — it names a credential, not the surface, and sits
beside `AdminApiKeyAuth`, which is a different scheme entirely. That
adjacency is also why `Admin` on the schemas had become actively
misleading rather than merely stale.

Folded in here rather than deferred because six more handlers are about
to be written against these types, and the caller's call sites are about
to multiply. It only gets more expensive.

The operationId prefix matters on a shared `ServerInterface`: without it
these seven sit in the same namespace as the dashboard's own operations,
and `UpsertProject` says nothing about which surface it answers on.

**Breaks the caller on its next spec sync** — seven method names and
four type names in `controlplanes/management.go`. All mechanical, all
compile failures, same coordination as the two required fields above.

Handler files are now one per operation, matching the rest of the
directory (`admin_users_delete.go`, `template_get.go`, `teams_list.go`)
rather than the single scaffolded file the seven arrived in. A PR
implementing one handler touches one file, and the surface is legible
from `ls`. `sendNotImplemented` moved to `utils_management.go` and loses
a caller with each handler that lands.

---

Also drops six `int64` conversions in `packages/auth/pkg/types/teams.go`
that became redundant when #3429 widened `TeamLimit`. They were
**failing lint on main** — after that change I checked `db` and
`dashboard-api` but not the package that consumes the type.
…hes (#3385)

A single bad 4K sector on a node's local NVMe crashed the whole
orchestrator and killed every sandbox on the node: the kernel cannot
return EIO for a memory access, so a failed page-in from an mmap'd cache
file delivers SIGBUS, which Go turns into an unrecoverable fatal error
inside the Cache.ReadAt memmove.

RunFaultSafe converts such faults into a regular error carrying the
fault address (debug.SetPanicOnFault + recover); other panics still
crash. It guards the in-process mmap copies: build segment reads, NBD
dispatch read/write/write-zeroes and the dedup compare. A bad block now
fails a single read or NBD request instead of the whole orchestrator,
and the affected file is just a local cache of remote storage anyway.
The guard is cheap enough to sit on the hot read path.

Recovered faults are logged and counted in
orchestrator.block.memory_fault, which should stay at zero on a healthy
node.

---------

Co-authored-by: Jakub Rojko <jakub@e2b.dev>
Adds a unit test that holds every distro profile's package set to a
shared capability map, and integration tests that build a template from
an Ubuntu, Fedora, Arch and Alpine base image plus one unsupported image
that must be rejected with a readable reason. Stacked on #3411 —
retarget to main once that merges. Heads-up on cost: the first run pulls
and provisions four new base images, so the templates package gets
slower until the base-layer cache warms.
…fork (#3419)

Closes #3418

## Summary

- Add early-exit `*body.Timeout <= 0` check in `PostSandboxes`
immediately after body parse — before the template cache lookup, so
invalid input is rejected with zero DB cost
- Add the same lower-bound guard in `PostSandboxesSandboxIDResume` and
`PostSandboxesSandboxIDFork` before their upper-bound checks
- All three handlers now return `400 Bad Request` with message `"Timeout
must be greater than 0"` for any `timeout <= 0`
- Add `sandbox_timeout_validation_test.go` with 7 parallel subtests
covering `0`, `-1`, and `-3600` across all three endpoints — no
testcontainers required since validation fires before any
template/orchestrator access

## Why

If a client sends `{"timeout": 0}` or `{"timeout": -10}`, the existing
code evaluates `body.Timeout != nil` as true and converts the value into
a zero or negative `time.Duration`. The only existing guard (`timeout >
MaxLengthHours`) passes because `-10s < 24h`, so the sandbox is created
with an `EndTime` already in the past. The orchestrator's background
expiry loop picks it up immediately and kills it before the caller can
use it. The fix adds an explicit lower-bound check that returns `400`
early and cheap.

For `PostSandboxes` specifically, the check was placed immediately after
body parse (before `templateCache.ResolveAlias`), which is also the
correct design order: fail fast on cheap input validation before doing
more expensive work.

Compare to `PostSandboxesSandboxIDTimeout` (the keepalive endpoint),
which already handles `body.Timeout < 0` correctly — this PR closes the
same gap in the lifecycle endpoints.

## Test Plan

- [ ] New tests pass: `go test -v -run
'TestSandbox.*RejectsNonPositiveTimeout'
./packages/api/internal/handlers/`
- [ ] Full handler suite passes: `go test -count=1 -timeout 300s
./packages/api/internal/handlers/`
- [ ] Build is clean: `go build ./packages/api/...`
- [ ] Linter passes: `make lint` (or `golangci-lint run
./packages/api/...`)
- [ ] Confirmed `timeout: 0` → `400 Bad Request: Timeout must be greater
than 0`
- [ ] Confirmed `timeout: -10` → `400 Bad Request: Timeout must be
greater than 0`
- [ ] Confirmed `timeout: nil` (omitted) → defaults correctly, no
regression
- [ ] Confirmed `timeout: 30` (valid) → proceeds through template lookup
and orchestrator as before

/cc @jakubno @dobrac @ValentaTomas @arkamar @tvi
Stacks on #3411. Adds a `nixos` distro profile — premade images whose
packages/services are declared in the image's own NixOS configuration
rather than installed at provision time — plus the E2B NixOS base-image
definition.

## Verification status

Verified at tip on real KVM: the image was built with a nix builder from
the committed `configuration.nix` (build.sh fixed — it was
non-executable and staged from a hardcoded out-of-repo path, so the
committed config was previously never the one built) and booted as
sandboxes through the template pipeline. All previously inspection-only
fixes now have observed evidence: envd unit parity (`Nice=-20`,
`OOMScoreAdjust=-1000`, `GOMEMLIMIT=512MiB`, IO/CPU/memory weights),
chrony source-select oneshot (pool branch live; PHC branch
replay-verified — a real PHC lock needs `kvm-ptp`, not exposed on the
verification host), firewall off with envd's port connected, CA bundle a
regular file + `https=200`, default user + NOPASSWD sudo, hostname/hosts
kept, shadow tools present, nix store DB valid (`--check-validity` OK,
532 paths).

Note: sshd accepts connections ~1–2s after sandbox start — NixOS
generates host keys per sandbox on first boot (unique keys per sandbox;
the package-manager families bake shared per-template keys at provision
time). Inherent to NixOS's design, left as-is.

## Known NixOS limitations

- **Sudoers for non-default users.** A `USER` build step naming an
account other than `user` fails: `commands/user.go` appends to
`/etc/sudoers`, which is a read-only store symlink on NixOS. The fix is
a drop-in under `/etc/sudoers.d`, which changes behaviour for every
family and belongs in its own PR.
- **No at-tip real-KVM coverage.** Building the premade image needs a
nix builder; none is available on the current box or dev VM, so the
NixOS-side changes here are reviewed but not runtime-verified.

Draft: on-the-fly (nix-build at template-build time) is still open
design; merge after #3411.
…d -gdb (#3108)

## Why

`resume-build -gdb` (guest-kernel debugging, added in #3040) didn't
actually run on a real orchestrator node: it staged the gdb-enabled
Firecracker by renaming the prod binary in place, but on a node the
FC-versions directory is a **read-only** mount (gcsfuse), so it failed
with `read-only file system`. It also located its `fc-debug.gdb` macro
library relative to the source tree, so a binary copied to a node
without its source failed with `macro library not found`. And bridging a
snapshot from another environment for debugging meant hand-copying the
FC + kernel debug artifacts.

This makes the `-gdb` path portable, adds a one-command artifact bridge,
and trims a debug-artifact fetch path that never fired in practice.

## What

1. **`fix(orch): make resume-build -gdb portable to read-only mounts and
standalone binaries`**
- Stage the gdb-enabled Firecracker into a **writable temp dir** and
point the factory's `FirecrackerVersionsDir` at it before the factory
captures the config, instead of overwriting the prod binary in the
read-only versions dir. The original dir is preserved only for
*resolving* the published `firecracker-debug`; the real dir is never
written. Removes the old backup/restore (and its interrupted-run
recovery).
- **Embed** `fc-debug.gdb` with `go:embed` so the binary is
self-contained (a colocated copy still wins for local iteration).
- **Backstop:** verify the binary about to launch actually contains the
gdb feature (`FIRECRACKER_GDB_SOCKET`), so a stale/wrong
`firecracker-debug` fails with a clear message instead of an opaque "gdb
socket never bound".

2. **`feat(orch): add copy-build -gdb to stage FC + kernel debug
artifacts`**
- `copy-build -gdb` reads the build's FC/kernel versions from
`metadata.json` and ensures the runtime + debug artifacts
(`firecracker`, `firecracker-debug`, `vmlinux.bin`, `vmlinux.debug`)
exist at the destination, deriving the versions/kernels buckets from the
template bucket. CRC-compares like the snapshot copy: skips identical
content, otherwise copies — **replacing** a divergent/stale artifact.
Requires `gs://` source/dest.

3. **`refactor(orch): drop the unused gdb debug-artifact URL
auto-fetch`**
- The by-version HTTP fetch never fired in practice (the artifacts
aren't published at the default URL, and the per-cluster bucket layout
doesn't match the URL scheme); artifacts always resolve locally or via
`copy-build -gdb`. Removes the fetch code/env-var and renames
`resolveOrFetch` → `resolveLocal`.

4. **`docs(orch): correct the gdb runbook artifact resolution`**
- Replaces the inaccurate "auto-fetched by version" claim with how
resolution actually works (local, next to the snapshot's FC/kernel;
staged by `copy-build -gdb`; or explicit `-gdb-fc`/`-gdb-symbols`).

## Testing

- End-to-end on a dev node against a real snapshot, with the FC-versions
directory mounted **read-only**: Firecracker bound the gdb socket and
launched from the temp staging dir (never the read-only mount); DWARF
source-level symbols, vCPU state, and page-fault profiling all worked.
- Verified the backstop both ways: a non-gdb binary fails with the clear
"is not gdb-enabled" error; a real gdb binary passes and runs.
- `copy-build -gdb` exercised both the skip-identical and the
copy/replace paths.
- `go build`, `go test`, `golangci-lint` clean; each commit builds on
its own.

## Scope / risk

Touches only the `resume-build` and `copy-build` developer side-tools —
no production service code path. The read-only-mount override is gated
on `-gdb` and confined to that run (only the FC-versions dir; the kernel
dir is untouched).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Nikita Kalyazin <nikita.kalyazin@e2b.dev>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…vd (#3444)

`make build-debug` builds envd with `-race`, which needs cgo, so the
binary the integration host injects into every guest is dynamically
linked against glibc — and a musl guest has no loader for it. envd never
answers, the base layer times out and the build dies with a masked
internal error, which is why the Alpine case was skipped. Linking it
statically fixes that; `osusergo,netgo` keep user and DNS lookups pure
Go as in the `CGO_ENABLED=0` binary we ship, so only the race
instrumentation differs. The detector is intact (297 `__tsan` symbols; a
deliberate racy program built with the same flags still reports on
musl), and guest console output reaches `orchestrator.log`, so the
workflow's data-race grep keeps covering envd.

Checked on a KVM dev stack, cold cache each time: with the old binary
ubuntu/fedora/arch pass and alpine fails on `wait for envd: syncing took
too long`; with the static one all four pass (alpine 30s). `envd
-version` also runs under `alpine:3.24`, where the old binary gives
`exec: no such file or directory`.
## Why

Every prefetch mechanism we have today accelerates only the **resume →
envd-init** prefix of a sandbox's life:

- Build-time `optimize` intersects two finalize resumes but runs only to
`WaitForEnvd`.
- The throwaway pause-resume harvest resumes a network-isolated copy
that *has no workload to replay*, so it structurally cannot see the
post-init working set.

So the *workload's* hot pages are never prefetched, and every resume of
a paused sandbox demand-faults them cold from object storage through
UFFD — the dominant resume-latency / time-to-responsive cost. Meanwhile
the live sandbox's `PrefetchTracker` accumulates the whole resume→pause
cycle's fault set for free and it is discarded when the uffd is torn
down.

This PR replays the last cycle's working set on the next resume. Those
pages are already recorded: they *are* the snapshot's own pause diff —
the merged `memfile.header` `BuildMap` entries whose `BuildId` is the
header's own build ID — so the set is derived from the header at resume
with no pause-time capture, no new metadata, and no extra I/O. Off by
default and strictly additive.

## What

### Last-cycle prefetch on resume, selected by a flag

`buildDiffMemoryPrefetchMapping(header)` builds the mapping from the
snapshot's own-`BuildId` blocks — deduped by block index (a merged/dedup
header repeats an offset across layers) and offset-sorted — i.e. the
last cycle's writes, read from the memfile header at resume.
`ResumeSandbox` picks what to replay via the new
`resume-prefetch-source` flag (`selectResumePrefetch`): `init` (default,
today's behavior), `last-cycle`, `both`, or `off`.

### Fetch-only replay, behind a barrier

The prefetcher gains an exported `Prefault` field (default `true`, so
existing callers are unchanged); last-cycle sets it `false`, warming the
shared chunk cache and letting the guest fault the now-warm pages
instead of `UFFDIO_COPY`-ing them in. When an init trace is also present
it runs first (prefaulted) and its `Start()` blocks on the fetch+copy
waitgroups, so last-cycle begins only afterward.

### Contiguous-fetch coalescing

`coalesceIndices` merges runs of contiguous block indices into one
larger `source.Slice`, capped by the new
`memory-prefetch-coalesce-max-mb` int flag (read at `Start()` like the
existing worker-count flags; default `0` = off). The copy phase stays
strictly per-page — a coalesced extent is split back into page-sized
sub-slices before `Prefault`, because `Userfaultfd.Prefault` installs
one page per call (`UFFDIO_COPY` itself takes a page-multiple length, so
batching the copy is possible future work). The flag is read for every
prefetcher, but only the offset-sorted last-cycle mapping coalesces
meaningfully; the fault-ordered init trace stays one extent per block
(`coalesceIndices` never reorders).

### Volume cap

`resume-last-cycle-prefetch-max-mib` (default `-1` = uncapped) bounds
how much of the last-cycle diff a resume prefetches; `capResumePrefetch`
keeps the first N MiB of blocks in offset order and leaves the rest to
demand-fault.

### Observability (for rollout)

Two signals ship with the feature so a cohort can be measured without a
follow-up:
- **`orchestrator.sandbox.uffd.prefetch.mapping_blocks`** — a histogram
of the replayed mapping size in blocks, tagged `mode=prefault` (init
trace) vs `mode=fetch` (last-cycle diff). The `fetch` series is the
recorded-working-set distribution that sizes the rollout; its bottom
bucket flags idle-at-pause sandboxes. A live metric, so it's
dashboard-able without Tempo sampling.
- **Resume span attributes** `resume.prefetch.source` /
`resume.prefetch.init_blocks` / `resume.prefetch.last_cycle_blocks` —
cohort a resume by the chosen source (guards against flag
misconfiguration) and see the per-resume set sizes.

Existing prefetcher metrics (`…prefetch.pages` by stage,
`…prefetch.duration` by phase, and the per-page `…prefault` metric)
already cover fetched/copied/skipped counts and how late the prefetch
ran.

New feature flags (all default to today's behavior):

| Flag | Type | Default | Effect |
|---|---|---|---|
| `resume-prefetch-source` | enum `off\|init\|last-cycle\|both` | `init`
| which trace resume replays; `init` is a no-op-equivalent |
| `memory-prefetch-coalesce-max-mb` | int (MiB) | `0` | coalesce
contiguous last-cycle blocks into ≤N MiB fetches; `0` = off |
| `resume-last-cycle-prefetch-max-mib` | int (MiB) | `-1` | cap
last-cycle replay volume; `-1` = uncapped |

## Key decisions

- **Last-cycle set is header-derived, not captured at pause.** The pause
diff already encodes the last cycle's writes, so it is read from the
memfile header at resume — no `Prefetch.LastCycle` metadata field and no
pause-path work. Persisting an explicit trace (and its harvest flag) is
deferred as unnecessary.
- **Last-cycle replays fetch-only, not prefault.** Prefaulting the
multi-GiB diff costs ~2.5 s of cold resume for no workload gain and
regresses warm resumes; fetch-only warms the cache and lets the guest
fault.
- **Init and last-cycle are sequenced behind a barrier, not unioned.**
Init (prefaulted) lands first and last-cycle follows it, keeping the big
last-cycle fetch off the resume-critical path; a concurrent union raised
resume latency for no workload gain.
- **Coalescing is opt-in (default off).** The measured win saturates by
~8 MiB, so a small cap captures it; larger caps buy nothing.
- **The cap applies to last-cycle only** (the init trace is tiny);
default uncapped.
- **Backwards-compatible by construction:** default
`resume-prefetch-source=init` is today's behavior, and `prefetch.New`'s
signature is unchanged (fetch-only is opted into via a new field
defaulting to prefault). No Firecracker-version coupling — capture and
replay are orchestrator-side and reuse the in-prod prefetcher.

## Validation

- **Unit:** `go test ./pkg/sandbox/ ./pkg/sandbox/uffd/prefetch/` passes
— table-driven tests for the mapping builder, source selection, cap, and
coalescing; a test that a coalesced multi-block fetch still yields one
page-sized copy per block; and a `Start()`-level test that a fetch-only
run never calls `Prefault` while still fetching every block. Each commit
builds + tests standalone; `golangci-lint` clean.
- **Dev cluster, cold GCS resume, 8 GiB template** (workload reads a 3.5
GiB working set that would otherwise demand-fault cold; `mem_seq` on the
first cold resume is the reliable metric):
- no workload prefetch ≈ **173 s**; last-cycle ≈ **14.9 s** (~12×); with
coalescing **7.65 s** (~24×). Coalescing sweep (off→8→16→32 MiB): 14.9 →
6.9 → 7.7 → 6.1 s — the win is the off→on step and **saturates by ~8
MiB**.
- recommended shape (init prefaulted + last-cycle fetch-only +
coalesced, behind the barrier): **~3.5 s resume / ~9 s workload** —
resume matches the init-only floor (~2.3 s vs ~7.2 s with no prefetch)
while the workload is ~19× faster.
- **fetch-only vs prefault** (both init-prefaulted): fetch-only **3462
ms / 9.23 s** vs prefault **6014 ms / 9.06 s** — prefaulting costs ~2.5
s of resume for no workload gain.
- **warm resume** (hot cache): proposed config 648 ms / 2.20 s vs
init-only baseline 763 ms / 2.29 s — **neutral, no regression**.
- **Smoke (dev cluster):** create → pause → resume → exec, both with the
default flag and with `source=last-cycle` forced — both pass (resume
healthy, marker preserved).
- Volume rationale: last-cycle prefetches only the last cycle's writes;
a production dirty-set analysis puts the median at ~15× fewer bytes than
prefetching all resident memory.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Nikita Kalyazin <nikita.kalyazin@e2b.dev>
🤖 I have created a release *beep* *boop*
---


## 0.0.1 (2026-07-29)


### Features

* **envd:** add --no-cgroups flag to disable cgroup management
([#2811](#2811))
([e10814c](e10814c))
* **envd:** add optional EntryInfo to watch FilesystemEvent
([#2930](#2930))
([bbbc7c8](bbbc7c8))
* **envd:** allow opting into watching network mounts
([#2982](#2982))
([9799dd0](9799dd0))
* **envd:** give envd realtime IO priority, reset for user processes
([#2681](#2681))
([f4bd1b2](f4bd1b2))
* **envd:** split collapse stats into real migrations vs already-huge
([#3021](#3021))
([0d77614](0d77614))
* **envd:** support user-defined file metadata via xattrs
([#2732](#2732))
([da8fbe4](da8fbe4))
* freeze user cgroup across pause/resume to keep envd /init responsive
([#2688](#2688))
([eceb741](eceb741))
* **orch:** collapse envd's heap into 2 MiB hugepages before pause to
cut cold-resume faults
([#2997](#2997))
([6677f73](6677f73))
* **orch:** distro-aware template base-image provisioning
([#3411](#3411))
([f8c7b5b](f8c7b5b))


### Bug Fixes

* added envd to artifact repository
([#3432](#3432))
([6c4f0e2](6c4f0e2))
* correct 3 CVES ([#3218](#3218))
([076823b](076823b))
* **envd:** avoid Start deadlock after request cancellation
([#3256](#3256))
([04317f8](04317f8))
* **envd:** bound the in-memory logs queue
([#2676](#2676))
([05c9939](05c9939))
* **envd:** discard output when no subscriber is connected
([#2639](#2639))
([8cf1795](8cf1795))
* **envd:** fall back to lazy unmount when forced NFS umount fails
([#2683](#2683))
([5346a0d](5346a0d))
* **envd:** ignore closed pty read errors
([#2769](#2769))
([6118672](6118672))
* **envd:** include suppressed count in exporter error logs
([#2680](#2680))
([35c1141](35c1141))
* **envd:** make /init lock ctx-aware to prevent retry pile-up
([#2702](#2702))
([173afd4](173afd4))
* **envd:** make CA install lock ctx-aware
([#2690](#2690))
([83ee89f](83ee89f))
* **envd:** replace env vars in /init instead of merging
([#2706](#2706))
([1b52e9a](1b52e9a))
* **envd:** replace time.Sleep with ticker in ScanAndBroadcast for
prompt shutdown ([#3374](#3374))
([002fd9f](002fd9f))
* **envd:** self-heal MMDS routing on /init lookup failure
([#2701](#2701))
([90944d5](90944d5))
* **envd:** stop freezing socat cgroup across pause/resume
([#2923](#2923))
([8b6f2b9](8b6f2b9))
* **envd:** stop misleading CA install cancel errors on rapid /init
([#3206](#3206))
([91d09e4](91d09e4))
* **envd:** suppress repeat MMDS poll failures
([#2678](#2678))
([73d691a](73d691a))
* **envd:** tolerate busy tmpfs cleanup in tests
([#2938](#2938))
([a485834](a485834))
* **envd:** use constant-time comparison for signature validation
([#3145](#3145))
([fcf92fa](fcf92fa))
* **envd:** use WithoutCancel for CA cleanup goroutine ctx
([#3207](#3207))
([ee7bf84](ee7bf84))


### Performance Improvements

* **envd:** stop logging streamed payload content
([#2755](#2755))
([db3868c](db3868c))
* **sandbox:** keep envd logging out of journald
([#2675](#2675))
([f6943ca](f6943ca))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: e2b-release-please[bot] <298072688+e2b-release-please[bot]@users.noreply.github.com>
…3446)

## Why

Follow-up to #3108. The `-gdb` fail-fast precondition sits next to
`flag.Parse()` specifically so a misconfigured invocation fails
**before** the multi-GB snapshot copy. It only checked the `gs://`
scheme, but `copyGdbArtifacts` → `deriveArtifactBuckets` has a second
precondition: the bucket must end in `-fc-templates` (to map the
template bucket to its `-fc-versions`/`-fc-kernels` siblings). That
suffix check only fired deep inside the run, so `copy-build -gdb -from
gs://a-fc-templates -to gs://some-other-bucket` passed the up-front
check, copied the entire snapshot chain, printed "Build copied", and
*then* `log.Fatal`ed — the exact wasted-bandwidth pattern the scheme
hoist was meant to prevent.

Impact is bounded (a developer-only side tool; wasted time/bandwidth, no
data or production impact), so this is a consistency fix rather than a
bug fix — completing the fail-fast for the sibling precondition.

## What

Replace the scheme-only check with
`deriveArtifactBuckets(*from)`/`(*to)` up front. That single validator
covers **both** the `gs://` scheme and the `*-fc-templates` suffix, and
its error names the offending side (`-gdb -from:` / `-gdb -to:`). The
redundant scheme check is dropped.

## Validation

- `go build` / `go vet` / `golangci-lint` (0 issues) / `gofmt` clean.
- Manual, confirming failure now precedes any copy:
- `-to gs://my-custom-bucket` → `-gdb -to: cannot derive
versions/kernels buckets from "gs://my-custom-bucket" (expected a
*-fc-templates bucket)` — before "Copying build".
- `-to /local/out` → `-gdb -to: -gdb requires a gs:// location, got
"/local/out"`.
- Both `*-fc-templates` → validation passes, proceeds to the copy as
before.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: Nikita Kalyazin <nikita.kalyazin@e2b.dev>
…3435)

Resolve a new volume's type from
`DEFAULT_PERSISTENT_VOLUME_TYPE_BY_REGION`, keyed by the single
`region=` node label of the ready nodes the team's scheduling labels
select — teams stay untouched; Terraform auto-appends the label to every
client cluster. Node labels answer *where* the team runs; the map is the
only policy source for *what* a volume there should be, so a region
mounting several types (e.g. a new default introduced next to a legacy
one) never needs runtime guessing.

Precedence: LaunchDarkly override (team-targetable) → region map →
deployment-wide default. Zero or multiple matched regions fail open to
the default.

Pairs with e2b-dev/terraform#873, which emits the map by deriving it at
plan time: a region's single type resolves automatically, the deployment
default wins when mounted in a multi-type region, and the plan fails
when a multi-type region has no explicit
`default_persistent_volume_type_by_region` entry.
Leaves 501 for every `/v1/management` operation except project deletion.
Completes the cluster-side half of the contract belt's workspace-api
calls (EN-1740, EN-1741, EN-1743).

## What lands

| Route | Behaviour |
|---|---|
| `PUT /projects/{id}` | Creates from a caller-supplied UUID on the
default tier, or reconciles. 201 vs 200 |
| `PUT`·`DELETE /projects/{id}/members/{userId}` | One shared
implementation |
| `POST /projects/{id}/members/batch` | Same implementation, both
directions in one transaction |
| `DELETE /users/{id}` | Memberships and access tokens; `public.users`
survives |

## Contract changes

**`email` is now required.** The caller synchronizes every property in
the upsert body and sends all of them on every push, so a reconcile is a
complete statement of the project rather than a patch. Runtime-breaking
for a caller that omits it.

**`project_type` is gone.** This side has no column for it and no
opinion about the caller's plan vocabulary — the tier is assigned once
at creation from a local default, and the limits that matter arrive
absolute through `upsertProjectLimits`. Nothing sets
`additionalProperties: false`, so a caller still sending the field has
it **ignored rather than rejected**; the break is at their next codegen,
not at runtime.

The handler branches on whether the project exists rather than on an
insert failing, which is what makes "a create assigns the tier, a
reconcile never touches it" visible in the code.

## Why membership writes moved out of the handlers

Auth caches a copy of the team **per member** under `<userID>-<teamID>`,
and `InvalidateTeamCache` discovers those keys by reading `users_teams`
— so a member already removed is one it structurally cannot see. Writing
and evicting from separate call sites leaves a revoked member
authenticating until the entry expires.

`internal/management` makes that unrepresentable: the write and its
evictions are one call. Three routes converge on it, so the per-member
and batch paths cannot answer differently for the same desired state.

Two consequences worth flagging:

- Eviction covers every user the request **named**, not the rows that
moved. A crash before the eviction leaves a stale entry only a retry
clears, and that retry finds the work already done — keying off what the
statement touched would make the recovery path a no-op.
- Membership changes evict pair keys only. Sweeping the team-wide
entries too would evict every API key on the project for a change none
of them observe.

Review caught the inverse of the first point in `PurgeUser`, fixed in
17128ae: it read the user's teams, then ran an unconditional delete,
then evicted for the earlier read. Under READ COMMITTED those are
separate statement snapshots. It is now `DELETE ... RETURNING team_id`,
so the evicted set is the deleted set by construction.

## Project deletion stays 501

`envs`, `snapshots` and `volumes` reference `teams` with `ON DELETE NO
ACTION` and template deletion only stamps `deleted_at`, so any project
that ever built one pins its team row. Releasing it needs the api
service's orchestrator connections, which this process does not have.

Reason and the three ways out are recorded in the handler, with a test
pinning the 501. Belt drops its caller separately.

## Notes

- `ADMIN_AUTH_PROVIDER_CONFIG` is still absent from
`iac/provider-gcp/dashboard-api.tf`, so these routes answer 401 in
production until EN-2007.
- Cross-service ordering gaps found during review are tracked in EN-2012
rather than fixed here — they need workspace-api live to observe, and
several want a decision rather than a fix.
- `internal/management` handles `project`, handlers handle `team` — the
contract's word meeting the cluster's, translated at the call site.
#3453)

Alpine's OpenRC init hardcodes -F 1, and its chrony build (-NTS -SECHASH
-DEBUG) takes a SIGSYS the moment the PHC refclock is driven — 'Bad
system call' right after 'Loaded seccomp filter (level 1)' — so chronyd
lands in OpenRC's crashed state with the clock unsynced. Only fires
where the host exposes /dev/ptp0, so the pool path and the systemd
families (which pass no -F at all) are unaffected. Verified on a host
with a real PHC: crashed -> started, chronyc shows PHC0 reachable with
±28ns samples.
On every header version, LoadHeader filled a zero BuildData for any
build the mapping referenced but Builds did not cover. That is fair on
V3, which has neither a Builds section nor compression, but on V4+ the
gap means the entry was lost, not that the build is uncompressed.

createDiff reads the zero entry as CompressionNone and composes the
suffix-less buildID/memfile. For a compressed ancestor that object does
not exist, and because a zero entry also latches
UncompressedFullFrameTable the source is never re-resolved, so every
fault into that ancestor fails for the lifetime of the diff.

Restricting the backfill on V4+ to the header's own build keeps the one
case that needs it — runV3 never writes a self entry and only runs with
compression off — while leaving ancestor gaps absent, so createDiff
resolves them from the build's own header instead. The trade is one
header GET per V3-era ancestor on first fault, giving back #2999.

Co-authored-by: Tomas Valenta <49156497+ValentaTomas@users.noreply.github.com>
## What

When we are snapshotting a sandbox, during pause()/snapshot() we create
a snapshot of both memory and rootfs.
Exporting the rootfs boils down to reflinking the overlay to the local
cache path and then (asynchronously) uploading the cache to GCS.

This exporting process happens synchronously. We have seen that,
_typically_ reflinking is very fast as it deals with metadata (rather
than moving data around).

This PR makes the export asynchronous (optional, guarded by a feature
flag).

## Why

We have seen that, whereas the reflink operation is typically fast, it
can experience long tails when there's a lot of dirty data in the page
cache that needs to be written to the backing storage.

For example, when we are taking a snapshot after writing 2GiB of data in
the disk immediately after the write finished (E1) and 30 seconds after
the write had completed and:

* E1 — pause immediately after write: the ~2 GB NVMe flush burst happens
inside the pause window (host Dirty ~2 GB → 0 during the pause). → pause
= 6.8 s
* E2 — 30 s settle before pause: the flush burst happens during the
settle; Dirty is already 0 when the pause fires. → pause = 0.9 s
* Same write, same node — only difference is whether the flush overlaps
the pause.

This nicely scales with amount of time we wait after the write:
* settle  0 s  →  ~4.2 s   (6.8 s tail)
* settle  5 s  →   3.78 s
* settle 10 s  →   3.71 s
* settle 20 s  →   1.53 s
* settle 30 s  →  ~0.7–0.9 s

---------

Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…#3464)

## What

Adds an object ID codec to `packages/shared/pkg/id`: `Encode(uuid.UUID)
string` and `Decode(string) (uuid.UUID, error)`, encoding any UUID as a
fixed-width 26-character lowercase base32 string with an internal
rotation.

New files:

- `packages/shared/pkg/id/objectid.go` — the codec
- `packages/shared/pkg/id/objectid_test.go` — its test suite

No existing code paths are touched; nothing calls the new functions yet.

## Format

The UUID's 16 bytes are base32-encoded with the RFC 4648 section 6
alphabet (`A-Z2-7`), lowercased, unpadded — 26 characters — and the
string is then rotated left by 16, so what was the first character is
now the 11th.

The rotation is the point of the format. A v7 UUID leads with a 48-bit
big-endian millisecond timestamp, so unrotated encodings of IDs minted
close together share a long common prefix and the leading characters are
nearly constant for months. Rotating moves those characters inward: the
string leads with 10 characters of random bits, the timestamp reads out
from index 10, and the remaining random bits follow. `Decode` undoes it
by rotating left by the other 10.

Example — five real v7s minted 37 ms apart share `agp2kg` from index 10
(the timestamp showing through) while their fronts differ:

```
uk75vf2v7iagp2kgn7pfze3car  019fa519-bf79-724d-8811-a2bfda9755fa
imkhttdroeagp2kgn7t53cvkiw  019fa519-bf9f-762a-a916-431479cc7171
uxl2sotjfiagp2kgn7yv4e3e4g  019fa519-bfc5-784d-9386-a5d7a93a692a
```

Properties:

- **Version-agnostic and total.** `Encode` never reads version bits and
cannot fail; any 128-bit value encodes, v4 and v7 alike, always 26
characters.
- **Canonical form enforced.** 26 base32 digits carry 130 bits and a
UUID has 128, so 2 slack bits exist that stdlib decoders silently
discard — giving every UUID four spellings that decode identically.
`Decode` accepts only the one `Encode` produces and returns
`ErrNotCanonical` for the other three (the slack digit lands at index 9
after rotation). Uppercase and wrong lengths are rejected too
(`ErrBadLength` for the latter), so encoded strings compare by simple
equality.
- **Interop needs only the Python standard library.** The alphabet is
exactly what `base64.b32encode` produces, so the other side needs no
tables, only the rotation. The working snippet is in the doc comment.
- **Sort order is intentionally destroyed.** The string leads with
random bits, so encoded order and v7 chronological order are unrelated;
don't build ordered indexes on these strings (pinned by
`TestOrderIsGone`).

## Tests

Ported the full suite (16 tests), including:

- **Golden vectors**: five fixed v7 → string pairs, computed
independently with Python's `base64.b32encode` rather than captured from
`Encode`, so they check the implementation instead of recording it. They
are the compatibility contract: any change to the alphabet, width, or
rotation breaks them.
- **Independent reference implementation** (`refEncode`): the same
format restated as big-integer arithmetic (value shifted up by the slack
bits, written in 26 base32 digits, rotated); `Encode` is checked against
it over a corpus that includes `uuid.Nil`, `uuid.Max`, every power of
two ±1 across all 128 bits (a digit boundary falls every 5 bits), and
thousands of random v4/v7/unversioned values.
- **Canonical-form exhaustion**: all four spellings of each corpus UUID
are constructed; exactly one must decode and the other three must return
`ErrNotCanonical`.
- **Rotation properties**: same-millisecond v7s share the 10 timestamp
digits at index 10 while their fronts diverge; fresh v7 batches show no
common prefix.
- **Python round trip**: runs the doc-comment snippet under `python3`
(skipped if not on PATH) and requires agreement in both directions over
388 IDs.
- **Rejection cases**: wrong lengths, padding, uppercase (whole string
and single character), characters outside the alphabet (`0`, `1`, `8`,
`9`, `-`), non-ASCII.

Adapted to repo conventions: `t.Parallel()` on every test,
`exec.CommandContext(t.Context(), ...)`, gofumpt formatting.

## Verification

- `go test ./pkg/id/ -count=1` — all tests pass, including the existing
`id.go` tests and the Python interop test
- `golangci-lint run ./pkg/id/` — 0 issues
…3459)

Switching provisioning to the declared os-release id dropped every
Debian derivative that used to work when we probed for a package manager
— Kali declares `ID=kali ID_LIKE=debian`. Unknown ids now retry each
`ID_LIKE` token and warn that the profile is a best-effort guess.

`rhel`/`ol`/`amzn` are guarded **before** the fallback — both Oracle and
Amazon Linux declare `ID_LIKE=fedora` and would otherwise be silently
re-admitted past the rejection the rhel profile documents.

Behavior after this change:
- Known id → unchanged.
- Unknown id with a matching `ID_LIKE` token → provisions as that
family, with a customer-visible `WARNING` that it's best effort and
untested.
- `rhel`/`ol`/`amzn` → still rejected fast with the kernel/kABI reason.
- Id matching nothing, or no `/etc/os-release` → still exits 1.

Covered by `TestUnknownIDFallsBackToIDLike` and
`TestRejectedIDsAreNotReachableViaIDLike` (guard must precede the
fallback).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
## What

Adds typed object IDs to `packages/shared/pkg/id`, on top of the codec
from #3464: a kind tag in front of the encoded UUID, and the same kind
in the Go type.

```
prj_uk75vf2v7iagp2kgn7pfze3car   a project
wrk_imkhttdroeagp2kgn7t53cvkiw   a workspace
```

New files:

- `packages/shared/pkg/id/object.go` — the types, the parsers, the width
invariant
- `packages/shared/pkg/id/object_test.go` — its test suite

One edit outside them: `ErrBadLength`'s doc comment in `objectid.go` now
covers the full ID width as well as a bare body, since both are reported
through it.

No existing code paths are touched; nothing calls the new types yet.

## API

```go
type ObjectKind uint8
const ( KindProject ObjectKind = iota + 1; KindWorkspace )
func (ObjectKind) Prefix() string   // "prj" / "wrk", "" for the zero value
func (ObjectKind) String() string

const ObjectIDLen = 30

type ObjectID[K kind] uuid.UUID
type ProjectID   = ObjectID[projectKind]
type WorkspaceID = ObjectID[workspaceKind]

func (ObjectID[K]) Kind() ObjectKind
func (ObjectID[K]) String() string

func ParseProjectID(string)   (ProjectID, error)
func ParseWorkspaceID(string) (WorkspaceID, error)
func MustParseProjectID(string)   ProjectID
func MustParseWorkspaceID(string) WorkspaceID

func ConvertTeamIDToProjectID(uuid.UUID) ProjectID
```

## The kind lives in the type

`ObjectID` is generic over a phantom type parameter that names one
`ObjectKind`, and `ProjectID` / `WorkspaceID` are that generic
instantiated. They are therefore distinct types that share one copy of
the parsing and formatting:

```go
var p ProjectID
var w WorkspaceID
w = p   // cannot use p (variable of array type ProjectID) as WorkspaceID value
```

The constraint's method is unexported, so the set of kinds is closed to
this package — no other package can instantiate `ObjectID` with a kind
of its own.

The underlying type is `uuid.UUID`, so getting in and out is a
conversion and needs no constructor and no accessor:

```go
pid := ProjectID(uuid.Must(uuid.NewV7()))
u := uuid.UUID(pid)
```

Conversion between two `ObjectID`s compiles too, their underlying types
being identical. That is deliberate: crossing kinds should be possible,
but only by writing it down.

## The width is fixed, and enforced

Every ID is exactly `ObjectIDLen` = 30 characters: `prefixLen` (3) for
the tag, one for the separator, `encodedLen` (26) for the body. Nothing
about it varies by kind, by UUID version, or over time, so a storage
column can be a fixed 30.

Holding the width means holding the prefixes to three characters, which
the compiler does rather than a test:

```go
var (
	_ [prefixLen]byte = [len(projectPrefix)]byte{}
	_ [prefixLen]byte = [len(workspacePrefix)]byte{}
)
```

A four-character prefix gives `cannot use [4]byte{} as [3]byte value`; a
two-character one gives `[2]byte`. Both were checked by temporarily
breaking the constants. Adding a kind means adding a line here, and
forgetting is a build failure.

Because the width is fixed, `parseObjectID` reads by position instead of
searching for the separator:

```go
if len(s) != ObjectIDLen         { ... ErrBadLength }
if s[separatorIndex] != prefixSeparator { ... ErrNoPrefix }
if prefix := s[:prefixLen]; prefix != want.Prefix() { ... ErrWrongKind }
u, err := Decode(s[bodyIndex:])
```

The width gate runs first, so both slices are always in range and
`Decode`'s own length check can no longer fire. A 30-character string
whose underscore is somewhere other than index 3 is rejected, which a
`strings.Cut` version would have accepted the prefix of.

## Errors

Two new sentinels, `ErrNoPrefix` and `ErrWrongKind`, joining
`ErrBadLength` and `ErrNotCanonical` from the codec. All four are
wrapped, so `errors.Is` works on any of them; the body's own errors are
wrapped rather than restated, being more precise than anything the outer
layer knows. A rejected string returns the zero ID, not a partly filled
one.

The point of the tag is the message it makes possible: `ParseProjectID`
on a workspace ID reports `id: wrong kind: "wrk_..." is a "wrk" id, want
"prj"`, naming both sides, rather than becoming a lookup that finds
nothing.

## `ConvertTeamIDToProjectID`

A project is a row in `public.teams` — the same identity under two
names, one internal and older than the other
(`docs/ARCHITECTURE.md:214`). The function is a conversion: it cannot
fail and moves no bits. It is spelled out rather than left to
`ProjectID(teamID)` so the places that cross the rename can be found by
searching for it. The inverse is `uuid.UUID(pid)` and is just as free,
so it has no named helper.

## Tests

11 tests, 164 subtests, reusing the codec suite's `goldenV7` vectors and
its `corpus` generator (`uuid.Nil`, `uuid.Max`, every power of two ±1
across all 128 bits, and thousands of random v4/v7/unversioned values):

- **Golden strings** — `ProjectID(u).String()` is `"prj_"` plus the
exact 26 characters the codec's independently computed vectors pin, and
likewise for workspaces, so the external form is held to the same
compatibility contract.
- **Width** — `ObjectIDLen == 30`, that it equals `prefixLen + 1 +
encodedLen`, that the indices follow from `prefixLen`, and that
`String()` on the extremes and a real v7 is 30 characters containing
exactly one underscore, at index 3.
- **Prefixes** — each is exactly `prefixLen` lowercase letters (`a`–`z`
only: no digit, no separator, no case to fold), they differ, and the
zero `ObjectKind` has no prefix and prints as `ObjectKind(0)` rather
than passing for one.
- **Kind coverage** — walks past the last known kind, so a kind added to
the enum but not to the test list fails rather than going silently
unchecked.
- **Round trip** — over the corpus, both kinds: whatever `String`
writes, `Parse` reads back unchanged.
- **Rejection**, 22 cases run against both parsers with the prefix
substituted, so neither can be laxer than the other: wrong widths
(empty, no separator, bare body, prefix only, body ±1, two-character
prefix, leading space, trailing newline, empty prefix, a raw UUID as the
body), right width with the separator missing or misplaced (three
cases), right width with the wrong tag (other kind, uppercase prefix,
space in prefix), and right shape with a body the codec refuses
(non-canonical, uppercase, `9`, `_`, `-`). Every case also has to name
the package in its message.
- **Cross-kind** — each parser refuses the other's ID with
`ErrWrongKind`, and the message names both kinds.
- **Zero on failure** — a rejected parse returns the nil UUID, so a
caller that ignores the error gets an obviously empty ID.
- **Type distinctness** — recorded with `reflect`, since a careless edit
could collapse both aliases onto one instantiation and nothing else
would notice.
- **`ConvertTeamIDToProjectID`** — over the corpus: the UUID survives
byte for byte, the kind is `KindProject`, and the printed form parses
back to the team ID it started as.
- **`MustParse`** — both succeed on valid input and panic on garbage and
on the wrong kind.

## Verification

- `go test -race -count=1 ./pkg/id/` — pass, including the pre-existing
`id.go` and codec suites
- `go vet ./pkg/id/` — clean
- `golangci-lint run ./pkg/id/...` — 0 issues
- Compile failures confirmed by hand for a mis-width prefix (both
directions) and for `WorkspaceID = ProjectID`

No `docs/ARCHITECTURE.md` change: this adds an unwired shared type and
alters no service, port, protocol, data store, flow, or topology.
…ere they matter (#3462)

All three compression configs ran the entire suite, so CI spent **50
runner-minutes** re-running every API-surface, egress-firewall and envd
test three times. The compression knobs (`COMPRESS_*`,
`TESTS_MEMFILE_DIFF_DEDUP_MODE`, `TESTS_DISABLE_MEMFD`) reach only four
code paths: template layer upload (`storage.UseCaseBuild`),
pause/snapshot upload (`storage.UseCasePause`), decompress-on-read in
the seekable chunk cache, and the dedup/memfd flags in `pkg/sandbox`.
Frames are a fixed 2 MiB, so the compressed writer does not care what a
layer contains. `uncompressed` still runs everything; `zstd1` and `lz4`
re-run `scripts/compression-tests.tsv` — an explicit allow-list of tests
whose subject is writing or reading back a snapshot, criteria documented
in the file. Any entry matching zero tests is a hard error, and `make
check-tests-allowlist` runs in CI to prove that guard still bites.

Measured on this PR: **50 -> 28.1 runner-minutes**, wall clock **20.1m
-> 10.9m**, end-to-end **21.5m -> 11.1m**. 166 of 227 tests no longer
re-run under compression; they still run in full under `uncompressed`.
Safety evidence: when the Alpine/musl envd break landed on #3437, all
three compression shards failed identically — compression-invariant
tests gain nothing from the extra two runs. The 26 egress-firewall tests
alone were 2560s of zstd1 time and are network-timeout bound.
tomasvarga and others added 12 commits July 30, 2026 13:44
…h telemetry (#3445)

# feat(orch): persist an fs-quiesced flag on filesystem-only pause, with
telemetry

## Why

A filesystem-only snapshot (`pause(memory:false)`) is only safe to
cold-boot — and to rewrite offline — if its rootfs was **frozen**
(`FIFREEZE`) at pause, not merely `sync`'d. The pause path already knows
which happened (`guestPrepareFsForPause` picks native `fsfreeze` for
envd ≥ 0.6.6, `fsfreeze-exec` for older envd, or a `sync` fallback), but
that fact is **thrown away**: it lives only in a transient
span/histogram attribute and is never persisted with the snapshot. So
nothing downstream can tell, for a given snapshot, whether its rootfs is
crash-consistent, and there is no fleet-level signal for *how many*
fs-only snapshots are frozen.

This is the building block for a later offline envd-upgrade feature
(separate PR): to upgrade an envd-unaware, paused sandbox by rewriting
its rootfs and cold-booting, that feature must restrict itself to
snapshots it *knows* were frozen — otherwise it would have to repair a
possibly-torn journal. This PR records and measures the fact; it does
not consume it.

## What

**Persist the flag in the snapshot's own metadata.** A new `FsQuiesced
bool` field (`json:"fs_quiesced,omitempty"`) on `metadata.Template`,
with `IsFsQuiesced()` / `MarkFsQuiesced()` helpers, sitting right next
to `filesystem_only` in the per-snapshot `metadata.json`.
`guestPrepareFsForPause` now returns whether a real freeze ran (`true`
for both the native and the exec-API paths; `false` only for the `sync`
fallback), and `Sandbox.Pause` stamps
`m.MarkFsQuiesced(filesystemSnapshot && frozen)`. It is re-stamped on
every pause, so it never goes stale, and is deliberately *not* carried
by the copy-constructors (like `filesystem_only`).

**Telemetry** — two signals so the flag is observable both in aggregate
and per-pause:

| signal | where | what it shows |
|---|---|---|
| `orchestrator.sandbox.pause.fs_quiesced{quiesced=true\|false}` counter
| Mimir | count of fs-only pauses split by frozen vs sync — the
eligible-snapshot population |
| `fs_quiesced` bool attribute on the `sandbox-snapshot` span | Tempo
(sampled) | per-pause value, for inspecting individual snapshots |

The counter increments only on fs-only pauses, so the headline query is
the eligible fraction:

```promql
sum(rate(orchestrator_sandbox_pause_fs_quiesced_total{quiesced="true"}[$__rate_interval]))
  / sum(rate(orchestrator_sandbox_pause_fs_quiesced_total[$__rate_interval]))
```

The counter is registered in `telemetry` (`meters.go`) with description
+ unit map entries, guarded by a meter-map test.

## Validation

- **Unit** (`go test`, orchestrator + shared): metadata round-trip test
asserting `fs_quiesced` survives serialize→deserialize on a
filesystem-only snapshot (no dedicated metadata version needed), that a
sync-fallback fs-only snapshot stays not-quiesced, and that a legacy
snapshot without the field deserializes as not-quiesced (the safe
default); the `#3420` fsfreeze-pause test adapted + strengthened to
assert the new `frozen` return (`true` on a successful native freeze,
`false` on an aborted one); a meter-map guard test. Build + `go vet`
clean, `gofmt` clean.
- **Dev-cluster e2e:** deployed this build to a dev orchestrator node,
funneled placement to it, created a sandbox (guest envd 0.6.13, native
`FIFREEZE`), and did a real `memory:false` pause. The metric appeared in
dev Mimir through the actual OTEL pipeline: `sum by
(quiesced)(orchestrator_sandbox_pause_fs_quiesced_total)` →
`{quiesced="true"} = 1`. Reverted afterwards.

## Key decisions

- **No new metadata version.** The field is only meaningful on fs-only
snapshots, which `MarkFilesystemOnly` already stamps at `>=
FilesystemOnlyVersion` — a version `deserialize()` fully unmarshals — so
the flag survives without its own version bump. Verified by the
round-trip test.
- **Backward-compatible by construction.** `omitempty` + the `false`
zero value mean a pre-existing/legacy snapshot (no field) reads as *not*
quiesced. For the future consumer that is the safe "not eligible, wait
for a fresh freezing pause" default; no migration.
- **Both freeze paths count as quiesced.** `quiesced=true` means "a real
`FIFREEZE` ran," native (≥0.6.6) or exec-API (<0.6.6) alike — both yield
an equally crash-consistent rootfs; only `sync` is `false`. The
native-vs-exec split remains available on the existing
`guest_sync.duration` `method` label, so it is not duplicated here.
- **Signature change:** `guestPrepareFsForPause` now returns `(frozen
bool, err error)`; its sole caller (`Sandbox.Pause`) and the existing
fsfreeze test are updated in this PR.
- **Deliberately scoped to record + measure.** The consumer of the flag
— the offline rootfs `envd` swap, the reboot-time gate, and the
version-remap flag — is intentionally *not* in this PR; it lands
separately on top of this building block.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Nikita Kalyazin <nikita.kalyazin@e2b.dev>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Babis Chalios <mail@bchalios.io>
…ub (#3477)

`build.sh` can already build the premade NixOS base image, but it
defaults to the local dev registry, so nothing pullable exists. This
adds a `workflow_dispatch` job that runs the same script on a hosted
runner (nix runs inside the `nixos/nix` container, so no KVM) and pushes
`docker.io/e2bdev/nixos:<tag>`. The namespace is its own input rather
than derived from `DOCKERHUB_USERNAME`, keeping the credential and the
image path independent; the job fails up front naming a missing secret.
`tag` is required and must be unused, and the rootfs tar is verified
before any push (`ID=nixos`, the boot symlinks and their targets, the
toplevel closure, a non-empty `db-registration`). The unused-tag guard
fails closed: a 429/5xx/DNS error is retried and then aborts rather than
being read as "tag is free".

`:latest` is tagged only after those checks, so it always names a
verified build — but it is a convenience pointer, not a reproducible
reference: `phases/base/hash.go` keys the base layer on the literal
image string, not the resolved digest, so a template built `FROM
e2bdev/nixos:latest` keeps its cache key when `latest` moves and goes on
using the stale base layer until force-rebuilt. The immutable `:<tag>`
remains the reproducible reference.

**Ordering — this merges, but don't publish from it yet.** `build.sh`
currently builds from `channel:nixos-24.05`, whose nixpkgs branch last
moved 2024-12-30; it is long EOL and receives no security updates.
Merging here is harmless (dispatch-only — it publishes nothing until
someone runs it). The 26.05 bump lands separately, and **the first tag
ever pushed to `e2bdev/nixos` should be built from a supported
release.** Publishing an EOL base as that repo's first image would
repeat E2B#1625 (`from_fedora_image` defaulting to an EOL Fedora 42) one
layer down.
#3478)

The premade NixOS base image built from `channel:nixos-24.05`, whose
nixpkgs branch stopped receiving commits on **2024-12-30** — long EOL.
Same defect class as E2B#1625 (`from_fedora_image` defaulting to an EOL
Fedora), one layer down, and it would otherwise have been the first tag
ever pushed to `e2bdev/nixos`. This pins an exact channel release
instead of a channel name: a channel resolves at build time, so the same
commit evaluated to a different closure every run. The release URL is
immutable, so it is a real pin. Bumping `NIXOS_SERIES`/`NIXPKGS_RELEASE`
is now the whole maintenance story.

Pinning alone doesn't boot. Up to 24.11 `$toplevel/init` was the stage-2
script — it mounted `/proc`, ran `activate` to populate `/etc`, then
exec'd systemd. From 25.05 that file **is** the systemd binary
(md5-identical to `systemd-260.2/lib/systemd/systemd`); activation moved
into the stage-1 initrd. We boot the rootfs directly with no initrd, so
PID 1 came up against an empty `/etc` and froze on `Unit default.target
not found`. The image now ships `/sbin/e2b-nixos-init`, which does what
stage 2 did, and the `nixos` profile points `InitBinary` at it. Two
silent traps: it needs an explicit `PATH` (no FHS userland
pre-activation, so `mount`/`install`/`ln` aren't found), and it must
mount `/proc` before activating — `nix-store` reads `/proc/self/exe`, so
the non-fatal `e2bNixDb` snippet was failing and leaving the store DB
unloaded.

Also fixes the pre-activation `/etc/os-release`, which still hardcoded
`24.05` — invisible to every check, since `ID` is the only field read.

**Verified on real KVM (x86_64, KVM slot): C1–C11 all pass on 26.05** —
envd active and 49983 connected, unit parity + GOMEMLIMIT, chrony, sshd,
CA bundle + `https_code=200`, default user + NOPASSWD sudo, hostname,
firewall clear, **C9 `check_validity=OK` / 568 valid paths**,
`os_release=nixos/NixOS 26.05 (Yarara)`, and C11 raw-envd exec resolving
git/jq/sh with reclaim and guestSync replicas `rc=0`. The container
build reproduced the host preflight toplevel byte-for-byte.

> Includes a merge of `main` to pick up #3477, whose rootfs check
asserted the old `sbin/init` target; the assertion now covers the shim.
)

## What

Rebase + extension of #3235 (cc @dobrac), squashed onto current main
after #3462 landed. Opened as a **draft for a measuring run** — the
shard timings from this PR's CI decide the final shard balance.

Two things changed since #3235 was written, and this branch reconciles
both:

- #3462 already landed the coverage philosophy (uncompressed runs the
whole suite once; zstd1/lz4 only re-run the allow-list). This rebase
**keeps that decision** and applies #3235's sharding on top — PRs now
shard the *uncompressed* config, not zstd1.
- The distro-family template tests (#3437/#3444) landed into what is now
templates shard 2, so the name-prefix split needs rebalancing from this
run's junits.

## PR-path matrix (8 jobs; push to main unchanged: 3 unsharded configs)

| Job | Shard | Runs |
|---|---|---|
| uncompressed-templates-1/2 | `^TestTemplateBuild` split by name prefix
| 11 + 9 real-build tests |
| uncompressed-sandboxes | sandboxes + metrics + volumes + proxies | 121
tests |
| uncompressed-rest | everything else, `-skip '^TestTemplateBuild'` | 86
tests |
| zstd1-templates / lz4-templates | templates package ∩ allow-list | 6
snapshot-build entries |
| zstd1-other / lz4-other | package complement ∩ allow-list | 33 entries
|

Static partition verified: 227 top-level tests land exactly once across
the uncompressed shards; templates/no-templates is an exact package
complement. `TESTS_ONLY` composes only with package-pure shards (the
name-split shards' own `-run` would collide — guarded, fails loudly).
Unknown shard names fail loudly instead of silently running the whole
suite.

## Carried from #3235 (unchanged in spirit)

- DB containers start right after checkout and overlap the Go builds
(`start-databases`); `start-services` waits with bounded health loops.
Docker Hub pre-pull retries (#3410) folded in, now covering the pinned
tags + otel.
- ClickHouse migrations via goose on the host (`migrate-host`) — no
migrator image build.
- Salted optimized Go cache; `DEBUG_GCFLAGS` opt-out (race stays on, `-N
-l` off in CI). envd keeps the musl-static link from #3444, gcflags
parametrized.
- 7 metadata/authz template tests use `RequestTemplateWithoutBuild`;
envd process tests poll instead of fixed sleeps.

## Expected (to be confirmed by this run)

Baseline on main (run 30626752076): integration jobs 8.4–9.9 min, PR
wall ~10–12 min. Expected here: **~5–5.5 min wall**, long pole likely
uncompressed-templates-2 (it inherited the distro suite). Follow-ups
after measurement: rebalance `TEMPLATE_BUILDS_SHARD1_RE`, cache the
built sandbox template keyed on inputs.

Relationship to #3235: this supersedes the mechanical parts; adopt into
the original PR or take this one over — @dobrac's call.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…time (#3440)

`provision.sh` probed `/dev/ptp0` at build time and baked either the PHC
refclock or the NTP pool into `chrony.conf`, assuming provisioning and
runtime share a host — build and sandbox nodes are separate pools, so
that probe doesn't belong in a durable template artifact (codex P1 on
#3411). The source line is now written on every boot by
`e2b-chrony-source` (systemd oneshot; OpenRC boot service on Alpine) and
pulled in with an `include`, mirroring what the NixOS base image already
does.

Rebased over #3453, whose seccomp workaround keys off exactly the
build-time PHC verdict this PR deletes — left alone it never fires, and
under `set -eu` the now-unset var breaks every distro's build. Alpine's
`-F 0` is therefore unconditional in the OpenRC init setup, which is
free: on the pool line chronyd runs fine under the filter, while against
a real PHC (an `igc` NIC clock) `-F 1` gives "Loaded seccomp filter
(level 1)" then "Bad system call", and `-F 1 -F 0` selects PHC0 at
±55ns. Alpine and ubuntu re-verified on real KVM — 7/7 each, plus 7/7
seccomp checks. Cold boots only: memory-resumed sandboxes keep their
snapshot's chronyd. Rollout needs a `build-provision-version` bump.
Adds the overlay primitives the in-place background rootfs seal needs, on top of
main's #3320 deferred-export code:

- Overlay: atomic writable-cache pointer + a "sealing" slot. SwapCache installs
  a fresh writable cache and moves the previous one to the sealing slot; ReadAt
  resolves writable -> sealing -> base. FoldSealing / ReleaseSealing collapse the
  sealing cache back once its background seal is done.
- Cache.FillMissingFrom copies the sealing cache's blocks the writable cache
  lacks (concurrency-safe writeAtIfAbsent keeps the guest's newer writes), so the
  writable cache is a complete diff again after a fold.

Cache.DiffMetadata already exists on main (from #3320) and is reused. Purely
additive; overlay.go was untouched by #3320.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>
Adds the NBD provider capabilities the in-place background seal needs, alongside
main's #3320 PrepareExportDiff (destroy-path eject+stop):

- ExportDiffInPlace: reflink the overlay's dirty blocks without ejecting the
  cache, so the sandbox keeps running on it (synchronous in-place fallback).
- SwapForBackgroundSeal: flush the device, swap a fresh writable cache onto the
  live overlay, return the frozen previous cache for background sealing.
- FoldSealed: fold the sealing cache back into the writable cache and detach it.

DirectProvider stubs all three (NBD-only). Reuses main's ejectAndStopSandbox.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>
Enable resuming the same Firecracker VM after a memory snapshot, with its memory
backing intact:

- Process.ResumeInPlace un-pauses the already-running FC process (no reconfigure,
  no UFFD handshake, no snapshot load) — the process/memory/rootfs are all live.
- Memory.PeekMemfd borrows the memfd without consuming it (Uffd loads instead of
  swapping; NoopMemory returns nil).
- block.NewCacheFromMemfdKeepOpen copies the dirty guest pages WITHOUT closing
  the memfd; ExportMemory + pauseProcessMemory + processMemorySnapshot gain a
  keepMemfdOpen flag that takes this path and disables dedup / inflight-serve /
  provisional serving (which would consume or double-manage the fd the live guest
  still faults on).

keepMemfdOpen is threaded but always false so far (Pause passes false) — no
behavior change until the in-place branch wires it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>
…ckpoint

Wire the in-place path on top of main's deferred-export machinery, gated by the
existing defer-rootfs-export flag (no new flag):

- pauseOptions gains maintainSandbox (WithMaintainSandbox). Pause registers a
  resume-on-error, snapshots memory with keepMemfdOpen, waits out any prior
  in-place seal, then after writing metadata ResumeInPlace + unfreeze/thaw +
  restart checks BEFORE starting the (possibly deferred) rootfs seal, so the
  reflink stays off the resume critical path.
- processRootfsSnapshot gets a maintainSandbox branch: with defer-rootfs-export
  it swaps a fresh COW cache (SwapForBackgroundSeal) and runs runInPlaceRootfsExport
  in the background (reflink -> resolve deferred diff -> FoldSealed + release,
  reusing main's sealCacheToDiff); without the flag it exports in place
  synchronously (RootfsDiffCreator with nil closeHook -> ExportDiffInPlace). A
  provider that can't swap (DirectProvider -> ErrDeferredExportNotSupported) falls
  back to the synchronous in-place export.
- waitForRootfsSeal serializes the next in-place checkpoint behind the prior
  seal's fold so the writable cache is a complete diff before it swaps again.
- Checkpoint handler resumes in place: drop MarkStopping/stopSandboxAsync and the
  resume-fresh (GetTemplate/ResumeSandbox/lifecycle/upgrade/markLive) path; the
  original sandbox keeps running and the upload targets it. Prefetch is omitted
  (live tracker over-prefetches). snapshotAndCacheSandbox gains maintainSandbox.

The base device stays on template T (no rebase), so the live sandbox never
depends on the not-yet-durable checkpoint build. fs-only in-place checkpoint
(proto/API) is a separate follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>
Wire a filesystem_only flag through the Checkpoint gRPC handler. Unlike the
earlier destroy-and-resume-fresh design, the in-place checkpoint needs no new
mode: fs-only checkpoint is just Pause with WithFilesystemSnapshot() +
WithMaintainSandbox() together, which the current Pause already handles — it
fsfreeze-quiesces the guest, skips the memory snapshot (mem stays NoDiff, so no
memfile is produced or uploaded), seals the rootfs in the background, resumes the
same VM in place, and thaws it (bestEffortFsthaw is already gated on envd
fsfreeze support, so no 404 teardown on old envd). The persisted template is
marked filesystem-only and cold-boots (reboots) on launch; the live sandbox keeps
its memory and running processes.

Only the handler flag mapping changes (was hardcoded false); prefetch is already
omitted for in-place checkpoints. No SnapshotMode enum, no build_upload guards.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>
Add an optional `memory` boolean (default true) to SandboxSnapshotRequest. When
false, PostSandboxesSandboxIDSnapshots derives FilesystemOnly and threads it
through SnapshotTemplateOpts -> the upsert params and the orchestrator
SandboxCheckpointRequest, producing a rootfs-only template whose sandboxes
cold-boot (reboot) from disk. Default behavior (full memory snapshot) is
unchanged. Regenerated api.gen.go and the integration client.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>
@cla-bot cla-bot Bot added the cla-signed label Jul 31, 2026
@cursor

cursor Bot commented Jul 31, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Snapshot/checkpoint semantics and volume create ordering touch production sandbox and storage paths; auth cache invalidation changes affect authorization freshness but are bounded and synchronous.

Overview
This PR extends snapshot and checkpoint flows so callers can persist only the rootfs (cold-boot on launch) via an optional memory flag on snapshot requests, wired through the API orchestrator path that previously always took full memory snapshots. Sandbox create, resume, and fork now share validation that rejects zero or negative timeouts before any orchestrator work.

Integration CI is reworked to start databases in the background, build Go packages in parallel with a dedicated optimized build cache, shard the uncompressed suite across runners on PRs (while main keeps three compression configs unsharded), and run ClickHouse migrations on the host instead of building a migrator image. Publish and release paths gain docker-bake for api/db-migrator, several binary components to GCS, a manual NixOS base-image workflow with pre-push tar checks, and release-please mappings for more packages.

On the API side, volume creation creates the orchestrator directory first (with optional type resolution from the orchestrator response) then inserts the DB row, with per-region default volume types derived from ready node region= labels when configured. Auth generalizes service JWT verification (JWKSVerifier), expands team cache invalidation to member session keys, and adds configurable authenticators plus OIDC identity verification without a user lookup.

Reviewed by Cursor Bugbot for commit fc71874. Bugbot is set up for automated code reviews on this repo. Configure here.

@bchalios
bchalios force-pushed the feat/in-place-resume branch from d4b28b7 to 20bbe96 Compare July 31, 2026 16:52
s.bestEffortUnfreeze(ctx)
if pauseOpts.filesystemSnapshot {
s.bestEffortFsthaw(ctx)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In-place fs thaw skips exec path

High Severity

On a successful filesystem-only in-place checkpoint, only bestEffortFsthaw runs. That no-ops when envd lacks native /fsfreeze and the rootfs was frozen via the exec path. Cleanup thaw is error-only, so the live guest can stay FIFREEZE'd after resume.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 20bbe96. Configure here.


return s.process.ResumeInPlace(ctx)
})
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checks not restarted after error resume

Medium Severity

Checks.Stop() runs at the start of Pause, but NewChecks/Start only run on the in-place success path. If snapshotting fails after FC pause, cleanup resumes the VM via ResumeInPlace and leaves health checks permanently stopped.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 20bbe96. Configure here.

Comment thread packages/orchestrator/pkg/sandbox/sandbox.go
…ecks)

Three fixes from the PR #3483 review of the in-place checkpoint path:

- Memfd consumed before peek (High): processMemorySnapshot called Memfd()
  unconditionally, which swaps the memfd out of uffd, so the keepMemfdOpen
  branch's PeekMemfd returned nil (broke the in-place keep-open export and
  leaked/dropped the taken fd). Now the in-place path uses PeekMemfd (borrow)
  and only the destroy path consumes via Memfd.
- In-place fs-thaw skips exec path (High): the success resume only ran the
  native bestEffortFsthaw, a no-op when the rootfs was frozen via the exec path
  (old envd) — leaving the live in-place guest's filesystem frozen. Now thaws to
  match guestPrepareFsForPause: native /fsthaw when supported, else exec
  fsfreeze -u. (The prior "frozen state discarded when the VM stops" assumption
  doesn't hold for in-place resume.)
- Checks not restarted after error resume (Medium): the resumeOnError cleanup
  resumed the VM but left health checks (stopped at Pause start) off. It now
  restarts NewChecks/Start after ResumeInPlace.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>
SandboxId: sbx.SandboxID,
BuildId: upsertResult.BuildID.String(),
Metadata: map[string]string{storageopts.ObjectMetadataTemplateID: snapshotTemplateEnvID},
FilesystemOnly: opts.FilesystemOnly,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checkpoint errors kill live sandbox

High Severity

In-place checkpoint keeps the sandbox running when upload (or other post-resume steps) fail, but the API still RemoveSandboxs on Checkpoint errors. That kills a healthy resumed sandbox after a template persistence failure.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit cefcc89. Configure here.

Spike benchmarks for the always-sync WP memory-CoW design. All are
root+hugepage gated and skip otherwise:

- TestSyncWPFaultLatency: per-fault round-trip for synchronous WP
  (WP_ASYNC off) on 2 MiB hugetlbfs, tracking-only handler.
- TestAsyncWPWriteLatency: the current WP_ASYNC mechanism (in-kernel
  clear, no handler) + pagemap dirty-set readout cost, for comparison.
- TestSyncWPConcurrentLatency: sweeps N concurrent writers (vCPUs)
  against a production-style reader + worker fan-out handler.

Measured on a dev-cluster orchestrator node (6.17-gcp): sync WP ~18us
p50 vs async ~0.9us; concurrent p50 degrades ~3.3x to ~120us at 8
writers with no throughput cliff.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>
@jakubno
jakubno force-pushed the feat/in-place-resume branch from cefcc89 to 2053638 Compare August 3, 2026 13:33
jakubno pushed a commit that referenced this pull request Aug 3, 2026
…ecks)

Three fixes from the PR #3483 review of the in-place checkpoint path:

- Memfd consumed before peek (High): processMemorySnapshot called Memfd()
  unconditionally, which swaps the memfd out of uffd, so the keepMemfdOpen
  branch's PeekMemfd returned nil (broke the in-place keep-open export and
  leaked/dropped the taken fd). Now the in-place path uses PeekMemfd (borrow)
  and only the destroy path consumes via Memfd.
- In-place fs-thaw skips exec path (High): the success resume only ran the
  native bestEffortFsthaw, a no-op when the rootfs was frozen via the exec path
  (old envd) — leaving the live in-place guest's filesystem frozen. Now thaws to
  match guestPrepareFsForPause: native /fsthaw when supported, else exec
  fsfreeze -u. (The prior "frozen state discarded when the VM stops" assumption
  doesn't hold for in-place resume.)
- Checks not restarted after error resume (Medium): the resumeOnError cleanup
  resumed the VM but left health checks (stopped at Pause start) off. It now
  restarts NewChecks/Start after ResumeInPlace.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>
Comment thread packages/orchestrator/pkg/sandbox/sandbox.go
@bchalios
bchalios force-pushed the feat/in-place-resume branch from 2053638 to fce9b16 Compare August 3, 2026 16:21

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 4 total unresolved issues (including 3 from previous reviews).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit fce9b16. Configure here.

"packages/envd": "0.0.1",
"packages/orchestrator": "0.0.1",
"packages/envd": "0.6.13",
"packages/orchestrator": "0.0.1",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate release-please manifest keys

High Severity

packages/envd and packages/orchestrator are each listed twice. Duplicate JSON keys are undefined across parsers, so release-please can pick 0.0.1 instead of the real envd version 0.6.13 and mis-tag or re-release components.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit fce9b16. Configure here.

For a sync-WP Firecracker build (guest RAM registered WITHOUT
UFFD_FEATURE_WP_ASYNC), a userfault write-protect event is no longer
fatal: resolveWriteProtect marks the page dirty and clears the
protection (waking the blocked writer). Tracking-only for now — the
copy-before-unprotect (background memory export) lands later.

Unconditional, no runtime toggle: with a WP_ASYNC Firecracker no WP
events are ever delivered, so this path is inert.

Validated on a dev-cluster node (kernel 6.17-gcp) via
TestSyncWPResolveDirectFault.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Babis Chalios <babis.chalios@e2b.dev>
@bchalios
bchalios force-pushed the feat/in-place-resume branch from fce9b16 to fc71874 Compare August 3, 2026 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.