fix(orchestrator): restore dpkg-owned dirs under /etc/ssl/certs before cert bundle - #3528
Open
AdaAibaby wants to merge 1 commit into
Open
Conversation
AdaAibaby
requested review from
ValentaTomas,
dobrac and
jakubno
as code owners
August 4, 2026 06:09
…nalize The finalize phase boots a fresh VM to properly wire the final rootfs path. On this fresh boot e2b-seed-certs re-seeds /etc/ssl/certs from ssl-certs.tar, which does not contain package-owned subdirectories that were only ever written into a prior build-layer's tmpfs (e.g. ca-certificates-java creates /etc/ssl/certs/java/ at install time, inside the tmpfs; it never reaches the NBD-backed rootfs and therefore never ends up in the tar). When packCertBundleCmd then runs update-ca-certificates, the jks-keystore hook activated by ca-certificates-java's dpkg trigger tries to write /etc/ssl/certs/java/cacerts, fails with FileNotFoundException because the directory is missing, and dpkg records the package as half-configured (iF). update-ca-certificates swallows the hook failure and exits 0, so the build reports success and the broken state is silently baked into the snapshot. Every subsequent apt-get inside any sandbox from the template then exits 100. Fix in two layers: 1. packCertBundleCmd (configure.go): before update-ca-certificates, walk dpkg's file database and recreate any missing directories under /etc/ssl/certs. Then run dpkg --configure -a to resolve pending triggers with their required paths present. Finally, fail the build explicitly if any package is still in iF state — preventing silent broken snapshots. 2. seed-certs.sh.tpl: seed the tmpfs additively: copy the underlying rootfs directory first, then overlay the tar on top. The tar remains the authoritative source for cert files; the rootfs copy ensures subdirectories that exist on the rootfs but were absent from an older tar survive the fresh-VM boot. Defense in depth against the same class of issue. Fixes e2b-dev#3518
AdaAibaby
force-pushed
the
fix/finalize-dpkg-half-configured-under-ssl-certs
branch
from
August 4, 2026 06:16
a55992d to
aefad6b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #3518
What was broken
Any template that installs a JRE (directly or transitively) ships a snapshot where:
causing every subsequent inside those sandboxes to exit with code 100.
Root cause
Three mechanisms interact (full analysis in #3518):
Fix
** — ** (primary fix):
Before :
After :
3. Check for remaining packages and fail the build explicitly — preventing silent broken snapshots going forward
**** (defense in depth):
Seed the tmpfs additively: copy the underlying rootfs directory first, then overlay the tar on top. The tar remains authoritative for cert files; the rootfs copy ensures subdirectories present on the rootfs but absent from an older tar survive the fresh-VM boot.
Testing
Reproduced the issue (template with ) and confirmed:
/cc @jakubno @dobrac @ValentaTomas @arkamar @tvi @tomassrnka Looking forward to your code review.