Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
that collapse is what let an unparseable version fall through as a pass.

### Fixed
- **`source_date_epoch` is honored outside extension images.** The key was
read only by `ext image`, which exports it inside its own script. The rootfs
build script has always passed `-T "${SOURCE_DATE_EPOCH:-0}"` to
`mkfs.erofs`, but nothing ever set the variable, so a project that
configured an epoch got a reproducibility stamp on its `.raw` extensions and
a silently ignored key everywhere else — including in `post_build` hooks,
which run in their own container. Rootfs, initramfs and `post_build` runs

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The post_build claim covers only one of the two post_build hooks

"Rootfs, initramfs and post_build runs now all carry it" reads as unqualified, but there are two run_post_build implementations in this tree and only the runtime one was wired. The extension hook (src/commands/ext/build.rs:1473) still passes bare env_vars: self.runtime_env_vars() at :1520 with no injection.

The runtime side is genuinely wired end to end, to be clear - config.source_date_epoch reaches run_post_build (runtime/build.rs:743), is injected into that container's env map (:2890), and RunConfig.env_vars reaches the container as -e KEY=VALUE on both the local (container.rs:1450) and runs_on remote (:1510) paths, carrying the same value the build run at :541 uses.

Failure path: avocado.yaml sets source_date_epoch: 1700000000; an extension declares post_build: scripts/bake.sh that gzips or tars a generated file into the ext sysroot. Per ext/build.rs:705 that hook runs before the .raw is sealed, so wall-clock timestamps land in the artifact's content - gzip and tar headers, .pyc - which mkfs.erofs -T cannot normalize afterwards. The .raw then differs build to build while this entry tells the user post_build is covered.

Within one avocado build, Phase 1 runs the ext post_build hook with no epoch and ext image then seals the .raw with one, so the value is not consistent across hooks in a single build. The ext hook already holds config: &Config in scope, so this is a one-line wiring rather than a plumbing constraint.

now all carry it.

Note the key still behaves two ways by design: absent config leaves the
variable unset for rootfs/initramfs/`post_build`, because
`SOURCE_DATE_EPOCH` changes the behavior of tools well beyond ours (gzip,
tar, python bytecode) and hooks should not inherit that unasked. `ext image`
continues to export `0`.
- **Rootfs and initramfs no longer reinstall on every run.** `avocado sdk
install` wiped and rebuilt both sysroots from scratch on every invocation,
even with nothing changed. Removal detection compared the lockfile against
Expand Down
5 changes: 5 additions & 0 deletions src/commands/initramfs/image.rs
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,11 @@ export AVOCADO_OS_VERSION_ID
if wrap_kab {
env_vars.insert("KAB_KEYSET_FILE".to_string(), "/tmp/kab.keyset".to_string());
}
// Reproducibility stamp. Inert for now — the generated initramfs script
// has no reader yet, since the mtime-normalization step that consumes
// the epoch lands separately. Set anyway so every image-building run
// carries the same env.
crate::utils::container::inject_source_date_epoch(&mut env_vars, config.source_date_epoch);

let container_args_with_keyset = if let Some(ref host_path) = kab_keyset_host_path {
let mut args = merged_container_args.clone().unwrap_or_default();
Expand Down
43 changes: 43 additions & 0 deletions src/commands/rootfs/image.rs
Original file line number Diff line number Diff line change
Expand Up @@ -398,6 +398,8 @@ export AVOCADO_OS_VERSION_ID
if wrap_kab {
env_vars.insert("KAB_KEYSET_FILE".to_string(), "/tmp/kab.keyset".to_string());
}
// Reproducibility stamp for `mkfs.erofs -T` in the build script above.
crate::utils::container::inject_source_date_epoch(&mut env_vars, config.source_date_epoch);

// Bind-mount the keyset into the container as a single -v arg
// appended to whatever the user / config already has.
Expand Down Expand Up @@ -517,6 +519,47 @@ export AVOCADO_OS_VERSION_ID
mod tests {
use super::*;

#[test]
fn test_rootfs_script_reads_source_date_epoch_from_the_env() {
// The other half of `inject_source_date_epoch`. Injection and
// consumption have to agree on the variable name, and a mismatch in
// either half fails silently — the script just falls back to 0 and the
// configured stamp is quietly ignored, which is the bug this pairing
// exists to fix. Pin the name on this side too.
let script = generate_rootfs_build_script(NAMESPACE_UUID, "erofs-lz4", None, "");
// Counted, not just `contains`. Both mkfs branches (erofs-zst and
// erofs-lz4) are emitted unconditionally, so a `contains` check passes
// even if one branch loses the flag — asserting the count is what
// actually catches that.
assert_eq!(
script.matches(r#"-T "${SOURCE_DATE_EPOCH:-0}""#).count(),
2,
"both mkfs.erofs branches must take their timestamp from $SOURCE_DATE_EPOCH"
);
}

/// The rest of the rootfs reproducibility contract. Nothing pinned these,
/// and each silently reintroduces per-build variance if dropped: the image
/// UUID would be randomized, and ownership would come from whoever ran the
/// build rather than being normalized to root.
#[test]
fn test_rootfs_image_reproducibility_flags_are_pinned() {
let script = generate_rootfs_build_script(NAMESPACE_UUID, "erofs-lz4", None, "");

assert_eq!(
script
.matches("-U 00000000-0000-0000-0000-000000000000")
.count(),
2,
"both mkfs.erofs branches must pin the image UUID"
);
assert_eq!(
script.matches("--all-root").count(),
2,
"both mkfs.erofs branches must normalize ownership to root"
);
}

#[test]
fn test_rootfs_script_guards_against_half_populated_sysroot() {
// A stale or interrupted build volume can leave the sysroot with
Expand Down
19 changes: 18 additions & 1 deletion src/commands/runtime/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -534,6 +534,12 @@ impl RuntimeBuildCommand {
env_vars.insert("AVOCADO_VERBOSE".to_string(), "1".to_string());
}

// Reproducibility stamp. This run executes the rootfs and initramfs
// image sections, so one insert covers both. Extension images are NOT
// built here -- this run only copies their pre-built artifacts, and
// `ext image` exports the epoch inside its own script.
crate::utils::container::inject_source_date_epoch(&mut env_vars, config.source_date_epoch);

if let Some(stone_paths) = config.get_stone_include_paths_for_runtime(
&self.runtime_name,
target_arch,
Expand Down Expand Up @@ -734,6 +740,7 @@ impl RuntimeBuildCommand {
merged_container_args,
container_helper,
runs_on_context,
config.source_date_epoch,
)
.await?;
}
Expand Down Expand Up @@ -2829,6 +2836,7 @@ rpm --root="$AVOCADO_EXT_SYSROOTS/{ext_name}" --dbpath=/var/lib/extension.d/rpm
/// - `AVOCADO_RUNTIME_BUILD_DIR`: `/opt/_avocado/<target>/runtimes/<runtime>` —
/// the runtime build dir the script can inspect or post-process.
#[allow(clippy::too_many_arguments)]
#[allow(clippy::too_many_arguments)]
async fn run_post_build(
&self,
script_path: &str,
Expand All @@ -2839,6 +2847,7 @@ rpm --root="$AVOCADO_EXT_SYSROOTS/{ext_name}" --dbpath=/var/lib/extension.d/rpm
merged_container_args: &Option<Vec<String>>,
container_helper: &SdkContainer,
runs_on_context: Option<&RunsOnContext>,
source_date_epoch: Option<u64>,
) -> Result<()> {
print_info(
&format!(
Expand Down Expand Up @@ -2872,7 +2881,15 @@ rpm --root="$AVOCADO_EXT_SYSROOTS/{ext_name}" --dbpath=/var/lib/extension.d/rpm
dnf_args: self.dnf_args.clone(),
sdk_arch: self.sdk_arch.clone(),
tui_context: self.tui_context.clone(),
env_vars: self.runtime_env_vars(),
env_vars: {
// post_build runs in its own container, so it needs its own
// injection -- the build run's env does not reach it. A hook
// that produces an artifact gets the same stamp the image
// steps do, rather than silently building against wall clock.
let mut env = self.runtime_env_vars().unwrap_or_default();
crate::utils::container::inject_source_date_epoch(&mut env, source_date_epoch);
Some(env)
},
..Default::default()
};

Expand Down
61 changes: 61 additions & 0 deletions src/utils/container.rs
Original file line number Diff line number Diff line change
Expand Up @@ -513,6 +513,26 @@ pub fn inject_repo_tls_env(env_vars: &mut std::collections::HashMap<String, Stri
}
}

/// Inject `SOURCE_DATE_EPOCH` into a container's env map when the project sets it.
///
/// Read today by the rootfs script's `mkfs.erofs -T "${SOURCE_DATE_EPOCH:-0}"`.
/// The initramfs path exports it too, inert until its mtime-normalization step
/// lands separately.
///
/// Left unset when the key is absent rather than defaulted to 0, because
/// `SOURCE_DATE_EPOCH` is honored by plenty of tools besides ours (gzip, tar,
/// python bytecode) and would change post_install behavior for projects that
/// never opted in. `ext image` differs — it exports `unwrap_or(0)` inside its
/// own script, so the key behaves one way there and another way here.
pub fn inject_source_date_epoch(
env_vars: &mut std::collections::HashMap<String, String>,
source_date_epoch: Option<u64>,
) {
if let Some(epoch) = source_date_epoch {
env_vars.insert("SOURCE_DATE_EPOCH".to_string(), epoch.to_string());
}
}

/// Configuration for running commands in containers
#[derive(Debug, Clone)]
pub struct RunConfig {
Expand Down Expand Up @@ -3113,6 +3133,47 @@ async fn read_output_stream<R: tokio::io::AsyncRead + Unpin>(
mod tests {
use super::*;

/// Regression: `source_date_epoch` was plumbed into extension images only.
/// The rootfs script reads `${SOURCE_DATE_EPOCH:-0}` for `mkfs.erofs -T` but
/// nothing set the variable, so a project that configured it got a
/// reproducibility stamp on its `.raw` extensions and a silently ignored
/// key everywhere else.
mod source_date_epoch {
use super::*;

#[test]
fn configured_value_reaches_the_container_env() {
let mut env_vars = std::collections::HashMap::new();
inject_source_date_epoch(&mut env_vars, Some(1700000000));
assert_eq!(
env_vars.get("SOURCE_DATE_EPOCH").map(String::as_str),
Some("1700000000")
);
}

#[test]
fn zero_is_a_real_value_not_an_absent_one() {
// `Some(0)` is an explicit opt-in to the epoch, distinct from unset.
let mut env_vars = std::collections::HashMap::new();
inject_source_date_epoch(&mut env_vars, Some(0));
assert_eq!(
env_vars.get("SOURCE_DATE_EPOCH").map(String::as_str),
Some("0")
);
}

#[test]
fn unset_config_leaves_the_var_absent() {
// Not defaulted to 0 on purpose: the build scripts carry their own
// `:-0` fallback, and SOURCE_DATE_EPOCH is honored by unrelated
// tools that may run in post_install hooks. Projects that never
// opted in must see the same container env as before.
let mut env_vars = std::collections::HashMap::new();
inject_source_date_epoch(&mut env_vars, None);
assert!(!env_vars.contains_key("SOURCE_DATE_EPOCH"));
}
}

/// Wiring tests for container stdio.
///
/// `utils::interactivity` tests the *decision*; these test that the
Expand Down
47 changes: 47 additions & 0 deletions tests/source_date_epoch_wiring.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
//! Guard: the image runs actually inject `SOURCE_DATE_EPOCH`.
//!
//! Both halves of this feature are unit-tested — `inject_source_date_epoch`'s
//! three-way `Option` behavior in `utils::container`, and the rootfs script's
//! `mkfs.erofs -T "${SOURCE_DATE_EPOCH:-0}"` in `rootfs::image` — but the call
//! that connects them was covered by neither. Deleting it leaves the whole
//! suite green while a configured epoch silently stops reaching the container,
//! which is the exact regression the feature exists to prevent.
//!
//! The check lives here rather than in a `mod tests` inside those files
//! because the needle would then appear in the file it scans, and the
//! assertion would hold with the real call site deleted. (Confirmed the hard
//! way.) Same reason `no_hand_rolled_stdio_flags.rs` sits out here.
//!
//! ponytail: pins the call's spelling, not its effect. Testing the effect
//! means lifting env-map construction out of the async run path in both
//! commands; worth doing when a third image type wants the same stamp.

use std::fs;
use std::path::PathBuf;

/// The call every image-building run has to make.
const INJECTION: &str = "inject_source_date_epoch(&mut env_vars, config.source_date_epoch)";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The guard cannot see either runtime/build.rs call site, including the new one

Four call sites spell the injection - rootfs/image.rs:402, initramfs/image.rs:318, runtime/build.rs:541, runtime/build.rs:2890 - and this guard pins the first two. runtime/build.rs:541 matches the needle byte-for-byte but is never scanned, and :2890, the call this commit adds, is spelled (&mut env, source_date_epoch) so the needle can never match it.

Demonstrated rather than argued: I deleted both runtime/build.rs injections and ran the full suite - 1395/1404 unit tests plus every integration target passed, this guard included.

What that leaves exposed is the primary path. runtime/build.rs:541 feeds the runtime build script avocado build actually runs, the one interpolating the rootfs and initramfs sections (:2318, :2327). A refactor dropping it stops a configured epoch reaching the rootfs image, with the suite green throughout - verbatim the failure this file's own header says it exists to prevent. The commit's headline wiring at :2890 has zero coverage.

Credit where due: this is not the generated-text pattern - it reads the real source files, and deleting rootfs/image.rs:402 does turn it red with a clear message. Two narrower notes for the fix: it is a contains on source text, so it also passes on a commented-out call; and an allowlist of two paths only covers what someone remembers to add, where the sibling guard it cites sweeps all of src/ so new files are covered on landing.


fn source(relative: &str) -> String {
let path = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(relative);
fs::read_to_string(&path).unwrap_or_else(|e| panic!("reading {}: {e}", path.display()))
}

#[test]
fn rootfs_image_run_injects_source_date_epoch() {
assert!(
source("src/commands/rootfs/image.rs").contains(INJECTION),
"the rootfs image run must inject SOURCE_DATE_EPOCH into its container env"
);
}

#[test]
fn initramfs_image_run_injects_source_date_epoch() {
// Inert on this base — the initramfs script has no reader until the
// mtime-normalization pass lands — so a deletion here would be entirely
// invisible without this.
assert!(
source("src/commands/initramfs/image.rs").contains(INJECTION),
"the initramfs image run must inject SOURCE_DATE_EPOCH into its container env"
);
}
Loading