Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,9 +73,8 @@ Only a scan that answers the same question is reused, which is stricter than
branch rather than a pull request, from an explicitly clean worktree, with no
scanner problems reported — and this run has to be a default whole-commit scan
itself. Anything else runs a real scan: nothing inside the window, only a failed
or still-running scan, a worktree that does not match the commit (including
files the index hides from `git status`), or a lookup the platform could not
answer.
or still-running scan, a worktree `git status` reports changes in, or a lookup
the platform could not answer.

Two things are hard errors instead. An unresolvable commit (not a git
repository, or no commits yet) exits 1 rather than silently scanning. And a run
Expand Down
2 changes: 1 addition & 1 deletion skills/corgea/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ An included image is enough on its own: when it is combined with `--only-uncommi

`--skip-if-commit-scanned-recently` reuses the project's most recent reusable scan of the current commit instead of starting a duplicate, when one ran inside the `--scanned-within` window (default `24h`; accepts `90s`, `30m`, `4h`, `7d`, and a bare number as hours). The reused scan takes the new scan's place for the rest of the command — results table, `--block-on` gate and its exit code, `--out-file` report — so the pipeline behaves the same either way. It prints `CORGEA_SCAN_SKIPPED=true` plus `CORGEA_SCAN_ID=<id>` on a reuse and `CORGEA_SCAN_SKIPPED=false` when a scan runs, so a later step can branch on it.

Reuse requires a candidate that answers the same question: a completed `corgea-blast` scan of that commit, on a branch rather than a pull request, from an explicitly clean worktree, reporting no scanner problems. Anything else runs a real scan (nothing in the window, a failed or still-running scan, a worktree that does not match the commit including files hidden from `git status`, or a failed lookup). `--ignore-dirty-worktree` (requires `--skip-if-commit-scanned-recently`) overrides the dirty-worktree half of that test: reuse proceeds even if this worktree is dirty or the prior scan recorded `worktree_dirty=true`. A prior scan that never reported the flag is still not reused. A new scan still reports the real dirty status. An unresolvable commit is a hard error (exit 1). Because the API exposes neither a scan's configured scan types and target policies nor whether it bundled a container image, a run that changes what gets scanned cannot be matched against a candidate, so the flag cannot be combined with `--scan-type`, `--policy`, `--include-image`, `--only-uncommitted`, or `--target`. `--exclude` is allowed but warns on a skip: what gets reused is a scan of the whole commit, so the results and the gate can cover files the run would have skipped (over-reporting, never under-reporting).
Reuse requires a candidate that answers the same question: a completed `corgea-blast` scan of that commit, on a branch rather than a pull request, from an explicitly clean worktree, reporting no scanner problems. Anything else runs a real scan (nothing in the window, a failed or still-running scan, a worktree `git status` reports changes in, or a failed lookup). `--ignore-dirty-worktree` (requires `--skip-if-commit-scanned-recently`) overrides the dirty-worktree half of that test: reuse proceeds even if this worktree is dirty or the prior scan recorded `worktree_dirty=true`. A prior scan that never reported the flag is still not reused. A new scan still reports the real dirty status. An unresolvable commit is a hard error (exit 1). Because the API exposes neither a scan's configured scan types and target policies nor whether it bundled a container image, a run that changes what gets scanned cannot be matched against a candidate, so the flag cannot be combined with `--scan-type`, `--policy`, `--include-image`, `--only-uncommitted`, or `--target`. `--exclude` is allowed but warns on a skip: what gets reused is a scan of the whole commit, so the results and the gate can cover files the run would have skipped (over-reporting, never under-reporting).

### Upload — `corgea upload [report]`

Expand Down
7 changes: 4 additions & 3 deletions src/scanners/blast.rs
Original file line number Diff line number Diff line change
Expand Up @@ -486,9 +486,10 @@ fn start_new_scan(
utils::terminal::set_text_color("", utils::terminal::TerminalColor::Green)
);
let repo_after = utils::generic::get_repo_info_for_scan("./").unwrap_or_default();
// Notice = visible status only (not index hide-bits / --target / SHA drift).
let worktree_dirty = repo_before.as_ref().is_some_and(|i| i.status_dirty)
|| repo_after.as_ref().is_some_and(|i| i.status_dirty);
// Notice = what `git status` shows, from the raw samples (so neither
// --target/--exclude nor SHA drift, which the upload flag also covers).
let worktree_dirty = repo_before.as_ref().is_some_and(|i| i.dirty)
|| repo_after.as_ref().is_some_and(|i| i.dirty);
if worktree_dirty {
let notice_sha = repo_after
.as_ref()
Expand Down
10 changes: 4 additions & 6 deletions src/skip_scan.rs
Original file line number Diff line number Diff line change
Expand Up @@ -125,11 +125,9 @@ pub fn resolve_reusable_scan(
exclude: Option<&str>,
ignore_dirty_worktree: bool,
) -> Option<ScanResponse> {
// `dirty`, not `status_dirty`: this asks whether the run would upload an
// exact snapshot of the commit, and that is the flag the upload itself
// sends. `status_dirty` is narrower — it is the user notice, and it cannot
// see assume-unchanged/skip-worktree files, dirty submodules, or an index
// it failed to read, all of which change what gets packaged.
// Dirtiness here is what `git status` reports, the same signal the upload
// sends and the same one the user can check for themselves before asking
// why a scan ran.
let commit = utils::generic::get_repo_info_for_scan("./")
.ok()
.flatten()
Expand All @@ -151,7 +149,7 @@ pub fn resolve_reusable_scan(
);
} else {
println!(
"Working tree does not match commit {} exactly (uncommitted changes, or files the index hides from git status), so no scan of that commit describes what would be scanned here - running a new scan.",
"Working tree does not match commit {} exactly (git status reports uncommitted changes), so no scan of that commit describes what would be scanned here - running a new scan.",
short
);
print_skipped_marker(None);
Expand Down
175 changes: 129 additions & 46 deletions src/utils/generic.rs
Original file line number Diff line number Diff line change
@@ -1,13 +1,38 @@
use crate::utils::terminal::{set_text_color, TerminalColor};
use git2::{IndexEntryExtendedFlag, IndexEntryFlag, Repository, StatusOptions};
use git2::{Repository, StatusOptions};
use globset::{Glob, GlobSetBuilder};
use ignore::WalkBuilder;
use std::env;
use std::fs::{self, File};
use std::io;
use std::path::{Path, PathBuf};
use std::process::Command;
use zip::{write::FileOptions, ZipWriter};

/// Environment variables through which git pins a subprocess to a specific
/// repository, index or config. Git exports them to hooks, so a `corgea` run
/// invoked from one would otherwise read the state the hook was handed instead
/// of the worktree it was pointed at. `deps::run` scrubs the same set for its
/// own git subprocesses; the library and binary crates share no module that
/// could hold one copy.
const GIT_LOCAL_ENV_VARS: &[&str] = &[
"GIT_ALTERNATE_OBJECT_DIRECTORIES",
"GIT_CONFIG",
"GIT_CONFIG_PARAMETERS",
"GIT_CONFIG_COUNT",
"GIT_OBJECT_DIRECTORY",
"GIT_DIR",
"GIT_WORK_TREE",
"GIT_IMPLICIT_WORK_TREE",
"GIT_GRAFT_FILE",
"GIT_INDEX_FILE",
"GIT_NO_REPLACE_OBJECTS",
"GIT_REPLACE_REF_BASE",
"GIT_PREFIX",
"GIT_SHALLOW_FILE",
"GIT_COMMON_DIR",
];

// Global exclude globs used across multiple functions
const DEFAULT_EXCLUDE_GLOBS: &[&str] = &[
"**/tests/**",
Expand Down Expand Up @@ -353,29 +378,46 @@ fn get_repo_info_inner(dir: &str, sample_dirty: bool) -> Result<Option<RepoInfo>
.map(|commit| commit.id().to_string())
});

let (dirty, status_dirty) = if sample_dirty {
worktree_dirty_flags(&repo)
} else {
(false, false)
};
let dirty = sample_dirty && worktree_is_dirty(&repo, Path::new(dir));

Ok(Some(RepoInfo {
branch,
repo_url: origin_url(&repo),
sha,
dirty,
status_dirty,
}))
}

/// `(upload_dirty, status_dirty)`.
/// `upload_dirty`: status changes, dirty submodules, or assume-unchanged /
/// skip-worktree (status hides those). Errors fail closed to dirty.
/// `status_dirty`: non-empty `statuses()` only (user notice).
fn worktree_dirty_flags(repo: &Repository) -> (bool, bool) {
let status_dirty = status_has_changes(repo);
let upload_dirty = index_hides_worktree(repo) || status_dirty;
(upload_dirty, status_dirty)
/// True when the worktree holds changes `git status` reports.
///
/// `git status` is what a user checks this answer against, so git itself is
/// asked and libgit2 only stands in when the git binary cannot answer. The two
/// disagree more often than it looks: libgit2 runs no clean filters (git-lfs
/// and friends), cannot read a sparse index, and knows nothing of a
/// `status.showUntrackedFiles` preference, and each disagreement surfaces as an
/// uncommitted change the user's own `git status` does not show. A status
/// nobody can produce still fails closed to dirty.
fn worktree_is_dirty(repo: &Repository, dir: &Path) -> bool {
git_status_has_changes(dir).unwrap_or_else(|| status_has_changes(repo))
}

/// Whether `git status` reports anything, or None when the git binary is
/// missing or the command failed - the only cases libgit2 answers instead.
fn git_status_has_changes(dir: &Path) -> Option<bool> {
let mut command = Command::new("git");
for var in GIT_LOCAL_ENV_VARS {
command.env_remove(var);
}
let output = command
// `--no-optional-locks` keeps this read from writing the user's index.
.args(["--no-optional-locks", "status", "--porcelain"])
.current_dir(dir)
.output()
.ok()?;
if !output.status.success() {
return None;
}
Some(output.stdout.iter().any(|b| !b.is_ascii_whitespace()))
}

fn status_has_changes(repo: &Repository) -> bool {
Expand All @@ -389,19 +431,6 @@ fn status_has_changes(repo: &Repository) -> bool {
.unwrap_or(true)
}

/// assume-unchanged / skip-worktree are omitted from `statuses()`.
fn index_hides_worktree(repo: &Repository) -> bool {
repo.index()
.map(|index| {
index.iter().any(|entry| {
IndexEntryFlag::from_bits_truncate(entry.flags).is_valid()
|| IndexEntryExtendedFlag::from_bits_truncate(entry.flags_extended)
.is_skip_worktree()
})
})
.unwrap_or(true)
}

/// Merge before/after packaging samples. Clean only if both exist, both clean,
/// same SHA; otherwise dirty. Prefer post-packaging branch/url/sha.
pub fn reconcile_repo_info_for_upload(
Expand All @@ -422,7 +451,6 @@ pub fn reconcile_repo_info_for_upload(
repo_url: after.repo_url.or(before.repo_url),
sha: after.sha.or(before.sha),
dirty: !stable_clean,
status_dirty: before.status_dirty || after.status_dirty,
})
}
}
Expand Down Expand Up @@ -538,8 +566,6 @@ pub struct RepoInfo {
pub sha: Option<String>,
/// Not an exact clean HEAD snapshot. Always false from [`get_repo_info`].
pub dirty: bool,
/// Non-empty git status (excludes index hide-bits). Drives user notice.
pub status_dirty: bool,
}

#[cfg(test)]
Expand All @@ -564,6 +590,21 @@ mod tests {
);
}

/// Dates a file so its recorded stat data no longer matches the index.
#[cfg(unix)]
fn touch(path: &std::path::Path) {
assert!(
Command::new("touch")
.args(["-t", "203001010000"])
.arg(path)
.status()
.unwrap()
.success(),
"touch {} failed",
path.display()
);
}

#[test]
fn get_repo_info_at_root_only_not_nested_cwd() {
let dir = tempfile::tempdir().unwrap();
Expand Down Expand Up @@ -607,7 +648,6 @@ mod tests {
.unwrap()
.expect("repo info");
assert!(info.dirty);
assert!(info.status_dirty);
}

#[test]
Expand All @@ -621,7 +661,6 @@ mod tests {
.unwrap()
.expect("repo info");
assert!(info.dirty);
assert!(info.status_dirty);
}

#[test]
Expand All @@ -634,7 +673,6 @@ mod tests {
.unwrap()
.expect("repo info");
assert!(info.dirty);
assert!(info.status_dirty);
}

#[test]
Expand All @@ -650,35 +688,70 @@ mod tests {
.unwrap()
.expect("repo info");
assert!(!info.dirty);
assert!(!info.status_dirty);
}

/// `git status` shows nothing for an assume-unchanged edit, so neither does
/// the CLI: the reported state is the one the user can check.
#[test]
fn get_repo_info_for_scan_dirty_when_assume_unchanged_hides_edit() {
fn get_repo_info_for_scan_clean_when_assume_unchanged_hides_edit() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
init_committed_repo(root);
fs::write(root.join("README"), "changed").unwrap();
git(root, &["update-index", "--assume-unchanged", "README"]);
// status clean; zip would still include the edit
let info = get_repo_info_for_scan(root.to_str().unwrap())
.unwrap()
.expect("repo info");
assert!(info.dirty);
assert!(!info.status_dirty);
assert!(!info.dirty);
}

/// Same for skip-worktree, which sparse checkouts set on every file they
/// leave out - a whole clean repository would otherwise read as dirty.
#[test]
fn get_repo_info_for_scan_dirty_when_skip_worktree() {
fn get_repo_info_for_scan_clean_when_skip_worktree() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
init_committed_repo(root);
git(root, &["update-index", "--skip-worktree", "README"]);
let info = get_repo_info_for_scan(root.to_str().unwrap())
.unwrap()
.expect("repo info");
assert!(info.dirty);
assert!(!info.status_dirty);
assert!(!info.dirty);
}

/// A clean filter (how git-lfs and similar tools store a file) makes the
/// worktree copy differ from the stored blob on purpose. `git status`
/// applies the filter and reports nothing; libgit2 cannot run it and reads
/// every such file as modified, which is why git is the one asked.
#[cfg(unix)]
#[test]
fn get_repo_info_for_scan_clean_when_a_clean_filter_rewrites_the_blob() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
init_committed_repo(root);
git(root, &["config", "filter.upper.clean", "tr a-z A-Z"]);
git(root, &["config", "filter.upper.smudge", "cat"]);
fs::write(root.join(".gitattributes"), "*.txt filter=upper\n").unwrap();
fs::write(root.join("payload.txt"), "lowercase\n").unwrap();
git(root, &["add", ".gitattributes", "payload.txt"]);
git(root, &["commit", "-m", "filtered"]);
// Stale timestamps are how a checkout out of a CI cache looks. Both
// implementations stop trusting the index's stat cache and compare
// content; only git runs the filter while doing it.
touch(&root.join("payload.txt"));

let repo = Repository::discover(root).unwrap();
assert!(
status_has_changes(&repo),
"libgit2 runs no clean filter, so it should read the file as modified"
);
let info = get_repo_info_for_scan(root.to_str().unwrap())
.unwrap()
.expect("repo info");
assert!(
!info.dirty,
"git status is empty, so the scan must not report a dirty worktree"
);
}

#[test]
Expand All @@ -691,19 +764,30 @@ mod tests {
.unwrap()
.expect("repo info");
assert!(!clean.dirty);
assert!(!clean.status_dirty);

fs::write(root.join("README"), "changed").unwrap();
let identity = get_repo_info(root.to_str().unwrap())
.unwrap()
.expect("repo info");
assert!(!identity.dirty);
assert!(!identity.status_dirty);
let scan = get_repo_info_for_scan(root.to_str().unwrap())
.unwrap()
.expect("repo info");
assert!(scan.dirty);
assert!(scan.status_dirty);
}

/// The libgit2 fallback answers only when the git binary cannot, so it is
/// exercised directly.
#[test]
fn libgit2_fallback_sees_a_modified_tracked_file() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
init_committed_repo(root);
let repo = Repository::discover(root).unwrap();
assert!(!status_has_changes(&repo));

fs::write(root.join("README"), "changed").unwrap();
assert!(status_has_changes(&repo));
}

fn sample_info(sha: &str, dirty: bool) -> RepoInfo {
Expand All @@ -712,7 +796,6 @@ mod tests {
repo_url: Some("https://github.com/org/repo.git".into()),
sha: Some(sha.into()),
dirty,
status_dirty: false,
}
}

Expand Down
Loading
Loading