🧪 Test gap analysis — 4 gaps found in runtime extension validate() unit tests

[github-actions]

Test Gap Analysis

Test suite snapshot: 1,347 unit tests, 87 integration tests (tests/compiler_tests.rs), 3 init tests, 8 MCP HTTP tests — 1,450 total. All pass. ✅

Previous issue #506 was closed as completed — the feed-url integration tests were added (test_python_runtime_with_feed_url_compiled_output, test_node_runtime_with_feed_url_compiled_output). However, the validate() unit test gaps from #506 were not addressed. All four runtime extension files still have zero unit tests, leaving every error/warning branch in their validate() implementations untested.

Progress on #376: 2 of its 3 original gaps are now fixed (init_logging() has 6 tests, MCP tools-list asserts all renamed tools). generate_lean_prompt() remains at 0 tests.

---

Priority Gaps

Module	Function/Path	Why It Matters	Suggested Test
runtimes/python/extension.rs	validate() — bash-disabled warning, config+feed-url bail!, config-only warning, invalid feed URL, version injection (5 branches), 0 unit tests	The mutually-exclusive config+feed-url bail! is a user-facing error; a regression could silently accept an invalid config and generate a broken pipeline	Unit test each branch with a synthetic CompileContext
runtimes/node/extension.rs	validate() — same 5-branch structure as Python, 0 unit tests	Identical risk: the config+feed-url bail! and reject_pipeline_injection on version are untested	Mirror python extension tests
runtimes/dotnet/extension.rs	validate() — 7 branches incl. global.json conflict detection and GLOBAL_JSON_SENTINEL skip logic, 0 unit tests	The global.json conflict path reads the filesystem (ctx.compile_dir.join("global.json").exists()) and bail!s — the most complex validation in any runtime extension, completely untested	Temp-dir test with/without global.json present
runtimes/lean/extension.rs	validate() — 1 warning branch (bash disabled), 0 unit tests	Bash-disabled warning path never exercised by any test	Single unit test with empty tools.bash

---

Suggested Test Cases

1. python/extension.rs — validate() error paths

#[cfg(test)]
mod tests {
    use super::*;
    use crate::compile::common::CompileContext;
    use crate::compile::types::FrontMatter;

    fn bare_ctx() -> FrontMatter { FrontMatter::default() }

    #[test]
    fn test_validate_config_and_feed_url_are_mutually_exclusive() {
        let ext = PythonExtension::new(PythonRuntimeConfig::WithOptions(PythonOptions {
            feed_url: Some("(pkgs.dev.azure.com/redacted),
            config: Some("pip.conf".into()),
            ..Default::default()
        }));
        let fm = bare_ctx();
        let ctx = CompileContext { agent_name: "agent".into(), front_matter: &fm, compile_dir: None };
        let err = ext.validate(&ctx).unwrap_err();
        assert!(err.to_string().contains("mutually exclusive"));
    }

    #[test]
    fn test_validate_bash_disabled_emits_warning() {
        use crate::compile::types::{ToolsConfig};
        let ext = PythonExtension::new(PythonRuntimeConfig::Enabled(true));
        let mut fm = FrontMatter::default();
        fm.tools = Some(ToolsConfig { bash: Some(vec![]), ..Default::default() });
        let ctx = CompileContext { agent_name: "agent".into(), front_matter: &fm, compile_dir: None };
        let warnings = ext.validate(&ctx).unwrap();
        assert!(!warnings.is_empty());
        assert!(warnings[0].contains("tools.bash is empty"));
    }

    #[test]
    fn test_validate_version_injection_rejected() {
        let ext = PythonExtension::new(PythonRuntimeConfig::WithOptions(PythonOptions {
            version: Some("3.12\n##vso[task.setvariable]x=y".into()),
            ..Default::default()
        }));
        let fm = FrontMatter::default();
        let ctx = CompileContext { agent_name: "agent".into(), front_matter: &fm, compile_dir: None };
        assert!(ext.validate(&ctx).is_err());
    }
}

2. dotnet/extension.rs — global.json conflict detection

#[test]
fn test_validate_global_json_conflict_bails() {
    use std::fs;
    let dir = tempfile::tempdir().unwrap();
    fs::write(dir.path().join("global.json"), r#"{"sdk":{"version":"8.0.100"}}"#).unwrap();

    let ext = DotnetExtension::new(DotnetRuntimeConfig::WithOptions(DotnetOptions {
        version: Some("9.0".into()),
        ..Default::default()
    }));
    let fm = FrontMatter::default();
    let ctx = CompileContext {
        agent_name: "agent".into(),
        front_matter: &fm,
        compile_dir: Some(dir.path()),
    };
    let err = ext.validate(&ctx).unwrap_err();
    assert!(err.to_string().contains("global.json"));
}

#[test]
fn test_validate_global_json_sentinel_accepted_with_file_present() {
    use std::fs;
    let dir = tempfile::tempdir().unwrap();
    fs::write(dir.path().join("global.json"), r#"{"sdk":{"version":"8.0.100"}}"#).unwrap();

    let ext = DotnetExtension::new(DotnetRuntimeConfig::WithOptions(DotnetOptions {
        version: Some("global.json".into()),
        ..Default::default()
    }));
    let fm = FrontMatter::default();
    let ctx = CompileContext {
        agent_name: "agent".into(),
        front_matter: &fm,
        compile_dir: Some(dir.path()),
    };
    assert!(ext.validate(&ctx).is_ok(), "global.json sentinel should not conflict");
}

3. lean/extension.rs — bash-disabled warning

#[test]
fn test_validate_lean_bash_disabled_emits_warning() {
    use crate::compile::types::{ToolsConfig};
    let ext = LeanExtension::new(LeanRuntimeConfig::Enabled(true));
    let mut fm = FrontMatter::default();
    fm.tools = Some(ToolsConfig { bash: Some(vec![]), ..Default::default() });
    let ctx = CompileContext { agent_name: "agent".into(), front_matter: &fm, compile_dir: None };
    let warnings = ext.validate(&ctx).unwrap();
    assert!(!warnings.is_empty());
    assert!(warnings[0].contains("tools.bash is empty"));
}

---

Coverage Summary

Module	validate() Branches	Unit Tests	Integration Tests	Coverage Gap
runtimes/python/extension.rs	5 (bash-warn, config+feed bail!, config-warn, feed-url validate, version inject)	0	happy-path + feed-url output	Error/warn branches untested
runtimes/node/extension.rs	5 (same structure)	0	happy-path + feed-url output	Error/warn branches untested
runtimes/dotnet/extension.rs	7 (bash-warn, config+feed bail!, feed-url validate, version inject, sentinel skip, global.json bail!, config inject)	0	happy-path + feed-url output	All error paths including filesystem check untested
runtimes/lean/extension.rs	1 (bash-warn)	0	happy-path only	Only warning branch untested

---

This issue was created by the automated test gap finder. Previous runs: #506 (closed 2026-05-11, feed-url integration tests fixed but validate() unit tests not added), #376 (still open, generate_lean_prompt gap). Total tests found: 1,450 (up from 1,439). Modules audited this cycle: all (full re-audit; only docs commit since last run).

Generated by Test Gap Finder · ● 960.2K · ◷