Skip to content

Add Kani tooling and local smoke targets (4.1.1)#305

Open
leynos wants to merge 3 commits into
mainfrom
4-1-1-kani-tooling-and-local-smoke-targets
Open

Add Kani tooling and local smoke targets (4.1.1)#305
leynos wants to merge 3 commits into
mainfrom
4-1-1-kani-tooling-and-local-smoke-targets

Conversation

@leynos
Copy link
Copy Markdown
Owner

@leynos leynos commented May 10, 2026

Summary

This branch implements roadmap task (4.1.1), adding the pinned local Kani tooling contract for Netsuke's formal-verification workflow.

It adds tools/kani/VERSION, scripts/install-kani.sh, and the local make kani, make kani-full, and make formal-pr targets. It also documents the developer workflow and marks the roadmap item complete after validation.

Roadmap task: (4.1.1)
Execplan: docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md

Review walkthrough

  • Start with the ExecPlan to review the completed decisions, validation evidence, and scope boundary.
  • Review tools/kani/VERSION and scripts/install-kani.sh for the pinned kani-verifier installation path.
  • Review Makefile for the opt-in Kani smoke, full-suite, and formal pull-request targets.
  • Review docs/developers-guide.md and docs/roadmap.md for the developer workflow and completed roadmap checkboxes.

Validation

  • scripts/install-kani.sh: passed, installed kani-verifier 0.67.0 and ran cargo kani setup
  • make kani: passed
  • make formal-pr: passed
  • make kani-full: passed; currently reports that no #[kani::proof] harnesses exist
  • mbake validate Makefile: passed
  • make check-fmt: passed
  • make lint: passed
  • make test: passed
  • make markdownlint: passed
  • make nixie: passed
  • coderabbit review --agent: passed with 0 findings on each milestone and final run

Notes

  • make fmt was attempted, but it still fails on pre-existing Markdown line-length and table issues in unrelated documentation. Formatter-only edits outside this branch's intended files were restored.
  • This PR intentionally does not add CI jobs, proof harnesses, Proptest coverage, or Verus proofs; those remain deferred to later roadmap items.

Summary by Sourcery

Add pinned Kani formal-verification tooling and local make targets for smoke and full runs, along with developer documentation and roadmap updates.

New Features:

  • Introduce pinned Kani version metadata under tools/kani/VERSION for formal verification tooling.
  • Add a Bash installer script to install and set up the repository-supported kani-verifier version.
  • Add make targets for Kani smoke checks, full verification runs, and a formal pull-request path.

Enhancements:

  • Document the Kani-based formal-verification workflow in the developer guide.
  • Mark roadmap item 4.1.1 and its subitems as completed and add a detailed execution plan document for Kani tooling and local smoke targets.

Documentation:

  • Add an execution plan document describing the design, constraints, risks, and outcomes for Kani tooling and local smoke targets.

Draft the approval-gated plan for roadmap item 4.1.1.

Capture the Kani version-pin, installer-script, and local Make target
workflow, along with validation, review, and rollback guidance.
Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @leynos, you have reached your weekly rate limit of 2500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 10, 2026

Review Change Stack

Walkthrough

This PR establishes Kani bounded model checker integration for Netsuke. It pins version 0.67.0, adds a Bash installer script, wires Make targets (kani, kani-full, formal-pr) for smoke checks and full verification, documents the workflow in the developers guide, and records the comprehensive execution plan with roadmap completion.

Changes

Kani tooling integration

Layer / File(s) Summary
Version pin and installer script
tools/kani/VERSION, scripts/install-kani.sh
Creates version pin at tools/kani/VERSION with 0.67.0. Installer script uses strict Bash (set -euo pipefail), validates version format, requires Cargo, installs pinned kani-verifier, runs setup, and verifies the installation via cargo kani --version.
Make targets for Kani verification
Makefile
Adds configurable variables (KANI, KANI_FLAGS, KANI_SMOKE_FLAGS) and three phony targets. kani runs a smoke check with --version flag, kani-full runs the full suite, and formal-pr delegates to kani via recursive Make.
Developer documentation
docs/developers-guide.md
Documents formal-verification tooling section explaining version source, installation refresh via the installer script, and usage of make kani, make kani-full, and make formal-pr targets. Clarifies that Kani is opt-in outside standard test/lint/format gates.
Execution plan and roadmap tracking
docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md, docs/roadmap.md
Records comprehensive ExecPlan 4.1.1 covering scope constraints (no harnesses, no CI), risks, decisions, staged plan, execution steps, acceptance criteria, testing strategy, interfaces, idempotence requirements, and external dependencies. Marks roadmap item 4.1.1 as complete.

Suggested labels

Roadmap

Poem

🔧 Kani's crowned—a bounded verifier stands,
Version pinned with care in trusted hands,
Make targets hum: smoke checks light and lean,
Documentation shows the way between,
Plans recorded, roadmap shines complete! ✨


Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

❌ Failed checks (2 errors, 1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Testing (Overall) ❌ Error Installer script and Makefile targets lack automated tests. The shell script contains non-trivial validation logic with zero test coverage despite manual validation. Add shell script tests covering: valid/invalid version formats, missing version file, missing cargo, successful installation, and error messages. Add Makefile target integration tests.
Module-Level Documentation ❌ Error The scripts/install-kani.sh module carries only a minimal single-line comment. Existing bash scripts provide multi-line documentation. This script does not meet the standard. Expand module-level documentation in scripts/install-kani.sh with multi-line comments explaining purpose, utility, functionality, and relationship to the Kani formal-verification workflow.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Performance And Resource Use ❓ Inconclusive No result was produced after verification. Marking as INCONCLUSIVE. Re-run the check or adjust instructions to produce a final result.
✅ Passed checks (14 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
User-Facing Documentation ✅ Passed Kani tooling is developer infrastructure, not user-facing functionality. Documentation properly placed in docs/developers-guide.md, not docs/users-guide.md. Netsuke CLI itself was not modified.
Developer Documentation ✅ Passed Kani tooling fully documented in developers' guide with installation instructions and Make targets. ExecPlan complete with decision log. Roadmap item 4.1.1 marked complete. No multilingual docs.
Testing (Unit And Behavioural) ✅ Passed PR implements Makefile targets, Bash installer, and documentation. ExecPlan explicitly exempts non-Rust infrastructure work from automated test requirements.
Testing (Property / Proof) ✅ Passed Adds Kani tooling infrastructure only. No proof harnesses, property tests, or invariants introduced. Proofs are deferred to later roadmap items as designed.
Testing (Compile-Time / Ui) ✅ Passed PR adds tooling integration (Makefile, Bash installer, version pinning, docs) with no new Rust code, compile-time behaviour, or output requiring snapshot tests. Custom check is not applicable.
Unit Architecture ✅ Passed Queries, commands, and fallible operations are properly separated. Error handling is explicit; dependencies are configured via files, not global state; side-effects are visible and isolated.
Domain Architecture ✅ Passed PR adds only build tooling (Makefile, installer script, documentation). Zero Rust domain code changes. Domain Architecture check does not apply to infrastructure.
Observability ✅ Passed Adds developer-only Kani tooling (installer, Makefile targets). No production operational behaviour introduced. Observability instrumentation not applicable.
Security And Privacy ✅ Passed Input validation prevents command injection. Shell safety best practices followed. No secrets, credentials, or sensitive data exposed. File permissions appropriate.
Concurrency And State ✅ Passed PR adds build tooling, version pinning, and sequential installer. No concurrent code, shared state, locks, async, or state transitions. Concurrency check not applicable.
Title check ✅ Passed The pull request title accurately reflects the main change—adding Kani tooling and local smoke targets—and includes the required roadmap item reference (4.1.1).
Description check ✅ Passed The pull request description comprehensively documents the changeset, providing roadmap context, a structured review walkthrough, validation evidence, and implementation notes that align with the actual changes made.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 4-1-1-kani-tooling-and-local-smoke-targets

Comment @coderabbitai help to get the list of available commands and usage tips.

codescene-delta-analysis[bot]

This comment was marked as outdated.

leynos added 2 commits May 11, 2026 01:35
Pin the supported Kani verifier version and add the local
installer script used by the formal-verification workflow.
Expose local Kani smoke and full-suite Make targets for the
formal-verification workflow, document their use, and mark
roadmap item 4.1.1 complete after validation.
@leynos leynos marked this pull request as ready for review May 13, 2026 22:39
@sourcery-ai
Copy link
Copy Markdown
Contributor

sourcery-ai Bot commented May 13, 2026

Reviewer's Guide

Adds pinned Kani formal-verification tooling and opt-in Make targets, plus documentation and roadmap updates, to establish a repeatable local Kani smoke workflow without impacting existing CI/test paths.

Sequence diagram for make formal-pr Kani smoke workflow

sequenceDiagram
  actor Developer
  participant Make
  participant cargo_kani

  Developer->>Make: make formal-pr
  Make->>Make: kani
  Make->>cargo_kani: cargo kani --version
Loading

Flow diagram for scripts/install-kani.sh installation steps

flowchart TD
  A[run scripts/install-kani.sh] --> B[require_command cargo]
  B --> C[read tools/kani/VERSION]
  C --> D[validate MAJOR.MINOR.PATCH]
  D --> E[cargo install --locked kani-verifier --version <version>]
  E --> F[cargo kani setup]
  F --> G[cargo kani --version]
Loading

File-Level Changes

Change Details Files
Introduce pinned Kani installer workflow and version contract.
  • Add tools/kani/VERSION as the single source of truth for the supported kani-verifier version (0.67.0).
  • Implement scripts/install-kani.sh to read the pinned version, validate it, install kani-verifier via cargo, run cargo kani setup, and smoke-check cargo kani --version with clear error handling.
tools/kani/VERSION
scripts/install-kani.sh
Add Make targets for Kani smoke testing and formal-PR workflow.
  • Extend .PHONY list and define KANI-related variables (KANI, KANI_FLAGS, KANI_SMOKE_FLAGS) to keep commands overrideable.
  • Add kani target as a fast smoke check running cargo kani with smoke flags (default --version).
  • Add kani-full target to run the full Kani verification suite via cargo kani.
  • Add formal-pr target that delegates to make kani as the formal-verification PR gate.
Makefile
Document developer-facing Kani workflow and mark roadmap item 4.1.1 complete.
  • Add a formal-verification tooling section describing how to install the pinned Kani version and how to use make kani, make kani-full, and make formal-pr, explicitly stating they are opt-in and not part of make test/lint/format/all.
  • Mark roadmap task 4.1.1 and its subitems as completed, referencing the existing formal-verification design document and clarifying that CI and harnesses are deferred.
docs/developers-guide.md
docs/roadmap.md
Add detailed ExecPlan documenting design, constraints, risks, decisions, and validation for Kani tooling.
  • Create an execution-plan document capturing scope, constraints, risks, progress log, decisions, validation criteria, and concrete steps for implementing Kani tooling and local smoke targets.
  • Record that make kani is currently a command smoke check (cargo kani --version) while make kani-full is wired for future proof harnesses, and that Kani is intentionally kept out of existing CI/test targets.
docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md" line_range="44" />
<code_context>
+- Keep Makefile additions consistent with the current target style: variables
+  near the top, `.PHONY` declarations, target descriptions using `##`, and
+  recipes that honour overrideable variables.
+- Preserve the existing OrthoConfig and localized CLI help surfaces. This
+  item adds developer tooling, not a user-facing Netsuke subcommand or flag.
+- If any Kani-specific Rust source is introduced later during implementation,
</code_context>
<issue_to_address>
**issue (typo):** Align "localized" with the stated en-GB-oxendict spelling ("localised").

This bullet references en-GB-oxendict spelling but uses the en-US form "localized"; please change it to "localised" for consistency with the documented style.

```suggestion
- Preserve the existing OrthoConfig and localised CLI help surfaces. This
```
</issue_to_address>

### Comment 2
<location path="docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md" line_range="276" />
<code_context>
+
+## Context and orientation
+
+Netsuke is a Rust build-system compiler. It reads a YAML `Netsukefile`, expands
+MiniJinja-controlled manifest logic, validates a static Intermediate
+Representation (IR), emits a deterministic Ninja file, and delegates execution
</code_context>
<issue_to_address>
**suggestion (review_instructions):** The acronym “YAML” is used without being expanded on first use.

Please expand YAML on first use, for example: “YAML Ain’t Markup Language (YAML) `Netsukefile`”. Later occurrences can use “YAML” alone.

<details>
<summary>Review instructions:</summary>

**Path patterns:** `**/*.md`

**Instructions:**
Define uncommon acronyms on first use.

</details>
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

- Keep Makefile additions consistent with the current target style: variables
near the top, `.PHONY` declarations, target descriptions using `##`, and
recipes that honour overrideable variables.
- Preserve the existing OrthoConfig and localized CLI help surfaces. This
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (typo): Align "localized" with the stated en-GB-oxendict spelling ("localised").

This bullet references en-GB-oxendict spelling but uses the en-US form "localized"; please change it to "localised" for consistency with the documented style.

Suggested change
- Preserve the existing OrthoConfig and localized CLI help surfaces. This
- Preserve the existing OrthoConfig and localised CLI help surfaces. This


## Context and orientation

Netsuke is a Rust build-system compiler. It reads a YAML `Netsukefile`, expands
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (review_instructions): The acronym “YAML” is used without being expanded on first use.

Please expand YAML on first use, for example: “YAML Ain’t Markup Language (YAML) Netsukefile”. Later occurrences can use “YAML” alone.

Review instructions:

Path patterns: **/*.md

Instructions:
Define uncommon acronyms on first use.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b9e59e9e6b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Makefile
nixie --no-sandbox

kani: ## Run the Kani local smoke check
$(KANI) $(KANI_SMOKE_FLAGS)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Enforce the pinned Kani version in smoke checks

When a developer or CI image already has cargo-kani installed at a different version, this target exits successfully because it only runs cargo kani --version and never compares that output with tools/kani/VERSION; formal-pr delegates to this target, so stale or newer Kani binaries can validate PRs despite the new pinned-version contract. Please have the smoke path read the version pin and fail on a mismatch, or route it through a script that performs that check.

Useful? React with 👍 / 👎.

@coderabbitai coderabbitai Bot added the Roadmap label May 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant