Himitsu Implementation Plan (Detailed)

This document defines an execution plan for the vNext architecture:

centralized store (~/.himitsu)
age-only secret model (vars/<env>/<KEY>.age)
transport-agnostic sharing protocol
GitHub PR inbox + Nostr send/receive
full Rust rewrite of current shell implementation

Phase 1 - Rust Project Scaffold

Phase 1 complete

Goals

Create Rust crate and executable himitsu.
Implement command parsing and logging framework.

Modules / files

Cargo.toml                        # Single crate, all dependencies
rust/src/
├── main.rs                       # Entry point, tracing init, clap dispatch
├── error.rs                      # HimitsuError enum (thiserror)
└── cli/
    ├── mod.rs                    # Top-level Cli enum + subcommand dispatch
    ├── init.rs                   # Stub
    ├── set.rs                    # Stub
    ├── get.rs                    # Stub
    ├── ls.rs                     # Stub
    ├── encrypt.rs                # Stub
    ├── decrypt.rs                # Stub
    ├── sync.rs                   # Stub
    ├── search.rs                 # Stub
    ├── recipient.rs              # Stub
    ├── group.rs                  # Stub
    ├── remote.rs                 # Stub
    ├── share.rs                  # Stub
    ├── inbox.rs                  # Stub
    ├── schema.rs                 # Stub
    ├── codegen.rs                # Stub
    └── import.rs                 # Stub
flake.nix                         # Updated: build Rust binary alongside shell
.github/workflows/rust.yml        # CI: fmt, clippy, test on macOS + Linux

Test cases

cargo test                        # All unit tests
cargo clippy -- -D warnings       # Lint
cargo fmt -- --check              # Format check

himitsu --help prints full command tree with all subcommands
himitsu --version prints version string
himitsu <subcommand> --help works for every subcommand stub
Binary builds on macOS (aarch64-apple-darwin)
Binary builds on Linux (x86_64-unknown-linux-gnu)
nix build produces both shell and Rust binaries

Acceptance Criteria

himitsu --help works with planned command tree.
Project builds on macOS and Linux.

Risks

Slow startup due to heavy crate selection.
Over-designing module boundaries too early.

Phase 2 - Core Runtime Parity (No Sharing Yet)

Phase 2 complete

Goals

Implement config loading and mode detection.
Implement remote resolution and local secret operations.
Add optional macOS Keychain storage for generated age private keys.
Add SOPS_AGE_KEY_CMD key resolution that checks Keychain first, then file fallback.

Modules / files

rust/src/
├── config/
│   ├── mod.rs                    # detect_mode() → ProjectMode | UserMode
│   ├── global.rs                 # GlobalConfig: parse ~/.himitsu/config.yaml
│   ├── project.rs                # ProjectConfig: parse <repo>/.himitsu.yaml
│   └── remote.rs                 # RemoteConfig: parse remote himitsu.yaml
├── keyring/
│   ├── mod.rs                    # KeyProvider trait + scope/fingerprint mapping
│   └── macos.rs                  # macOS Keychain adapter via `security` CLI
├── remote/
│   ├── mod.rs                    # Remote discovery, resolution, list known remotes
│   └── store.rs                  # Secret file I/O: read/write vars/<env>/<KEY>.age
├── git.rs                        # Git CLI wrapper: clone, commit, push, pull, status
├── crypto/
│   ├── mod.rs                    # Trait defs: Encryptor, Decryptor
│   └── age.rs                    # age crate: keygen, encrypt, decrypt, parse recipients
├── index/
│   ├── mod.rs                    # SecretIndex: open db, query, upsert
│   └── schema.sql                # CREATE TABLE statements (embedded via include_str!)
├── cli/
│   ├── init.rs                   # Full implementation
│   ├── set.rs                    # Full implementation
│   ├── get.rs                    # Full implementation
│   ├── ls.rs                     # Full implementation
│   ├── encrypt.rs                # Full implementation
│   ├── decrypt.rs                # Full implementation
│   ├── search.rs                 # Full implementation
│   ├── recipient.rs              # Full implementation
│   ├── group.rs                  # Full implementation
│   └── remote.rs                 # Full implementation (add/push/pull/status)
src/lib/
├── common.sh                     # `SOPS_AGE_KEY_CMD` keychain-first lookup helper
└── init.sh                       # Optional keychain save when generating age key
tests/
├── integration/
│   ├── init_test.rs              # CLI integration tests for init
│   ├── set_get_test.rs           # set → get roundtrip
│   ├── ls_test.rs                # ls output format
│   ├── encrypt_decrypt_test.rs   # encrypt → decrypt roundtrip
│   ├── recipient_test.rs         # recipient add/rm/ls
│   ├── group_test.rs             # group add/rm/ls
│   ├── remote_test.rs            # remote add/push/pull/status
│   └── search_test.rs            # cross-remote search
└── fixtures/                     # (from Phase 0)

Keychain indexing (when enabled)

Scope pointer item: service=io.darkmatter.himitsu.agekey.scope.v1, account=gh:<org>:<repo>:<group> with value <fingerprint>.
Key material item: service=io.darkmatter.himitsu.agekey.byfp.v1, account=<fingerprint> with value AGE-SECRET-KEY-....
Resolution order for SOPS_AGE_KEY_CMD: scope pointer → fingerprint key item → SOPS_AGE_KEY_FILE fallback.
Scope values are normalized (org/repo lowercase, group escaped) to ensure unique matching across all org/repo/group combos.

Test cases

cargo test                        # Unit + integration
cargo test --test '*'             # Integration tests only

Unit tests (inline `#[cfg(test)]`)

Integration tests (`tests/integration/`)

Acceptance Criteria

Core local commands produce expected filesystem results.
Equivalent flows succeed on baseline fixtures.
himitsu search returns results across multiple remotes.
Keychain mode stores generated age keys and decrypts via SOPS_AGE_KEY_CMD without plaintext key files required.
Key lookup remains uniquely addressable for all <org>/<repo>/<group> scopes.

Risks

Edge cases with path expansion and symlinked directories.
Value quoting/newline handling in set.
Keychain access prompts/ACL behavior may break CI or non-interactive sessions.
Scope normalization bugs could cause key lookup misses.

Phase 3 - Recipient Policy Engine

Phase 3 complete

Goals

Replace env-only recipient model with path policy resolution.
Support local groups and remote refs in one normalized pipeline.

Modules / files

rust/src/
├── policy/
│   ├── mod.rs                    # PolicyEngine: load policies, resolve for path
│   └── resolver.rs               # RecipientRef parsing, expansion, dedup, ordering

Test cases

cargo test policy                 # Run policy module tests only

Acceptance Criteria

Recipient resolution snapshots are deterministic.
Policy tests cover include/exclude precedence.

Risks

Policy complexity creep.
Ambiguous behavior at path boundaries.

Phase 4 - Protocol + GitHub PR Inbox

Phase 4 complete

Goals

Implement signed envelope protocol.
Ship first complete external sharing path via GitHub PR inbox.

Modules / files

rust/src/
├── crypto/
│   └── signing.rs                # Ed25519 keygen, sign, verify
├── protocol/
│   ├── mod.rs                    # Shared types, canonical JSON helpers
│   ├── envelope.rs               # Envelope struct, serde, sign/verify methods
│   ├── payload.rs                # Payload struct, serde
│   └── profile.rs                # Profile struct, serde
├── transport/
│   ├── mod.rs                    # Transport trait: send, list, fetch
│   └── github_pr.rs              # GitHub PR inbox: create PR, list inbox files
├── inbox/
│   ├── mod.rs                    # Accept/reject pipeline orchestration
│   └── replay.rs                 # Replay DB: SQLite envelope-id tracking
├── cli/
│   ├── share.rs                  # Full implementation
│   └── inbox.rs                  # Full implementation

Test cases

cargo test protocol               # Protocol struct tests
cargo test transport              # Transport adapter tests
cargo test inbox                  # Inbox pipeline tests

Unit tests

Integration tests

Full send flow: share send --to github:org/repo --path ... --value ... creates PR with envelope JSON (mocked GitHub API via wiremock)
Envelope JSON in PR body is valid and parseable
Full receive flow: inbox list shows pending envelopes (mocked)
inbox accept <id> verifies signature, decrypts, writes .age file
inbox accept with duplicate envelope id is rejected
inbox accept with expired envelope is rejected
inbox reject <id> records envelope as processed without writing secret

Acceptance Criteria

Sender can share to external inbox repo via PR.
Receiver can verify, decrypt, and apply encrypted output.
Duplicate envelope IDs are rejected.

Risks

GitHub auth/token scope complexity.
Canonicalization bugs causing signature mismatch.

Phase 5 - Nostr Send and Receive

Phase 5 complete

Goals

Add full Nostr roundtrip delivery.

Modules / files

rust/src/
├── transport/
│   └── nostr.rs                  # Nostr relay adapter: publish, subscribe, parse

Test cases

cargo test transport::nostr       # Nostr transport tests

Acceptance Criteria

share send --to nostr:... publishes valid event.
inbox list --transport nostr returns envelopes.
inbox accept succeeds end-to-end.

Risks

Relay reliability and event propagation latency.
Metadata inconsistencies between relays.

Phase 6 - Identity Resolvers (Email + ENS + Nostr)

Phase 6 complete

Goals

Implement external identity resolution beyond GitHub.

Modules / files

rust/src/
├── identity/
│   ├── mod.rs                    # Resolver trait, ResolvedProfile, cache layer
│   ├── github.rs                 # GitHub keys repo: fetch team keys, parse .pub files
│   ├── email.rs                  # HTTP fetch /.well-known/himitsu.json, parse profile
│   ├── ens.rs                    # ENS text record lookup (feature-gated)
│   └── nostr.rs                  # npub normalization, optional profile metadata

Test cases

cargo test identity               # All resolver tests

All network tests use wiremock for HTTP mocking. ENS tests mock the RPC endpoint.

Acceptance Criteria

share send --to email:... resolves profile and sends.
share send --to ens:... resolves profile and sends.
Resolver output is cached and reproducible.

Risks

Resolver trust and TOFU pitfalls.
Network timeout behavior degrading CLI UX.

Phase 7 - Schema and Developer UX

Phase 7 complete

Goals

Add schema-backed validation and autocomplete support.

Modules / files

rust/src/
├── schema/
│   ├── mod.rs                    # Orchestration: generate static + dynamic schemas
│   ├── static_schema.rs          # Generate schemas/himitsu.schema.json
│   └── dynamic.rs                # Generate schemas/recipients.schema.json from live data
├── cli/
│   └── schema.rs                 # Full implementation

Test cases

cargo test schema                 # Schema generation tests

Static schema is valid JSON Schema draft 2020-12
Static schema validates a correct himitsu.yaml
Static schema rejects missing required fields
Dynamic schema includes local group names as enum values
Dynamic schema includes remote team refs as enum values
schema refresh regenerates dynamic schema from current state
Config validation errors include path + field + clear message

Acceptance Criteria

YAML editors can autocomplete groups/remote refs.
Invalid config points to path+field with clear message.

Risks

Dynamic schema becoming stale without refresh.
Large remote identity sets impacting schema size.

Phase 8 - Sync, Codegen, and Import

Phase 8 complete

Goals

Implement sync destinations for project-level encrypted secret delivery.
Implement typed codegen for downstream consumers.
Implement import from external secret stores.

Modules / files

rust/src/
├── cli/
│   ├── sync.rs                   # Full implementation (sync destinations + autosync)
│   ├── codegen.rs                # Full implementation
│   └── import.rs                 # Full implementation
├── codegen/
│   ├── mod.rs                    # Orchestration: detect lang, merge envs, write output
│   ├── typescript.rs             # TypeScript interface + const generation
│   ├── golang.rs                 # Go struct generation
│   └── python.rs                 # Python dataclass generation
├── import/
│   ├── mod.rs                    # Import trait, dispatch by source type
│   ├── sops.rs                   # SOPS YAML/JSON parser: decrypt via sops binary, extract keys
│   └── onepassword.rs            # 1Password: shell out to `op`, parse item fields

Test cases

cargo test codegen                # Codegen output tests
cargo test import                 # Import source tests
cargo test --test sync_test       # Sync integration tests

Sync

sync writes encrypted .age files to project directory
sync does not write plaintext anywhere
sync is idempotent (running twice produces same result)
autosync_on: set triggers sync after himitsu set
autosync_on: push triggers sync after himitsu remote push
Context isolation: set in project mode only writes to project's remote

Codegen

TypeScript output is syntactically valid (parseable by tsc)
Go output is syntactically valid (parseable by go vet)
Python output is syntactically valid (parseable by python -c)
Codegen merges common env with specific env (common first, env overrides)
Codegen respects app-scoped extraction from data.json
Snapshot tests: output matches expected for each language

Import

import --sops file.sops.yaml --env prod extracts all keys
Each extracted key is written as vars/prod/<KEY>.age
import --sops handles nested YAML keys (flattens with _ separator)
import --sops with --overwrite replaces existing secrets
import --sops without --overwrite skips existing secrets
import --op "op://vault/item/field" --env prod --key TOKEN writes one secret
import --op "op://vault/item" --env prod writes all fields from item
import --op fails gracefully when op CLI is not installed

Acceptance Criteria

sync writes encrypted files to project without plaintext.
Autosync triggers correctly based on configured event.
Codegen produces valid typed output for each supported language.
SOPS import decrypts and re-encrypts all keys into vars/<env>/<KEY>.age.
1Password import fetches and encrypts items into remote's format.

Risks

SOPS format variations (YAML vs JSON, nested vs flat keys).
1Password CLI version/auth differences across platforms.
Autosync timing edge cases (concurrent mutations from multiple devices).

Testing Infrastructure

How to run tests

# All tests (unit + integration)
cargo test

# Unit tests only (fast, no I/O)
cargo test --lib

# Integration tests only
cargo test --test '*'

# Specific module tests
cargo test config
cargo test crypto
cargo test policy
cargo test protocol
cargo test index

# With output (see println! in tests)
cargo test -- --nocapture

# Snapshot tests (update snapshots after intentional changes)
cargo insta test
cargo insta review

# Lint and format
cargo clippy -- -D warnings
cargo fmt -- --check

CI pipeline (`.github/workflows/rust.yml`)

jobs:
  test:
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
      - run: cargo fmt -- --check
      - run: cargo clippy -- -D warnings
      - run: cargo test

Test dependencies

Crate	Purpose
`assert_cmd`	Run CLI binary, assert stdout/stderr/exit code
`predicates`	Fluent assertions for CLI output matching
`tempfile`	Isolated temp directories per test
`insta`	Snapshot testing for output format stability
`wiremock`	HTTP mocking for GitHub API, .well-known, ENS RPC
`rusqlite`	Already a runtime dep; used directly in index tests

Test directory layout

tests/
├── integration/
│   ├── helpers/
│   │   └── mod.rs                # Shared setup: temp dir, init himitsu, create remote
│   ├── init_test.rs
│   ├── set_get_test.rs
│   ├── ls_test.rs
│   ├── encrypt_decrypt_test.rs
│   ├── recipient_test.rs
│   ├── group_test.rs
│   ├── remote_test.rs
│   ├── search_test.rs
│   ├── policy_test.rs
│   ├── envelope_test.rs
│   ├── inbox_test.rs
│   ├── sync_test.rs
│   ├── codegen_test.rs
│   └── import_test.rs
├── fixtures/
│   ├── golden/                   # Captured shell outputs for parity
│   ├── configs/                  # Sample YAML configs
│   ├── remotes/                  # Sample remote directory layouts
│   └── envelopes/                # Sample signed envelope JSON files

Test helper pattern

Every integration test follows this pattern:

use assert_cmd::Command;
use tempfile::TempDir;

fn himitsu() -> Command {
    Command::cargo_bin("himitsu").unwrap()
}

#[test]
fn set_get_roundtrip() {
    let home = TempDir::new().unwrap();
    let project = TempDir::new().unwrap();

    // init
    himitsu()
        .env("HOME", home.path())
        .arg("init")
        .assert()
        .success();

    // set
    himitsu()
        .env("HOME", home.path())
        .current_dir(project.path())
        .args(["set", "prod", "API_KEY", "secret123"])
        .assert()
        .success();

    // get
    himitsu()
        .env("HOME", home.path())
        .current_dir(project.path())
        .args(["get", "prod", "API_KEY"])
        .assert()
        .success()
        .stdout("secret123\n");
}

Cross-cutting Workstreams

Security

Envelope signature verification tests.
Replay DB integrity checks.
Sender allowlist enforcement.
Secret redaction in logs and errors.

Observability

Structured logs for send/accept operations.
Debug mode with trace IDs per envelope.

Performance

Cache remote identity lookups.
Batch decrypt/encrypt where safe.
Avoid blocking relay/network calls on unrelated commands.

Milestone Checklist

M1: Rust scaffold builds, --help works (Phase 1)
M2: Local secret parity: init/set/get/ls/encrypt/decrypt/sync/remote/search (Phase 2)
M3: Recipient policy engine with include/exclude (Phase 3)
M4: GitHub PR inbox send/receive end-to-end (Phase 4)
M5: Nostr send/receive end-to-end (Phase 5)
M6: Email/ENS/Nostr identity resolvers with caching (Phase 6)
M7: Schema validation and autocomplete (Phase 7)
M8: Sync + codegen + import (Phase 8)

Definition of Done

The rewrite is complete when:

1. Rust CLI fully replaces shell runtime.

FilesExpand file tree

TODO.md

Latest commit

History

TODO.md

File metadata and controls

Himitsu Implementation Plan (Detailed)

Phase 1 - Rust Project Scaffold

Goals

Modules / files

Test cases

Acceptance Criteria

Risks

Phase 2 - Core Runtime Parity (No Sharing Yet)

Goals

Modules / files

Keychain indexing (when enabled)

Test cases

Unit tests (inline #[cfg(test)])

Integration tests (tests/integration/)

Acceptance Criteria

Risks

Phase 3 - Recipient Policy Engine

Goals

Modules / files

Test cases

Acceptance Criteria

Risks

Phase 4 - Protocol + GitHub PR Inbox

Goals

Modules / files

Test cases

Unit tests

Integration tests

Acceptance Criteria

Risks

Phase 5 - Nostr Send and Receive

Goals

Modules / files

Test cases

Acceptance Criteria

Risks

Phase 6 - Identity Resolvers (Email + ENS + Nostr)

Goals

Modules / files

Test cases

Acceptance Criteria

Risks

Phase 7 - Schema and Developer UX

Goals

Modules / files

Test cases

Acceptance Criteria

Risks

Phase 8 - Sync, Codegen, and Import

Goals

Modules / files

Test cases

Sync

Codegen

Import

Acceptance Criteria

Risks

Testing Infrastructure

How to run tests

CI pipeline (.github/workflows/rust.yml)

Test dependencies

Test directory layout

Test helper pattern

Cross-cutting Workstreams

Security

Observability

Performance

Milestone Checklist

Definition of Done

Unit tests (inline `#[cfg(test)]`)

Integration tests (`tests/integration/`)

CI pipeline (`.github/workflows/rust.yml`)