validate-harborize-instrument: free rungs 0-5 plus canary and mechanical packages (tasks 1-8) by cameronraysmith · Pull Request #2747 · cameronraysmith/vanixiets

cameronraysmith · 2026-08-18T20:08:58Z

Grounding

OpenSpec change validate-harborize-instrument validates the harborize
instrument (frozen at 0.2.1) through a seven-rung cost ladder. This PR executes
rungs 0 through 5 plus the two free authoring sections — tasks.md sections 1-8 —
from the tip of harborize-instrument. Rungs 4 and 5 are Docker time with zero
model calls. Sections 9 and 10 (the one metered codex trial and close-out) are
deliberately not run and stay open on this PR.

Three settled decisions were folded into the change documents before execution:
CLI installs track PyPI latest stable (2026-08-15) rather than sha-pinned source
checkouts; the metered adapter is codex via the ChatGPT-subscription path
(CODEX_FORCE_AUTH_JSON / CODEX_AUTH_JSON_PATH), superseding the earlier
claude-code first-cell choice; and the Gate-1 word harbor run -k 5 was already
encoded identically in tasks.md 8.1 and the spec delta, so no edit was needed.

Review round

An independent two-axis review of this PR reported 12 Standards findings and 13
Spec findings. All 25 are addressed here — fixed where the change owns the
artifact, and recorded on task 10.4's deferred list where the artifact is inside
the instrument that decision D5 freezes. The two load-bearing ones:

The mechanical verifier could not fail on an oracle bug (S1).
verifier/test.sh computed its expectation with code character-identical to
oracle/solve.sh, so the two agreed by construction. It now holds a
hand-derived literal and reads only /logs/verifier/summary.json, which retires
verifier/_deps/ along with two of the fixture's four copies. The median defect
the shared derivation hid is fixed with it: task.md asked for "the median"
while both sides took the upper median durations[n // 2]. Reproduced against
the pre-change pair on a 14-record fixture — the task-correct answer 475 scores
0 and the oracle's 500 scores 1. task.md now states the even-count convention
and the oracle implements it; the shipped 15-record answer is 450 either way, so
no recorded evidence moves. Severity of the new gate was exercised against seven
independently mutated oracles, each scoring 0 where the correct one scores 1.

Task 5.6 was ticked while both heads resolved to SHARED (P1). Both READMEs
recorded the mechanical package's fork as "separate" while Harbor resolved
SHARED, because the mode comes from verifier.environment_mode or
[verifier.environment] and from nothing else — a verifier/Dockerfile infers
nothing. Implementing separate was investigated and rejected on two independent
grounds, both reproduced:

benchflow 0.7.4 does not fall back to shared. "separate verifier sandboxes are parsed but not executed" (runtime_capabilities.py:186-192) is the
reason string of a fail-closed refusal (sandbox/setup.py:676, :819-842),
so a package declaring it does not launch at all. bench tasks check --level runtime-capability --sandbox docker exits 0 on both packages as shipped and
reports that issue the moment sandbox_mode: separate is added. The proposal
requires this package to pass under both runners.
Harbor's separate verifier empties /logs/verifier before running
(trial.py:599) through the same host bind it mounts at :686-692, so the
summary.json channel would be deleted and every agent including the oracle
would score 0.

So the record was corrected rather than the fork implemented: both packages are
shared, verifier/Dockerfile is dropped, D11 is rewritten with the evidence,
and separate mode is stated to be exercised nowhere in this change. Task 5.6 now
requires the fork to be verified by resolving it from the exported head, which
is what let a directory layout stand in for a declaration that was never made.
It stays ticked because the tree now supports it:
resolve_task_verifier_mode(Task(<head>).config) is SHARED on both heads,
matching both READMEs.

Two further corrections fell out of that investigation, neither in the review.
The change recorded that a shared fork puts the verifier's answer key within a
running agent's reach; it does not, because both runners upload the verifier's
directory during the verification phase, after the agent phase has ended (harbor
verifier/verifier.py:147-153, phase order trial/single_step.py:41 then
:52; benchflow task/verifier_core.py:385). That is also what makes the
literal expectation above safe. Rung 6 still declines to read the canary's
reward, for the reason that a model-driven trial's reward conflates delivery
with the model's own behaviour.

The remaining findings, by disposition:

Fixed in the corpus: the canary oracle's unguarded $HOME under set -u,
which aborted before writing any output and read downstream as a false canary
alarm (S5, reproduced in-container: exit 1 before, exit 0 and reward 1 after);
base images pinned by multi-arch index digest, the floating tag having already
drifted past what this host held; checks/resolve_check.py given a docstring,
a main() guard and an argument check, and its recorded invocation corrected
from a bare python3 that never resolved harbor to the CLI entrypoint's own
interpreter (S9); the corpus README split into a permanent half and a
change-scoped rung log whose removal task 10.6 now carries (S10), carrying the
freeze-digest recipe that previously would have archived with tasks.md (P13);
injection-canary/README.md:19's false "the token lives only in the injected
SKILL.md" (P6).
Fixed in the change documents: D3 narrowed against the settled metered-cell
decision with the mechanism that makes it sound (P2 — harbor's codex adapter
derives cost_usd from token counts against LiteLLM's list rates rather than
reading a billed charge, so the figure is a metered-rate computation
regardless of how the trial authenticated; the confounds D3 names act on
throughput, and the rung is one non-concurrent trial); the cost extraction's
job name corrected from the pre-fold cc-registration, and its provenance
anchor from claude-code-only to codex's own path; six fold-residue sites (P3);
the marketplace-program.md:46 citation, which is a cell roster that names
claude-code first and records no supersession; the freeze recipe's
__pycache__ exclusion and path precondition (P5); what the flake guard's
nix eval does and does not prove (P7); proposal.md's false "Unchanged … nix
skill composition" and the marketplace-versus-nix-delivery asymmetry (P4, P8);
the .gitignore and .gitattributes edits now named with their reasons
(P10); three stale citations (P11).
Recorded as deferred, not fixed: twelve instrument-side findings — the
unquoted shell interpolation in design_matrix.py, the leakage audit's
quote-pairing extractor and substring matcher and its suffix and
oracle-comparison gaps, analyze_lattice.py's interval-ignoring tag and
silent --units typo, three input-validation defects in census.py,
collect_rewards.py and materialize_conditions.py, the uppercase and
one-sentence-per-line breaches, and the type-annotation inconsistency. All sit
inside the frozen instrument, which proposal.md's Non-goals and D5 forbid
editing during this change; task 10.4 exists to carry exactly this list and
now holds all of it with anchors.
Not addressed, deliberately: proposal.md's linear_story_state
frontmatter, which reads Todo against 34 checked boxes (P12). The Linear
mirror is owned outside this worktree. Task 1.8's jj revision (P9) remains a
git commit with the substitution disclosed in place, since the rung executed
in a plain git worktree. Four commit subjects exceed 72 characters (S12); all
four predate this review round and rewriting them would rewrite the
harborize-instrument chain this branch is stacked on. The four new commits
are 56, 60, 62 and 55 characters.

Blast radius

Additive. git diff --name-only over the review-round commits touches only
.gitattributes, modules/home/ai/evals/harborize/, and
openspec/changes/validate-harborize-instrument/. No AGENTS.md or
CLAUDE.md is created or modified anywhere in the diff, and no
linear_story_* frontmatter field is changed.

Correcting an earlier claim in this description: the branch does touch a nix
module — modules/home/ai/skills/default.nix gains eight issues-beads*
exclusions in 5f33c36e, which lands inside the change window and belongs to
the beads retirement rather than to any rung. proposal.md now records that
rather than declaring it unchanged. And "no published surface" was wrong:
excluding a skill from nix delivery does not exclude it from apm marketplace
publication, which is package-scoped through apm.yml and
.github/plugin/marketplace.json, so the instrument remains published inside
the testing-and-quality plugin. That asymmetry is now recorded in the
proposal's Impact.

The instrument freeze is intact: no commit after 432e3573 touches the
harborize skill directory, and its content digest recomputes to
3fdd30d1fa2a69a5e53c8d34474c107a516c32e29dcc5087b9bd7738b22ccd4e.

Rungs 4 and 5

Both ran on this host against benchflow 0.7.4 and harbor 0.21.0, oracle-only,
with no model calls.

Rung 4 failed on its first run with the skill provably delivered, and the cause
is a channel that only one runner preserves. Both packages routed their
oracle-to-verifier deliverable through /logs/verifier, chosen from Harbor's
behaviour alone. BenchFlow clears that directory's contents on the agent
container immediately before the verifier runs, unconditionally:
harden_before_verify executes _CLEAR_VERIFIER_DIR_CMD
(sandbox/lockdown.py:775-784, call at :1205-1212), while the mount-aware
conditional clear in _verify_test_script (task/verifier_core.py:360-372) is
a different branch that never reached the file. The bind survives and the
content does not, so the canary scored 1 under Harbor and 0 under BenchFlow with
no error and no diagnostic beyond a grep miss — the runner-dependent silent
zero this corpus exists to detect, reached from the opposite direction to the
one D11 predicted. Reproduced in-container both ways before the fix.

Both deliverables now go to /logs/artifacts, which both runners bind for the
whole trial (sandbox/docker.py:186-201; models/trial/paths.py:38,
:195-202), which no hardening step touches, and which is additionally the one
path that would survive Harbor's separate fork, whose wipe is followed by an
artifact re-upload. D11, the plan and task 5.3 are corrected with that evidence
rather than quietly re-pointed.

Rung 4 after the fix: the canary with --skill-mode with-skill scores reward 1
with zero errors, effective_skills_dir equal to the host condition directory
that was passed, and no fidelity exception raised (7.1, 7.2); the no-skill
control scores 0, so the canary can fail (7.3); one dir(C) shape exists, so
7.1 covers it (7.5). The mechanical package was re-run the same way and scores
1.

The fidelity control (7.4) could not be built the way the task prescribed, and
that is a finding rather than an execution detail. BenchFlow itself stages the
host skills directory into environment/_deps/skills and appends the
COPY _deps/skills /skills/ line the recipe wanted hand-written
(sandbox/setup.py:529-568, rollout/__init__.py:979-1016), working on a temp
copy of the task, so a hand-authored baked set is overwritten before the build;
a second attempt that baked a decoy at a discovery path also scored 1, because
the runtime link step runs rm -rf <dest> && ln -sfn /skills <dest>
(agents/install.py:61-90). The control that does provoke
experiment_fidelity/skill_deployment_missing puts a symlinked skill directory
in the condition directory: the host glob follows the symlink while the staging
copy drops symlinked entries by design (the #411 fix), so the in-container
catalogue is genuinely short one skill. The exception was raised naming the
expected set. Throwaway packages and condition directories lived outside the
repository and were deleted after the runs.

Rung 5: five trials per package under harbor run -k 5, reward 1.0 on all five
for each, zero errored trials, pass@2/4/5 all 1.0 (8.1). No
agent/exit-code.txt exists under any of the fifteen trial directories, which
is the passing reading because OracleAgent.run writes that file only on a
nonzero return (agents/oracle.py:149-151) (8.2). The canary head takes
--skill <dir(C)> on the Harbor line; without it the same head scores 0 across
five trials, which is the Harbor-side counterpart of the no-skill control and
was run first. No trial errored in any job, so the errored-trial triage (8.3)
was never entered and is recorded as vacuous rather than as performed.

One candidate entry for task 10.4's deferred instrument list is recorded in the
corpus README rather than acted on: the instrument's emitter reference presents
/logs/verifier only as the reward-file path, which is correct, and nowhere
warns that the same directory is cleared before the verifier under BenchFlow.

Verification

Selected as what would fail if these changes were wrong, plus the immediate
consumers of what they touch. Everything below was run on this branch after the
review-round commits; the pre-existing rung evidence is unchanged except where
named above.

bench tasks check --level structural and --level runtime-capability --sandbox docker: exit 0 on both native packages. The second level is new to
this round and is what establishes that neither package declares a feature
BenchFlow would refuse to launch.
harbor.models.task.task.Task(<head>) constructs both exported heads, and
resolve_task_verifier_mode returns SHARED for both, matching what every
README and task now records.
bench tasks export --overwrite: lossless, 0 losses, both heads.
audit_leakage.py: exit 0 on the mechanical package, exit 1 on the canary
with exactly the expected check-1 token flag.
Extension audit plus nix eval on both nixosConfigurations and
darwinConfigurations: exit 0.
shellcheck clean on all six corpus shell files; python3 -m py_compile
clean on resolve_check.py; rung 2 re-run reproduces the recorded digest
sha256:47016a2e…5cd9 and all three malformed-root raises, plus a usage line
and exit 2 where a bare invocation previously raised IndexError.
Container dry runs under the pinned base images: mechanical correct summary
scores 1, a summary with one digit of the median altered scores 0, a missing
summary.json scores 0; canary with dir(C) mounted scores 1 and without it
scores 0; canary with HOME unset now exits 0 and scores 1.
openspec validate validate-harborize-instrument --strict passes.

Re-run after the channel change, as the checks that would fail if it were wrong
or if it broke something upstream of it: bench tasks check at schema,
structural and runtime-capability --sandbox docker on both native trees,
exit 0 each; Task() construction and resolve_task_verifier_mode on both
re-exported heads, still SHARED; bench tasks export --overwrite, lossless on
both; audit_leakage.py, unchanged at exit 0 on the mechanical package and the
expected single check-1 flag on the canary; shellcheck clean on all eight
corpus shell files; the extension audit and nix eval .#nixosConfigurations,
exit 0; and openspec validate --strict, passing.

The runner-level runs recorded under Rungs 4 and 5 are the primary evidence,
since they exercise the delivery path end to end under both runners.
nix flake check was deliberately not run: this round's diff touches no nix
file, and CI covers it.

Deliberately not run: sections 9 and 10, which need model spend. Deliberately
not re-run: the rung 0 and rung 1 evidence, which nothing in this round
touches.

An earlier version of this description claimed bench tasks export was lossless
"twice"; only the most recent report is retained per head, so that was not
checkable from the tree. The current claim is what the tree shows.

Base

Transplanted onto origin/harborize-instrument (9d382e89), which was rebased
onto current main and force-pushed while this PR was open. Only this branch's
own twelve commits were replayed, with
git rebase --onto origin/harborize-instrument 5f33c36e; the six inherited
commits keep the ids 9d382e89 published, and origin/main is now a direct
ancestor of this branch. Every path this PR owns is byte-identical across the
transplant (git diff <pre> <post> -- .gitattributes modules/home/ai/evals/harborize openspec/changes/validate-harborize-instrument is empty), and the instrument tree object is unchanged at 64bc599c….

Verification round

After the review-round commits landed, a four-lane adversarial sweep was run
against the diff. It found sixteen defects, most of them introduced by those
commits rather than inherited, and all are fixed in the final commit. The three
that mattered:

The rewritten verifier did not write a reward on every path, which is worse
than scoring 0 — both runners read a missing reward file as a trial-level
crash. integers_only recurses and sat outside the try, so a submission
nested about 500 deep raised RecursionError with no reward written, and an
oversized one hit the timeout or the OOM killer the same way. It now writes
0 before reading anything and upgrades to 1 only on success.
The freeze check is redesigned around the instrument's git tree object,
64bc599c…, with the content digest demoted to a secondary check. A
.DS_Store flips the digest while git status stays clean, so the
__pycache__ exclusion had fixed one instance of a class and left six; the
digest is also blind to mode changes and symlinks, and does not survive a
rebase of the base branch — which stopped being hypothetical when this
branch's base was rebased.
Four residue sites the review-round commits missed, each contradicting a
sibling those same commits did rewrite, plus a sentence introduced into the
canary README that read as the opposite of the mechanism.

Also corrected: eight anchors and counts verified wrong against the installed
trees (cost_usd sits on JobStats, not JobResult; set -euo pipefail is at
design_matrix.py:357; the "444 non-nix files" figure survived in two documents
after being replaced in a third, and its replacement was itself stale). Two
scope preconditions are now recorded rather than implied: the no-wipe claim
holds for single-step trials only, since multi_step.py:202 resets the same
directory before every shared step verifier; and neither package clears
--level publication-grade, for a reason unrelated to D7's argument.

…d-quality Place the harborize skill into the apm marketplace under the testing-and-quality plugin group, establishing a versioned source of truth for an instrument that previously existed only as an unmanaged directory in ~/.claude/skills. The placement is verbatim from the authoring tree. Known defects in the original -- census.py mode 644, a hardcoded /tmp output path, and falsified Harbor and BenchFlow CLI contracts -- are preserved so that subsequent fixes land as tracked edits against a clean baseline.

Repair four falsified upstream contracts verified against harbor ac398bbda7c4c1073461797d3b95c2455cc671b5, benchflow d30527b82027a416e72014920cdf43a534967ad3 and skillsbench 9a1f4dd5f7659f75707435da3ce854b6e48321d1. Harbor skill injection used --ak skills_dir=, a container-side path, so a host path silently failed to copy and every condition collapsed to the empty condition while the run exited zero. It now uses --skill, which uploads per trial and SHA-pins each skill into the trial lock file. The removed harbor tasks check gate is replaced. The BenchFlow arm no longer emits a --run-id option that does not exist. The separate verifier mode becomes an explicit fork carrying the image requirements it actually has. Add a results collector so the analysis half of the pipeline can be reached from the emission half, branch the interval estimator on reward type, and record that a cluster bootstrap over a single task carries no interval. Census parses YAML folded scalars, takes an output path, and emits a plugin-membership map so a plugin unit is a derived aggregate over the flat deployed tree. Scripts carry uv script headers. Encode the settled decisions: skill-level units over the deployed tree with plugin as a derived aggregate, mechanical contracts by default with one judge-validation package as a gate, metered pricing for every budget figure, and instrument versioning at 0.2.0 with revisions confined to the interval between evaluation rounds. Ignore Python bytecode repository-wide so running these scripts leaves no __pycache__ artifacts for a later snapshot to track.

…ration harborize is iterated in place through a hand-made symlink from ~/.claude/skills/harborize into this repository, which gives a tight edit-and-reload loop with every change tracked. Home-manager's backup pass moves an existing non-symlink target aside before linking, and because the symlink is an intermediate path component that move resolves into the repository and renames the skill's own sources to *.before-home-manager. Excluding the skill at the delivery layer leaves composition untouched and keeps the symlink authoritative. Remove the symlink before removing this exclusion.

…n-consuming adapters The trial lock file is written in Trial.__init__ before the skills are resolved and long before they are uploaded, so it records host-side resolution and proves nothing about delivery into the container. Two places claimed otherwise. Both now state what is actually proven and name what it does not cover. Twenty-two of thirty-nine registered Harbor adapters consume an injected skill and seventeen ignore it with no error, no warning and no log line, which yields a clean-looking negative result. The design matrix now refuses to emit a manifest for a cell naming a non-consuming adapter or an ACP registry shorthand, which catches the case before any container spend. Author BenchFlow-native and derive the Harbor head by export, because publication-grade rejects the co-present layout and SkillsBench CI forbids exactly the paths Harbor requires. Record BenchFlow's oracle rollout as the free delivery gate, the per-adapter destination paths, and the two prerequisites that otherwise read as authoring errors.

Scope Gate 1 of the harborize instrument: prove the repaired instrument produces packages whose skill injection is verified end to end, and measure the per-run cost constant that every later budget depends on. The tasks follow a six-rung ladder that puts the money last. An adapter allowlist check, host-side skill resolution, and static task validation are free. BenchFlow's oracle rollout proves delivery to five discovery paths with no model calls, and Harbor's oracle proves inhabitation the same way. Only the final rung, which asserts Harbor's per-adapter registration copy, is metered, and it is reached after four cheaper rungs have derisked it. Evaluation packages live under modules/home/ai/evals/harborize, which import-tree ignores because its filter admits only files ending in .nix. A fixture carrying that extension would be evaluated as a flake-parts module, so the constraint is recorded with a check rather than a comment.

github-actions · 2026-08-18T23:00:03Z

Triggered from #2747 by @cameronraysmith.

Checking if we can fast forward main (25d10e7) to fm/vx-harborize-rung0 (50af830).

Target branch (main):

commit 25d10e7cc6efcf78b8f89ce9557778345df6c5ad (HEAD -> main, origin/main)
Author: Cameron Smith <cameron.ray.smith@gmail.com>
Date:   Tue Aug 18 16:00:36 2026 -0400

    feat(darwin): declare sshd server enablement for stibnite
    
    Nothing in the repo enabled the ssh server on any darwin host; rosegold
    and argentum answer ssh only via a manual macOS Remote Login toggle from
    setup day (services.openssh.enable defaults to null = macOS-managed).
    
    Add modules/darwin/sshd-server.nix, which sets services.openssh.enable
    = true (nix-darwin boots Apple's com.openssh.sshd launchd daemon,
    listening on all interfaces including zerotier), and import it from
    stibnite. Per-user keys ride the existing users pattern: stibnite
    already wires users.users.crs58.openssh.authorizedKeys.keys from
    users.crs58.meta.sshKeys, rendered via nix-darwin's
    101-authorized-keys.conf AuthorizedKeysCommand. No new host keys or
    clan sshd role changes; NixOS hosts' sshd declarations untouched.
    
    rosegold/argentum should migrate onto this declaration later (out of
    scope).
    
    Validation: nix build .#darwinConfigurations.stibnite.system passes;
    activation script contains 'launchctl bootstrap system
    /System/Library/LaunchDaemons/ssh.plist' and crs58's two meta.sshKeys
    keys; nix eval of openssh.enable: stibnite true, rosegold/argentum
    unchanged (null), cinnabar/magnetite unchanged (true); nix build
    checks.darwin-stibnite and checks.clan-inventory-consistency pass;
    nix fmt clean.

Pull request (fm/vx-harborize-rung0):

commit 50af830d4ddb3357608e111e8cff58087d932904 (pull_request/fm/vx-harborize-rung0)
Author: Cameron Smith <cameron.ray.smith@gmail.com>
Date:   Tue Aug 18 18:59:41 2026 -0400

    docs(openspec): complete the deferred instrument-defect list
    
    Task 10.4's list carried seven of the review's instrument findings. Adds the
    five remaining, so the list the next instrument revision reads is the whole
    set rather than the subset one commit happened to reach.
    
    New entries: `census.load_membership`'s unguarded `data["skills"]` read
    against its sibling's validation of the same input; `collect_rewards`'s
    early return before the `errors_out` write, which drops the diagnostic in
    the all-errored case; `materialize_conditions`'s stray guard passing an
    empty non-skill `--dest` to `rmtree` and raising on a symlinked one;
    `SKILL.md`'s 27 multi-sentence lines against a peer corpus at 0 to 1; and
    type annotations present in one of six instrument scripts, which is an
    inconsistency inside one instrument rather than a missing-tooling
    observation.
    
    All remain deferred under D5 rather than fixed: the instrument is frozen at
    0.2.1 and its digest still recomputes to `3fdd30d1…`.
    
    `openspec validate validate-harborize-instrument --strict` passes.

Can't fast forward main (25d10e7) to fm/vx-harborize-rung0 (50af830). main (25d10e7) is not a direct ancestor of fm/vx-harborize-rung0 (50af830). Branches appear to have diverged at 623ec73:

* 50af830d4ddb3357608e111e8cff58087d932904 docs(openspec): complete the deferred instrument-defect list
* 612fcaf837f5244e586557b0614ded0e1f09de9f docs(openspec): fold the review round into the change documents
* beb12bcc25ea9c9c0e575e6b2bfd7cb72b86aca7 fix(evals): repair the harborize corpus under two-axis review
* 86edf4a836f314c3eb1fdddf6949ab37388a3eb0 chore(repo): mark the generated harbor heads as generated
* b08c4168b8c308443db13a79d32e06996b6451dd feat(evals): harborize rung 3 — static task validation
* 28f216a09aa39e27cc234c05b7d73c6869cceec1 feat(evals): harborize task packages, dual head
* 6e1db1539efb2dfe37e5767f26e89aa737c0c989 feat(evals): harborize rung 2 — host-side resolution
* 361aed974167940780901a8b8eef2adf55b619bc feat(evals): harborize rung 1 — adapter allowlist gate
* 4329bc32b5ea89bbdd32d8bc511053874e0bf215 feat(evals): harborize canary condition directory
* fb493bc566f8b9172ba83e8c02485f0b036bde3d feat(evals): harborize rung 0 — prerequisites green
* 0300b717bff5ca64862795966adc4aed2701cd78 docs(openspec): fold settled rung decisions into validate-harborize-instrument
* 5f33c36e23e2d4f36b23aefe7b9217c4d22ff3f6 chore(skills): exclude retired issues-beads* skills from nix delivery
* 1dcdac2e8ea14f8c739fde419667bb4fd1df0462 docs(openspec): add validate-harborize-instrument change
* 432e3573b79a14cfcc60f96f92d6f9f15a51a0f1 fix(skills): correct harborize container-boundary claims and guard non-consuming adapters
* 817ae8b426f6b373e024ab20f784c532db6b7e46 chore(skills): exclude harborize from nix delivery during in-tree iteration
* a1e9f5e6291fdb12d11c0a51548d7c5454fe4be4 feat(skills): repair harborize contracts and encode evaluation decisions
* 66b4c9e71165bce166af8f576feadd35f33860d4 feat(skills): add harborize evaluation-instrument skill to testing-and-quality
| * 25d10e7cc6efcf78b8f89ce9557778345df6c5ad feat(darwin): declare sshd server enablement for stibnite
| * f744232479c7f0364da324df6d3a181e9faeb4e7 docs(docs): add ADR-0022 on committed per-repository agent context
| * 108387f46cdd01dc1a10b3e837abe65096f09694 fix(hooks): don't gate kill word inside quoted search patterns
|/  
* 623ec732430b9ea37bf3dd361d3a86f0486456e8 fix(wezterm): stop masking Shift+Enter for omp kitty protocol

commit 623ec732430b9ea37bf3dd361d3a86f0486456e8
Author: Cameron Smith <cameron.ray.smith@gmail.com>
Date:   Tue Aug 18 14:07:04 2026 -0400

    fix(wezterm): stop masking Shift+Enter for omp kitty protocol

Rebase locally, and then force push to fm/vx-harborize-rung0.

…nstrument Fold three settled decisions into the change documents where they conflicted: - CLI installs track PyPI latest stable (captain decision 2026-08-15), not sha-pinned source checkouts; the upstream pins remain reading pins. tasks 1.5/1.6, plan Task 1 Step 3 + Global Constraints, proposal Impact. - The metered adapter is codex via the ChatGPT-subscription path (CODEX_FORCE_AUTH_JSON / CODEX_AUTH_JSON_PATH), recorded on-branch at the instrument's references/marketplace-program.md:46; supersedes the claude-code first-cell choice. tasks 9.4, plan Task 3 Step 4 cells file + Task 9 Steps 3-4 + Task 10 cost record, design D8, proposal Impact, spec registration/auth scenarios. - Gate-1 word 'harbor run -k 5' verified already encoded in tasks.md 8.1 and spec.md; no conflict, no edit.

Docker daemon answering; kernel probe exit 0 (egress control available, agent-phase no-network stays); harbor 0.21.0 and bench 0.7.4 installed from PyPI with all three ghq clones verified clean at their pins; workspace README recording pins, installed versions, freeze digest 3fdd30d1 (13 files, baseline 5f33c36), probe branch and attribution, claude-code install caveat; flake-evaluation guard clean (no .nix under the corpus, nixosConfigurations eval exit 0). tasks.md 1.1-1.11 ticked.

dir(C) at conditions/canary/ with exactly harborize-injection-canary (token HARBORIZE-CANARY-9F3A21, no deployed-skill collision); README records the by-design leakage flag and the shared-verifier-fork consequence. tasks.md 2.1-2.4 ticked.

Three design_matrix.py runs: aider negative control exit 1 with no manifest, acp: shorthand negative control exit 1 with no manifest, real codex cells file exit 0 with throwaway manifest discarded. tasks.md 3.1-3.3 ticked.

resolve_skills over dir(C) yields one entry with sha256 digest; missing-path, file-not-dir, and child-without-SKILL.md controls each raise on the host with exit 1; README labels the evidence resolution and request, never delivery, with the lock.json ordering that refutes it as delivery proof. tasks.md 4.1-4.3 ticked.

injection-canary (shared fork, empty skills dir, public baseline + agent-phase no-network) and pipeline-event-summary (separate verifier fork with own fixture copy, single binary reward sampling the preferences-json-querying contract); both exported lossless to -harbor heads; leakage audit mechanical exit 0 (summary key renamed per_pipeline to clear a check-1 false positive) and canary's expected check-1 flag recorded; package READMEs stamped 0.2.1 + three pins; extension audit clean and flake eval exit 0 after export. Container dry runs prove each oracle/verifier pair rewards 1 on the solved state and 0 otherwise. tasks.md 5.1-5.10 ticked.

bench tasks check structural exit 0 on both natives; harbor Task() constructs both exported heads; README records the _validate_tests early-return blind spot, re-confirmed at installed 0.21.0 against the pin anchor. tasks.md 6.1-6.3 ticked.

`modules/home/ai/evals/harborize/*-harbor/**` is emitted by `bench tasks export` and carries no marker of its own. A marker cannot be written into the emitted content: the exporter copies `environment/`, `oracle/` and `verifier/` verbatim and rebuilds `task.toml` through `tomli_w`, which emits no comments (benchflow `task/export.py:268-273`, `:367-388`), so a header would vanish on the next export and the committed tree would stop equalling a fresh one. `linguist-generated` is where the repository already puts this fact for `flake.lock`, `bun.lock` and `machines/*/facter.json`. Verified: `git check-attr linguist-generated` reports `true` for `pipeline-event-summary-harbor/task.toml` and `injection-canary-harbor/tests/test.sh`, and `unspecified` for the authored native `pipeline-event-summary/task.md`.

Applies the corpus-side findings of the review at `fm-review-2747/report.md`. Each claim below was reproduced before and after. The mechanical verifier no longer re-derives its expectation through the oracle's algorithm (S1). `verifier/test.sh` computed `expected` with code character-identical to `oracle/solve.sh`, so the gate could not fail on an oracle defect and any ambiguity in `task.md` resolved identically on both sides. It now holds a hand-derived literal and reads only `/logs/verifier/summary.json`, which also retires `verifier/_deps/` — two of the fixture's four copies. The median defect the shared derivation hid is fixed with it. `task.md:45` asks for "the median" while both sides took `durations[n // 2]`, the upper median. Reproduced on a 14-record fixture against the pre-change pair: a submission computing the true median (475) scores 0 and the oracle's upper median (500) scores 1. `task.md` now states the even-count convention and `oracle/solve.sh` implements it; the shipped 15-record answer is 450 under either convention, so the recorded evidence is unaffected. Severity of the new gate, exercised against seven independently mutated oracles: median off by one, counting success instead of failed, renaming `per_pipeline`, emitting a float median, emitting a bool for a count, adding an extra key, and writing no output all score 0; the correct oracle scores 1. Both packages' verifier fork is corrected to shared, and `pipeline-event-summary/verifier/Dockerfile` is dropped (P1). Task 5.6 was ticked and both READMEs recorded "separate", while both heads resolved to `VerifierEnvironmentMode.SHARED` — Harbor reads the mode from `verifier.environment_mode` or `[verifier.environment]` and from nothing else (`models/task/verifier_mode.py:10-21`), so shipping a `verifier/Dockerfile` infers nothing. Implementing separate was rejected on two independent grounds rather than on preference: - benchflow 0.7.4 refuses to launch a package declaring it rather than falling back to shared. "separate verifier sandboxes are parsed but not executed" (`runtime_capabilities.py:186-192`) is the reason string of a refusal, and `raise_for_task_runtime_support` is a fail-closed pre-launch gate (`sandbox/setup.py:676`, `:819-842`). Reproduced: `bench tasks check --level runtime-capability --sandbox docker` exits 0 on both packages as shipped and reports that issue the moment `sandbox_mode: separate` is added. proposal.md requires this package to pass under both runners. - Harbor's separate verifier empties `/logs/verifier` before running (`trial.py:599`) through the same host bind it mounts at `:686-692`, so the `summary.json` channel would be deleted and every agent including the oracle would score 0. `_run_shared_verifier` (`trial.py:536-567`) does not wipe. The canary oracle no longer aborts on an unset `HOME` (S5). Four of six discovery roots were `$HOME`-relative under `set -euo pipefail`, so a container exporting no `HOME` failed at line 6 before writing any output and the verifier read that as a canary alarm. Reproduced in-container: exit 1 before, exit 0 and reward 1 after. Base images are pinned by index digest, `ubuntu:24.04@sha256:561618e2…538ea`, the image the recorded evidence ran against. The floating tag had already moved to a later build than this host held, so the instrument was comparing measurements from different environments. The multi-arch index digest is pinned rather than a platform manifest so the line resolves on arm64 and amd64 alike. `checks/resolve_check.py` gains a docstring, a `main()` guard and an argument check; a bare invocation raised `IndexError` and importing it ran the whole body. The house PEP-723 header is deliberately not added: `uv run --script` always builds an isolated environment, so declaring `harbor` as a dependency would resolve a second copy and stop measuring the CLI under test. The recorded invocation is corrected to the harbor entrypoint's own interpreter, which is what the evidence used and what the plan's bare `python3` never was. Both READMEs are corrected where they were false and reflowed to one sentence per line. `injection-canary/README.md:19` claimed the token "lives only in the injected SKILL.md" while it is also in the verifier (P6). The opposite claim, that the shared fork puts the token within a running agent's reach, is also false: both runners upload the verifier's directory during the verification phase, after the agent phase ends (harbor `verifier/verifier.py:147-153`, phase order `trial/single_step.py:41` then `:52`; benchflow `task/verifier_core.py:385`). That is also why the literal expectation above is not a leak. The corpus README is split into a permanent half and a change-scoped rung log whose removal task 10.6 now carries (S10), and it carries the freeze-digest recipe, which previously lived only in the change's tasks.md and would have archived with it (P13). Verification run, all green: `bench tasks check --level structural` and `--level runtime-capability --sandbox docker` on both natives; `Task()` construction plus `resolve_task_verifier_mode` on both heads, `SHARED` each, matching what the READMEs now record; `bench tasks export --overwrite` lossless with 0 losses on both; `audit_leakage.py` exit 0 mechanical and exit 1 canary with the expected token flag; the extension audit and `nix eval` on both `nixosConfigurations` and `darwinConfigurations`; shellcheck clean on all six shell files; and container dry runs under the pinned images covering the mechanical package correct/tampered/absent and the canary with-skill/no-skill/HOME-unset. Not run, and out of scope for these rungs: sections 7 through 10, which need runner time or model spend. The instrument freeze is unaffected — the digest recomputes to `3fdd30d1…` and `git status` over the instrument directory is empty.

Applies the spec-axis findings of the review at `fm-review-2747/report.md`. Every anchor below was re-read in the installed harbor 0.21.0 and benchflow 0.7.4 trees rather than only at the reading pins. D11 and task 5.6 are corrected rather than left contradicting the tree (P1). The change recorded the mechanical package's fork as separate and "where separate mode gets exercised", built on reading benchflow's "separate verifier sandboxes are parsed but not executed" as a silent fallback. It is the reason string of a fail-closed refusal, so a package declaring separate does not launch at all; and Harbor's separate verifier empties `/logs/verifier` before running, which destroys the channel both packages depend on. Separate mode is exercised nowhere in this change and D11 now says so with the evidence. Task 5.6 additionally requires the fork to be verified by resolving it from the exported head rather than inferred from the source layout, which is what let a `verifier/Dockerfile` stand in for a declaration that was never made. D3 no longer stands against the settled metered-cell decision (P2). D3 excludes subscription-authenticated cells from any reported figure while the fold made the change's headline cost constant come from one. The narrowing is recorded with the mechanism that makes it sound rather than asserted: harbor's codex adapter has no billed-cost field and derives `cost_usd` from token counts against LiteLLM's pricing table (`codex.py:724-780`, `:1120-1124`, `:1194`), unlike claude-code which parses an authoritative `total_cost_usd` from its stream (`claude_code.py:858-879`, `:1463`, `:1525`). The reported figure is therefore a list-rate computation that does not vary with how the trial authenticated, and the confounds D3 names — throttling and concurrency caps — act on throughput, not on token counts, against a rung that is one short trial on one cell. The record now carries the pricing basis beside the auth mode, and a billed-rate figure is named as the dependent change's own measurement. Two mechanical failures compounding that are fixed: the extraction read `logs/harborize/gate1/cc-registration` while the run emits `codex-registration`, so the guard would have failed indistinguishably from a genuinely absent cost; and the provenance anchor was claude-code-only for a codex rung. LiteLLM's table was confirmed to carry `gpt-5.6-sol` and `gpt-5.6-luna` in the installed harbor tool environment, so the figure is obtainable; task 10.1 records how to re-check before spending. Fold residue is cleared at all six sites (P3). The Gate-1 verdict table and D10 named claude-code's `/logs/agent` bind mount as the registration witness for a codex cell whose destination sits in no bind mount; tasks.md 9.3 and plan.md attributed the no-network install failure to claude-code specifically when the mechanism is the phase boundary; the section 9 preamble anchored the install-only refutation on claude-code alone and now carries codex's own chain (`codex.py:1199-1207`, `:1413`, `:1415`, inside `async def run` at `:1333`); and the rung-0 preamble still offered the scratch-`UV_PROJECT_ENVIRONMENT` install path the settled decision removed and still described both CLIs as uninstalled. The citation is corrected in all four places that carried it. The instrument's `references/marketplace-program.md:46` is a cell roster that lists claude-code first, records no supersession, and does not mention `CODEX_AUTH_JSON_PATH`; its `:52-53` still carries the pre-decision costing policy. It is now cited only for the auth forms and model strings it does supply, the decision is attributed to itself, and the stale instrument lines are added to task 10.4's deferred list since the instrument is frozen. Task 1.8's freeze recipe excludes `__pycache__` and states its path precondition (P5). Tasks 2.3, 5.8 and section 3 all run the instrument's own `scripts/*.py`, CPython writes bytecode beside them, and this change's own `.gitignore` addition hides it — so complying with those tasks would have made task 10.3 report a freeze violation that never happened. Reproduced: the recorded recipe changes digest under a single stray `.pyc` and the corrected one does not. The digest is also a function of the path string `find` was given, which the recorded value did not say. Task 1.11's guard says what its `nix eval` proves (P7). `builtins.attrNames` forces the attrset spine and no module body, so it evidences import-tree's enumeration and that every module file parses — the hazard 1.10 names — and not that the skill composition or its exclusion list evaluates. proposal.md's Impact is corrected where it was false (P4, P8, P10). The "Unchanged … nix skill composition" line was untrue under either reading, since `5f33c36e` edits `excludedSkills` inside the change window; that edit is recorded rather than reverted. Two limits of that exclusion are recorded because it reads wider than it is: it does not withhold anything from marketplace publication, which is package-scoped, and `allSkills // extraSkills` means an `extraSkillDirs` entry with a colliding leaf name would silently re-add an excluded skill. The `.gitignore` and `.gitattributes` edits are now named with their reasons instead of arriving unexplained. Three stale citations are corrected (P11): the "444 non-nix files" figure matched no revision and is replaced with two reproducible counts at named revisions; `out.mkdir` is at `design_matrix.py:443`, not `:442`; and the File Structure block gains the four paths named elsewhere in plan.md. Task 10.4's deferred list is seeded with the seven instrument defects the review surfaced, each with its anchors, and task 10.6 is added to carry the corpus README's change-scoped half into verify.md at archive. `openspec validate validate-harborize-instrument --strict` passes. Checkbox counts: 34 checked across sections 1-6 and 20 open across 7-10, one more than before because 10.6 is new. No checkbox was ticked that the tree does not support; task 5.6's tick is now backed by `resolve_task_verifier_mode(Task(<head>).config) == SHARED` on both heads matching what both READMEs record. Not addressed here, deliberately: proposal.md's `linear_story_state` frontmatter, which P12 flags as reading `Todo` against 34 checked boxes. The Linear mirror is owned outside this worktree and this task is instructed not to edit it.

Task 10.4's list carried seven of the review's instrument findings. Adds the five remaining, so the list the next instrument revision reads is the whole set rather than the subset one commit happened to reach. New entries: `census.load_membership`'s unguarded `data["skills"]` read against its sibling's validation of the same input; `collect_rewards`'s early return before the `errors_out` write, which drops the diagnostic in the all-errored case; `materialize_conditions`'s stray guard passing an empty non-skill `--dest` to `rmtree` and raising on a symlinked one; `SKILL.md`'s 27 multi-sentence lines against a peer corpus at 0 to 1; and type annotations present in one of six instrument scripts, which is an inconsistency inside one instrument rather than a missing-tooling observation. All remain deferred under D5 rather than fixed: the instrument is frozen at 0.2.1 and its digest still recomputes to `3fdd30d1…`. `openspec validate validate-harborize-instrument --strict` passes.

A four-lane adversarial verification of the review-round commits found sixteen defects, most of them introduced by those commits rather than inherited. Each claim below was reproduced before and after. The mechanical verifier did not write a reward on every path, which is worse than scoring 0: both runners treat a missing reward file as a trial-level crash rather than as a score, so a hostile submission voided the trial. Two things sat outside the try — `integers_only`, which recurses, and the reward write itself. A submission nested about 500 deep raised RecursionError with no reward file written; an oversized one hit the verifier timeout or the OOM killer with the same result. The verifier now writes `0` before reading anything and upgrades to `1` only on success, and the comparison moved inside the guard. Reproduced: 1500-deep nesting was exit 1 with no reward file, now exit 0 with reward 0, while all nine correctness cases are unchanged. The freeze check is redesigned around the git tree object the instrument path resolves to, `64bc599c…`, with the content digest demoted to a secondary working-tree check. Three reasons, all reproduced. A `.DS_Store` flips the digest while `git status` stays clean — on macOS one Finder visit plants one, and `*.log`, `result*`, `*~`, `*.swp` and a `.pyc` outside a `__pycache__` directory behave the same, so the `__pycache__` exclusion fixed one instance of a class and left six. The digest is blind to mode changes, symlinks and empty directories, since it hashes regular files only. And it survives a rebase of the base branch, where a commit id does not — which is not hypothetical: the recorded baseline `5f33c36e` was superseded while this branch was open, and the tree object is identical at the old base, the new one, and here. The claim that a trailing slash changes the digest is false and is removed; `find` normalizes it on all three implementations present. The other three path forms do change it and are kept. Residue the review-round commits missed, each contradicting a sibling they did rewrite: task 5.3 still asserted the channel survives a fork change, which 5.2 one line above now refutes; `plan.md` Task 9 Step 4 still asserted a real agent can reach the token, which the same commit corrected two hundred lines earlier; a spec scenario heading kept its canary scope while its body was generalized, and its parent requirement is widened to match; and `plan.md` carried the uncorrected freeze recipe at both Task 1 Step 5 and Task 10 Step 4, so the operator-facing document still had the defect its commit message claimed to have fixed. A sentence introduced into the canary README read as the opposite of the mechanism — the token is exactly what the injected skill tree carries — and sat one line above a sentence presupposing the correct version. Rewritten to say what is true: the token exists in the injected `SKILL.md` and in this package's own verifier, and nowhere in the skill corpus under evaluation. Anchor and count corrections, each verified in the installed trees: `cost_usd` sits on `JobStats` (`result.py:28`, field at `:41`), not on `JobResult` (`:236`), so the accessor is `result.stats.cost_usd`; `set -euo pipefail` is at `design_matrix.py:357`, not `:359`; the audit_leakage suffix filter is at `:136-137`, not `:137-138`; `verifier_core.py:385` is in `_verify_test_script` (`:346`) reached from `verify()` (`:260`), not lexically inside it; `design_matrix.py:101-107` is `cond_dir`'s own docstring, whose scoped claim holds at all three call sites, so that sub-claim is withdrawn; the "444 non-nix files" figure survived in `design.md` and `plan.md` after being replaced only in `tasks.md`, and the replacement counts were themselves stale — 433 at `origin/main` and 480 here, of which 34 are the corpus; the 10.4 preamble said seven where its list holds nine; and the "2 of 14 literals" evidence is now dated to the revision that produced it, since repairing the verifier moved it. `_find_skill_dirs` errors only on a stray non-hidden *directory*; a stray file is filtered by the `child.is_dir()` test at `skills.py:396`. The condition directory's emptiness is therefore maintained rather than enforced, and both places that cited it now say so. Two scope preconditions are recorded rather than left implied. The no-wipe claim is true of `_run_shared_verifier` and of single-step trials only: `multi_step.py:202` calls `_reset_shared_step_verifier_dirs` (`:338-342`) before every shared step verifier, so a package that grows `[[steps]]` loses the channel; `SingleStepTrial.__init__` raises on a stepped task (`single_step.py:28-29`), so the two cannot be confused silently. And neither package clears `bench tasks check --level publication-grade`, for a reason unrelated to D7's co-present argument — `structural_checks.py:224-226` wants a `verifier.md` neither package authors — recorded so publication-grade is not later cited as met. Two documents still described the prerequisites rung as currently unmet while all eleven of its boxes are checked and both CLIs are installed. Verification, all green: `bench tasks check` at `--level structural` and `--level runtime-capability --sandbox docker` on both natives; both committed heads byte-identical to a fresh export (`diff -r`, including the export report); `resolve_task_verifier_mode` `SHARED` on both; `audit_leakage.py` exit 0 mechanical and exit 1 canary; shellcheck clean on all eight corpus shell files (the previous commit body said six — there are eight, and all pass); `py_compile` clean; extension audit and `nix eval` on both configuration sets; `openspec validate --strict` valid; instrument tree `64bc599c…` and digest `3fdd30d1…` both unchanged with an empty `git status` over the instrument.

The first rung-4 run scored 0 with the skill delivered, and the cause is a channel that only one runner preserves. Both packages routed their oracle-to-verifier deliverable through `/logs/verifier`, chosen from Harbor's behaviour alone: a shared verifier there performs no wipe, so the file is present when the verifier greps for it. BenchFlow clears that directory's contents on the agent container immediately before the verifier runs, unconditionally. `harden_before_verify` executes `_CLEAR_VERIFIER_DIR_CMD` (`sandbox/lockdown.py:775-784`, call at `:1205-1212`) whether or not the directory is a bind mount; the mount-aware conditional clear in `_verify_test_script` (`task/verifier_core.py:360-372`) is a different branch and never reached the file. The bind survives and the content does not, so the canary scored 1 under Harbor and 0 under BenchFlow with no error and no diagnostic beyond a `grep` miss — the runner-dependent silent zero this corpus exists to detect, arrived at from the opposite direction to the one predicted. Reproduced before and after against benchflow 0.7.4: an instrumented copy of the canary printed the mount table and its own written file inside the container, then the verifier reported the same path missing. The same copy with the deliverable moved scored 1. `/logs/artifacts` is bind-mounted for the whole trial by both runners (`sandbox/docker.py:186-201`; `models/trial/paths.py:38`, `:195-202`), no hardening step touches it, and it is additionally the one path that would survive Harbor's separate fork, whose wipe at `trial.py:599` is followed by an artifact re-upload (`trial.py:601-607`). `/logs/verifier` keeps its one role under both runners: the reward file the verifier itself writes, after the clear. Both Harbor heads are re-exported rather than hand-edited, which also rewrites `source_task_dir` in each export report to this checkout, as that field records.

Rung 4 ran under benchflow 0.7.4 and rung 5 under harbor 0.21.0, both oracle-only with no model calls. Canary with the condition directory: reward 1, zero errors, with `effective_skills_dir` equal to the host directory passed and no fidelity exception. Falsifiability control with no skill: reward 0. Harbor, five trials per package: reward 1.0 on all five for each, zero errored trials, and no `agent/exit-code.txt` anywhere, which is the passing reading because `OracleAgent.run` writes that file only on a nonzero return (`agents/oracle.py:149-151`). The canary head takes `--skill <dir(C)>` on the Harbor line; without it the same head scores 0 across five trials, which is the Harbor-side counterpart of the no-skill control. No trial errored in any job, so the errored-trial triage was never entered and is recorded as vacuous rather than as performed. Two corrections the runs forced, each recorded with the anchors rather than quietly applied. D11's channel argument is inverted: it preferred `/logs/verifier` over a workspace path to avoid a package that passes under one runner and fails under the other, and `/logs/verifier` is exactly such a path. And task 7.4's fidelity control cannot be built the way it prescribed, because BenchFlow itself stages the host skills directory into `environment/_deps/skills` and appends the `COPY` line the recipe wanted to hand-write (`sandbox/setup.py:529-568`, `rollout/__init__.py:979-1016`), then destroys any pre-baked discovery directory with `rm -rf <dest> && ln -sfn` (`agents/install.py:61-90`). Both constructions scored 1. The control that does provoke `experiment_fidelity/skill_deployment_missing` puts a symlinked skill directory in the condition directory, where the host glob follows the symlink and the staging copy drops it by design (`_stage_ignore`, the #411 fix), so the in-container catalogue is genuinely short one skill. Selection: the runner-level runs above are what would fail if the channel change were wrong, since they exercise the delivery path end to end under both runners. Re-run alongside them after the edit: `bench tasks check` at schema, structural and runtime-capability on both native trees; `Task()` construction and `resolve_task_verifier_mode` on both re-exported heads, still SHARED; the leakage audit, unchanged at exit 0 on the mechanical package and the expected single check-1 flag on the canary; and the flake-evaluation guard, no `.nix` under the corpus and `nix eval .#nixosConfigurations` exit 0. `nix flake check` was deliberately not run: this diff touches no nix file, and CI covers it.

mergify Bot assigned cameronraysmith Aug 18, 2026

cameronraysmith added 6 commits August 18, 2026 16:20

chore(skills): exclude retired issues-beads* skills from nix delivery

9d382e8

cameronraysmith added 12 commits August 18, 2026 19:59

feat(evals): harborize rung 1 — adapter allowlist gate

d6e34ee

Three design_matrix.py runs: aider negative control exit 1 with no manifest, acp: shorthand negative control exit 1 with no manifest, real codex cells file exit 0 with throwaway manifest discarded. tasks.md 3.1-3.3 ticked.

cameronraysmith force-pushed the fm/vx-harborize-rung0 branch from 50af830 to a3b2995 Compare August 19, 2026 00:00

cameronraysmith added 2 commits August 18, 2026 20:59

cameronraysmith changed the title ~~validate-harborize-instrument: free rungs 0-3 plus canary and mechanical packages (tasks 1-6)~~ validate-harborize-instrument: free rungs 0-5 plus canary and mechanical packages (tasks 1-8) Aug 19, 2026

Navigation Menu

validate-harborize-instrument: free rungs 0-5 plus canary and mechanical packages (tasks 1-8) - #2747

validate-harborize-instrument: free rungs 0-5 plus canary and mechanical packages (tasks 1-8)#2747
cameronraysmith wants to merge 20 commits into
mainfrom
fm/vx-harborize-rung0

cameronraysmith commented Aug 18, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

cameronraysmith commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Grounding

Review round

Blast radius

Rungs 4 and 5

Verification

Base

Verification round

Uh oh!

github-actions Bot commented Aug 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

cameronraysmith commented Aug 18, 2026 •

edited

Loading