experiment: OpenShell filesystem hierarchical policy

[ben-alkov]

Summary

• Test whether OpenShell's Landlock-based filesystem policy enforces hierarchical path precedence (read_only child inside read_write parent)
• 24 assertions across overlap, read-write, read-only, deny, and edge-case categories
• Key finding: Landlock unions permissions additively — read_only inside read_write is not enforced (H0 refuted, 20/24 passed)

Test plan

• Ran ./run.sh with synthetic fixtures on Fedora 44 / kernel 7.0.12
• Results documented in findings.md

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 3:12 PM UTC · Completed 3:23 PM UTC
Commit: bb4d904 · View workflow run →

[fullsend-ai-review]

Review

Findings

Medium

• [error-handling-gap] openshell-filesystem-hierarchical-policy/run.sh:175 — run_probe() merges stdout and stderr into probe-output.jsonl via sandbox_exec (which does 2>&1 internally). Any SSH banner, Python traceback, or stderr warning will be written as a non-JSON line into the JSONL file. parse_results() pipes each line through python3 json.load which will crash with JSONDecodeError on the first non-JSON line, aborting the result table and losing all subsequent results.
  Remediation: Stop merging stderr in sandbox_exec. In run_probe, redirect only stdout to the JSONL file and send stderr to a separate log: sandbox_exec "python3 /sandbox/probe.py" > "$RESULTS_DIR/probe-output.jsonl" 2>"$RESULTS_DIR/probe-stderr.log".

Low

• [technical-accuracy] openshell-filesystem-hierarchical-policy/findings.md:157 — The ABI V2 downgrade speculation ("may explain why read_only within read_write is not enforced") is self-correcting within the same paragraph but could be clearer. The additive permission model is fundamental to all ABI versions. Recommendation 3 should focus on the 2 skipped rules specifically rather than ABI version as a potential root cause.

• [command-injection] openshell-filesystem-hierarchical-policy/run.sh:103 — $REPO from --repo flag is interpolated into SSH command strings without format validation. Mitigated by gh api pre-validation (non-matching values fail before reaching sandbox_exec) and sandbox isolation, making this a self-attack scenario. Matches the existing pattern in openshell-policy-bypass/run.sh.
  Remediation: Add a strict pattern check after parsing: [[ "$REPO" =~ ^[a-zA-Z0-9._-]+/[a-zA-Z0-9._-]+$ ]].

• [edge-case] openshell-filesystem-hierarchical-policy/src/probe.py:97 — try_read calls os.path.isdir() which can itself raise PermissionError on Landlock-restricted paths. Caught by the outer except OSError, but the error attribution is imprecise (reported as "read" when it was a stat check). No impact on current results.

• [data-exposure] openshell-filesystem-hierarchical-policy/results/sandbox-logs.txt — Sandbox logs with ephemeral UUIDs, container IPs (10.200.0.x), and SSH channel IDs are committed. Low risk since artifacts are ephemeral, and this matches the existing pattern in openshell-policy-bypass.

• [documentation-consistency] openshell-filesystem-hierarchical-policy/HOW_TO.md:59 — Expected Output section doesn't mention findings.md where detailed results are documented. Consider adding a pointer.

Previous run

Review

Findings

Medium

• [command-injection] openshell-filesystem-hierarchical-policy/run.sh:103 — The $REPO variable from user input (--repo flag) is interpolated into a string passed to sandbox_exec (ssh), which forwards it to a remote shell. A crafted value could execute arbitrary commands inside the sandbox. Mitigated by sandbox isolation and gh api pre-validation, but no format validation is applied to the input.
  Remediation: Validate $REPO against a strict pattern (e.g., '^[a-zA-Z0-9._-]+/[a-zA-Z0-9._-]+$') immediately after parsing.

• [error-handling-gap] openshell-filesystem-hierarchical-policy/run.sh:175 — run_probe() merges stdout and stderr into probe-output.jsonl via 2>&1. Any SSH warnings or Python tracebacks will interleave with JSONL output, causing parse_results() to crash on non-JSON lines.
  Remediation: Redirect only stdout to the JSONL file; send stderr to a separate log or /dev/null.

• [test-integrity] openshell-filesystem-hierarchical-policy/findings.md — The document speculates the ABI V2 downgrade "may explain why read_only within read_write is not enforced." This is misleading — ABI V2 added LANDLOCK_ACCESS_FS_REFER for cross-directory renames, unrelated to hierarchical override semantics. The additive permission model is fundamental to all ABI versions. Recommendation 3 ("Investigate ABI V2 downgrade") compounds this by directing investigation toward a non-factor.
  Remediation: Remove or clearly dismiss the ABI-downgrade speculation. Recommendation 3 should focus on the 2 skipped rules specifically, not on the ABI version as a root cause.

Low

• [logic-error] openshell-filesystem-hierarchical-policy/run.sh:8 — REPO="${1:-}" initializes from $1, but main() parses --repo as a named flag. The initial assignment is dead code that could mask bugs if the parser changes. Change to REPO="".

• [data-consistency] openshell-filesystem-hierarchical-policy/.gitignore:1 — The .gitignore is empty (0 bytes) while the implementation plan specifies it should contain results/. The results/ directory is committed with sandbox logs and probe output. If results are intentionally committed for reproducibility, this is fine — but the empty .gitignore file should either be populated or removed.

• [data-exposure] openshell-filesystem-hierarchical-policy/results/sandbox-logs.txt — Sandbox logs with ephemeral UUIDs, container IPs (10.200.0.x), and SSH channel IDs are committed. Low risk since artifacts are ephemeral, but unnecessary infrastructure metadata in version control.

• [data-exposure] openshell-filesystem-hierarchical-policy/superpowers/2026-06-12-1630-openshell-filesystem-policy-experiment-implementation.md:4 — Session docs expose developer local paths (/home/balkov/...), including CA key location and GPG keyring path. Consider redacting to ~ or $HOME.

• [edge-case] openshell-filesystem-hierarchical-policy/src/probe.py:97 — try_read calls os.path.isdir() which can itself raise PermissionError on Landlock-restricted paths. Caught by the outer except OSError, but the error attribution is imprecise (reported as "read" when it was a stat check). No impact on current results.

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 3:55 PM UTC · Completed 4:11 PM UTC
Commit: 2f23a86 · View workflow run →