chore: check and report four-eyes to Kosli

[mbevc1]

Add four eyes evaluation and attest result to Kosli.

[claude]

Claude finished @mbevc1's task in 6m 5s —— View job

---

Review of PR #891 — Four-eyes evaluation and attestation

• Run static analysis (go build, go vet, go mod tidy)
• Read changed files and understand current state
• Review code quality, correctness, and edge cases
• Post inline comments and summary

---

Static Analysis

go build ./..., go vet ./..., and go mod tidy all pass cleanly. No Go code changes in this PR (workflow YAML, Rego policy, JSON schema, shell script only).

Key Finding

YAML parsing bug in workflow steps (high priority): The new Evaluate trails and Report four-eyes attestation steps use run: as a plain YAML scalar with \ line continuations and embedded JSON (--params '{"attestation_name": "pr"}'). This combination has two problems:

1. The ": " inside the JSON triggers YAML's mapping indicator detection, causing a parse error (verified with a spec-compliant YAML parser).
2. \ in plain scalars doesn't produce shell line continuation — YAML folds newlines to spaces, turning \ into \ (escaped space), which can corrupt argument parsing.

Fix: Use run: | (block literal scalar) for these steps. See inline comment for the suggested replacement.

Other Notes

• Default attestation_name mismatch (low priority): The Rego policy defaults to "pr-review" (line 16) but the workflow uses "pr". The --params override handles CI correctly, but the default is misleading for local testing. Inline comment.

What looks good

• Previous review feedback is well addressed: --org flags added, error swallowing replaced with proper JSON fallback, scratch file removed, --params for attestation name added.
• Clean separation of violation_reasons (detection) and violations (formatting) in the Rego policy — ensures a sprintf failure can't flip allow to true.
• Setup script has proper set -euo pipefail, KOSLI_API_TOKEN validation, and now accepts org as $1.
• The || echo '{"allow":false,...}' fallback on evaluate failure is a smart pattern — command failures produce a valid (failing) attestation instead of breaking the pipeline.
• release-flow-template.yml change is clean and minimal.

[claude]

Bug: YAML plain scalar breaks on embedded JSON colon.

This run: step is a YAML plain scalar (no |). The ": " inside --params '{"attestation_name": "pr"}' is interpreted by the YAML parser as a mapping indicator, causing a parse error. I verified this with Python's PyYAML (spec-compliant) and it fails:

ScannerError: mapping values are not allowed here
  --params '{"key": "val"}'
                        ^

Additionally, the \ line continuations don't work as intended in plain scalars — YAML folds newlines to spaces, so \ + newline becomes \ (escaped space in bash), which merges with the next argument (e.g., --policy becomes --policy with a leading space).

Fix: Use a block literal scalar (run: |) so newlines are preserved and \ works as shell line continuation:

Suggested change

	- name: Evaluate trails for four-eyes to Kosli
	if: ${{ inputs.report_to_kosli == 'all' }}
	env:
	KOSLI_API_TOKEN: ${{ secrets.kosli_api_token }}
	run: kosli evaluate trails ${{inputs.trail_name}} \
	--policy "./bin/never_alone/four-eyes-policy.rego" \
	--show-input \
	--flow ${{inputs.flow_name}} \
	--org ${{inputs.kosli_org}} \
	--no-assert \
	--params '{"attestation_name": "pr"}' \
	--output json > "4eyes-eval-${{inputs.trail_name}}.json" || echo '{"allow":false,"violations":["evaluate command failed"]}' > "4eyes-eval-${{inputs.trail_name}}.json"
	run: |
	kosli evaluate trails ${{inputs.trail_name}} \
	--policy "./bin/never_alone/four-eyes-policy.rego" \
	--show-input \
	--flow ${{inputs.flow_name}} \
	--org ${{inputs.kosli_org}} \
	--no-assert \
	--params '{"attestation_name": "pr"}' \
	--output json > "4eyes-eval-${{inputs.trail_name}}.json" \
	|| echo '{"allow":false,"violations":["evaluate command failed"]}' > "4eyes-eval-${{inputs.trail_name}}.json"

The existing kosli begin trail and kosli attest pullrequest steps work because they don't use \ or embedded JSON — YAML folding produces a clean single-line command. But this step needs run: |.

[claude]

Same issue: needs run: | for consistency and correctness.

While this step doesn't have the JSON colon problem, it still uses \ in a plain scalar. Switch to block literal for safety:

Suggested change

	- name: Report four-eyes attestation to Kosli
	if: ${{ inputs.report_to_kosli == 'all' }}
	env:
	KOSLI_API_TOKEN: ${{ secrets.kosli_api_token }}
	run: kosli attest custom \
	--type "four-eyes-result" \
	--name "four-eyes-result" \
	--attestation-data "4eyes-eval-${{inputs.trail_name}}.json" \
	--attachments "./bin/never_alone/four-eyes-policy.rego" \
	--trail ${{inputs.trail_name}} \
	--flow ${{inputs.flow_name}} \
	--org ${{inputs.kosli_org}}
	run: |
	kosli attest custom \
	--type "four-eyes-result" \
	--name "four-eyes-result" \
	--attestation-data "4eyes-eval-${{inputs.trail_name}}.json" \
	--attachments "./bin/never_alone/four-eyes-policy.rego" \
	--trail ${{inputs.trail_name}} \
	--flow ${{inputs.flow_name}} \
	--org ${{inputs.kosli_org}}

[claude]

Nit: Default doesn't match the workflow.

The workflow passes --params '{"attestation_name": "pr"}', so this default won't fire in CI. But anyone running the policy standalone (local testing, debugging) gets "pr-review", which doesn't match the actual attestation name "pr". Consider aligning:

Suggested change

	} else := "pr-review"
	} else := "pr"