Showcase: automated template compatibility testing + AI debugging workflow alignment

[kalanyuz]

PR: Showcase automated template compatibility testing and AI debugging workflow alignment

Summary

Brief overview of what this PR implements:

1. Template Compatibility Guardrail: Adds a unified compatibility suite for template IDs 1..5 (including IDs 3 and 5) with a drift canary.
2. Process and Operational Hardening: Adds CI path-gated enforcement, documents embedded-vs-dynamic source modes, and codifies repeatable QA/template/TUI agent workflows.

flowchart LR
    A[Template or creinit change] --> B[Path Filter Job]
    B --> C[TestTemplateCompatibility]
    C --> D[PR Signal]
    D --> E[QA + Skills + Docs Alignment]

Loading

Changes

1. Template Compatibility Automation

File: test/template_compatibility_test.go

What changed:

• Added TestTemplateCompatibility table-driven suite for template IDs 1..5.
• Added explicit cases for TS_HelloWorld_Template3 and TS_ConfHTTP_Template5.
• Added drift canary TestTemplateCompatibility_AllTemplatesCovered.

Result: Template init/build/simulate regressions are caught deterministically and coverage drift is flagged early.

2. CI Enforcement Path

File: .github/workflows/pull-request-main.yml

What changed:

• Added template-compat-path-filter.
• Added ci-test-template-compat on Linux + Windows.
• Runs go test -v -timeout 20m -run TestTemplateCompatibility ./test/ when template-impacting paths change.

Result: Compatibility checks become a targeted PR guardrail instead of ad-hoc local validation.

3. Template Repository Operational Setup

Files:

• submodules.yaml
• scripts/setup-submodules.sh
• Makefile
• .gitignore

What changed:

• Added clone/update/clean workflow for external cre-templates reference checkout.
• Added make targets and managed gitignore behavior.

Result: External template workspace setup is reproducible without using Git submodules.

4. Agent and Framework Alignment

Files:

• AGENTS.md
• .claude/skills/cre-add-template/**
• .claude/skills/cre-qa-runner/**
• .claude/skills/cre-cli-tui-testing/**
• .claude/skills/using-cre-cli/**
• testing-framework/*.md
• .claude/skills/skill-audit-report.md

What changed:

• Documented current embedded template source and branch-gated upcoming dynamic pull mode.
• Updated skills with clearer invocation boundaries and dynamic-mode handling guidance.
• Aligned testing-framework docs to dual-source model and advisory-first dynamic validation rollout.

Result: Consistent operator + agent guidance across docs, skills, and CI expectations.

Impact

Modified (selected files)

• ✅ test/template_compatibility_test.go - New unified compatibility suite + canary.
• ✅ .github/workflows/pull-request-main.yml - Path filter + compatibility job.
• ✅ AGENTS.md - Source-mode architecture and maintenance workflow.
• ✅ submodules.yaml - External template repo relationship config.
• ✅ scripts/setup-submodules.sh - Deterministic clone/update/clean script.
• ✅ .claude/skills/cre-add-template/SKILL.md - Template workflow and checklist boundaries.
• ✅ .claude/skills/cre-qa-runner/SKILL.md - QA runbook/reporting + source provenance guidance.
• ✅ testing-framework/README.md - Embedded baseline + branch-gated dynamic framing.

Affected (no runtime behavior change)

• ✅ cmd/creinit/* runtime flow remains embedded-template baseline.
• ✅ Dynamic-template behavior is documented as branch-gated, not active default behavior.

Testing

Compatibility Suite

• ✅ go test -v -timeout 20m -run TestTemplateCompatibility ./test/
• ✅ Includes IDs 1,2,3,4,5 and canary coverage guard.

Skill/Docs Consistency Checks

• ✅ Skill description overlap scan (rg "^description:" .claude/skills/*/SKILL.md).
• ✅ Cross-doc alignment pass across AGENTS.md, testing-framework/*.md, and updated skill docs.

Deployment Notes

Pre-Deployment Requirements

• No production deployment steps required.
• CI workflow changes require standard Actions permissions already present in repo workflow.

Deployment Order

1. Merge PR.
2. Let PR workflow enforce template-compat job on relevant path changes.
3. Use updated skills/runbooks for subsequent template additions and pre-release QA.

Backward Compatibility

• Backward compatible.
• No CLI API/runtime behavior changes introduced by this PR.

Verification Checklist

• Unified compatibility test covers template IDs 1..5.
• Canary guard exists for registry/test-table drift.
• CI path filter + compatibility job added.
• Skills updated for invocation clarity and dynamic-mode branch gating.
• Testing framework docs aligned to embedded-now + dynamic-next model.

Rollback Plan

If issues occur:

1. Revert workflow additions in .github/workflows/pull-request-main.yml.
2. Revert compatibility test file test/template_compatibility_test.go.
3. Revert docs/skills updates in AGENTS.md, .claude/skills/**, and testing-framework/*.md.

To fix the problem, add an explicit permissions block to the template-compat-path-filter job so its GITHUB_TOKEN is scoped to the least privileges it needs. This job only checks out code and runs local git diff using the cloned repo; it doesn’t need to write to GitHub or access special scopes like id-token. Thus, contents: read is sufficient and matches CodeQL’s suggested minimal configuration.

Concretely, in .github/workflows/pull-request-main.yml, under jobs: template-compat-path-filter:, add a permissions: section at the same indentation level as runs-on: and outputs:, with contents: read. No additional imports or methods are needed; it is a pure YAML configuration change and does not alter the job’s functional behavior.