docs: add sample quality and model E2E gates

[Tsuyoshi Ushio (TsuyoshiUshio)]

Summary

This draft proposes explicit sample-design and sample-validation gates for medium and larger features. It separates that discussion from PR #174 so the ExecPlan proposal can be reviewed independently.

The goal is to prevent runnable samples from becoming overloaded system-test fixtures and to ensure that model-authored structured inputs are validated across the real model/runtime boundary.

Proposed changes

• Add a sample-design checkpoint to the add-feature skill: identify the target learner, teach one primary capability, define a user-centered story and observable result, and explicitly decide whether to extend an existing sample or create a dedicated one.
• Add an independent sample-review checkpoint that verifies the documented path is runnable and that extracting a capability does not break the original sample.
• Require opt-in real-model E2E when an LLM authors tool arguments, dependency graphs, configuration, or equivalent structured runtime input. Unit and integration tests that bypass the model remain necessary, but do not substitute for this boundary.
• Require uniquely named Foundry environment variables, a clear skip when not opted in, an actionable failure for partial configuration, exact non-secret evidence, refusal to overwrite user files, and cleanup limited to files owned by the sample script.
• Extend the FRD template with a dedicated Sample / user journey section and corresponding test-plan items.

Motivation and examples

PR #170 is the motivating example. It exposed a gap only when a human ran the sample through the full natural-language → model → tool call → workflow path. PR #170 now proposes a concrete opt-in harness at samples/workflow-retry-policy/scripts/run-e2e.py, including Skill-resource loading, safe local.settings.json creation, real Foundry execution, terminal-status checks, and ownership-safe cleanup.

This process PR does not depend on PR #170 merging. Its requirements are intentionally generic and apply beyond workflows. The exact guidance can be refined after #170 merges and the concrete harness receives review.

PR #174 remains focused only on whether to adopt ExecPlans.

Opt-in behavior

The live E2E runs only when all sample-specific Foundry variables are present. With none present, it skips clearly. Partial configuration fails with an actionable message. A sample-owned script may generate ignored local settings only if it can prove ownership, must not overwrite existing or concurrently created files, and must remove only resources it created.

Feedback requested

1. Is one primary capability per sample the right default, and when should extending an existing sample be preferred?
2. Is the natural-language → model → structured tool input → parser/runtime → terminal-result boundary the right E2E requirement for model-authored behavior?
3. Are the opt-in environment, skip, evidence, and cleanup rules practical for local development and CI-assisted validation?
4. Should the concrete PR design: define Dynamic Workflow task execution policy #170 harness become the recommended template after it merges?