Skip to content

Commit 0a2663c

Browse files
zh1pengzh1peng
committed
Add psyflow-validate contracts and validator gates
1 parent c6293f9 commit 0a2663c

28 files changed

Lines changed: 2060 additions & 5 deletions

‎.github/workflows/ci.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,13 @@ jobs:
1212
- uses: actions/setup-python@v5
1313
with:
1414
python-version: "3.10"
15+
- name: Install minimal validator dependency
16+
run: python -m pip install pyyaml
1517
- name: Compile Python sources
1618
run: |
1719
python -m compileall -q psyflow tests
1820
python -m py_compile setup.py
21+
- name: Run contracts validator smoke
22+
run: python -m psyflow.validate 'psyflow/templates/cookiecutter-psyflow/{{cookiecutter.project_name}}'
1923
- name: Run unit tests
2024
run: python -m unittest discover -s tests -p "test_*.py"

‎ChangLog.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,29 @@
11
# psyflow change log
22

3+
## 0.1.9 (2026-02-16)
4+
5+
### Summary
6+
- Added versioned task contracts at `psyflow/contracts/v0.1.0` for standardized psyflow/TAPS task development and audit.
7+
- Added new validator CLI: `psyflow-validate <task_path>` with per-contract PASS/WARN/FAIL feedback and actionable suggestions.
8+
- Added contract adoption tracking in task metadata via `taskbeacon.yaml -> contracts.psyflow_taps`.
9+
- Updated cookiecutter template to include `taskbeacon.yaml`, `CHANGELOG.md`, and `assets/README.md` so new tasks match required structure by default.
10+
- Expanded contract coverage with explicit checks for:
11+
- `.gitignore` task artifact rules (`outputs/*` / `data/*` migration-safe),
12+
- stimulus schema and asset-backed path conventions (`image`/`movie`/`sound` from `assets/`),
13+
- responder/sampler plugin standards (`sim.responder.type`, local `responders/` class checks).
14+
- Added stricter config validation semantics across contracts (`mandatory` / `optional` / `recommended` keys and value specs).
15+
- Updated CI to install minimal validator dependency (`pyyaml`) and run contract validation smoke checks against the task template.
16+
- Added validator regression tests for asset-backed stimuli and responder contract failures.
17+
18+
### New modules/files
19+
- `psyflow/validate.py`
20+
- `psyflow/contracts/v0.1.0/*`
21+
- `tests/test_validate.py`
22+
23+
### Packaging/CLI
24+
- New console script: `psyflow-validate`
25+
- Included `contracts/**/*` in package data.
26+
327
## 0.1.8 (2026-02-16)
428

529
### Summary

‎docs/tutorials/cli_usage.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ For convenience, psyflow also provides task launch shortcuts:
2323
| `psyflow-run <task>` | Run task in human mode via shortcut | `psyflow-run T000006-mid` |
2424
| `psyflow-qa <task>` | Run task in QA mode via shortcut | `psyflow-qa T000006-mid --config config/config_qa.yaml` |
2525
| `psyflow-sim <task>` | Run task in simulation mode via shortcut | `psyflow-sim T000006-mid --config config/config_sim.yaml` |
26+
| `psyflow-validate <task>` | Validate task structure/contracts | `psyflow-validate T000006-mid` |
2627

2728
## 1. Creating a New Project
2829

@@ -69,3 +70,9 @@ Each shortcut calls the task's `main.py` with explicit mode, so task-local argum
6970
```bash
7071
psyflow-qa T000006-mid --config config/config_qa.yaml --set-maturity smoke_tested
7172
```
73+
74+
Validate contract compliance (file structure + metadata + config + runtime wiring):
75+
76+
```bash
77+
psyflow-validate T000006-mid
78+
```

‎psyflow/contracts/v0.1.0/README.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# psyflow contracts v0.1.0
2+
3+
These contracts define practical standards for building auditable psyflow/TAPS tasks.
4+
5+
## What is enforced
6+
- Task file/folder skeleton
7+
- `.gitignore` artifact-ignore rules
8+
- Task metadata (`taskbeacon.yaml`)
9+
- Config structure and explicit value/type constraints
10+
- mandatory/optional/recommended keys and value specs
11+
- stimulus type standards and asset-backed path conventions
12+
- Runtime entrypoint pattern (`main.py`)
13+
- Trial runtime pattern (`src/run_trial.py`)
14+
- Responder/sampler plugin standards (`config_sim.yaml` + `responders/`)
15+
- README metadata rows
16+
- Changelog format
17+
- QA/sim artifact presence conventions
18+
19+
## How to validate
20+
- `psyflow-validate <task_path>`
21+
- `psyflow-validate <task_path> --strict-warn`
22+
- `psyflow-validate <task_path> --json-report <path>`
23+
24+
## Pattern references
25+
- `main_pattern.md`
26+
- `run_trial_pattern.md`
27+
28+
The goal is standardization without overdesign: clear expectations, easy audits, and room for task-specific logic.

‎psyflow/contracts/v0.1.0/artifacts.yaml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
name: artifacts
2+
qa_required_filenames:
3+
- qa_report.json
4+
- static_report.json
5+
- contract_report.json
6+
- qa_trace.csv
7+
- qa_events.jsonl
8+
sim_recommended_patterns:
9+
- '*_sim_events.jsonl'
10+
- '*_qa_events.jsonl'
11+
notes:
12+
- Artifact checks are warning-level unless a QA output directory exists and is expected.

‎psyflow/contracts/v0.1.0/changelog.yaml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
name: changelog
2+
file: CHANGELOG.md
3+
required_patterns:
4+
- '^#\s+CHANGELOG'
5+
- '^##\s+\[[^\]]+\]\s+-\s+\d{4}-\d{2}-\d{2}'
6+
recommended_sections:
7+
- Added
8+
- Changed
9+
- Fixed

‎psyflow/contracts/v0.1.0/config.yaml‎

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
name: config_base
2+
file: config/config.yaml
3+
required_sections:
4+
- window
5+
- task
6+
- timing
7+
- stimuli
8+
- triggers
9+
mandatory_nested_keys:
10+
- task.task_name
11+
- task.key_list
12+
- triggers.map
13+
- triggers.driver.type
14+
optional_nested_keys:
15+
- controller
16+
recommended_nested_keys:
17+
- task.trial_per_block
18+
mandatory_value_specs:
19+
window.size:
20+
type: list_int
21+
exact_len: 2
22+
window.units:
23+
type: str
24+
min_length: 1
25+
window.screen:
26+
type: int
27+
min: 0
28+
window.bg_color:
29+
type: str
30+
min_length: 1
31+
window.fullscreen:
32+
type: bool
33+
task.task_name:
34+
type: str
35+
min_length: 1
36+
task.total_blocks:
37+
type: int
38+
min: 1
39+
task.total_trials:
40+
type: int
41+
min: 1
42+
task.conditions:
43+
type: list_str
44+
min_items: 1
45+
task.key_list:
46+
type: list_str
47+
min_items: 1
48+
task.seed_mode:
49+
type: str
50+
allowed:
51+
- same_across_sub
52+
- same_within_sub
53+
timing:
54+
type: mapping
55+
stimuli:
56+
type: mapping
57+
min_items: 1
58+
triggers.map:
59+
type: mapping
60+
min_items: 1
61+
triggers.driver.type:
62+
type: str
63+
allowed:
64+
- serial_url
65+
- serial_port
66+
- mock
67+
- callable
68+
optional_value_specs:
69+
controller:
70+
type: mapping
71+
controller.initial_duration:
72+
type: number
73+
min: 0.01
74+
controller.min_duration:
75+
type: number
76+
min: 0.01
77+
controller.max_duration:
78+
type: number
79+
min: 0.01
80+
controller.step:
81+
type: number
82+
min: 0.001
83+
controller.target_accuracy:
84+
type: number
85+
min: 0.0
86+
max: 1.0
87+
recommended_value_specs:
88+
task.delta:
89+
type: number
90+
task.trial_per_block:
91+
type: int
92+
min: 1
93+
stim_supported_types:
94+
- text
95+
- textbox
96+
- circle
97+
- rect
98+
- polygon
99+
- image
100+
- shape
101+
- movie
102+
- sound
103+
stim_asset_types:
104+
- image
105+
- movie
106+
- sound
107+
stim_asset_path_keys_any:
108+
- file
109+
- filename
110+
- image
111+
- path
112+
stim_asset_path_keys_by_type:
113+
image:
114+
- image
115+
- file
116+
- path
117+
movie:
118+
- filename
119+
- file
120+
- path
121+
sound:
122+
- file
123+
- filename
124+
- path
125+
stim_asset_path_prefixes:
126+
- assets/
127+
- ./assets/
128+
- assets\\
129+
- .\\assets\\
130+
notes:
131+
- controller is optional.
132+
- if controller is configured, keep implementation under src/utils.py.
133+
- stimuli entries must include a valid type.
134+
- asset-backed stimuli (image/movie/sound) should load from assets/ paths.
135+
- if an asset-backed stimulus path is set dynamically in runtime code, keep a clear comment in run_trial.py.

‎psyflow/contracts/v0.1.0/config_qa.yaml‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
name: config_qa
2+
file: config/config_qa.yaml
3+
mandatory_nested_keys:
4+
- qa.output_dir
5+
- qa.acceptance_criteria.required_columns
6+
mandatory_value_specs:
7+
qa.output_dir:
8+
type: str
9+
min_length: 1
10+
qa.acceptance_criteria.required_columns:
11+
type: list_str
12+
min_items: 1
13+
optional_nested_keys:
14+
- qa.enable_scaling
15+
- qa.timing_scale
16+
- qa.min_frames
17+
- qa.strict
18+
- qa.max_wait_s
19+
- qa.acceptance_criteria.expected_trial_count
20+
- qa.acceptance_criteria.allowed_keys
21+
- qa.acceptance_criteria.triggers_required
22+
- qa.acceptance_criteria.trigger_codes_expected
23+
optional_value_specs:
24+
qa.enable_scaling:
25+
type: bool
26+
qa.timing_scale:
27+
type: number
28+
min: 0.05
29+
qa.min_frames:
30+
type: int
31+
min: 1
32+
qa.strict:
33+
type: bool
34+
qa.max_wait_s:
35+
type: number
36+
min: 0.1
37+
qa.acceptance_criteria.expected_trial_count:
38+
type: int
39+
min: 1
40+
qa.acceptance_criteria.allowed_keys:
41+
type: list_str
42+
min_items: 1
43+
qa.acceptance_criteria.triggers_required:
44+
type: bool
45+
qa.acceptance_criteria.trigger_codes_expected:
46+
type: list_int
47+
min_items: 1
48+
recommended_nested_keys: []
49+
recommended_value_specs: {}

‎psyflow/contracts/v0.1.0/config_sim.yaml‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
name: config_sim
2+
file: config/config_sim.yaml
3+
mandatory_nested_keys:
4+
- sim.output_dir
5+
- sim.seed
6+
- sim.policy
7+
- sim.responder.type
8+
mandatory_value_specs:
9+
sim.output_dir:
10+
type: str
11+
min_length: 1
12+
sim.seed:
13+
type: int
14+
sim.policy:
15+
type: str
16+
allowed:
17+
- strict
18+
- warn
19+
- coerce
20+
sim.responder.type:
21+
type: str
22+
min_length: 1
23+
optional_nested_keys:
24+
- sim.default_rt_s
25+
- sim.clamp_rt
26+
- sim.participant_id
27+
- sim.session_id
28+
- sim.log_path
29+
- sim.responder.kwargs
30+
optional_value_specs:
31+
sim.default_rt_s:
32+
type: number
33+
min: 0.01
34+
sim.clamp_rt:
35+
type: bool
36+
sim.participant_id:
37+
type: str
38+
min_length: 1
39+
sim.session_id:
40+
type: str
41+
min_length: 1
42+
sim.log_path:
43+
type: str
44+
min_length: 1
45+
sim.responder.kwargs:
46+
type: mapping
47+
recommended_nested_keys:
48+
- sim.participant_id
49+
- sim.session_id
50+
- sim.log_path
51+
- sim.responder.kwargs
52+
recommended_value_specs:
53+
sim.participant_id:
54+
type: str
55+
min_length: 1
56+
sim.session_id:
57+
type: str
58+
min_length: 1
59+
sim.log_path:
60+
type: str
61+
min_length: 1
62+
sim.responder.kwargs:
63+
type: mapping
64+
min_items: 1
65+
responder_builtin_types:
66+
- scripted
67+
- null
68+
responder_local_module_prefixes:
69+
- responders.
70+
responder_required_methods:
71+
- act
72+
responder_optional_methods:
73+
- start_session
74+
- on_feedback
75+
- end_session
76+
notes:
77+
- built-in responder types: scripted, null
78+
- custom responders should use import path in sim.responder.type.
79+
- for task-specific samplers, use responders.<module>:<ClassName>.

0 commit comments

Comments
 (0)