[DBA-295] Migrate validate-agent-guidance from Bash to Python#79
[DBA-295] Migrate validate-agent-guidance from Bash to Python#79
validate-agent-guidance from Bash to Python#79Conversation
There was a problem hiding this comment.
Pull request overview
Migrates the agent guidance/skills Tier-1 validator from a Bash script to a typed Python module, and updates local tooling (Makefile + pre-commit) to invoke the new validator via uv.
Changes:
- Added
scripts/validate_agent_guidance.pyimplementing skill structure, cross-reference, and eval JSON validation (plus YAML frontmatter checks). - Removed
scripts/validate-agent-guidance.sh. - Updated
make lint-skillsand thevalidate-agent-guidancepre-commit hook to run the Python validator.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
scripts/validate_agent_guidance.py |
New Python implementation of the guidance/skills static validator (structure, references, eval JSON, frontmatter). |
scripts/validate-agent-guidance.sh |
Deleted legacy Bash validator. |
Makefile |
Points lint-skills at the new Python validator. |
.pre-commit-config.yaml |
Updates the local hook to run the new Python validator via uv. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
SimonKaran13
left a comment
There was a problem hiding this comment.
As copilot suggested, scripts/smoke-test-skills.sh still references validate-agent-guidance.sh. Let's make sure it's not referenced anywhere before merging
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
7815c57 to
a812709
Compare
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
# Conflicts: # scripts/validate-agent-guidance.sh
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
docs/testing-strategy.md:47
docs/testing-strategy.mdremoves the Make Targets section (including how to runmake lint-skills), but later the doc still references “skill validation” as part of what runs in CI/pre-commit. This makes the document internally inconsistent and the removal isn’t mentioned in the PR description; consider restoring the section (or replacing it with a briefmake lint-skills/make smoke-skillsnote) and updating thelint-skillscommand to the new Python validator.
## What Does NOT Run in CI / Pre-commit
Everything above (lint, type-check, unit tests, skill validation) is
**LLM-free** and runs automatically in CI and pre-commit hooks.
| # Invalid YAML frontmatter: treat as missing frontmatter to avoid crashing. | ||
| return {} | ||
| if not isinstance(parsed, dict): | ||
| return {} | ||
| return {str(k): str(v) for k, v in parsed.items()} |
There was a problem hiding this comment.
MdFile.frontmatter coerces all YAML values to str (return {str(k): str(v) ...}), which can incorrectly treat null, booleans, lists, etc. as valid strings (e.g., name: null becomes 'None') and defeats later type validation. Consider returning the parsed mapping without coercion (e.g., dict[str, Any]) and validating name/description are actual non-empty strings at the call site.
| for field in ("name", "description"): | ||
| if field not in fm: | ||
| errors.append(f"{skill_name}/SKILL.md frontmatter missing required field '{field}'") | ||
| continue | ||
| value = fm.get(field) | ||
| # Require non-empty string values for required fields |
There was a problem hiding this comment.
validate_skill_structure() currently only checks that name/description keys exist in frontmatter; empty values (or name: null after string coercion) will still pass, and the directory-name match is skipped when name is falsy. If these fields are required, enforce they are non-empty strings and always validate name against skill_name when present (including empty/whitespace).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Summary
Replaces the Bash validate-agent-guidance script with a typed Python module
that is easier to extend and maintain. Adds
MdFileandClaudeDirdataclasses for structured, lazy-parsed access to skills and agents.
Changes
Python rewrite of the validator
Replaced
scripts/validate-agent-guidance.shwithscripts/validate_agent_guidance.py.Introduced
MdFile(lazy-parsed markdown with cached properties) andClaudeDir(.claude/directory withskill_dirs,skill_files,agent_fileslistings). Adds YAML frontmatter validation:nameanddescriptionfields required,namemust match the skill directory name.Files
scripts/validate_agent_guidance.pyscripts/validate-agent-guidance.sh(deleted)Tooling update
Updated
make lint-skillsand the pre-commit hook to invoke the Python script.Files
Makefile.pre-commit-config.yamlTest Plan
make lint-skillspassesmake checkpasses (ruff + mypy + pre-commit hook)🤖 Generated with Claude Code