Skip to content

[DBA-295] Migrate validate-agent-guidance from Bash to Python#79

Merged
gasparian merged 11 commits intomainfrom
ls/validate_agent_guidance_py
Mar 31, 2026
Merged

[DBA-295] Migrate validate-agent-guidance from Bash to Python#79
gasparian merged 11 commits intomainfrom
ls/validate_agent_guidance_py

Conversation

@catstrike
Copy link
Copy Markdown
Collaborator

Summary

Replaces the Bash validate-agent-guidance script with a typed Python module
that is easier to extend and maintain. Adds MdFile and ClaudeDir
dataclasses for structured, lazy-parsed access to skills and agents.

Changes

Python rewrite of the validator

Replaced scripts/validate-agent-guidance.sh with scripts/validate_agent_guidance.py.
Introduced MdFile (lazy-parsed markdown with cached properties) and
ClaudeDir (.claude/ directory with skill_dirs, skill_files,
agent_files listings). Adds YAML frontmatter validation: name and
description fields required, name must match the skill directory name.

Files
  • scripts/validate_agent_guidance.py
  • scripts/validate-agent-guidance.sh (deleted)

Tooling update

Updated make lint-skills and the pre-commit hook to invoke the Python script.

Files
  • Makefile
  • .pre-commit-config.yaml

Test Plan

  • make lint-skills passes
  • make check passes (ruff + mypy + pre-commit hook)

🤖 Generated with Claude Code

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Migrates the agent guidance/skills Tier-1 validator from a Bash script to a typed Python module, and updates local tooling (Makefile + pre-commit) to invoke the new validator via uv.

Changes:

  • Added scripts/validate_agent_guidance.py implementing skill structure, cross-reference, and eval JSON validation (plus YAML frontmatter checks).
  • Removed scripts/validate-agent-guidance.sh.
  • Updated make lint-skills and the validate-agent-guidance pre-commit hook to run the Python validator.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File Description
scripts/validate_agent_guidance.py New Python implementation of the guidance/skills static validator (structure, references, eval JSON, frontmatter).
scripts/validate-agent-guidance.sh Deleted legacy Bash validator.
Makefile Points lint-skills at the new Python validator.
.pre-commit-config.yaml Updates the local hook to run the new Python validator via uv.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown
Member

@SimonKaran13 SimonKaran13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As copilot suggested, scripts/smoke-test-skills.sh still references validate-agent-guidance.sh. Let's make sure it's not referenced anywhere before merging

Copilot AI review requested due to automatic review settings March 27, 2026 10:15
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@catstrike catstrike force-pushed the ls/validate_agent_guidance_py branch from 7815c57 to a812709 Compare March 27, 2026 11:06
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 27, 2026 11:08
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI review requested due to automatic review settings March 27, 2026 11:29
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 27, 2026 11:34
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Andrei Gasparian and others added 3 commits March 27, 2026 15:35
# Conflicts:
#	scripts/validate-agent-guidance.sh
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 31, 2026 14:47
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@gasparian gasparian self-requested a review March 31, 2026 14:47
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

docs/testing-strategy.md:47

  • docs/testing-strategy.md removes the Make Targets section (including how to run make lint-skills), but later the doc still references “skill validation” as part of what runs in CI/pre-commit. This makes the document internally inconsistent and the removal isn’t mentioned in the PR description; consider restoring the section (or replacing it with a brief make lint-skills/make smoke-skills note) and updating the lint-skills command to the new Python validator.
## What Does NOT Run in CI / Pre-commit

Everything above (lint, type-check, unit tests, skill validation) is
**LLM-free** and runs automatically in CI and pre-commit hooks.

Comment on lines +59 to +63
# Invalid YAML frontmatter: treat as missing frontmatter to avoid crashing.
return {}
if not isinstance(parsed, dict):
return {}
return {str(k): str(v) for k, v in parsed.items()}
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MdFile.frontmatter coerces all YAML values to str (return {str(k): str(v) ...}), which can incorrectly treat null, booleans, lists, etc. as valid strings (e.g., name: null becomes 'None') and defeats later type validation. Consider returning the parsed mapping without coercion (e.g., dict[str, Any]) and validating name/description are actual non-empty strings at the call site.

Copilot uses AI. Check for mistakes.
Comment on lines +115 to +120
for field in ("name", "description"):
if field not in fm:
errors.append(f"{skill_name}/SKILL.md frontmatter missing required field '{field}'")
continue
value = fm.get(field)
# Require non-empty string values for required fields
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

validate_skill_structure() currently only checks that name/description keys exist in frontmatter; empty values (or name: null after string coercion) will still pass, and the directory-name match is skipped when name is falsy. If these fields are required, enforce they are non-empty strings and always validate name against skill_name when present (including empty/whitespace).

Copilot uses AI. Check for mistakes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@gasparian gasparian merged commit f9c774e into main Mar 31, 2026
6 checks passed
@gasparian gasparian deleted the ls/validate_agent_guidance_py branch March 31, 2026 14:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants