Skip to content

docs(skills): add retrieval models skill#2305

Draft
oliverholworthy wants to merge 3 commits into
mainfrom
oholworthy/docs/retrieval-models-skill
Draft

docs(skills): add retrieval models skill#2305
oliverholworthy wants to merge 3 commits into
mainfrom
oholworthy/docs/retrieval-models-skill

Conversation

@oliverholworthy
Copy link
Copy Markdown
Contributor

What does this PR do ?

Adds a retrieval-models agent skill for NeMo AutoModel bi-encoder and cross-encoder work, including guidance for retrieval wrappers, bidirectional backbones, recipes, collator shape contracts, validation, and live skill evals.

Changelog

  • Add skills/retrieval-models/SKILL.md with retrieval model workflows, trigger boundaries, validation commands, and troubleshooting guidance.
  • Add skills/retrieval-models/PITFALLS.md for common retrieval wrapper, metadata, and recipe shape issues.
  • Register the retrieval skill files in packaging metadata and the skills README.
  • Add skills/retrieval-models/evals/evals.json for astra live-agent evaluation coverage.

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?

Additional Information

Validation run:

  • astra-skill-eval evaluate skills/retrieval-models --static --markdown -> A, 100/100, no issues
  • astra-skill-eval validate skills/retrieval-models -> all checks passed
  • astra-skill-eval evaluate skills/retrieval-models --agent-eval -a codex --env-mode local --n-attempts 1 --skip-baseline --copy-repo --results-dir /tmp/astra-retrieval-models-eval-clean --markdown -> pass@1 3/4, accuracy 1.00
  • Focused retrieval unit subset passed with one environment-sensitive flash-attn case deselected: 29 passed, 1 deselected

Draft PR because the live eval still shows one implementation-heavy scenario below threshold and should get a reviewer pass before marking ready.

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented May 22, 2026

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant