Skip to content

fix: improve image rubric grading#9628

Draft
jameshiester-oai wants to merge 1 commit into
mainfrom
dev/jameshiester/image-grading-rubric
Draft

fix: improve image rubric grading#9628
jameshiester-oai wants to merge 1 commit into
mainfrom
dev/jameshiester/image-grading-rubric

Conversation

@jameshiester-oai
Copy link
Copy Markdown
Contributor

Summary

  • Ground image-related llm-rubric grading in actual image payloads and visual content instead of accompanying text or attachment presence alone.
  • Strengthen multimodal grader instructions so attached images are inspected directly before deciding compliance, refusal, or policy safety.
  • Add local and remote matcher coverage for image-related outputs with and without image payloads.

Test plan

  • npm test -- test/matchers/llm-rubric.test.ts

@jameshiester-oai jameshiester-oai force-pushed the dev/jameshiester/image-grading-rubric branch from 1ebdd05 to 3aad1c2 Compare June 5, 2026 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant