feat(guard): llm-guard I/O security for agent input & output#17
Open
ngoclam9415 wants to merge 2 commits into
Open
feat(guard): llm-guard I/O security for agent input & output#17ngoclam9415 wants to merge 2 commits into
ngoclam9415 wants to merge 2 commits into
Conversation
Integrate protectai/llm-guard as a pluggable GuardService securing STARAgent input and output at the query/aquery choke point. - Input: block-listed scanners (default prompt_injection) refuse the request; secrets/PII findings sanitize + pass. - Output: gated two-stage scrub - rule scanners strip/redact, LLM scrub fires only when a scanner flags. Scrubbed text overwrites the persisted AGENT_RESPONSE so raw data never lands on disk or replays to the LLM. - Pluggable scanner registry (mirrors llm_provider); injectable via STARAgent(guard_instance=...). Fail-open: missing/broken models no-op + warn. Thread-safe lazy scanner init. - Audit to structlog + timeline.json metadata. - Config via DANA_GUARD_* env vars. llm-guard added as core dependency (downgrades transformers->4.51.3 per its constraints); guard disabled by default in tests. Tests: 32 guard unit tests; full core suite green. Claude-Session: https://claude.ai/code/session_01WdeJ8nFuNdE7yuLKBcZx37
llm-guard keys results by scanner CLASS name (PromptInjection), but block_on / findings / config use snake_case (prompt_injection), so the input block branch was dead for every default scanner — detected injections downgraded to SANITIZED and reached the LLM unchanged. - scanner_factory: build_* now also return a class-name -> snake_case registry-key map (joined via type(scanner).__name__). - LLMGuardService: translate llm-guard's class-name keys to snake_case when building findings/triggered, so they match config out of the box. - Block matching normalized (case/underscore-insensitive) so block_on also tolerates the class-name spelling. - Tests: pin the default-config injection -> BLOCKED path with the real class-name vocabulary (the prior stubs used snake_case and masked the bug); add name-map + unmapped-key coverage. Claude-Session: https://claude.ai/code/session_01WdeJ8nFuNdE7yuLKBcZx37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Integrate protectai/llm-guard as a pluggable
GuardServicethat securesSTARAgentinput & output at the singlequery/aquerychoke point.Behavior
prompt_injection) → request refused (STAR loop skipped, refusal + attempt recorded). Secrets/PII findings → sanitize + pass.AGENT_RESPONSEso raw sensitive data never lands intimeline.jsonor replays to the LLM next turn.llm_provider: scanner registry +STARAgent(guard_instance=...)injection.timeline.jsonmetadata.Architecture
New
dana/core/guard/:GuardServiceProtocol (3 methods) +LLMGuardService,NoOpGuardService,OutputSanitizer, scanner registry, env-drivenGuardConfig, result types. Wired intoSTARAgent.Config (env)
DANA_GUARD_ENABLED,_INPUT_SCANNERS,_OUTPUT_SCANNERS,_SANITIZE_LLM_ENABLED,_FAIL_MODE,_BLOCK_ON,_BLOCK_MESSAGE.Dependency note⚠️
llm-guardadded as a core dependency (per design decision). Its constraints downgradetransformers 5.3.0→4.51.3,tokenizers,regex,pyopenssland add spacy. It pulls torch/transformers (~GB) —uv syncnot yet run in this branch; guard is fail-open so runtime works without it until synced.Tests
tests/conftest.py.test_llm_providers.pyfailures are unrelated (fail on clean base).Review
Adversarial code review run; fixed CRITICAL (raw response persisted to timeline → leak/replay), thread-safety, scanner-registry clobbering, wrong-entry annotation.
Test plan
uv syncto install llm-guard, then exercise with a real providertransformersdowngrade doesn't regress other pathshttps://claude.ai/code/session_01WdeJ8nFuNdE7yuLKBcZx37