feat(storage): add content_filters to DynamoDbChatStorage - #537
feat(storage): add content_filters to DynamoDbChatStorage#537nuthalapativarun wants to merge 2 commits into
Conversation
Adds an optional content_filters parameter — a list of (original, placeholder) pairs — to DynamoDbChatStorage. Sensitive values are replaced with their placeholders before writing to DynamoDB and restored on every read path (fetch_chat, fetch_chat_with_timestamp, fetch_all_chats). Closes 2FastLabs#131
|
Hey @nuthalapativarun, a few things need to be fixed before this can merge. There is a bug in One thing worth calling out in the PR description: the filter replacement works by doing a |
…ts filter guard - save_chat_message now runs the early-return (consecutive same-role duplicate) path through _reverse_content_filters before returning, instead of leaking the raw placeholder-masked stored content. - fetch_all_chats now always reverses content filters instead of only when content is a list, so non-list message content no longer leaks masked placeholder text into the returned chat list. - Add regression tests for both fixes. - Document the content_filters parameter/behavior in the DynamoDB storage docs. Addresses review feedback on 2FastLabs#537.
|
Thanks for the review, @cornelcroi! Pushed a follow-up commit addressing all four points:
Ready for re-review whenever you get a chance. |
Issue Link (REQUIRED)
Fixes #131
Summary
Changes
Adds an optional
content_filtersparameter toDynamoDbChatStorage— a list of(original, placeholder)string pairs. When provided:save_chat_message,save_chat_messages): each original value is replaced by its placeholder before the message is written to DynamoDB.fetch_chat,fetch_chat_with_timestamp,fetch_all_chats): placeholders are restored to their original values before returning to the caller.This eliminates the need for a custom storage implementation when sensitive content (e.g. company names, tokens) must not be persisted in plain text.
User experience
Before: users had to subclass
DynamoDbChatStorageor write a custom storage class to mask sensitive values.After:
DynamoDB stores
#COMPANY#; the agent always seesacme-corp.Checklist
Acknowledgment
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.