Skip to content

fix: handle binary files in read tool (images/PDFs)#1638

Open
vihaan-kk wants to merge 2 commits into
anthropics:mainfrom
vihaan-kk:fix/read-tool-unicode-decode-error
Open

fix: handle binary files in read tool (images/PDFs)#1638
vihaan-kk wants to merge 2 commits into
anthropics:mainfrom
vihaan-kk:fix/read-tool-unicode-decode-error

Conversation

@vihaan-kk
Copy link
Copy Markdown

@vihaan-kk vihaan-kk commented Jun 2, 2026

Fixes #1637

Changes

The beta_read_tool in agent_toolset.py called target.read_text() on every file, which decodes bytes as UTF-8. Reading a binary file (image or PDF) raises an uncaught UnicodeDecodeError since only ToolError and OSError were caught, UnicodeDecodeError is a ValueError and propagated uncaught to the model as a raw tool error.

Fix

Added a _binary_media_type helper that detects binary files by extension. When a binary file is detected, read returns a base64-encoded image or document content block instead of attempting text decoding. The existing text path is unchanged.

Supported types:

  • Images: .jpg, .jpeg, .png, .gif, .webpimage block
  • Documents: .pdfdocument block

Testing

  • All existing tests pass (3216 Pydantic v2, 3119 Pydantic v1 on Python
    3.9; 3253 on Python 3.14)
  • ./scripts/lint passes clean (pyright, mypy, ruff)
  • ./scripts/format applied

Notes

  • The view_range slicing logic is text-only and is correctly bypassed for binary files since the binary path returns early
  • The return type annotation on read was updated from str to Any to reflect that binary files return a list content block
  • The 256 KiB size cap still applies to binary files before the binary check, a separate larger cap for images could be a follow-up

Copilot AI review requested due to automatic review settings June 2, 2026 23:39
@vihaan-kk vihaan-kk requested a review from a team as a code owner June 2, 2026 23:39
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds binary-aware handling to the read tool so it can return base64-encoded images/PDFs instead of attempting to decode them as text.

Changes:

  • Introduces a binary media type lookup (_BINARY_MEDIA_TYPES + _binary_media_type).
  • Updates read to return base64 “image”/“document” blocks for supported binary types.
  • Changes read return annotation from str to Any.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/anthropic/lib/tools/agent_toolset.py
Comment thread src/anthropic/lib/tools/agent_toolset.py
Comment on lines +531 to 536
media = _binary_media_type(target)
if media is not None:
data = base64.standard_b64encode(target.read_bytes()).decode("ascii")
kind = "document" if media == "application/pdf" else "image"
return [{"type": kind, "source": {"type": "base64", "media_type": media, "data": data}}]
text = target.read_text()
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, noted this in the PR description as a follow-up. A separate lower size cap for base64 responses makes sense but felt out of scope for this fix.

Comment on lines 528 to +535
f"read: {file_path} is {st.st_size} bytes, exceeds {limit}-byte limit. "
"Use bash (head/tail/sed) to read a slice."
)
media = _binary_media_type(target)
if media is not None:
data = base64.standard_b64encode(target.read_bytes()).decode("ascii")
kind = "document" if media == "application/pdf" else "image"
return [{"type": kind, "source": {"type": "base64", "media_type": media, "data": data}}]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Self-hosted agent_toolset read tool raises UnicodeDecodeError on binary files (images/PDFs) instead of returning content blocks

2 participants