Skip to content

feat(sdk): improve read_url error diagnostics and html table extraction - #1122

Open
nilaymallikk wants to merge 1 commit into
CodebuffAI:mainfrom
nilaymallikk:feature/read-url-diagnostics
Open

feat(sdk): improve read_url error diagnostics and html table extraction#1122
nilaymallikk wants to merge 1 commit into
CodebuffAI:mainfrom
nilaymallikk:feature/read-url-diagnostics

Conversation

@nilaymallikk

Copy link
Copy Markdown

Summary

Closes #1121.

Enhances the read_url tool in @codebuff/sdk and common tool schemas:

  • Structured Error Diagnostics: Adds typed error classifications (errorCode such as HTTP_ERROR, BLOCKED_ADDRESS, INVALID_URL, TOO_MANY_REDIRECTS, UNSUPPORTED_CONTENT_TYPE, RESPONSE_TOO_LARGE, TIMEOUT, ABORTED) and forwards HTTP status codes (e.g. 404, 429, 500) so caller agents can differentiate retryable rate-limits from permanent errors.
  • HTML Table Extraction: Added formatHtmlTables pre-processor during HTML readable text extraction to transform <table> elements into structured GitHub-Flavored Markdown tables instead of flattening cells into unformatted lines.

Scope

All changes are strictly scoped within:

  • sdk/src/tools/read-url.ts
  • sdk/src/__tests__/read-url.test.ts
  • common/src/tools/params/tool/read-url.ts

Verification

  • Passed unit tests: bun test sdk/src/__tests__/read-url.test.ts (21 pass)
  • Passed full SDK test suite: bun test sdk/src/__tests__/ (470 pass)
  • Passed CI build and packaging: bun run ci
  • Passed binary smoke test: bun cli/scripts/smoke-binary.ts cli/bin/freebuff

@codebuff-team

Copy link
Copy Markdown
Contributor

Good instinct and correctly scoped to sdk/common (not touching any forbidden paths). The errorCode/status additions to read_url are backward-compatible (optional fields) and well-tested for the HTTP_ERROR/UNSUPPORTED_CONTENT_TYPE/INVALID_URL/BLOCKED_ADDRESS/TOO_MANY_REDIRECTS paths.

Two things need fixing before this is portable:

  1. Table cell escaping is missing. In formatHtmlTables (read-url.ts ~L200-235), cell text is joined with | but never escapes literal | characters that may appear in the extracted text (e.g. a table cell containing a|b). That will silently corrupt the resulting markdown table structure. Needs cell.replace(/\|/g, '\|') or similar before joining.

  2. Error classification via string matching is fragile. isInvalid (L449-452) and isTooLarge (L534-537) key off substrings of thrown error messages ('Only http:// and https:// URLs are supported', 'Response is too large', 'Response exceeded') rather than typed errors from assertUrlAllowed/fetch. If those message strings ever change, the errorCode silently degrades to the wrong classification with no test to catch it. Prefer having assertUrlAllowed throw a typed error (e.g. {code, message}) rather than matching text.

  3. Test coverage gap: you added TIMEOUT, ABORTED, and RESPONSE_TOO_LARGE to the error-code union and wrote classification logic for them, but there's no test exercising any of those three paths — exactly the paths most likely to regress given point 2.

Also worth a nested-table caveat: the regex-based <table>...</table> matcher will misbehave on nested tables (rare in practice, but worth a comment/guard).

Address the pipe-escaping bug and add tests for the untested error codes and this becomes straightforward to port.

@codebuff-team codebuff-team added bot:triaged Classified by the community triage bot pr:needs-work Right idea, not mergeable as written labels Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:triaged Classified by the community triage bot pr:needs-work Right idea, not mergeable as written

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve error diagnostics and tabular content extraction in read_url tool

2 participants