Skip to content

fix: resolve leaked extract() element IDs on z.string() URL fields - #2784

Open
antonvishal wants to merge 2 commits into
browserbase:mainfrom
antonvishal:fix/extract-string-url-ids
Open

fix: resolve leaked extract() element IDs on z.string() URL fields#2784
antonvishal wants to merge 2 commits into
browserbase:mainfrom
antonvishal:fix/extract-string-url-ids

Conversation

@antonvishal

@antonvishal antonvishal commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • extract() leaked accessibility-tree IDs ([0-74], 0-74) when the schema used plain z.string() for URL fields. injectUrls only ran for z.url() / format: uri.
  • Agents and callers that extract links as strings (for example { stories: z.string() } or { rank1_url: z.string(), ... } on a page like Hacker News) got raw element IDs back. z.url() fields already resolved correctly.
  • Fix: tell the model to return frameId-backendId only for URL values, then replaceElementIdsWithUrls maps leaked IDs through combinedUrlMap. Bracketed IDs rewrite in any string; bare IDs only on URL-ish field names.

Before

{ "stories": "[0-74]\n[0-112]\n[0-150]\n[0-188]\n[0-226]" }

{
"rank1_title": "GrapheneOS in 2027 available on high-end Motorola phones",
"rank1_url": "0-74",
"rank2_url": "0-112",
"rank3_url": "0-150",
"rank4_url": "0-188",
"rank5_url": "0-226"
}

After (this fix)

{
"stories": "https://grapheneos.social/@GrapheneOS/117078064184215730\nhttps://yassa9.github.io/osint/gralhix-004/\nhttps://www.raphaelbauer.com/posts/postgresql-everything/\nhttps://sprocketfox.io/xssfox/2026/08/19/sondehub-and-war/\nhttps://openlogi.org/en"
}

{
"rank1_title": "GrapheneOS in 2027 available on high-end Motorola phones",
"rank1_url": "https://grapheneos.social/@GrapheneOS/117078064184215730",
"rank2_url": "https://yassa9.github.io/osint/gralhix-004/",
"rank3_url": "https://www.raphaelbauer.com/posts/postgresql-everything/",
"rank4_url": "https://sprocketfox.io/xssfox/2026/08/19/sondehub-and-war/",
"rank5_url": "https://openlogi.org/en"
}

Test plan

  • pnpm exec vitest run --root . packages/extension/tests/extract.test.ts
  • extract() on a page with links using z.object({ title: z.string(), url: z.url() }) still returns real hrefs
  • Same instruction with z.object({ title: z.string(), url: z.string() }) returns real hrefs, not element IDs
  • extract("Extract the first 5 story URLs", z.object({ urls: z.string() })) returns real URLs, not [0-74] / 0-74
  • Non-URL strings that look like IDs (e.g. score: "0-74") are left unchanged

Summary by cubic

Resolves leaked accessibility-tree IDs from extract() when URL fields use z.string(). Before, only z.url() fields resolved to hrefs; now both string and z.url() URL fields return real URLs, and non-URL fields never return element IDs.

  • Updates the system prompt: for any value that should be a URL, return the link element ID as frameId-backendId (no brackets); for all other fields, return only visible text and never return IDs.
  • Adds replaceElementIdsWithUrls in the extraction flow to map leaked IDs to hrefs via the snapshot combinedUrlMap. Bracketed IDs ([0-74]) are replaced anywhere; bare IDs (0-74) are replaced only on URL-like field names (url, href, link, uri). Unknown IDs are left as-is.
  • Extends transformSchema to treat url, uri, and uri-reference as URL formats, preserving injectUrls behavior for typed z.url() fields.
  • Tests cover prompt wording and end-to-end resolution for z.string() URL outputs.
  • No API changes. Migration: if any caller relied on raw element IDs in string fields, update that logic to accept real URLs.

Written for commit e9bf389. Summary will update on new commits.

Review in cubic

@changeset-bot

changeset-bot Bot commented Aug 20, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: e9bf389

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions

Copy link
Copy Markdown
Contributor

This PR is from an external contributor and must be approved by a stagehand team member with write access before CI can run.
Approving the latest commit mirrors it into an internal PR owned by the approver.
If new commits are pushed later, the internal PR stays open but is marked stale until someone approves the latest external commit and refreshes it.

@github-actions github-actions Bot added external-contributor Tracks PRs mirrored from external contributor forks. external-contributor:awaiting-approval Waiting for a stagehand team member to approve the latest external commit. labels Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external-contributor:awaiting-approval Waiting for a stagehand team member to approve the latest external commit. external-contributor Tracks PRs mirrored from external contributor forks.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant