Add concurrent scraping example with structured output#112
Open
MagMueller wants to merge 1 commit intomainfrom
Open
Add concurrent scraping example with structured output#112MagMueller wants to merge 1 commit intomainfrom
MagMueller wants to merge 1 commit intomainfrom
Conversation
5/10 creative test agents needed concurrency patterns but couldn't find any in the docs. Added: - Full concurrent scraping example to structured output page showing asyncio.gather (Python) and Promise.all (TypeScript) with typed schemas - Compact concurrency snippet to both llms.txt files so LLMs discover the pattern immediately Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Preview deployment for your docs. Learn more about Mintlify Previews.
|
There was a problem hiding this comment.
2 issues found across 3 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="docs/cloud/llms.txt">
<violation number="1" location="docs/cloud/llms.txt:33">
P2: The JavaScript example references `tasks` without defining it, so the snippet will fail when copied as-is.</violation>
</file>
<file name="docs/llms.txt">
<violation number="1" location="docs/llms.txt:33">
P2: The JavaScript snippet references an undefined `tasks` variable, so the example is not runnable as written.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.
| ``` | ||
| ```javascript | ||
| const results = await Promise.all( | ||
| tasks.map((task) => client.run(task, { schema: MySchema })) |
There was a problem hiding this comment.
P2: The JavaScript example references tasks without defining it, so the snippet will fail when copied as-is.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At docs/cloud/llms.txt, line 33:
<comment>The JavaScript example references `tasks` without defining it, so the snippet will fail when copied as-is.</comment>
<file context>
@@ -21,6 +21,19 @@ export BROWSER_USE_API_KEY=bu_your_key_here
+```
+```javascript
+const results = await Promise.all(
+ tasks.map((task) => client.run(task, { schema: MySchema }))
+);
+```
</file context>
Suggested change
| tasks.map((task) => client.run(task, { schema: MySchema })) | |
| ["Scrape site A", "Scrape site B", "Scrape site C"].map((task) => client.run(task, { schema: MySchema })) |
| ``` | ||
| ```javascript | ||
| const results = await Promise.all( | ||
| tasks.map((task) => client.run(task, { schema: MySchema })) |
There was a problem hiding this comment.
P2: The JavaScript snippet references an undefined tasks variable, so the example is not runnable as written.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At docs/llms.txt, line 33:
<comment>The JavaScript snippet references an undefined `tasks` variable, so the example is not runnable as written.</comment>
<file context>
@@ -21,6 +21,19 @@ export BROWSER_USE_API_KEY=bu_your_key_here
+```
+```javascript
+const results = await Promise.all(
+ tasks.map((task) => client.run(task, { schema: MySchema }))
+);
+```
</file context>
Suggested change
| tasks.map((task) => client.run(task, { schema: MySchema })) | |
| ["Scrape site A", "Scrape site B", "Scrape site C"].map((task) => client.run(task, { schema: MySchema })) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
5/10 creative test agents needed concurrency patterns but couldn't find any in the docs.
Changes
Structured output page — added a full "Scrape multiple sites concurrently" section with:
asyncio.gather+ Pydantic schema (3 sites, typed products)Promise.all+ Zod schema (same pattern)llms.txt (both root + cloud) — added compact concurrency snippet so LLMs discover the pattern immediately.
The example scrapes 3 e-commerce sites for laptop prices concurrently, returning typed
Productobjects. Real, practical, copy-paste ready.🤖 Generated with Claude Code
Summary by cubic
Added a concurrent scraping example with structured output in Python and TypeScript. Lets users run multiple
client.run()sessions in parallel and return typedProductresults.Productlist).asyncio.gatherwithpydantic; TypeScript usesPromise.allwithzod.docs/llms.txtanddocs/cloud/llms.txtfor quick discovery.Written for commit 981afe4. Summary will update on new commits.