Skip to content

Commit ecaa105

Browse files
six-ddcclaude
andcommitted
refactor(skills): rename skills, fix bugs, and expand coverage
- Rename research → browser-cli-research, site-guide → browser-cli-site-guide to avoid naming conflicts with other skills - Fix runtime bug: extractCompanyInfo → extractCompany in research skill - Remove duplicate Category URLs table in HN site guide - Add Weibo, Quora, Discord as research sources (sections 11-13) - Expand HN section with navigateTo/extractPostDetail/formatCommentTree/nextPage - Expand LinkedIn section with searchContent/searchCompanies/extractProfile - Add Web Component + CSS Module patterns to site-guide selector preference - Add reddit/weibo/discord to site-guide Existing Guides table - Expand Weibo guide Recipe debugging boilerplate - Fix Discord guide language consistency (Chinese → English) - Expand Error Recovery and Pagination tables in research skill - Register all three skills in marketplace.json Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 90d9ee7 commit ecaa105

10 files changed

Lines changed: 300 additions & 72 deletions

File tree

.claude-plugin/marketplace.json

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,11 @@
1111
"description": "Automate a real browser from the command line — navigate, interact, query, manage tabs/cookies/storage, intercept network requests, and capture snapshots/screenshots. Uses a Chrome/Firefox extension + daemon architecture (no Playwright required).",
1212
"strict": false,
1313
"keywords": ["browser", "automation", "cli", "chrome", "extension", "scraping"],
14-
"skills": ["./skills/browser-cli"]
14+
"skills": [
15+
"./skills/browser-cli",
16+
"./skills/browser-cli-research",
17+
"./skills/browser-cli-site-guide"
18+
]
1519
}
1620
],
1721
"metadata": {

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ The CLI sends commands to a background daemon, which relays them over WebSocket
2929

3030
- Navigate (goto, back, forward, reload), take screenshots
3131
- Extract clean readable Markdown via [Defuddle](https://github.com/nickersoft/defuddle) — strips nav, ads, and boilerplate
32-
- Accessibility snapshots with element refs (`@e1`, `@e2`) for precise interaction
32+
- Accessibility snapshots with element refs (`@e1`, `@e2`), baseline saving & diffing to detect page changes
3333
- Evaluate JavaScript in page context
3434

3535
**Interaction**
Lines changed: 172 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,12 @@
11
---
2-
name: research
2+
name: browser-cli-research
33
description: >
44
Deep research and information gathering using browser-cli to search, extract, and synthesize
55
information from the real web. Covers Google search, YouTube transcript extraction, Reddit/HN
6-
community discussions, LinkedIn professional insights, WeChat/Xiaohongshu Chinese-language sources,
7-
X/Twitter social posts, Google Scholar academic papers, article content extraction (markdown),
8-
and custom JS-based scraping for arbitrary websites.
6+
community discussions, LinkedIn professional insights, WeChat/Xiaohongshu/Weibo Chinese-language
7+
sources, X/Twitter social posts, Google Scholar academic papers, Quora Q&A, Discord tech
8+
communities, article content extraction (markdown), and custom JS-based scraping for arbitrary
9+
websites.
910
Use this skill whenever the user wants to research a topic, investigate a question, gather
1011
opinions from multiple sources, find discussions or community sentiment, compare viewpoints,
1112
collect references, or do any kind of information retrieval that goes beyond what a simple
@@ -48,11 +49,12 @@ Before touching the browser, decompose the research question:
4849
3. **Select sources** — For each sub-question, decide which sources are most likely to have
4950
good answers:
5051
- **Factual/technical**: Google → articles (markdown) → Google Scholar
51-
- **Community opinion**: Reddit, HN, X/Twitter
52+
- **Community opinion**: Reddit, HN, X/Twitter, Quora
5253
- **Professional/industry**: LinkedIn content search, LinkedIn company pages
53-
- **Chinese perspective**: WeChat, Xiaohongshu
54+
- **Chinese perspective**: WeChat, Xiaohongshu, Weibo
5455
- **Expert explanation**: YouTube transcripts
55-
- **Current events**: Google News, X/Twitter
56+
- **Current events**: Google News, X/Twitter, Weibo trending
57+
- **Tech communities**: HN, Discord (open-source projects, dev communities)
5658
4. **Report the plan** — Tell the user: "I'll research X by looking into [sub-questions]
5759
across [sources]. Let me get started."
5860

@@ -140,7 +142,7 @@ use the absolute path derived from this SKILL.md's location:
140142
SCRIPT_PATH = <this SKILL.md's directory>/../browser-cli/scripts
141143
```
142144

143-
For example, if this file is at `/Users/me/skills/research/SKILL.md`, then:
145+
For example, if this file is at `/Users/me/skills/browser-cli-research/SKILL.md`, then:
144146
`SCRIPT_PATH=/Users/me/skills/browser-cli/scripts`
145147

146148
### 1. Article Content Extraction (any website)
@@ -302,22 +304,30 @@ No login required for reading.
302304
**Script path**: `SCRIPT_PATH/hn.mjs`
303305

304306
```bash
305-
# Browse categories
306-
browser-cli --tab <tabId> navigate 'https://news.ycombinator.com/' # Top stories
307-
browser-cli --tab <tabId> navigate 'https://news.ycombinator.com/ask' # Ask HN
308-
browser-cli --tab <tabId> navigate 'https://news.ycombinator.com/show' # Show HN
307+
# Browse categories (or use navigateTo: top/new/ask/show/jobs)
308+
browser-cli --tab <tabId> script SCRIPT_PATH/hn.mjs --call navigateTo -- --category "top"
309309
browser-cli --tab <tabId> script SCRIPT_PATH/hn.mjs --call extractPosts
310310

311-
# Read a discussion
312-
browser-cli --tab <tabId> navigate 'https://news.ycombinator.com/item?id=<id>'
311+
# Next page of posts
312+
browser-cli --tab <tabId> script SCRIPT_PATH/hn.mjs --call nextPage
313+
browser-cli --tab <tabId> script SCRIPT_PATH/hn.mjs --call extractPosts
314+
315+
# Navigate to a specific post and get metadata
316+
browser-cli --tab <tabId> script SCRIPT_PATH/hn.mjs --call navigateToPost -- --id "<postId>"
317+
browser-cli --tab <tabId> script SCRIPT_PATH/hn.mjs --call extractPostDetail
318+
319+
# Extract comments (structured tree with depth)
313320
browser-cli --tab <tabId> script SCRIPT_PATH/hn.mjs --call extractComments
321+
# Or as a human-readable indented tree (ideal for research reports):
322+
browser-cli --tab <tabId> script SCRIPT_PATH/hn.mjs --call formatCommentTree
314323
```
315324

316325
**Research workflow**:
317326

318327
1. Search Google with `site:news.ycombinator.com <topic>` to find relevant threads
319328
2. Or use HN's built-in search: `https://hn.algolia.com/?q=<query>`
320-
3. Extract comments from high-engagement threads for expert opinions
329+
3. Navigate to the post with `navigateToPost`, get metadata with `extractPostDetail`
330+
4. Extract comments — use `formatCommentTree` for a readable indented view with usernames
321331

322332
### 7. LinkedIn (Professional & Industry Insights)
323333

@@ -330,19 +340,26 @@ industry trends, and professional discussions. Login required.
330340
# Check login (required)
331341
browser-cli --tab <tabId> script SCRIPT_PATH/linkedin.mjs --call detectLogin
332342

333-
# Search for content/posts on a topic
334-
browser-cli --tab <tabId> navigate 'https://www.linkedin.com/search/results/content/?keywords=<encoded-query>'
335-
browser-cli --tab <tabId> wait 3000
343+
# Search for content/posts on a topic (with extraction)
344+
browser-cli --tab <tabId> script SCRIPT_PATH/linkedin.mjs --call searchContent -- --keywords "AI trends 2025"
336345

337346
# Search for people (experts in a field)
338347
browser-cli --tab <tabId> script SCRIPT_PATH/linkedin.mjs --call searchPeople -- --keywords "AI researcher"
339348

349+
# Extract a person's profile
350+
browser-cli --tab <tabId> script SCRIPT_PATH/linkedin.mjs --call navigateProfile -- --username "johndoe"
351+
browser-cli --tab <tabId> script SCRIPT_PATH/linkedin.mjs --call extractProfile
352+
340353
# Search for companies
341-
browser-cli --tab <tabId> navigate 'https://www.linkedin.com/search/results/companies/?keywords=<encoded-query>'
354+
browser-cli --tab <tabId> script SCRIPT_PATH/linkedin.mjs --call searchCompanies -- --keywords "autonomous driving"
342355

343356
# Extract company info
344-
browser-cli --tab <tabId> navigate 'https://www.linkedin.com/company/<slug>/'
345-
browser-cli --tab <tabId> script SCRIPT_PATH/linkedin.mjs --call extractCompanyInfo
357+
browser-cli --tab <tabId> script SCRIPT_PATH/linkedin.mjs --call navigateCompany -- --slug "openai"
358+
browser-cli --tab <tabId> script SCRIPT_PATH/linkedin.mjs --call extractCompany
359+
360+
# Job market research
361+
browser-cli --tab <tabId> script SCRIPT_PATH/linkedin.mjs --call searchJobs -- --keywords "ML engineer" --location "San Francisco"
362+
browser-cli --tab <tabId> script SCRIPT_PATH/linkedin.mjs --call extractJobDetail
346363

347364
# Extract feed posts (industry discussions)
348365
browser-cli --tab <tabId> navigate 'https://www.linkedin.com/feed/'
@@ -351,10 +368,10 @@ browser-cli --tab <tabId> script SCRIPT_PATH/linkedin.mjs --call extractFeed
351368

352369
**Research use cases**:
353370

354-
- Company due diligence: company page → about, size, industry, recent posts
355-
- Expert identification: people search → profile extraction
356-
- Industry trends: content search → post extraction with engagement metrics
357-
- Job market research: job search → listings and descriptions
371+
- Company due diligence: `searchCompanies``navigateCompany``extractCompany`
372+
- Expert identification: `searchPeople``navigateProfile``extractProfile`
373+
- Industry trends: `searchContent` → post extraction with engagement metrics
374+
- Job market research: `searchJobs``extractJobDetail` for listings and descriptions
358375

359376
**Notes**: Login required for all LinkedIn pages. If not logged in, ask the user to log in first.
360377

@@ -422,7 +439,115 @@ browser-cli --tab <tabId> navigate 'https://x.com/<handle>'
422439
browser-cli --tab <tabId> script SCRIPT_PATH/x.mjs --call extractTweets
423440
```
424441

425-
### 11. Visual Evidence (Screenshots)
442+
### 11. Weibo (Chinese Public Discourse + Trending)
443+
444+
Weibo is China's largest microblogging platform — essential for Chinese public opinion, trending
445+
topics, breaking news, and celebrity/brand discussions. Login required for full access.
446+
447+
**Script path**: `SCRIPT_PATH/weibo.mjs`
448+
449+
```bash
450+
# Check login
451+
browser-cli --tab <tabId> script SCRIPT_PATH/weibo.mjs --call detectLogin
452+
453+
# Search for a topic
454+
browser-cli --tab <tabId> script SCRIPT_PATH/weibo.mjs --call search -- --keyword "人工智能"
455+
browser-cli --tab <tabId> script SCRIPT_PATH/weibo.mjs --call extractSearchResults
456+
457+
# Extract trending topics (热搜榜)
458+
browser-cli --tab <tabId> navigate 'https://weibo.com/hot/search'
459+
browser-cli --tab <tabId> wait 3000
460+
browser-cli --tab <tabId> script SCRIPT_PATH/weibo.mjs --call extractTrending
461+
462+
# Extract feed posts (homepage or hot)
463+
browser-cli --tab <tabId> navigate 'https://weibo.com/hot'
464+
browser-cli --tab <tabId> wait 3000
465+
browser-cli --tab <tabId> script SCRIPT_PATH/weibo.mjs --call extractFeed
466+
467+
# Read a post's comments
468+
browser-cli --tab <tabId> navigate '<post-detail-url>'
469+
browser-cli --tab <tabId> script SCRIPT_PATH/weibo.mjs --call openComments
470+
browser-cli --tab <tabId> script SCRIPT_PATH/weibo.mjs --call extractComments
471+
```
472+
473+
**Research workflow**:
474+
475+
1. Check `extractTrending` for what's currently hot in China
476+
2. Search specific topics with `search` + `extractSearchResults`
477+
3. For deeper sentiment, read comments on popular posts
478+
4. Comments use virtual scrolling — use `scrollComments` + `getComments` for more
479+
480+
**Notes**: Weibo has strong anti-bot measures. All interactions must go through page clicks
481+
(not API calls). Add delays (500-2000ms) between operations. Login required.
482+
483+
### 12. Quora (Q&A Knowledge)
484+
485+
Quora is useful for expert answers, explanations, and diverse perspectives on questions.
486+
487+
**Script path**: `SCRIPT_PATH/quora.mjs`
488+
489+
```bash
490+
# Search for questions
491+
browser-cli --tab <tabId> script SCRIPT_PATH/quora.mjs --call navigateToSearch -- --query "best practices for X"
492+
browser-cli --tab <tabId> script SCRIPT_PATH/quora.mjs --call extractSearchResults
493+
494+
# Read answers on a specific question
495+
browser-cli --tab <tabId> script SCRIPT_PATH/quora.mjs --call navigateToQuestion -- --url "<question-url>"
496+
browser-cli --tab <tabId> script SCRIPT_PATH/quora.mjs --call extractQuestionMeta
497+
browser-cli --tab <tabId> script SCRIPT_PATH/quora.mjs --call expandAllAnswers
498+
browser-cli --tab <tabId> script SCRIPT_PATH/quora.mjs --call extractAnswers
499+
500+
# Load more answers if needed
501+
browser-cli --tab <tabId> script SCRIPT_PATH/quora.mjs --call loadMoreAnswers
502+
browser-cli --tab <tabId> script SCRIPT_PATH/quora.mjs --call extractAnswers
503+
```
504+
505+
**Research workflow**:
506+
507+
1. Search Google with `site:quora.com <topic>` or use `navigateToSearch` directly
508+
2. Navigate to high-quality questions, expand all answers
509+
3. Extract answers with upvote counts and author credentials
510+
511+
### 13. Discord (Tech Communities + Open Source)
512+
513+
Discord communities are valuable for real-time discussions in open-source projects, dev tools,
514+
crypto, gaming, and niche tech communities. Login required; read-only operations.
515+
516+
**Script path**: `SCRIPT_PATH/discord.mjs`
517+
518+
```bash
519+
# Check login
520+
browser-cli --tab <tabId> script SCRIPT_PATH/discord.mjs --call detectLogin
521+
522+
# List joined servers
523+
browser-cli --tab <tabId> script SCRIPT_PATH/discord.mjs --call listServers
524+
525+
# Navigate to a server and list channels
526+
browser-cli --tab <tabId> script SCRIPT_PATH/discord.mjs --call navigateServer -- --name "Open WebUI"
527+
browser-cli --tab <tabId> script SCRIPT_PATH/discord.mjs --call listChannels
528+
529+
# Read messages in a channel
530+
browser-cli --tab <tabId> script SCRIPT_PATH/discord.mjs --call navigateChannel -- --url "<channel-url>"
531+
browser-cli --tab <tabId> script SCRIPT_PATH/discord.mjs --call extractMessages
532+
533+
# Search messages across the server
534+
browser-cli --tab <tabId> script SCRIPT_PATH/discord.mjs --call searchMessages -- --query "bug report"
535+
536+
# View pinned messages
537+
browser-cli --tab <tabId> script SCRIPT_PATH/discord.mjs --call extractPinnedMessages
538+
```
539+
540+
**Research workflow**:
541+
542+
1. Navigate to the relevant server and channel
543+
2. Search for topic-specific messages with `searchMessages`
544+
3. Extract messages and pinned messages for context
545+
4. Check member list with `extractMembers` for community size/roles
546+
547+
**Notes**: Login required. Discord search uses Draft.js contentEditable input (handled by
548+
the recipe script). Read-only operations only.
549+
550+
### 14. Visual Evidence (Screenshots)
426551

427552
For research involving charts, infographics, or visual data that `markdown` can't capture:
428553

@@ -434,7 +559,7 @@ browser-cli --tab <tabId> screenshot --path /tmp/research-evidence.png
434559
browser-cli --tab <tabId> screenshot --selector '.chart-container' --path /tmp/chart.png
435560
```
436561

437-
### 12. Custom JS Extraction (Arbitrary Websites)
562+
### 15. Custom JS Extraction (Arbitrary Websites)
438563

439564
For websites without a dedicated site guide, use `eval` or `script -` to extract data directly.
440565

@@ -482,6 +607,10 @@ When a source fails, don't stop — fall back to alternatives:
482607
| X/Twitter requires login | Search Google with `site:x.com <topic>` for indexed tweets |
483608
| WeChat article expired | Search for the article title on Google or Baidu for cached versions |
484609
| LinkedIn not logged in | Ask user to log in; meanwhile use Google `site:linkedin.com` for public profiles |
610+
| Discord not logged in | Ask user to log in; no public fallback available |
611+
| Weibo anti-bot blocks | Add longer delays (2-5s) between operations; avoid rapid sequential requests |
612+
| Scholar CAPTCHA triggered | Wait 30-60s and retry; reduce query frequency; try alternative search terms |
613+
| Quora blocks extraction | Search Google with `site:quora.com <topic>` and read via `markdown` |
485614

486615
## Handling Pagination
487616

@@ -500,6 +629,22 @@ browser-cli --tab <tabId> navigate 'https://www.google.com/search?q=topic&start=
500629
# WeChat: use nextPage
501630
browser-cli --tab <tabId> script SCRIPT_PATH/weixin.mjs --call nextPage
502631

632+
# HN: click "More" link for next page
633+
browser-cli --tab <tabId> script SCRIPT_PATH/hn.mjs --call nextPage
634+
635+
# Scholar: next page of results
636+
browser-cli --tab <tabId> script SCRIPT_PATH/scholar.mjs --call nextPage
637+
638+
# Weibo: use scroll collector for feed/search pages
639+
browser-cli --tab <tabId> script SCRIPT_PATH/weibo.mjs --call initScrollCollector
640+
browser-cli --tab <tabId> script SCRIPT_PATH/weibo.mjs --call scrollAndCollect
641+
browser-cli --tab <tabId> script SCRIPT_PATH/weibo.mjs --call getCollected
642+
643+
# LinkedIn: use feed collector for infinite scroll
644+
browser-cli --tab <tabId> script SCRIPT_PATH/linkedin.mjs --call initFeedCollector
645+
browser-cli --tab <tabId> script SCRIPT_PATH/linkedin.mjs --call scrollAndCollect
646+
browser-cli --tab <tabId> script SCRIPT_PATH/linkedin.mjs --call getCollectedPosts
647+
503648
# General infinite scroll: scroll + wait + extract
504649
browser-cli --tab <tabId> scroll down --amount 3000
505650
browser-cli --tab <tabId> wait 2000

0 commit comments

Comments
 (0)