feat(fetch): add proxy support via standard proxy environment variables

[mact24]

Summary

• The fetch pipeline step (used by all strategy: public adapters like HackerNews, Wikipedia, arXiv, etc.) previously called Node.js native fetch directly, which ignores system proxy settings
• Added proxy support using undici ProxyAgent, reading standard proxy environment variables at runtime

How it works

When any of the following env vars are set, Node-side fetch calls are routed through the proxy. Browser-side fetch paths are unaffected (Chrome uses its own proxy settings).

HTTPS_PROXY=http://127.0.0.1:7890 opencli hackernews top        
HTTP_PROXY=http://127.0.0.1:7890 opencli arxiv search --query "LLM"                                                                                                                                                                                                                                                       
ALL_PROXY=socks5://127.0.0.1:7891 opencli wikipedia search --query "AI"                                                                                                                                                                                                                                                   
                                                                                                                                                                                                                                                                                                                          
Supported variables (in priority order): HTTPS_PROXY, https_proxy, HTTP_PROXY, http_proxy, ALL_PROXY, all_proxy                                                                                                                                                                                                           
                                                                                                                                                                                                                                                                                                                          
Changes                                                                                                                                                                                                                                                                                                                   
                                                                
- src/pipeline/steps/fetch.ts: added getProxyFetch() helper that returns a proxy-aware fetch when env vars are present, falls back to native fetch otherwise                                                                                                                                                              
- package.json / package-lock.json: added undici dependency (the same library that powers Node.js built-in fetch)

[Astro-Han]

Good idea — proxy support for Chinese users is a real need. Found some issues with the current implementation:

Critical

SOCKS5 will crash at runtime

undici ProxyAgent only supports HTTP/HTTPS CONNECT proxies. ALL_PROXY=socks5://127.0.0.1:7891 (common with Clash, V2Ray, etc.) will throw a TypeError on first request. SOCKS5 support is still an open issue in undici.

Proxy selection ignores request scheme

getProxyFetch() picks one proxy URL and uses it for all requests. When both HTTP_PROXY and HTTPS_PROXY are set, http:// requests incorrectly go through HTTPS_PROXY. Also NO_PROXY is not respected.

Recommended fix: Replace the manual proxy resolution with undici's built-in EnvHttpProxyAgent, which handles scheme routing, NO_PROXY, and env var priority correctly out of the box. For SOCKS5, either detect and skip with a warning, or add socks-proxy-agent as a fallback.

Warnings

Node version conflict — undici@^7.24.5 requires Node >=20.18.1, but the project declares >=20.0.0 in package.json.

ProxyAgent created per-request — getProxyFetch() creates a new ProxyAgent on every call. In batch scenarios (mapConcurrent), this means N separate connection pools with no cleanup. Should be cached as a module-level singleton.

Only covers YAML pipeline fetch — 10+ TS adapters call Node.js fetch() directly (weread, wikipedia, arxiv, google, substack, etc.) and won't benefit. Consider undici.setGlobalDispatcher() as a follow-up, or at least note this as a known limitation.

Minor

• No validation for malformed proxy URLs — will throw raw undici errors instead of CliError
• package-lock.json resolved URLs point to registry.npmmirror.com instead of registry.npmjs.org
• No tests added