Make curl_cffi the sole HTTP dependency, remove requests and httpx

Facebook blocks any HTTP client without Chrome-like TLS fingerprints,
making requests and httpx dead-weight fallbacks that fail with 403.
curl_cffi is now required (not optional), and the [stealth] and [async]
extras are removed. Installation is simplified to just `pip install
meta-ads-collector`.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: promisingcoder

CHANGELOG.md

@@ -5,6 +5,20 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.3.0] - 2026-02-21
+
+### Changed
+- **curl_cffi is now the sole HTTP dependency.** Removed `requests` and `httpx` entirely. Facebook blocks any HTTP client without Chrome-like TLS fingerprints, so `curl_cffi` (with `impersonate="chrome"`) is the only backend that actually works. Both `requests` and `httpx` were dead-weight fallbacks that failed with HTTP 403.
+- **Simplified installation.** `pip install meta-ads-collector` now includes everything -- no more `[stealth]` or `[async]` extras needed.
+- **Async client simplified.** Removed the `_AsyncResponse` wrapper and httpx fallback code paths. The async client now uses `curl_cffi.AsyncSession` directly.
+- Renamed `ProxyPool.get_requests_proxies()` to `get_proxy_dict()`.
+
+### Removed
+- `requests` dependency (was required, now removed)
+- `httpx` dependency and `[async]` optional extra
+- `[stealth]` optional extra (curl_cffi is now always installed)
+- `types-requests` from dev dependencies
+
 ## [1.2.0] - 2026-02-21
 
 ### Fixed
@@ -90,10 +104,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Optional batch mode for webhook sends
 
 #### Async Support
-- `AsyncMetaAdsClient` with `curl_cffi.AsyncSession` (preferred) or `httpx.AsyncClient` (fallback)
+- `AsyncMetaAdsClient` with `curl_cffi.AsyncSession`
 - `AsyncMetaAdsCollector` mirroring the sync API with `async for` generators
 - Async `search()`, `collect()`, `collect_to_json()`, `collect_to_csv()`, `search_pages()`
-- Optional dependency: `pip install meta-ads-collector[stealth]` (recommended) or `pip install meta-ads-collector[async]`
 
 #### Proxy Support
 - Single proxy configuration (host:port or host:port:user:pass)

README.md

@@ -49,28 +49,18 @@ meta-ads-collector -q "solar panels" -c US -n 10 -o ads.json
 pip install meta-ads-collector
 ```
 
-With stealth TLS fingerprinting (recommended, also enables async support):
-
-```bash
-pip install meta-ads-collector[stealth]
-```
-
-With async support only (uses [httpx](https://www.python-httpx.org/)):
-
-```bash
-pip install meta-ads-collector[async]
-```
-
 From source:
 
 ```bash
 git clone https://github.com/promisingcoder/MetaAdsCollector.git
-cd meta-ads-collector
-pip install -e ".[dev,async,stealth]"
+cd MetaAdsCollector
+pip install -e ".[dev]"
 ```
 
 **Requirements:** Python 3.9+
 
+`curl_cffi` is installed automatically and provides Chrome-like TLS fingerprints so requests are indistinguishable from a real browser.
+
 ## Features
 
 - **Search & Collection** -- keyword search, exact phrase, page-level collection by URL/name/ID
@@ -79,13 +69,13 @@ pip install -e ".[dev,async,stealth]"
 - **Media Downloads** -- download images, videos, and thumbnails from ad creatives
 - **Ad Enrichment** -- fetch additional detail data from the ad snapshot endpoint
 - **Events & Webhooks** -- 7 lifecycle events with callback registration, webhook POST integration
-- **Async Support** -- full async/await API using curl_cffi (preferred) or httpx (fallback)
+- **Async Support** -- full async/await API using curl_cffi
 - **Proxy Support** -- single proxy, proxy rotation with failure tracking and dead-proxy cooldown
 - **Structured Logging** -- text or JSON log format, optional file output
 - **Collection Reporting** -- summary statistics with throughput metrics
 - **Export Formats** -- JSON, CSV, JSONL
 - **Stream Mode** -- yield lifecycle events alongside ads through a single iterator
-- **Detection Avoidance** -- browser fingerprint randomization, TLS fingerprint impersonation (via `curl_cffi`), dynamic token extraction, session management
+- **Detection Avoidance** -- browser fingerprint randomization, Chrome TLS fingerprint impersonation via `curl_cffi`, dynamic token extraction, session management
 
 ---
 
@@ -381,14 +371,6 @@ with MetaAdsCollector() as collector:
 
 Full async API with the same TLS fingerprint impersonation as the sync client.
 
-```bash
-# Recommended: uses curl_cffi for TLS fingerprinting (same as sync client)
-pip install meta-ads-collector[stealth]
-
-# Alternative: uses httpx (may be detected by Facebook)
-pip install meta-ads-collector[async]
-```
-
 ```python
 import asyncio
 from meta_ads_collector.async_collector import AsyncMetaAdsCollector
@@ -405,7 +387,7 @@ async def main():
 asyncio.run(main())
 ```
 
-The async collector mirrors the sync API: `search()`, `collect()`, `collect_to_json()`, `collect_to_csv()`, `search_pages()`, `get_stats()`. When `curl_cffi` is installed, the async client uses `curl_cffi.AsyncSession` with Chrome TLS impersonation. Otherwise it falls back to `httpx.AsyncClient`.
+The async collector mirrors the sync API: `search()`, `collect()`, `collect_to_json()`, `collect_to_csv()`, `search_pages()`, `get_stats()`. The async client uses `curl_cffi.AsyncSession` with Chrome TLS impersonation.
 
 ---
 
@@ -718,7 +700,7 @@ All exceptions inherit from `MetaAdsError`.
 ## Development
 
 ```bash
-# Install with dev dependencies
+# Install with dev dependencies (curl_cffi is included automatically)
 pip install -e ".[dev]"
 
 # Run tests

docs/api-reference.md

@@ -173,8 +173,6 @@ Close the collector and release resources.
 
 **Module:** `meta_ads_collector.async_collector`
 
-**Requires:** `pip install meta-ads-collector[stealth]` (recommended) or `pip install meta-ads-collector[async]`
-
 ### class AsyncMetaAdsCollector
 
 Async mirror of `MetaAdsCollector`. All methods are `async def` and iterators are `async for`.
@@ -247,11 +245,9 @@ Close the session.
 
 **Module:** `meta_ads_collector.async_client`
 
-**Requires:** `pip install meta-ads-collector[stealth]` (recommended) or `pip install meta-ads-collector[async]`
-
 ### class AsyncMetaAdsClient
 
-Async mirror of `MetaAdsClient` using `curl_cffi.AsyncSession` (preferred) or `httpx.AsyncClient` (fallback). Handles Facebook's 403 verification challenges automatically. All HTTP methods are async, while pure-logic methods are delegated to the sync client.
+Async mirror of `MetaAdsClient` using `curl_cffi.AsyncSession`. Handles Facebook's 403 verification challenges automatically. All HTTP methods are async, while pure-logic methods are delegated to the sync client.
 
 Same method signatures as `MetaAdsClient` with `async def`.
 
@@ -544,7 +540,7 @@ Download images, videos, and thumbnails from ad creatives.
 | Parameter | Type | Default | Description |
 |---|---|---|---|
 | `output_dir` | `str \| Path` | (required) | Download directory |
-| `session` | `requests.Session \| None` | `None` | HTTP session to reuse |
+| `session` | `curl_cffi.requests.Session \| None` | `None` | HTTP session to reuse |
 | `timeout` | `int` | `30` | Per-request timeout |
 | `max_retries` | `int` | `2` | Retry attempts per download |
 
@@ -594,7 +590,7 @@ Round-robin proxy pool with failure tracking and cooldown recovery.
 | `mark_success(proxy)` | Record successful request |
 | `mark_failure(proxy)` | Record failed request |
 | `reset()` | Reset all counters and revive all proxies |
-| `get_requests_proxies(proxy_url) -> dict` | Convert to `requests`-compatible proxy dict |
+| `get_proxy_dict(proxy_url) -> dict` | Convert proxy URL to `{"http": url, "https": url}` dict |
 
 #### Properties
 

docs/architecture.md

@@ -28,7 +28,7 @@ This document describes the internal architecture of `meta-ads-collector`. It is
 | Module | Layer | Responsibility |
 |---|---|---|
 | `collector.py` | Collector | High-level search, pagination, export, media, enrichment |
-| `async_collector.py` | Collector | Async mirror of `collector.py` using `httpx` |
+| `async_collector.py` | Collector | Async mirror of `collector.py` using `curl_cffi` |
 | `client.py` | Client | Session management, token extraction, GraphQL requests |
 | `async_client.py` | Client | Async mirror of `client.py`, delegates logic to sync client |
 | `models.py` | Data | `Ad`, `AdCreative`, `PageInfo`, `PageSearchResult`, etc. |
@@ -172,9 +172,9 @@ Sessions become stale after `MAX_SESSION_AGE` (30 minutes by default). Before ea
 
 `_refresh_session()` performs a full re-initialization:
 
-1. Close the old session (`requests.Session` or `curl_cffi.requests.Session`)
+1. Close the old session (`curl_cffi.requests.Session`)
 2. Generate a new `BrowserFingerprint`
-3. Create a fresh session with new headers (preferring `curl_cffi` when available)
+3. Create a fresh session with new headers
 4. Re-run `initialize()` to get new tokens
 5. Track consecutive refresh failures to prevent infinite loops
 
@@ -204,7 +204,7 @@ All headers derived from the fingerprint are self-consistent -- the Chrome versi
 
 ### TLS Fingerprint Impersonation
 
-When the optional `curl_cffi` dependency is installed (`pip install meta-ads-collector[stealth]`), the client uses `curl_cffi.requests.Session(impersonate="chrome")` instead of `requests.Session`. This provides Chrome-like TLS fingerprints (JA3/JA4) at the connection level, making requests indistinguishable from a real Chrome browser to TLS-based bot detection systems. If `curl_cffi` is not installed, the library falls back to `requests.Session` transparently.
+The client uses `curl_cffi.requests.Session(impersonate="chrome")` which provides Chrome-like TLS fingerprints (JA3/JA4) at the connection level, making requests indistinguishable from a real Chrome browser to TLS-based bot detection systems.
 
 ### Request Mimicry
 
@@ -265,7 +265,7 @@ Seven lifecycle events:
 self._logic = MetaAdsClient.__new__(MetaAdsClient)  # No __init__ call
 ```
 
-All pure-logic methods (token extraction, payload building, response parsing) are delegated to `_logic`. Only HTTP methods are reimplemented using `curl_cffi.AsyncSession` (preferred) or `httpx.AsyncClient` (fallback). The async client also handles Facebook's 403 verification challenges automatically, matching the sync client's behavior.
+All pure-logic methods (token extraction, payload building, response parsing) are delegated to `_logic`. Only HTTP methods are reimplemented using `curl_cffi.AsyncSession`. The async client also handles Facebook's 403 verification challenges automatically, matching the sync client's behavior.
 
 `AsyncMetaAdsCollector` mirrors `MetaAdsCollector` method-for-method, using `async def` and `async for` throughout.
 

docs/async.md

@@ -1,21 +1,15 @@
 # Async Usage Guide
 
-`meta-ads-collector` provides a full async API for use with `asyncio`. The async client uses the same TLS fingerprint impersonation as the sync client to avoid detection.
+`meta-ads-collector` provides a full async API for use with `asyncio`. The async client uses `curl_cffi.AsyncSession` with Chrome TLS fingerprint impersonation, matching the sync client's behavior.
 
 ## Installation
 
-The async client uses `curl_cffi` (recommended) for TLS fingerprint impersonation, or falls back to `httpx`:
+Async support is included out of the box -- no extra install required:
 
 ```bash
-# Recommended: uses curl_cffi (same TLS fingerprinting as sync client)
-pip install meta-ads-collector[stealth]
-
-# Alternative: uses httpx (may get blocked by Facebook's TLS fingerprint detection)
-pip install meta-ads-collector[async]
+pip install meta-ads-collector
 ```
 
-If both `curl_cffi` and `httpx` are installed, `curl_cffi` is preferred automatically.
-
 ## AsyncMetaAdsCollector
 
 The async collector mirrors the sync `MetaAdsCollector` API with `async def` methods and `async for` generators.
@@ -141,7 +135,7 @@ async with AsyncMetaAdsClient() as client:
 
 ## Notes
 
-- The async client prefers `curl_cffi.AsyncSession` for TLS fingerprint impersonation, falling back to `httpx.AsyncClient` if curl_cffi is not installed
+- The async client uses `curl_cffi.AsyncSession` with Chrome TLS fingerprint impersonation
 - Facebook's 403 verification challenges are handled automatically (same as the sync client)
 - Rate limiting uses `asyncio.sleep()` instead of `time.sleep()`
 - Session initialization is performed asynchronously on the first request

docs/contributing.md

@@ -22,7 +22,7 @@ source .venv/bin/activate  # Linux/macOS
 # .venv\Scripts\activate   # Windows
 
 # Install in editable mode with all dev dependencies
-pip install -e ".[dev,async,stealth]"
+pip install -e ".[dev]"
 ```
 
 Or use the Makefile:
@@ -35,10 +35,8 @@ make install-dev
 
 | Group | What it includes | When to install |
 |---|---|---|
-| (none) | `requests>=2.28.0` | Always (core dependency) |
-| `async` | `httpx>=0.24.0` | Async fallback when curl_cffi is unavailable |
-| `stealth` | `curl_cffi>=0.7.0` | TLS fingerprint impersonation for sync and async clients (recommended) |
-| `dev` | `pytest`, `pytest-cov`, `pytest-asyncio`, `ruff`, `mypy`, `types-requests` | Always for development |
+| (none) | `curl_cffi>=0.7.0` | Always (core dependency, provides Chrome TLS fingerprinting) |
+| `dev` | `pytest`, `pytest-cov`, `pytest-asyncio`, `ruff`, `mypy` | Always for development |
 
 ## Running Tests
 

docs/proxy.md

@@ -120,7 +120,7 @@ pool.reset()
 | `reset()` | Revive all proxies, reset all counters |
 | `alive_proxies` | List of currently usable proxy URLs |
 | `from_file(path)` | Create pool from a text file |
-| `get_requests_proxies(url)` | Convert URL to `{"http": url, "https": url}` dict |
+| `get_proxy_dict(url)` | Convert URL to `{"http": url, "https": url}` dict |
 
 ## Disabling proxies
 

docs/quickstart.md

@@ -8,11 +8,7 @@ Get from zero to collecting ads in under 60 seconds.
 pip install meta-ads-collector
 ```
 
-For stealth TLS fingerprinting (recommended for production use):
-
-```bash
-pip install meta-ads-collector[stealth]
-```
+This installs `curl_cffi` automatically, providing Chrome-like TLS fingerprinting so requests are indistinguishable from a real browser.
 
 ## Python API
 
@@ -89,6 +85,6 @@ meta-ads-collector -q "test" --proxy "host:port:user:pass" -o ads.json
 - [Deduplication guide](deduplication.md) -- avoid collecting the same ad twice
 - [Media downloads](media.md) -- download images and videos from ad creatives
 - [Events & webhooks](events.md) -- react to collection lifecycle events
-- [Async usage](async.md) -- use with asyncio (curl_cffi or httpx)
+- [Async usage](async.md) -- use with asyncio
 - [Proxy configuration](proxy.md) -- proxy rotation and failure handling
 - [CLI reference](cli.md) -- complete list of CLI flags

meta_ads_collector/__init__.py

@@ -32,7 +32,7 @@
 from .url_parser import extract_page_id_from_url
 from .webhooks import WebhookSender
 
-__version__ = "1.2.0"
+__version__ = "1.3.0"
 __all__ = [
     # Models
     "Ad",