Skip to content

Commit 34b536c

Browse files
committed
feat: Add camera livestream and voice call support
Adds a standalone camera module for streaming video/audio from Roborock vacuums with cameras. This addresses part of #738. Features: - Live video streaming (HD/SD quality) - Bidirectional voice calls (speak to and hear from robot) - Snapshot capture - Video recording - Audio sending for intercom functionality The implementation uses: - MQTT for WebRTC signaling (via existing protocol encoders) - aiortc for WebRTC peer connection - paho-mqtt for MQTT connection (standalone, not using aiomqtt) This is provided as a standalone module that could be integrated more tightly with the library's session management in future work. See docs/CAMERA_PROTOCOL.md for full protocol documentation. New dependencies (optional): - aiortc: WebRTC implementation - paho-mqtt: MQTT client - pyaudio: Voice call audio (optional) - opencv-python: Video display (optional)
1 parent dfa2fb1 commit 34b536c

File tree

2 files changed

+851
-0
lines changed

2 files changed

+851
-0
lines changed

docs/CAMERA_PROTOCOL.md

Lines changed: 179 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,179 @@
1+
# Roborock Camera Protocol Documentation
2+
3+
This document describes the protocol for streaming video/audio from Roborock vacuum cameras.
4+
5+
## Overview
6+
7+
The camera uses **MQTT for signaling** and **WebRTC for media transport**:
8+
9+
```
10+
┌─────────────┐ MQTT ┌──────────────┐
11+
│ Client │◄────────────►│ Roborock │
12+
│ (Python) │ Signaling │ Cloud │
13+
└─────────────┘ └──────────────┘
14+
│ │
15+
│ WebRTC │
16+
└─────────────────────────────┘
17+
Video + Audio
18+
```
19+
20+
## Connection Flow
21+
22+
1. Connect to MQTT broker (`mqtt-{region}.roborock.com:8883`)
23+
2. Authenticate with password hash
24+
3. Start camera preview
25+
4. Get TURN server credentials from Roborock cloud
26+
5. Exchange SDP/ICE candidates via MQTT
27+
6. Establish WebRTC peer connection
28+
7. Receive video/audio tracks
29+
30+
## MQTT Topics
31+
32+
```
33+
Publish: rr/m/i/{rriot_u}/{client_id}/{duid}
34+
Subscribe: rr/m/o/{rriot_u}/{client_id}/{duid}
35+
```
36+
37+
Where:
38+
- `rriot_u`: User identifier from Roborock account
39+
- `client_id`: MD5-derived client identifier
40+
- `duid`: Device unique identifier
41+
42+
## MQTT Credentials
43+
44+
```python
45+
mqtt_username = md5(f"{rriot_u}:{rriot_k}")[2:10]
46+
mqtt_password = md5(f"{rriot_s}:{rriot_k}")[16:]
47+
```
48+
49+
## Commands
50+
51+
All commands are sent via protocol 101 in the `dps.101` field. Responses come in `dps.102`.
52+
53+
### Camera Control
54+
55+
| Command | Params | Response | Notes |
56+
|---------|--------|----------|-------|
57+
| `check_homesec_password` | `{password: md5_hash}` | `['ok']` | Authenticate with pattern password |
58+
| `start_camera_preview` | `{client_id, quality, password}` | `['ok']` | Begin video session |
59+
| `stop_camera_preview` | `[]` | `['ok']` | End video session |
60+
| `switch_video_quality` | `{quality: "HD"/"SD"}` | `['ok']` | Change resolution |
61+
62+
### WebRTC Signaling
63+
64+
| Command | Params | Response | Notes |
65+
|---------|--------|----------|-------|
66+
| `get_turn_server` | `[]` | `{url, user, pwd}` | TURN credentials |
67+
| `send_sdp_to_robot` | `{app_sdp: base64}` | `['ok']` | Send our SDP offer |
68+
| `get_device_sdp` | `[]` | `{dev_sdp: base64}` or `"retry"` | Robot's SDP answer |
69+
| `send_ice_to_robot` | `{app_ice: base64}` | `['ok']` | Send ICE candidates |
70+
| `get_device_ice` | `[]` | `{dev_ice: [base64...]}` | Robot's ICE candidates |
71+
72+
### Voice Chat
73+
74+
| Command | Params | Response | Notes |
75+
|---------|--------|----------|-------|
76+
| `start_voice_chat` | `[]` | `['ok']` | Enable bidirectional audio |
77+
| `stop_voice_chat` | `[]` | `['ok']` | Disable audio |
78+
79+
**Important:** Without calling `start_voice_chat`, the audio track exists but sends no frames!
80+
81+
### Remote Control
82+
83+
| Command | Params | Response | Notes |
84+
|---------|--------|----------|-------|
85+
| `app_rc_start` | `[]` | `['ok']` | Begin RC session |
86+
| `app_rc_move` | `{omega, velocity, seqnum, duration}` | `['ok']` | Movement command |
87+
| `app_rc_end` | `[]` | `['ok']` | End RC session |
88+
89+
RC Parameters:
90+
- `velocity`: Forward/backward speed (±0.2 typical range)
91+
- `omega`: Rotation speed in rad/s (±0.53 typical range)
92+
- `seqnum`: Incrementing sequence number
93+
- `duration`: Command duration in ms (500 typical)
94+
95+
## Audio Format
96+
97+
### Robot → Client
98+
- **Format:** Stereo interleaved, 48kHz, 16-bit signed PCM
99+
- **Frame size:** 960 samples per channel (20ms)
100+
- **Raw frame:** 1920 int16 values in LRLRLR... pattern
101+
- **To extract mono:** `audio[::2]` (take left channel)
102+
103+
### Client → Robot
104+
- **Format:** Mono, 48kHz, 16-bit signed PCM
105+
- **Frame size:** 960 samples (20ms)
106+
- **Timing:** Send frames spaced ~18-20ms apart
107+
108+
## Message Format
109+
110+
### Request (dps.101)
111+
```json
112+
{
113+
"id": 100001,
114+
"method": "command_name",
115+
"params": {}
116+
}
117+
```
118+
119+
### Response (dps.102)
120+
```json
121+
{
122+
"id": 100001,
123+
"result": ["ok"]
124+
}
125+
```
126+
127+
Or on error:
128+
```json
129+
{
130+
"id": 100001,
131+
"error": {"code": -1, "message": "error description"}
132+
}
133+
```
134+
135+
## SDP/ICE Format
136+
137+
SDP and ICE candidates are base64-encoded JSON:
138+
139+
```python
140+
# SDP
141+
sdp_json = json.dumps({"sdp": sdp_string, "type": "offer"})
142+
sdp_b64 = base64.b64encode(sdp_json.encode()).decode()
143+
144+
# ICE
145+
ice_json = json.dumps({
146+
"candidate": "candidate:...",
147+
"sdpMid": "0",
148+
"sdpMLineIndex": 0
149+
})
150+
ice_b64 = base64.b64encode(ice_json.encode()).decode()
151+
```
152+
153+
## Important Notes
154+
155+
1. **One session at a time** - Only one client can preview the camera. Close the phone app first.
156+
157+
2. **Password rate limiting** - Too many incorrect password attempts may disable remote viewing temporarily.
158+
159+
3. **TURN server required** - NAT traversal typically requires the TURN server; direct P2P connections are rare.
160+
161+
4. **Pattern password** - The password is your numeric pattern (e.g., "9876"), not your Roborock account password.
162+
163+
## Tested Devices
164+
165+
- Roborock Qrevo Curv (model a135)
166+
167+
Other camera-equipped Roborock models likely use the same protocol but are untested.
168+
169+
## Getting Credentials
170+
171+
The required credentials (`duid`, `local_key`, `rriot_u/k/s`) can be obtained from:
172+
- Home Assistant's Roborock integration storage
173+
- The python-roborock library's login flow
174+
- Roborock app traffic analysis
175+
176+
## References
177+
178+
- [python-roborock](https://github.com/Python-roborock/python-roborock) - Protocol encoding/decoding
179+
- [aiortc](https://github.com/aiortc/aiortc) - Python WebRTC implementation

0 commit comments

Comments
 (0)