Skip to content

Commit c7eb4e9

Browse files
committed
Add serial (UART) distribution support
Implement distributed Erlang over serial/UART connections, enabling disterl between devices connected via serial lines. Also fix net_kernel accept handler to support alternative carriers by wildcarding the family/protocol atoms instead of hardcoding inet/tcp. Also add `os.beam` to `estdlib.avm` fixing dialyzer errors. Signed-off-by: Paul Guyot <pguyot@kallisys.net>
1 parent 72d635d commit c7eb4e9

14 files changed

Lines changed: 1691 additions & 3 deletions

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1313
- Added generic unix support for uart using POSIX nifs
1414
- Added RISC-V 64-bit (RV64IMAC) JIT backend
1515
- Added DWARF debug information support for JIT-compiled code
16+
- Added Erlang distribution over serial (uart)
1617

1718
### Changed
1819
- ~10% binary size reduction by rewriting module loading logic

doc/src/distributed-erlang.md

Lines changed: 149 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,15 @@ Distribution is currently available on all platforms with TCP/IP communication,
1414
- ESP32
1515
- RP2 (Pico)
1616

17-
Two examples are provided:
17+
Distribution over serial (UART) is also available for point-to-point
18+
connections between any two nodes, including microcontrollers without
19+
networking (e.g. STM32). See [Serial distribution](#serial-distribution).
20+
21+
Three examples are provided:
1822

1923
- disterl in `examples/erlang/disterl.erl`: distribution on Unix systems
2024
- epmd\_disterl in `examples/erlang/esp32/epmd_disterl.erl`: distribution on ESP32 devices
25+
- serial\_disterl in `examples/erlang/serial_disterl.erl`: distribution over serial (ESP32 and Unix)
2126

2227
## Starting and stopping distribution
2328

@@ -94,6 +99,149 @@ fun (DistCtrlr, Length :: pos_integer(), Timeout :: timeout()) -> {ok, Packet} |
9499
95100
AtomVM's distribution is based on `socket_dist` and `socket_dist_controller` modules which can also be used with BEAM by definining `BEAM_INTERFACE` to adjust for the difference.
96101

102+
## Serial distribution
103+
104+
AtomVM supports distribution over serial (UART) connections using the
105+
`serial_dist` module. This is useful for microcontrollers that lack
106+
WiFi/TCP (e.g. STM32) but have UART, and for testing distribution
107+
locally using virtual serial ports.
108+
109+
### Quick start
110+
111+
```erlang
112+
{ok, _} = net_kernel:start('mynode@serial.local', #{
113+
name_domain => longnames,
114+
proto_dist => serial_dist,
115+
avm_dist_opts => #{
116+
uart_opts => [{peripheral, "UART1"}, {speed, 115200},
117+
{tx, 17}, {rx, 16}],
118+
uart_module => uart
119+
}
120+
}).
121+
```
122+
123+
On Unix, the `peripheral` is a device path such as `"/dev/ttyUSB0"` and
124+
the `uart_module` is `uart` from the `avm_unix` library.
125+
126+
### serial\_dist options
127+
128+
- `uart_opts` — proplist passed to `UartModule:open/1` (see `uart_hal`
129+
for common parameters: `peripheral`, `speed`, `data_bits`, `stop_bits`,
130+
`parity`, `flow_control`)
131+
- `uart_module` — module implementing the `uart_hal` behaviour. Defaults
132+
to `uart`.
133+
134+
### Wire protocol
135+
136+
Serial distribution layers three protocols on the raw UART byte stream:
137+
138+
**1. Sync frames (link layer)**
139+
140+
Both sides periodically send 2-byte sync frames (`<<16#AA, 16#55>>`) on
141+
the UART. These serve two purposes:
142+
143+
- **Liveness detection**: a node knows its peer is alive when it receives
144+
sync frames.
145+
- **Garbage collection**: after a failed handshake attempt, stale bytes
146+
from the old handshake remain in the UART buffer. Sync frames are
147+
stripped from all received data, so stale data that happens to contain
148+
`16#AA 16#55` pairs is harmlessly consumed, and non-sync stale bytes
149+
eventually get discarded when the next handshake attempt interprets
150+
them as an invalid packet.
151+
152+
The sync magic (`16#AA 16#55`) is chosen so it cannot appear as the
153+
first two bytes of a valid length-prefixed handshake message (handshake
154+
messages have a 16-bit big-endian length prefix, and the longest
155+
handshake message is well under 100 bytes, so the first byte is always
156+
`0x00`).
157+
158+
**2. Handshake packets (2-byte length prefix)**
159+
160+
During the Erlang distribution handshake, messages are framed as:
161+
162+
```
163+
<<Length:16/big, Payload:Length/binary>>
164+
```
165+
166+
This matches the TCP distribution handshake format. The handshake
167+
follows the standard Erlang distribution protocol (send\_name,
168+
send\_status, send\_challenge, send\_challenge\_reply,
169+
send\_challenge\_ack).
170+
171+
**3. Data packets (4-byte length prefix)**
172+
173+
After the handshake completes, distribution data packets use a 4-byte
174+
length prefix:
175+
176+
```
177+
<<Length:32/big, Payload:Length/binary>>
178+
```
179+
180+
Tick (keepalive) messages are sent as `<<0:32>>` (4 zero bytes).
181+
182+
### Peer-to-peer connection model
183+
184+
Unlike TCP distribution which uses a client/server model (one side
185+
listens, the other connects), serial is point-to-point: both nodes
186+
share a single UART link.
187+
188+
A **link manager** process on each node is the sole owner of UART
189+
reads. On each iteration it:
190+
191+
1. Checks its mailbox for a `setup` request from `net_kernel`
192+
(non-blocking).
193+
2. Sends a sync frame.
194+
3. Reads from the UART with a short timeout.
195+
4. Strips sync frames from received data.
196+
5. If handshake data remains, enters the **accept** path (responder).
197+
6. If a `setup` request was pending, enters the **setup** path
198+
(initiator).
199+
7. Otherwise, loops.
200+
201+
This design ensures only one process reads from the UART at any time,
202+
avoiding the race condition that would occur if separate accept and
203+
setup processes competed for the same byte stream.
204+
205+
If a handshake fails (the distribution controller process exits), the
206+
link manager flushes stale `setup` messages from its mailbox and
207+
restarts the loop, allowing retries.
208+
209+
### Testing with socat
210+
211+
On Unix, `socat` can create virtual serial port pairs for testing:
212+
213+
```bash
214+
socat -d -d pty,raw,echo=0 pty,raw,echo=0
215+
```
216+
217+
This creates two pseudo-terminal devices (e.g. `/dev/ttys003` and
218+
`/dev/ttys004`) connected back-to-back. Each AtomVM node uses one side:
219+
220+
```erlang
221+
%% Node A
222+
{ok, _} = net_kernel:start('a@serial.local', #{
223+
name_domain => longnames,
224+
proto_dist => serial_dist,
225+
avm_dist_opts => #{
226+
uart_opts => [{peripheral, "/dev/ttys003"}, {speed, 115200}],
227+
uart_module => uart
228+
}
229+
}).
230+
231+
%% Node B (separate AtomVM process)
232+
{ok, _} = net_kernel:start('b@serial.local', #{
233+
name_domain => longnames,
234+
proto_dist => serial_dist,
235+
avm_dist_opts => #{
236+
uart_opts => [{peripheral, "/dev/ttys004"}, {speed, 115200}],
237+
uart_module => uart
238+
}
239+
}).
240+
241+
%% From Node B, trigger autoconnect:
242+
{some_registered_name, 'a@serial.local'} ! {self(), hello}.
243+
```
244+
97245
## Distribution features
98246

99247
Distribution implementation is (very) partial. The most basic features are available:

examples/erlang/CMakeLists.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,3 +45,4 @@ pack_runnable(http_client http_client estdlib eavmlib avm_network)
4545
pack_runnable(disterl disterl estdlib)
4646
pack_runnable(i2c_scanner i2c_scanner eavmlib estdlib DIALYZE_AGAINST avm_esp32 avm_rp2 avm_stm32)
4747
pack_runnable(i2c_lis3dh i2c_lis3dh eavmlib estdlib DIALYZE_AGAINST avm_esp32 avm_rp2 avm_stm32)
48+
pack_runnable(serial_disterl serial_disterl eavmlib estdlib DIALYZE_AGAINST avm_esp32 avm_unix)

examples/erlang/serial_disterl.erl

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
%
2+
% This file is part of AtomVM.
3+
%
4+
% Copyright 2026 Paul Guyot <pguyot@kallisys.net>
5+
%
6+
% Licensed under the Apache License, Version 2.0 (the "License");
7+
% you may not use this file except in compliance with the License.
8+
% You may obtain a copy of the License at
9+
%
10+
% http://www.apache.org/licenses/LICENSE-2.0
11+
%
12+
% Unless required by applicable law or agreed to in writing, software
13+
% distributed under the License is distributed on an "AS IS" BASIS,
14+
% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
% See the License for the specific language governing permissions and
16+
% limitations under the License.
17+
%
18+
% SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later
19+
%
20+
21+
%% @doc Example: distributed Erlang over serial (UART).
22+
%%
23+
%% This example starts distribution using a UART connection instead of
24+
%% TCP/IP. It works on ESP32 and Unix (using a real serial device or a
25+
%% virtual serial port created with socat).
26+
%%
27+
%% <h3>ESP32 wiring</h3>
28+
%%
29+
%% ```
30+
%% ESP32 TX (GPIO 17) -> Peer RX
31+
%% ESP32 RX (GPIO 16) -> Peer TX
32+
%% ESP32 GND -> Peer GND
33+
%% '''
34+
%%
35+
%% <h3>Unix with socat</h3>
36+
%%
37+
%% Create a virtual serial port pair:
38+
%% ```
39+
%% socat -d -d pty,raw,echo=0 pty,raw,echo=0
40+
%% '''
41+
%% Then set the SERIAL_DEVICE environment variable to one of the pty
42+
%% paths before running this example. The peer node uses the other pty.
43+
%%
44+
%% <h3>Connecting</h3>
45+
%%
46+
%% The node name is derived from the serial device and hostname, e.g.
47+
%% `ttys003@myhost' or `uart1@esp32-abc'. Once both nodes are running,
48+
%% trigger autoconnect from either side:
49+
%% ```
50+
%% {serial_disterl, 'ttys003@myhost'} ! {hello, node()}.
51+
%% '''
52+
-module(serial_disterl).
53+
54+
-export([start/0]).
55+
56+
start() ->
57+
UartOpts = uart_opts(),
58+
NodeName = make_node_name(UartOpts),
59+
{ok, _NetKernelPid} = net_kernel:start(NodeName, #{
60+
name_domain => longnames,
61+
proto_dist => serial_dist,
62+
avm_dist_opts => #{
63+
uart_opts => UartOpts
64+
}
65+
}),
66+
io:format("Distribution started over serial~n"),
67+
io:format("Node: ~p~n", [node()]),
68+
net_kernel:set_cookie(<<"AtomVM">>),
69+
io:format("Cookie: ~s~n", [net_kernel:get_cookie()]),
70+
register(serial_disterl, self()),
71+
io:format("Registered as 'serial_disterl'. Waiting for messages.~n"),
72+
io:format("From the peer:~n"),
73+
io:format(" {serial_disterl, '~s'} ! {hello, node()}.~n", [node()]),
74+
loop().
75+
76+
%% Build a node name from the serial device.
77+
%% e.g. "UART1" -> 'uart1@serial'
78+
%% "/dev/ttys003" -> 'ttys003@serial'
79+
make_node_name(UartOpts) ->
80+
Peripheral = proplists:get_value(peripheral, UartOpts, "serial"),
81+
BaseName = basename(Peripheral),
82+
list_to_atom(string:to_lower(BaseName) ++ "@serial.local").
83+
84+
basename(Path) ->
85+
case lists:last(string:split(Path, "/", all)) of
86+
[] -> Path;
87+
Name -> Name
88+
end.
89+
90+
%% Platform-specific UART configuration.
91+
uart_opts() ->
92+
case erlang:system_info(machine) of
93+
"ATOM" ->
94+
case atomvm:platform() of
95+
esp32 ->
96+
[{peripheral, "UART1"}, {speed, 115200}, {tx, 17}, {rx, 16}];
97+
generic_unix ->
98+
Device = os:getenv("SERIAL_DEVICE"),
99+
case Device of
100+
false ->
101+
io:format("Error: set SERIAL_DEVICE env var to a serial port path~n"),
102+
io:format(" e.g. /dev/ttyUSB0 or a socat pty~n"),
103+
exit(no_serial_device);
104+
_ ->
105+
[{peripheral, Device}, {speed, 115200}]
106+
end;
107+
Other ->
108+
io:format("Error: unsupported platform ~p~n", [Other]),
109+
exit({unsupported_platform, Other})
110+
end;
111+
"BEAM" ->
112+
io:format("Error: this example requires AtomVM~n"),
113+
exit(beam_not_supported)
114+
end.
115+
116+
loop() ->
117+
receive
118+
quit ->
119+
io:format("Received quit, stopping.~n"),
120+
ok;
121+
{hello, From} ->
122+
io:format("Hello from ~p!~n", [From]),
123+
loop();
124+
Other ->
125+
io:format("Received: ~p~n", [Other]),
126+
loop()
127+
end.

libs/estdlib/src/CMakeLists.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,7 @@ set(ERLANG_MODULES
5959
maps
6060
math
6161
net
62+
os
6263
proc_lib
6364
sys
6465
logger
@@ -67,6 +68,8 @@ set(ERLANG_MODULES
6768
queue
6869
sets
6970
socket
71+
serial_dist
72+
serial_dist_controller
7073
socket_dist
7174
socket_dist_controller
7275
ssl

libs/estdlib/src/net_kernel.erl

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -285,7 +285,10 @@ handle_cast(_Message, State) ->
285285
{noreply, State}.
286286

287287
%% @hidden
288-
handle_info({accept, AcceptPid, SocketPid, inet, tcp}, #state{proto_dist = ProtoDist} = State) ->
288+
handle_info(
289+
{accept, AcceptPid, SocketPid, _Family, _Protocol},
290+
#state{proto_dist = ProtoDist} = State
291+
) ->
289292
Pid = ProtoDist:accept_connection(AcceptPid, SocketPid, State#state.node, [], ?SETUPTIME),
290293
AcceptPid ! {self(), controller, Pid},
291294
{noreply, State};

0 commit comments

Comments
 (0)