Skip to content

Declare mosh on every machine for connection-resilient terminal sessions - #2774

Merged
cameronraysmith merged 1 commit into
mainfrom
fm/vx-mosh-fleet
Aug 19, 2026
Merged

Declare mosh on every machine for connection-resilient terminal sessions#2774
cameronraysmith merged 1 commit into
mainfrom
fm/vx-mosh-fleet

Conversation

@cameronraysmith

Copy link
Copy Markdown
Owner

A terminal session over a phone's network dies whenever the TCP connection does: switching between wifi and cellular, a tunnel dropping, the handset sleeping. mosh survives all three, because the client and server exchange state over UDP and neither end cares that the peer's address changed.

mosh-server is spawned by an ordinary SSH login, so this needs no separate authentication path. It reuses the keys and certificate trust already declared for sshd. The only new surface is the UDP return path, which is why it is a separate change from the sshd work rather than folded into it: it touches the firewall on every machine.

modules/system/mosh.nix contributes to both the NixOS and darwin base namespaces, following caches.nix and zram-swap.nix, so every machine that already imports base picks it up with no per-host edit.

Which UDP range, and why

60001-60999, scoped to zt+.

The range is what mosh-server actually binds between, not what the documentation rounds it to. src/network/network.h defines PORT_RANGE_LOW = 60001 and PORT_RANGE_HIGH = 60999; network.cc scans upward from the low end for the first free port and takes one port per live session. The man page and nixpkgs both say "60000 to 61000", which is two ports wider than anything ever binds.

programs.mosh.openFirewall is therefore left off. Upstream's implementation of that option adds allowedUDPPortRanges = [{from = 60000; to = 61000;}], which is global — every interface, including whatever a cloud host has facing the public internet. Instead the range goes in networking.firewall.interfaces."zt+", matching the interface-scoping convention already used for the https rule in cinnabar/caddy.nix and the admin ports on magnetite. In the built closure this renders as exactly one new rule:

ip46tables -A nixos-fw -p udp --dport 60001:60999 -j nixos-fw-accept -i zt+

which covers v4 and v6 and is inert on a machine with no mesh interface. No host gained a global opening; magnetite's pre-existing allowedUDPPortRanges are unchanged.

programs.mosh.enable also installs the utempter setgid wrapper, which is upstream's default and is what lets who see a mosh session.

macOS

Checked rather than assumed. On stibnite pf reports Status: Enabled, but the loaded ruleset is only Apple's own anchors — com.apple/200.AirDrop, com.apple/250.ApplicationFirewall, com.apple.internet-sharing and its network_isolation child — and none of them carry a block rule; the top-level ruleset is scrub and anchor declarations only. The application firewall is separately off (socketfilterfw --getglobalstate returns state 0, stealth mode off). So inbound UDP already passes and there is nothing to declare; shipping the binary is the whole darwin change.

Also verified that the binary will be found: /etc/zshenv (written by nix-darwin) puts /run/current-system/sw/bin on the PATH of a non-interactive zsh, which is the shell sshd runs the mosh-server new command under. Confirmed by env -i zsh -c 'echo $PATH'.

The other three darwin machines were not reachable to check directly. They run the same nix-darwin configuration and stock macOS firewall defaults, so the same reasoning applies, but that is inference and not measurement. If the application firewall is ever turned on somewhere, it will need mosh-server allowed as an incoming-connection exception; that is a per-machine setting with no declarative surface here.

Validation

This changes the evaluated configuration of every machine, so the selection is all of them: nix build of the six nixosConfigurations toplevels and the four darwin-<host> checks, plus clan-inventory-consistency, naming-conventions, eval-md-format, and nix fmt -- --ci. All pass. Not run: the k8s, nixidy, and package checks, which nothing in this diff reaches.

The two claims worth checking were checked in the built output rather than inferred from the option values: the firewall rule quoted above comes from cinnabar's rendered firewall-start, and bin/mosh-server is present in both cinnabar's and stibnite's system-path.

What activation requires

Nothing beyond a normal deploy of each machine. On NixOS the firewall unit reloads itself as part of switching; there is no new daemon, and nothing listens on any of those ports until a client starts a session. On darwin it is a package appearing in the system profile.

To use it, from a client with mosh installed:

mosh <host>.zt

or, to land straight in a session:

mosh <host>.zt -- tmux new -A -s main

The client needs mosh installed on its side too; on iOS, Blink Shell speaks the protocol natively. A first connection is the real test of the path — the declarations here only establish that the server side is present and reachable.

@mergify

mergify Bot commented Aug 19, 2026

Copy link
Copy Markdown

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

Terminal sessions over a mobile link die with the TCP connection. mosh
keeps the session across address changes and sleep, and bootstraps over
an ordinary SSH login, so it reuses the keys and certificate trust
already declared rather than adding an authentication path.

modules/system/mosh.nix contributes to both base namespaces, the way
caches.nix and zram-swap.nix already do, so every machine that imports
base picks it up without a per-host edit.

On NixOS, programs.mosh.enable installs the package and the utempter
setgid wrapper (upstream default, lets `who` see mosh sessions).
openFirewall is off because upstream opens 60000-61000 on every
interface; the rule here is instead scoped to zt+, matching the
existing interfaces."zt+" convention in cinnabar/caddy.nix and
magnetite. The range is 60001-60999, the PORT_RANGE_LOW/PORT_RANGE_HIGH
constants mosh-server actually binds between (src/network/network.h);
network.cc scans upward from the low end for the first free port, one
per live session.

On darwin the package is all that is needed: pf is enabled on stibnite
but loads only Apple's own anchors, none of which carry a block rule,
and the application firewall is off, so inbound UDP already passes.
nix-darwin's /etc/zshenv puts /run/current-system/sw/bin on the PATH of
a non-interactive zsh, which is how the SSH-spawned mosh-server is
found.

Validation: the change touches every machine's evaluated config, so the
selection is all ten of them - nix build of the six nixosConfigurations
toplevels and the four darwin-<host> checks, plus
clan-inventory-consistency, naming-conventions, eval-md-format, and nix
fmt --ci; all pass. Verified in the built closures rather than inferred:
cinnabar's firewall-start renders exactly one new rule, `ip46tables -A
nixos-fw -p udp --dport 60001:60999 -j nixos-fw-accept -i zt+`, and both
cinnabar's and stibnite's system-path carry bin/mosh-server. Also
checked that no host gained a global 60000-61000 opening; magnetite's
pre-existing allowedUDPPortRanges are unchanged. Not run: the k8s,
nixidy, and package checks, which no part of this diff reaches.

Activation is a normal deploy per host; no service restart beyond the
firewall unit NixOS reloads itself, and nothing listens until a client
starts a session.
@cameronraysmith
cameronraysmith merged commit a1639fd into main Aug 19, 2026
6 checks passed
@cameronraysmith
cameronraysmith deleted the fm/vx-mosh-fleet branch August 19, 2026 17:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant