feat(ssh): generate a root password when none is given - #19
Merged
Conversation
`--ssh.enable` on its own used to refuse the pivot: sshd needs a password or a key, and given neither it logged "no auth method configured" and never bound the port. Refusing was better than the silent dead port it replaced, but it still pushed the operator into inventing a password on a command line, under time pressure, on the box they were about to take apart. Three random words beat that every time. xmorph now generates one — three words from the EFF short wordlist, hyphenated, ~31 bits. Short enough to read off a serial console and retype, which is the only channel that exists once the pivot is done. It is printed twice on purpose. The pre-pivot banner goes to the terminal that ran the command, which is usually the SSH session the pivot is about to kill; the post-pivot one goes to the console, where a serial line will still have it. A password the operator supplied is never printed and never logged — it may be reused somewhere that matters, so it stays theirs; SSHPasswordGenerated is what draws that line. The VM test now takes the password the way an operator does: it reads the banner off the console and logs in with it, then checks a wrong password is refused, so an sshd that accepted anything could not pass. Also close the gap that let a broken _test.go build clean on macOS and fail only in CI: `nix run .#build` now compiles the Linux test binaries.
The VM test hung for 900s waiting for a banner that had already been
printed. wait_for_console_text reads *forward* from a queue, and the
password banner comes out before "serving; no entrypoint to supervise" —
so the earlier wait had already drained past it and the second one could
never match. Poll get_console_log() instead, which is the whole log since
boot; the test then does not care what order xmorph logs things in.
Then it hung a second time, and that is the more expensive bug: the
assertions ran inside a script whose only target.crash() was on the happy
path. When the script raises, the driver still runs execute("sync") on
every machine that is_up(), which waits on a backdoor that died with the
old root — so a legible one-line failure became a 29-minute job that got
scored as a hang. Both pivot tests now crash the guest on the failure
path too.
The feature itself was fine throughout: both banners printed
throb-thorn-stash and sshd bound [::]:22.
wait_for_console_text drains its queue with one non-blocking get() per retry iteration, and retry sleeps a second between iterations. That is one console line per second, against a NixOS boot that emits hundreds — so whether it matches inside the timeout is a race with the backlog, not a question of whether the text was printed. The same call matched in 33s in one CI run and blew a 180s timeout in the next, with the text present and correct on the console both times. full_console_log holds everything since boot and reading it consumes nothing, so poll that instead. wait_console() does it in one place and every console assertion in both pivot tests goes through it.
The VM test logged in with the generated password, ran `echo logged-in`,
and never got a prompt back. Auth was fine — the server logged
`sshd: accepted user=root`. The session was the problem.
runSession set cmd.Stdin = ch. os/exec copies a non-*os.File stdin on its
own goroutine and makes Wait() block until that copy returns, and it
returns only when the client closes the channel. A client closes it when
its own stdin hits EOF, which for `ssh host cmd` run from a terminal or a
live pipe is never. So the command exited, the output arrived, and the
session hung — on a rescue box, at the worst possible moment.
StdinPipe instead: Wait closes it on exit, which unblocks the copy.
Every existing test in sshd_linux_test.go missed this, and would have kept
missing it. x/crypto/ssh's Session sends channel EOF immediately when
Stdin is nil, so sess.Run("true") passes against a server that deadlocks.
The new test gives it an io.Pipe that never closes, which is what a real
terminal looks like. Reverting the fix hangs it for the full 10s budget.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
--ssh.enableon its own used to refuse the pivot. sshd needs a password or akey; given neither it logged
no auth method configuredand never bound theport. Refusing (#18) was better than the silent dead port it replaced, but it
still pushed the operator into inventing a password on a command line, under
time pressure, on the box they were about to take apart.
xmorph now generates one.
What it is
Three words from the EFF short wordlist, hyphenated. That list is picked for
exactly this job: every word is 3–5 letters, common, and distinguishable from
the others by its first three characters — it has to survive a phone camera
pointed at a serial console. One entry,
yo-yo, is dropped because a hypheninside a word makes a hyphenated phrase ambiguous to read back, leaving 1295.
~31 bits. Not a key, and not meant to be one: it protects a root shell for the
minutes-to-hours a rescue lasts, against an attacker who has to guess online.
passphrase.DefaultWordsis a one-line change and each extra word adds ~10.3bits.
docs/rescue.mdstates the number and says to use--ssh.authorized-keysfor anything facing the open internet for long.Printed twice, on purpose
The pre-pivot banner goes to the terminal that ran the command — which is
usually the SSH session the pivot is about to kill. The post-pivot one goes to
the console, where a serial line will still have it.
A password the operator supplied is never printed and never logged. It may
be one they reuse somewhere that matters, so it stays theirs. Only the
generated one — worthless anywhere else, useless if unreadable — goes to the
console.
SSHPasswordGeneratedis what draws that line, and two tests hold it.Three bugs the new test found
nix/tests/lifecycle.nixnow takes the password the way an operator does:pivots with bare
--ssh.enable, scrapes the banner off the console, and logsin over real
ssh(1)from the second VM with password auth forced. Then itchecks a wrong password is refused, so an sshd that accepted anything could not
pass. Doing that turned up three things nothing else would have:
ssh host cmdhung. Auth succeeded,echo logged-inran, and the sessionnever returned.
runSessionsetcmd.Stdin = ch, and os/exec makesWait()block until that copy finishes — which happens only when the client closes the
channel, which for a command run from a terminal is never.
cmd.StdinPipe()instead;
Waitcloses it on exit. Every existing sshd test missed this andwould have kept missing it: x/crypto/ssh's
Sessionsends channel EOFimmediately when
Stdinis nil, sosess.Run("true")passes against a serverthat deadlocks. The new test gives it an
io.Pipethat never closes; revertingthe fix hangs it for the full 10s budget.
wait_for_console_textis a race. It drains its queue with onenon-blocking
get()perretryiteration, andretrysleeps a second — oneconsole line per second, against a NixOS boot that emits hundreds. The same
call matched in 33s in one CI run and blew a 180s timeout in the next, with the
text present and correct both times. All console assertions now poll
get_console_log(), which holds everything since boot and is not consumed byreading.
A failed assertion became a hang. The driver's end-of-script
execute("sync")waits on a backdoor that died with the old root, so anyexception turned a one-line error into a 29-minute job scored as a timeout.
Both pivot tests now crash the guest on the failure path too.
Also
internal/passphrasehas wordlist invariants (data files rot silently) and avariance check; both mutation-tested — a deterministic generator fails
TestGenerateVaries.nix run .#buildon macOS compiled the main binary but no test binaries, so abroken
_test.gobuilt clean locally and failed only in CI. It now compilesthe Linux test binaries. Verified it bites.