Skip to content

test_respawn_pane_replaces_shell races the respawned exec under CI parallelism #107

Description

@tony

test_respawn_pane_replaces_shell reads pane_current_command immediately after respawn_pane returns, with nothing waiting for the new process to exec. It normally wins that race and passes; under CI's parallelism it sometimes loses, consuming reruns and masking itself behind --reruns=2.

Where

tests/test_pane_tools.py#L1938-L1954respawn_pane(...) is called and assert "sleep" in result.pane_current_command runs against whatever tmux reports at that instant.

Pre-existing, not introduced by #100 — the test body is unchanged since d5362a9 (2026-04-20) apart from a parameter rename in bec9ab1.

Field evidence

Observed in tests run 30168336319, where it reran on two independent tmux jobs in the same run:

build (3.14, 3.5)   [gw1] [ 43%] RERUN tests/test_pane_tools.py::test_respawn_pane_replaces_shell
build (3.14, 3.3a)  [gw0] [ 42%] RERUN tests/test_pane_tools.py::test_respawn_pane_replaces_shell

--reruns=2 is in addopts, so this does not turn CI red — it just quietly costs attempts. A green run is therefore weaker evidence than it looks.

Recreation

respawn-pane returns as soon as tmux has forked; pane_current_command reports the new command only once it has exec'd. Reading with no delay in between shows how wide the window is:

./repro_respawn.sh 200
repro_respawn.sh
#!/usr/bin/env bash
# Reproduce the race in test_respawn_pane_replaces_shell: the test reads
# pane_current_command immediately after respawn_pane returns, with no
# wait for the new process to exec. Mirrors the test exactly -- a session
# on its default shell, a SPLIT pane running the old command, then a
# respawn of that split.
set -uo pipefail
SOCK="repro-respawn-$$"
N=${1:-200}
new=0; stale=0; other=0
tmux -L "$SOCK" kill-server 2>/dev/null
tmux -L "$SOCK" new-session -d -s s
for _ in $(seq 1 "$N"); do
  pane=$(tmux -L "$SOCK" split-window -d -P -F '#{pane_id}' -t s "sleep 3600")
  tmux -L "$SOCK" respawn-pane -k -t "$pane" "sleep 7200"
  cmd=$(tmux -L "$SOCK" display-message -p -t "$pane" '#{pane_current_command}')
  case "$cmd" in
    sleep)  new=$((new+1))   ;;   # test passes
    "")     other=$((other+1));;
    *)      stale=$((stale+1));;  # test FAILS: 'sleep' not in cmd
  esac
  tmux -L "$SOCK" kill-pane -t "$pane" 2>/dev/null
done
tmux -L "$SOCK" kill-server 2>/dev/null; rm -f "/tmp/tmux-1000/$SOCK"
echo "reports 'sleep'  (test passes): $new/$N"
echo "reports stale    (test FAILS):  $stale/$N"
echo "reports empty:                  $other/$N"

Measured on tmux 3.7b:

reports 'sleep'  (test passes): 3/200
reports stale    (test FAILS):  197/200
reports empty:                  0/200

That is the mechanism in isolation, not the test's real rate. The Python path does more work between the respawn and the read — pane resolution and result construction — and normally that is enough: the real test passed 40/40 at idle on a6e4117d39ffe633d50eb1ad3a6fcd55a71e5e60. The failures only appear under CI's -n auto parallelism, which is exactly what the reruns above show.

for i in $(seq 1 40); do env -u VIRTUAL_ENV uv run pytest tests/test_pane_tools.py -q --reruns 0 -p no:randomly -k test_respawn_pane_replaces_shell; done

Background

pane_current_command resolves through tmux's process-name lookup for the pane's foreground process group, so it reflects whatever is running at the moment of the query — see osdep-linux.c and the pane_current_command entry in format.c (locally: ~/study/c/tmux/osdep-linux.c, ~/study/c/tmux/format.c, checkout at tag 3.7b). There is no tmux event that says "the respawned command has started", so a poll is the only correct way to observe it.

Suggested fix

Poll instead of asserting on the first read. The project already uses retry_until from libtmux.test.retry for exactly this shape elsewhere in the same file:

retry_until(lambda: "sleep" in (new_pane.pane_current_command or ""), 5, raises=True)

Keep an assertion on result.pane_id unconditionally — that part is not racy. Only the command read needs settling.

Worth checking whether any other test asserts on pane_current_command straight after a spawn or respawn; the same shape would have the same race.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions