Skip to content

[1.4] core: ardupilot_manager: Fix Pixhawk restart reconnect - #4114

Open
joaoantoniocardoso wants to merge 2 commits into
bluerobotics:1.4-devfrom
joaoantoniocardoso:fix_pixhawk_restart_reconnect
Open

[1.4] core: ardupilot_manager: Fix Pixhawk restart reconnect#4114
joaoantoniocardoso wants to merge 2 commits into
bluerobotics:1.4-devfrom
joaoantoniocardoso:fix_pixhawk_restart_reconnect

Conversation

@joaoantoniocardoso

@joaoantoniocardoso joaoantoniocardoso commented Aug 11, 2026

Copy link
Copy Markdown
Member

Summary

  • Fix serial autopilot restart so Pixhawk boards reconnect after USB re-enumeration instead of leaving the MAVLink router on a stale serial path
  • Wait for the board to drop and return, then reopen the link on the freshly detected path
  • Increase the frontend /restart timeout to cover the backend reconnect window

Fixes #4109 for 1.4. Will be ported to master on another PR.

Test plan

  • Connect a Pixhawk to BlueOS (Pi3, 1.4.4-beta.15)
  • Trigger Restart Autopilot from the Power menu and via POST /ardupilot-manager/v1.0/restart
  • Confirm the FC reboots, USB re-enumerates, and heartbeat returns within ~5s
  • Repeat restart 3 times via API without failures
  • Confirm only one mavlink-routerd process remains after restart

@joaoantoniocardoso
joaoantoniocardoso marked this pull request as draft August 11, 2026 15:00
@github-actions

github-actions Bot commented Aug 11, 2026

Copy link
Copy Markdown

Automated PR Review

0. Summary

  • Verdict: MINOR SUGGESTIONS ✏️

Fixes serial autopilot restart on boards that USB-re-enumerate (Pixhawk) by (a) serializing restart_ardupilot calls behind an asyncio.Lock, (b) waiting for the board to drop and come back after reboot_vehicle(), (c) reopening the router on the freshly detected path via mavlink_manager.stop() + start_serial() instead of a full kill/start cycle so runtime-only endpoints survive, and (d) bumping the frontend /restart timeout from 10 s to 60 s to cover the backend reconnect window. The core change is well-scoped and the comments do a good job explaining the "why" behind each step.

1. Correctness & Implementation Bugs

  • 1.1 [minor] core/services/ardupilot_manager/autopilot_manager.py:236 (existing) vs. new restart path — during the serial reboot wait, auto_restart_ardupilot keeps polling on its 5 s cycle. Because is_running() still returns self.should_be_running == True for PlatformType.Serial (line 199), the heartbeat branch on line 217 stays active and _heartbeat_fail_count will accumulate real failures across the ~40 s disconnect+reconnect window (up to 30 s reconnect alone is 6 checks). If the count was already non-zero when the user hit Restart, it can hit _max_heartbeat_failures = 10 mid-restart and trigger a second restart_ardupilot() that then queues on _restart_lock. The lock prevents corruption but you get a redundant reboot immediately after the first one lands. Consider resetting self._heartbeat_fail_count = 0 on entry (or on successful reconnect) inside the serial branch.
  • 1.2 [minor] core/services/ardupilot_manager/autopilot_manager.py:611_detect_serial_board matches on detected.platform == board.platform only. On a bench setup with two boards of the same platform, the wrong device could be picked as the "reconnected" board. The pre-reboot board.path is known here; filtering detected.path == board.path first (falling back to platform match if the path changed, which is the whole reason for the reconnect wait) would be more precise. Edge-case, not blocking.
  • 1.3 [nit] Timing budget: disconnect_deadline (10 s) + reconnect_deadline (30 s) + mavlink_manager.stop() + start_serial() can plausibly run 40–50 s. The frontend timeout is 60 s, so the margin is tight; if start_mavlink_manager ever stalls a few seconds the UI will show a failed request even though the backend completed. Either shave the deadlines a little or bump the frontend timeout to ~75 s for headroom.

4. Performance

  • 4.1 [nit] core/services/ardupilot_manager/autopilot_manager.py:604 — polling available_boards() every 500 ms during the disconnect wait runs full BoardDetector.detect(True) twice a second. That likely enumerates USB and probes serial ports each time. Not a real problem at this cadence, but a 1 s interval would halve the load without changing behavior meaningfully.

6. Code Quality & Style

  • 6.1 [nit] core/services/ardupilot_manager/autopilot_manager.py:606-614 — the while … else: construct on the disconnect wait is valid but easy to misread; a small helper (await self._wait_for_board_disconnect(board, timeout=10.0) returning bool) or an explicit disconnected = False flag would be clearer at the call site.
  • 6.2 [nit] core/services/ardupilot_manager/autopilot_manager.py:616-620 — the reconnect loop sleeps before the first _detect_serial_board check. That's intentional (give the board time to enumerate after dropping), but if the disconnect wait already timed out because the board never dropped, the extra 0.5 s wait here is wasted. Optional: check once eagerly if the disconnect loop timed out.
  • 6.3 [nit] core/services/ardupilot_manager/autopilot_manager.py:626-632 — the outer except Exception swallows cancellation as well. If this ever gets awaited in a task that's cancelled, you'd log a spurious "Failed to stop Mavlink manager…" warning instead of propagating the cancel. Consider except (Exception,) as error: if isinstance(error, asyncio.CancelledError): raise; …, or narrower catches. Low priority.

7. Tests

  • 7.1 [nit] No unit tests added. The manual test plan in the PR body is thorough, and there isn't existing pytest coverage of AutoPilotManager.restart_ardupilot to regress against, so this is acceptable — flagging only so the pre-push hook expectation is on record.

8. Documentation

Comments here are a good example of the AGENTS.md "explain why, not what" rule — the block at autopilot_manager.py:597-599 and the frontend-timeout block on AutopilotManagerUpdater.ts:166-168 both document non-obvious constraints. No changes suggested.

Generated by PR Review Bot. This is advisory, a human reviewer must still approve.

@joaoantoniocardoso
joaoantoniocardoso marked this pull request as ready for review August 11, 2026 15:20
@joaoantoniocardoso
joaoantoniocardoso force-pushed the fix_pixhawk_restart_reconnect branch from 9901d93 to 7afdd86 Compare August 11, 2026 15:20
@joaoantoniocardoso
joaoantoniocardoso requested a review from a team August 11, 2026 16:34
@joaoantoniocardoso joaoantoniocardoso changed the title core: ardupilot_manager: Fix Pixhawk restart reconnect [1.4] core: ardupilot_manager: Fix Pixhawk restart reconnect Aug 11, 2026
…nect

After a Pixhawk reboot the USB device re-enumerates with a new path while
the MAVLink router keeps a stale serial handle. Wait for the board to drop
and return, then reopen the link on the freshly detected path. On failure,
stop the router and clear should_be_running so the stale handle is not left
running with its watchdog disarmed.
Serial board restarts wait for USB re-enumeration before reopening the
MAVLink link, which can take tens of seconds.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant