fix(setup-dpdk): ip link down each NIC + mount hugetlbfs at /mnt/huge1g#85
Conversation
Two operator-side speedbumps surfaced on c6in.metal during anygpt-52 (PR #65 issuecomment-4339242358): 1. dpdk-devbind refuses to bind active interfaces: Warning: routing table indicates that interface is active. Not modifying. Operators had to `ip link set <ifc> down` on every NIC by hand before the bind step succeeded. 2. Reserving 1 GiB hugepages was not enough for rte_eal_init: EAL: No available 1048576 kB hugepages reported on node 0/1 Operators had to `mount -t hugetlbfs -o pagesize=1G nodev /mnt/ huge1g` themselves before the scanner could find the pages. Fix: - bdf_to_iface() walks /sys/bus/pci/devices/<bdf>/net/ to map BDF → kernel iface name. cmd_bind invokes `ip link set <ifc> down` on each target before invoking dpdk-devbind, with best-effort failure semantics (missing ip command, missing iface, already-down iface all proceed to the bind). - ensure_hugetlbfs_mount() mounts a hugetlbfs of the matching pagesize at the configured path after a successful nr_hugepages reservation. Default targets are /mnt/huge1g (1 GiB) and /mnt/huge2m (2 MiB); ANYSCAN_DPDK_HUGEPAGES_1G_MOUNT / _2M_MOUNT override or set them to "" to opt out (operators provisioning hugetlbfs via fstab). Idempotent: detects existing hugetlbfs of the right pagesize via findmnt / /proc/mounts and skips remount. - ANYSCAN_DPDK_LOAD_ONLY=1 hook lets test-setup-dpdk.sh source the script for hermetic helper testing without triggering the argv dispatch. Tests (new tools/test-setup-dpdk.sh): - cmd_bind invokes `ip link set <ifc> down` BEFORE dpdk-devbind --bind=vfio-pci (order verified by line numbers in a single cmd log). - ensure_hugetlbfs_mount calls `mount -t hugetlbfs -o pagesize=1G nodev <path>`. - ensure_hugetlbfs_mount is a no-op when target is already hugetlbfs with the matching pagesize. - ensure_hugetlbfs_mount is a no-op with an empty mount path. - bdf_to_iface returns iface for a populated fake /sys tree. - bdf_to_iface returns empty when net/ dir is missing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 88f3bdcc4d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ANYSCAN_DPDK_HUGEPAGES_1G_MOUNT="${ANYSCAN_DPDK_HUGEPAGES_1G_MOUNT:-/mnt/huge1g}" | ||
| ANYSCAN_DPDK_HUGEPAGES_2M_MOUNT="${ANYSCAN_DPDK_HUGEPAGES_2M_MOUNT:-/mnt/huge2m}" |
There was a problem hiding this comment.
Preserve empty mount override when reading env defaults
The script documents that operators can disable auto-mounting by setting ANYSCAN_DPDK_HUGEPAGES_1G_MOUNT / _2M_MOUNT to an empty string, but these assignments use ${VAR:-default} so an empty value is treated as unset and replaced with /mnt/huge1g or /mnt/huge2m. In practice, the opt-out path is unreachable and ensure_hugetlbfs_mount will still try to create/mount those paths, which can cause unwanted mount attempts on hosts where hugetlbfs is managed externally (e.g., via fstab).
Useful? React with 👍 / 👎.
Summary
Two operator-side speedbumps surfaced on c6in.metal during anygpt-52 (PR #65 issuecomment-4339242358):
1.
dpdk-devbindrefuses to bind active interfaces:Operators had to
ip link set <ifc> downon every NIC by hand before the bind succeeded.2. Reserving 1 GiB hugepages was not enough for
rte_eal_init:Operators had to
mount -t hugetlbfs -o pagesize=1G nodev /mnt/huge1gmanually before the scanner could find the pages.Fix
bdf_to_iface()walks/sys/bus/pci/devices/<bdf>/net/to map BDF → iface name.cmd_bindinvokesip link set <ifc> downon each target before invoking dpdk-devbind. Best-effort: missingip, missing iface (already vfio-pci), and already-down ifaces all proceed to the bind.ensure_hugetlbfs_mount()mounts a hugetlbfs of the matching pagesize at the configured path after a successful nr_hugepages reservation. Default targets are/mnt/huge1g(1 GiB) and/mnt/huge2m(2 MiB); override viaANYSCAN_DPDK_HUGEPAGES_1G_MOUNT/_2M_MOUNT, or set to empty to opt out (operators provisioning hugetlbfs via fstab). Idempotent.ANYSCAN_DPDK_LOAD_ONLY=1hook lets the test source the script without triggering its argv dispatch.Test plan
bash tools/test-setup-dpdk.sh— 9/9 passip link set <ifc> downBEFOREdpdk-devbind --bind=vfio-pci(order verified by line numbers in a single cmd log)ensure_hugetlbfs_mountcallsmount -t hugetlbfs -o pagesize=1G nodev <path>ensure_hugetlbfs_mountis no-op when target is already hugetlbfs with the matching pagesizeensure_hugetlbfs_mountis no-op with empty mount pathbdf_to_ifaceresolution against a fake/systree (positive + negative)🤖 Generated with Claude Code