Skip to content

Fix: Parse MTU to Firecracker network configuration - #836

Open
HARSHVARANDANI wants to merge 3 commits into
urunc-dev:mainfrom
HARSHVARANDANI:fix/mtu-parsing
Open

Fix: Parse MTU to Firecracker network configuration#836
HARSHVARANDANI wants to merge 3 commits into
urunc-dev:mainfrom
HARSHVARANDANI:fix/mtu-parsing

Conversation

@HARSHVARANDANI

Copy link
Copy Markdown

Description

This PR fixes an issue where the host veth's MTU was not propagated to Firecracker virtual machines.
The dynamic network implementation correctly reads the host veth's MTU, but the static network mode was leaving it unset (defaulting to 0). Furthermore, the Firecracker VMM driver completely lacked support for configuring the MTU in the generated fc.json config file. Consequently, the guest Linux kernel defaulted to an MTU of 1500 regardless of the MTU configured on the host.
Specifically, this change:

  • Updates pkg/network/network_static.go to populate the MTU field in the returned Interface struct from redirectLink.Attrs().MTU.
  • Adds an mtu field to the FirecrackerNet JSON structure in pkg/unikontainers/hypervisors/firecracker.go and sets it using args.Net.MTU. This enables VIRTIO_NET_F_MTU feature negotiation with the guest.
  • Adds a new unit test suite (firecracker_test.go) covering the command construction, JSON config building, and JSON serialization for Firecracker.

Related issues

How was this tested?

The unit tests written in firecracker_test.go verify that the Firecracker command-line constructor (BuildExecCmd) correctly compiles and generates JSON configuration files. The test suite uses a table-driven approach to validate baseline parameters (like boot settings, seccomp filters, custom memory sizes, and virtual CPUs), verify the serialization of both default (1500) and custom (9000) MTU values, assert that zero-value MTUs are omitted from the serialized JSON configuration using the omitempty struct tag, and test the integration of auxiliary configurations such as vsock devices and custom initramfs images.

LLM usage

Used Google Antigravity for writing tests and scoping the issue. All changes have been manually reviewed.

Checklist

  • I have read the contribution guide.
  • The linter passes locally (make lint).
  • The e2e tests of at least one tool pass locally (make test_ctr, make test_nerdctl, make test_docker, make test_crictl).
  • If LLMs were used: I have read the llm policy.

Ensure the MTU is retrieved from the container interface and populated in the Interface struct, preventing it from defaulting to 0.

Signed-off-by: HARSHVARANDANI <hpvarandani@gmail.com>
Add MTU field to FirecrackerNet config struct and serialize it to the Firecracker config JSON to enable VIRTIO_NET_F_MTU negotiation with the guest.

Signed-off-by: HARSHVARANDANI <hpvarandani@gmail.com>
… building

Add test cases covering MTU parsing, JSON serialization, and baseline configuration generation for Firecracker.

Signed-off-by: HARSHVARANDANI <hpvarandani@gmail.com>
@netlify

netlify Bot commented Jul 18, 2026

Copy link
Copy Markdown

Deploy Preview for urunc canceled.

Name Link
🔨 Latest commit 2623e40
🔍 Latest deploy log https://app.netlify.com/projects/urunc/deploys/6a5bab5952e3490008a2f173

@cmainas

cmainas commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Hello @HARSHVARANDANI ,

thank you for opening the PR. Regarding the test method, we will need to verify that this works end-to-end, meaning that the Firecracker guest getc configured with the MTU we specify. A unit test is not enough for the testing.

You can take a look at #564 on how to test this end-to-end.

@HARSHVARANDANI

Copy link
Copy Markdown
Author

@cmainas
To validate this end-to-end, I built a test image (Firecracker + initrd + BusyBox) where the guest's run.sh captures its own ip addr show output and serves it over httpd, so the result reported below is read directly from the guest kernel itself, not inferred from the host-side veth.

I tested three distinct MTU scenarios:

1. Default network (no MTU pinned)

sudo nerdctl --address "$CONTAINERD_ADDR" --namespace "$NAMESPACE" \
    run -ti --rm -p 8890:80 --runtime io.containerd.urunc.v2 busybox-fc-test:latest

Result: eth0 MTU = 1500 (default)

2. k3s + Flannel network

Ran against k3s's containerd socket, explicitly pointed at Flannel's CNI config so the container joins the real overlay network:

sudo nerdctl --address /run/k3s/containerd/containerd.sock --namespace k8s.io \
    --cni-netconfpath /var/lib/rancher/k3s/agent/etc/cni/net.d \
    --cni-path <k3s CNI bin_dir> \
    run -ti -p 8887:80 --net cbr0 --runtime io.containerd.urunc.v2 busybox-fc-test:latest

Cross-checked the expected value against Flannel's own computed MTU on the node:

cat /run/flannel/subnet.env   # FLANNEL_MTU=1450

Result: eth0 MTU = 1450 — matches Flannel's computed value exactly

3. Custom nerdctl bridge network (arbitrary MTU)

To rule out coincidence, created a synthetic network pinned to an unrelated MTU value:

sudo nerdctl --address /run/k3s/containerd/containerd.sock --namespace k8s.io \
    --cni-netconfpath /var/lib/rancher/k3s/agent/etc/cni/net.d \
    network create urunc-mtu-1200 --subnet 10.99.0.0/24 --opt mtu=1200

sudo nerdctl --address /run/k3s/containerd/containerd.sock --namespace k8s.io \
    --cni-netconfpath /var/lib/rancher/k3s/agent/etc/cni/net.d \
    run -ti --rm -p 8887:80 --net urunc-mtu-1200 --runtime io.containerd.urunc.v2 busybox-fc-test:latest

Result: eth0 MTU = 1200

Summary

Scenario Host-side veth MTU Guest-reported eth0 MTU
Default 1500 1500
Flannel (real cluster) 1450 1450
Custom bridge 1200 1200

All three independent, distinct MTU values are correctly negotiated and applied inside the Firecracker guest, confirming the MTU fix works end-to-end for this VMM. Screenshots of each curl output attached below.
image
image
image

@cmainas

cmainas commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Thank you @HARSHVARANDANI for the testing. Could you also provide the image you created to replicate the testing/

@HARSHVARANDANI

Copy link
Copy Markdown
Author

Thank you @HARSHVARANDANI for the testing. Could you also provide the image you created to replicate the testing/

Here is the image link https://hub.docker.com/r/harshpvarandani/firecracker-busybox-initrd-ipconfig
there might be some trouble with the image running properly sometimes it just hangs so you migh have to restart it I haven't been able to figure out what goes wrong but when it does run you can just curl whatever localhost port you bind to the container's port 80 in the command

@gntouts

gntouts commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Thanks for taking the time to look into this. The MTU propagation gap in Firecracker is real and worth fixing. However, the current approach would break backwards compatibility with Firecracker versions older than v1.16.0.

To verify this, I wrote an fc.json matching the config urunc generates, then ran the official Firecracker release binaries directly, without urunc or containerd:

gntouts@lima-ubuntu:~/fctest$ jq '."network-interfaces"' fc-mtu.json
[
  {
    "iface_id": "net1",
    "guest_mac": "02:00:00:00:00:01",
    "host_dev_name": "tap_urunc",
    "mtu": 9000
  }
]
gntouts@lima-ubuntu:~/fctest$ ./fc-v1.7.0 --version
Firecracker v1.7.0

2026-07-30T22:08:42.259477810 [anonymous-instance:main] Firecracker exiting successfully. exit_code=0
gntouts@lima-ubuntu:~/fctest$ ./fc-v1.7.0 --no-api --no-seccomp --config-file ~/fctest/fc-mtu.json | grep mtu
Error: RunWithoutApiError(BuildMicroVMFromJson(ParseFromJson(InvalidJson(Error("unknown field `mtu`, expected one of `iface_id`, `host_dev_name`, `guest_mac`, `rx_rate_limiter`, `tx_rate_limiter`", line: 4, column: 107)))))
2026-07-30T22:08:47.053781041 [anonymous-instance:main] RunWithoutApiError error: Failed to build MicroVM from Json: Configuration for VMM from one single json failed: Invalid JSON: unknown field `mtu`, expected one of `iface_id`, `host_dev_name`, `guest_mac`, `rx_rate_limiter`, `tx_rate_limiter` at line 4 column 107
gntouts@lima-ubuntu:~/fctest$ ./fc-v1.16.0 --version
Firecracker v1.16.0

2026-07-30T22:08:59.524630750 [anonymous-instance:main] Firecracker exiting successfully. exit_code=0
gntouts@lima-ubuntu:~/fctest$ ./fc-v1.16.0 --no-api --no-seccomp --config-file ~/fctest/fc-mtu.json | grep mtu
Error: RunWithoutApiError(BuildMicroVMFromJson(StartMicroVM(Kvm(Kvm(Error(2))))))

(ignore the error in the v1.16.0 invocation, I run this experiment on a Lima VM running on an Apple M1 Pro, which has no nested virtualization)

We should find a way to only set this field when a supported Firecracker is installed in the system. Perhaps we could add a new option in urunc.toml to the existing [monitors.firecracker] section:

[monitors.firecracker]
default_memory_mb = 256
default_vcpus = 1
advertise_mtu = false   # requires Firecracker >= 1.16.0

@HARSHVARANDANI

HARSHVARANDANI commented Jul 31, 2026

Copy link
Copy Markdown
Author

I agree we should add backwards compatibility for firecracker versions.

We should find a way to only set this field when a supported Firecracker is installed in the system. Perhaps we could add a new option in urunc.toml to the existing [monitors.firecracker] section:

As I understand, please correct me if I'm wrong, to add an option in urunc.toml, the user would have to update the file manually with the firecracker version. Instead, we already have semver/v3 as a dependency in the repo, we can create a function to detect firecracker version and store as a field in the firecracker struct, and in the BuildExecCmd function we just check if the version is compatible for passing MTU and accordingly include/omit the MTU field.

Does this sound good?

@gntouts

gntouts commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

This would add the overhead of one extra call to Firecracker for every run. Could you try it out and measure the added latency this check will cost us? If it is very low, we can use your approach.

@cmainas

cmainas commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Except of the extra overhead in the hot path, there are not any cases where a cluster might have multiple Firecracker versions. On the contrary, Firecracker is installed once (and then updated). We do not need to check its version at runtime, since this information is static.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Set the MTU in the interface inside the sandbox

3 participants