Feature/tunnel wireguard for FRR by jbemmel · Pull Request #3569 · ipspace/netlab

jbemmel · 2026-07-02T19:41:30Z

IPv4 or IPv6
default routing table or in VRF
WireGuard package dynamically installed
keys auto-generated if WireGuard tools are installed
MTU auto-derived and validated

Tested:

./device-module-test -d frr -p clab tunnel *wire*
./device-module-test -d frr -p libvirt tunnel *wire* (needed the default route fixes)

Notes:

PR generalizes Linux package installation to support both apk and apt-get; needed for FRR containers
The ipv6 link local address is needed for OSPFv3, made this a util method in routing
add_linux_packages is a candidate for general util method

ipspace

It looks like you (or your AI friend) copied the utils/tunnel.py code, resulting in duplicate functionality. Please fix that.

And yes, I know you'll tell me something about AFs, but we can deal with that.

ipspace · 2026-07-03T15:12:07Z

+def _interface_has_af(intf: Box, af: str) -> bool:
+  return af in intf and isinstance(intf[af],str)
+
+def _interface_matches_constraints(


And why exactly did you reinvent the utility function? There's a reason I put it into utils, not into tunnel.gre

I instructed AI to implement a Wireguard tunnel plugin similar to the GRE tunnel plugin - it took this a bit too literally

ipspace · 2026-07-03T15:17:17Z

+  we derive a deterministic address that the initial config script assigns when
+  the interface is created.
+  '''
+  if intf.get('tunnel.af') != 'ipv6' or 'ipv6' not in intf:


If the wireguard tunnels are not dual-stacked (I know I could check), then you have to throw an error, not just check the AF, shrug, and return. If they are dual-stacked, then I don't understand why you're checking tunnel.af (which impacts the underlay interface selection)

jbemmel · 2026-07-04T05:27:56Z

PR now uses the common get_tunnel_source proposing a minor change in the AF filtering (i.e. if tunnel.af is set, use it to filter valid interfaces) and adding the MTU of the source interface in the response

ipspace · 2026-07-04T14:16:48Z

I would suggest you use the new utility functions from #3576, which might make your code cleaner (and with less duplicate code). Also, the new source selection function should give you all the information you need to figure out AF and MTU.

jbemmel · 2026-07-05T01:45:19Z

I would suggest you use the new utility functions from #3576, which might make your code cleaner (and with less duplicate code). Also, the new source selection function should give you all the information you need to figure out AF and MTU.

ok, updated:

Moved IPv6 link-local address generation to a FRR device quirk (rather than the Wireguard tunnel plugin)
Moved tunnel device creation to FRR initial script (also for GRE -> requires initializing the full tunnel config)
Suggested change for tunnel_source to yield each source interface such that specific plugins can configure each interface if needed

add_linux_packages doesn't belong in the Wireguard plugin, suggested alternative:

# tunnel/wireguard/defaults.yml (plugin declares dependency)
netlab_linux_packages:
  tunnel@wireguard:
    wireguard-tools: wg

with logic in augment/devices.py to merge it per device that runs the included modules or plugins

ipspace

Please stop "fixing" imaginary problems, hard-coding stuff where include files work, and changing things for no documented reason.

There are probably other things that have to be fixed; I had enough.

ipspace · 2026-07-05T11:07:04Z

 fi
-{% elif netlab_mgmt_vrf|default(True) %}
+{% else %}
+{%   if netlab_linux_packages|default({}) %}


Why would you install packages only in FRR container but not in FRR VM?

Both clab and libvirt providers pass the integration tests, what led you to believe this would only apply to FRR container?

Stupid me. We install Linux packages unconditionally in libvirt path and check whether netlab_linux_packages is defined in clab path. Maybe we should unify them once this gets merged?

It's the asymmetric redundant if condition that may have confused you, I'll make it the same

ipspace · 2026-07-05T11:16:56Z

  if [ ! -e /sys/devices/virtual/net/{{ i.ifname }} ]; then
 {%   if i.type in ['lag','bond'] %}
 {{     create_bond_dev(i,node_provider) }}
+{%   elif i.tunnel.mode|default('') == 'gre' %}


No. Why are you "fixing" a problem that does not exist, and why are you doing that in an unrelated PR?

As requested, I moved device creation to the initial template (including 'tunnel' similar to 'bond'). This in turn broke the GRE plugin, because now tunnel devices got created twice

I can refactor things and use the per-plugin include pattern instead

How about creating tunnels in the custom config templates? Why do you think you have to create it during the initial device configuration? FRR seems patient...

ipspace · 2026-07-05T11:18:38Z

+      dev {{ i.tunnel._source.ifname }} ttl 255
 {%   else %}
-    ip link add {{ i.ifname }} type dummy
+    ip link add {{ i.ifname }} type {{ 'wireguard' if i.tunnel.mode|default('') == 'wireguard' else 'dummy' }}


No. We have a nice solution in #3566. Why do you think it makes sense to hard-code "wireguard" in main FRR initial template just to fix your immediate challenge?

ipspace · 2026-07-05T11:19:34Z

-ip link set dev {{ intf.ifname }} up
-{% endfor %}
-exit 0
+# Empty file, GRE tunnels are created during initial FRR device configuration


Please stop fixing imaginary problems

ipspace · 2026-07-05T11:22:30Z

+
+  ip link set dev "${ifname}" mtu "${mtu}" up
+
+  wait_wg_peer "${ifname}" "${peer_key}"


How do you expect this to work between a FRR VM and a FRR container? Also, what happens if the two containers are configured in different batches?

ipspace · 2026-07-05T11:24:02Z


+      yield ndata,intf
+
+def tunnel_source(


Why was this change necessary?

For the Wireguard plugin I needed to customize each tunnel beyond what the generic code does.

There are many ways to do that, I could simply iterate again too. I thought this might be a nice way to do it - but if you don't like it, we can change it

Premature optimization at its best... Let's add a generator, and a stub function, with all the accompanying docstrings, just so we don't need an extra for loop 🤦‍♂️ in code that will be executed once every other blue moon.

ipspace · 2026-07-05T11:34:17Z

add_linux_packages doesn't belong in the Wireguard plugin, suggested alternative:

Why not? It's either there or in the device quirks.

with logic in augment/devices.py to merge it per device that runs the included modules or plugins

So we'll modify core logic to address the needs of one device for one plugin? Well, no.

Even if it were more common, it's still applicable only to Linux and friends (FRR, BIRD).

jbemmel · 2026-07-06T16:22:15Z

Moved ipv6 link local back to the plugin
Reverted changes to tunnel/init.py
Applied VyOS initial plugin for loop include pattern (separate PR [FRR] Move FRR GRE tunnel netdev setup into plugin initial templates #3585)

Install wireguard-tools during initial configuration on clab FRR nodes before the management VRF is created, and add integration tests for global and transport-VRF underlay scenarios. Co-authored-by: Cursor <cursoragent@cursor.com>

Add shared tunnel utilities, OSPFv3 link-local assignment, VRF routing rules for WireGuard handshakes, and integration tests for IPv6 transport. Co-authored-by: Cursor <cursoragent@cursor.com>

Use tunnel-AF-specific fwmark rules so IPv6 transport handshakes work over VRF underlays instead of peer-specific policy routing. Co-authored-by: Cursor <cursoragent@cursor.com>

Move WireGuard netdev creation out of the plugin's custom config (which runs last) and into the initial FRR config, so the interface exists before FRR and the routing modules are configured. A generic _linux_device_type interface attribute tells the initial script which kind of netdev to create, and the deterministic IPv6 link-local (_ipv6_link_local) needed for OSPFv3 is assigned at the same time (kernel address auto-generation is impossible for the ARPHRD_NONE WireGuard device). Drop the optional cryptography fallback; key generation now relies solely on wireguard-tools, and the integration tests use explicit static keys. Docs updated accordingly. Co-authored-by: Cursor <cursoragent@cursor.com>

The global WireGuard listening socket could not receive encrypted packets arriving on a VRF underlay interface for IPv6 tunnels, so handshakes never completed. net.ipv4.udp_l3mdev_accept governs l3mdev socket matching for both IPv4 and IPv6 UDP (there is no net.ipv6 counterpart), but it was only set in the IPv4 branch. Set it unconditionally and collapse the per-AF policy-rule handling into a single command. Co-authored-by: Cursor <cursoragent@cursor.com>

Default tunnel.allowed_ips to ::/0 for IPv6 tunnels (0.0.0.0/0 for IPv4) so IPv6 tunnels work without setting it explicitly. Add an Allowed IPs section explaining WireGuard cryptokey routing and its inbound/outbound semantics, and trim the examples to the plain and VRF/IPv6 cases. Co-authored-by: Cursor <cursoragent@cursor.com>

Co-authored-by: Cursor <cursoragent@cursor.com>

Set _default for tunnel.af, tunnel.mtu, and tunnel.persistent_keepalive in defaults.yml instead of hard-coding them in post_transform. Only the allowed_ips default stays in code because it depends on the resolved af. Co-authored-by: Cursor <cursoragent@cursor.com>

Replace hardcoded default routes with a dedicated core address pool so the static routes reference the pool rather than 0.0.0.0/0 and ::/0. Co-authored-by: Cursor <cursoragent@cursor.com>

Schema _default values for tunnel.af, mtu, and persistent_keepalive are materialized onto the per-node interface entries during link validation, before link->interface propagation. They then win over an explicit link-level override (e.g. tunnel.af: ipv6) during the interface merge, breaking the tunnel source lookup. Resolve these defaults in post_transform where the interface data is already merged. Also iterate node_iflist with .items() instead of re-indexing by key. Co-authored-by: Cursor <cursoragent@cursor.com>

Infer tunnel.af from the underlay source interface (IPv4 when available, IPv6 for IPv6-only underlays) so IPv6-only links no longer need an explicit tunnel.af. Derive the tunnel interface link-local address from the low 64 bits (interface identifier) of the overlay address via a new routing.get_ipv6_link_local helper, guaranteeing distinct link-locals on both tunnel endpoints. Drop the now-redundant tunnel.af/allowed_ips from the IPv6 integration tests and update the docs. Co-authored-by: Cursor <cursoragent@cursor.com>

…fig. Tunnel netdevs were created in the first loop but skipped the down/sysctl/up cycle that enables accept_ra on host interfaces. Co-authored-by: Cursor <cursoragent@cursor.com>

Defer link-local assignment until after the IPv6 sysctl loop and only bring loopback interfaces up during netdev creation so tunnel down/up does not drop addresses needed for OSPFv3. Co-authored-by: Cursor <cursoragent@cursor.com>

Match the VyOS initial-config pattern for FRR, restore shared tunnel_source in _p2p, and drop WireGuard peer handshake waiting that breaks partial deploys. Co-authored-by: Cursor <cursoragent@cursor.com>

Map tunnel.gre-style config names to slash-separated plugin directories so frr.initial.j2 templates are found during initial device configuration. Co-authored-by: Cursor <cursoragent@cursor.com>

Derive _ipv6_link_local in wireguard_intf_defaults instead of an FRR device quirk, and iterate node_iflist directly in post_transform. Co-authored-by: Cursor <cursoragent@cursor.com>

Inline tunnel AF derivation and scope af_active to the allowed_ips default block. Co-authored-by: Cursor <cursoragent@cursor.com>

Co-authored-by: Cursor <cursoragent@cursor.com>

Restore GRE from dev and fold WireGuard netdev creation into frr.j2, removing the frr.initial.j2 indirection from initial device config. Co-authored-by: Cursor <cursoragent@cursor.com>

Keep tunnel-specific IPv6 link-local assignment in the WireGuard plugin template instead of the shared initial device configuration. Co-authored-by: Cursor <cursoragent@cursor.com>

Run linux/packages.j2 on containerlab nodes again so per-node packages like wireguard-tools are installed during initial config. Co-authored-by: Cursor <cursoragent@cursor.com>

Use a single interface loop with inlined wg set commands instead of a separate setup_wg_tunnel wrapper function. Co-authored-by: Cursor <cursoragent@cursor.com>

Assume vrf_table and rule_pref are set whenever tunnel.vrf is configured. Co-authored-by: Cursor <cursoragent@cursor.com>

Set peer_ip from the tunnel AF first, then build peer_endpoint from IP and port. Co-authored-by: Cursor <cursoragent@cursor.com>

Compute rule_pref only when tunnel.vrf is set and inline the fwmark conditional. Co-authored-by: Cursor <cursoragent@cursor.com>

Co-authored-by: Cursor <cursoragent@cursor.com>

Move destination variables below VRF underlay setup and use intf.tunnel.af directly instead of a separate af alias. Co-authored-by: Cursor <cursoragent@cursor.com>

tunnel_destination can run after interface defaults even when earlier steps reported validation errors. Co-authored-by: Cursor <cursoragent@cursor.com>

jbemmel · 2026-07-07T15:06:34Z

Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away...

ipspace

Please try taking out the LLA hack. The current setup works for me without adding manual LLA to the tunnel interface.

Drop the deterministic fe80:: derivation and restore routing.py to dev. Co-authored-by: Cursor <cursoragent@cursor.com>

ipspace · 2026-07-08T13:12:32Z

+
+  for node in node_iflist:
+    ndata = topology.nodes[node]
+    for intf in _tunnel.interfaces(ndata,'wireguard'):


You could have used node_iflist[node] here, but it doesn't matter.

ipspace

Works like a charm ;) Thank you!

snuffy22 reviewed Jul 2, 2026

View reviewed changes

Comment thread netsim/extra/tunnel/wireguard/__init__.py

ipspace requested changes Jul 3, 2026

View reviewed changes

jbemmel marked this pull request as draft July 3, 2026 22:32

jbemmel marked this pull request as ready for review July 4, 2026 05:28

jbemmel marked this pull request as draft July 4, 2026 15:23

jbemmel force-pushed the feature/tunnel-wireguard branch from 8a275f9 to e4d2c21 Compare July 4, 2026 15:26

snuffy22 reviewed Jul 4, 2026

View reviewed changes

Comment thread tests/integration/tunnel/03-wireguard.yml

jbemmel marked this pull request as ready for review July 5, 2026 01:46

ipspace requested changes Jul 5, 2026

View reviewed changes

jbemmel marked this pull request as draft July 5, 2026 15:19

jbemmel mentioned this pull request Jul 6, 2026

[FRR] Move FRR GRE tunnel netdev setup into plugin initial templates #3585

Closed

jbemmel marked this pull request as ready for review July 6, 2026 16:22

jbemmel marked this pull request as draft July 7, 2026 14:08

jbemmel and others added 12 commits July 7, 2026 09:09

Add tunnel.wireguard plugin for FRR WireGuard tunnels.

2286332

Install wireguard-tools during initial configuration on clab FRR nodes before the management VRF is created, and add integration tests for global and transport-VRF underlay scenarios. Co-authored-by: Cursor <cursoragent@cursor.com>

Enhance tunnel.wireguard with IPv6, VRF underlay, and key autogen.

a6160a4

Add shared tunnel utilities, OSPFv3 link-local assignment, VRF routing rules for WireGuard handshakes, and integration tests for IPv6 transport. Co-authored-by: Cursor <cursoragent@cursor.com>

Fix WireGuard VRF underlay routing with fwmark policy rules.

224798f

Use tunnel-AF-specific fwmark rules so IPv6 transport handshakes work over VRF underlays instead of peer-specific policy routing. Co-authored-by: Cursor <cursoragent@cursor.com>

Update docs

a09d91d

Clarify tunnel.wireguard supports both clab and libvirt providers.

279cd53

Co-authored-by: Cursor <cursoragent@cursor.com>

Use address pool for WireGuard tunnel static routes in tests

f5ee2be

Replace hardcoded default routes with a dedicated core address pool so the static routes reference the pool rather than 0.0.0.0/0 and ::/0. Co-authored-by: Cursor <cursoragent@cursor.com>

jbemmel and others added 8 commits July 7, 2026 09:09

Apply IPv6 RA sysctl settings to tunnel interfaces in FRR initial con…

ee867a2

…fig. Tunnel netdevs were created in the first loop but skipped the down/sysctl/up cycle that enables accept_ra on host interfaces. Co-authored-by: Cursor <cursoragent@cursor.com>

Move FRR tunnel netdev setup into plugin initial templates.

f8b90ad

Match the VyOS initial-config pattern for FRR, restore shared tunnel_source in _p2p, and drop WireGuard peer handshake waiting that breaks partial deploys. Co-authored-by: Cursor <cursoragent@cursor.com>

Fix FRR plugin initial template include paths.

47c76e5

Map tunnel.gre-style config names to slash-separated plugin directories so frr.initial.j2 templates are found during initial device configuration. Co-authored-by: Cursor <cursoragent@cursor.com>

Move WireGuard IPv6 link-local handling back to the plugin.

9938d50

Derive _ipv6_link_local in wireguard_intf_defaults instead of an FRR device quirk, and iterate node_iflist directly in post_transform. Co-authored-by: Cursor <cursoragent@cursor.com>

Tidy WireGuard interface defaults in the tunnel plugin.

4ee5d74

Inline tunnel AF derivation and scope af_active to the allowed_ips default block. Co-authored-by: Cursor <cursoragent@cursor.com>

remove asymmetric redundant if condition, fix indentation

bb048ae

Co-authored-by: Cursor <cursoragent@cursor.com>

Move FRR tunnel netdev setup back into plugin config templates.

8cc58f6

Restore GRE from dev and fold WireGuard netdev creation into frr.j2, removing the frr.initial.j2 indirection from initial device config. Co-authored-by: Cursor <cursoragent@cursor.com>

jbemmel force-pushed the feature/tunnel-wireguard branch from d3fe3e1 to 8cc58f6 Compare July 7, 2026 14:14

jbemmel and others added 9 commits July 7, 2026 09:21

Restore FRR initial config from dev and move link-local to WireGuard.

3734e13

Keep tunnel-specific IPv6 link-local assignment in the WireGuard plugin template instead of the shared initial device configuration. Co-authored-by: Cursor <cursoragent@cursor.com>

Restore FRR clab package install and simplify WireGuard netdev check.

eec749a

Run linux/packages.j2 on containerlab nodes again so per-node packages like wireguard-tools are installed during initial config. Co-authored-by: Cursor <cursoragent@cursor.com>

Simplify FRR WireGuard config template.

09c47bf

Use a single interface loop with inlined wg set commands instead of a separate setup_wg_tunnel wrapper function. Co-authored-by: Cursor <cursoragent@cursor.com>

Simplify WireGuard VRF conditionals in FRR template.

dfb68f5

Assume vrf_table and rule_pref are set whenever tunnel.vrf is configured. Co-authored-by: Cursor <cursoragent@cursor.com>

Split WireGuard peer endpoint construction in FRR template.

3f58c00

Set peer_ip from the tunnel AF first, then build peer_endpoint from IP and port. Co-authored-by: Cursor <cursoragent@cursor.com>

Tidy WireGuard VRF setup in FRR template.

7ee2458

Compute rule_pref only when tunnel.vrf is set and inline the fwmark conditional. Co-authored-by: Cursor <cursoragent@cursor.com>

Trim whitespace after WireGuard fwmark conditional in FRR template.

173ac21

Co-authored-by: Cursor <cursoragent@cursor.com>

Reorder WireGuard peer endpoint setup in FRR template.

33e70e0

Move destination variables below VRF underlay setup and use intf.tunnel.af directly instead of a separate af alias. Co-authored-by: Cursor <cursoragent@cursor.com>

Remove redundant error check in WireGuard post_transform.

e68c71f

tunnel_destination can run after interface defaults even when earlier steps reported validation errors. Co-authored-by: Cursor <cursoragent@cursor.com>

jbemmel marked this pull request as ready for review July 7, 2026 15:06

ipspace requested changes Jul 7, 2026

View reviewed changes

Comment thread netsim/extra/tunnel/wireguard/frr.j2 Outdated

jbemmel marked this pull request as draft July 7, 2026 16:29

Remove WireGuard IPv6 link-local workaround.

e9e34da

Drop the deterministic fe80:: derivation and restore routing.py to dev. Co-authored-by: Cursor <cursoragent@cursor.com>

jbemmel marked this pull request as ready for review July 7, 2026 16:32

ipspace reviewed Jul 8, 2026

View reviewed changes

Minimize the number of integration tests

c096020

ipspace added a commit that referenced this pull request Jul 8, 2026

FRR integration tests for #3569

fd7aa66

ipspace approved these changes Jul 8, 2026

View reviewed changes

ipspace merged commit c24f73e into ipspace:dev Jul 8, 2026
6 checks passed

snuffy22 pushed a commit to snuffy22/netlab that referenced this pull request Jul 9, 2026

Wireguard tunnel for FRR (ipspace#3569)

6ea561a


		ip link set dev "${ifname}" mtu "${mtu}" up

		wait_wg_peer "${ifname}" "${peer_key}"

Conversation

jbemmel commented Jul 2, 2026

Uh oh!

Uh oh!

ipspace left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jbemmel commented Jul 4, 2026

Uh oh!

ipspace commented Jul 4, 2026

Uh oh!

Uh oh!

jbemmel commented Jul 5, 2026

Uh oh!

ipspace left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ipspace commented Jul 5, 2026

Uh oh!

jbemmel commented Jul 6, 2026

Uh oh!

jbemmel commented Jul 7, 2026

Uh oh!

ipspace left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ipspace left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants