Add UDP GSO/GRO support

[marcosfsch]

• Version of iperf3 (or development branch, such as master or
  3.1-STABLE) to which this pull request applies:
  master

• Issues fixed (if any):
  Rebase of PR 1309 including missing declarations on "iperf.h" and exception when using without "-l"

• Brief description of code changes (suitable for use as a commit message):
  Add UDP GSO/GRO support

[gegles]

What's the status on this? My team and I are very interested in using this feature! Thx!

[gegles]

Just FYI, after spending a little bit of time on this and an easy rebase on master, I tried running it and got the following core dumped:

./src/iperf3 -c localhost -u -b 20G --gsro
Floating point exception (core dumped)

I will see if I can debug... anybody else has seen this?

Has anybody got this feature working correctly?

[davidBar-On]

The exception happens in iperf_api.c/iperf_parse_arguments():

        test->settings->gso_dg_size = blksize;
        /* use the multiple of datagram size for the best efficiency. */
        test->settings->gso_bf_size = (test->settings->gso_bf_size / test->settings->gso_dg_size) * test->settings->gso_dg_size;

The problem is that for UDP tests, when -l is not set, blksize is 0. (As stated some lines above these lines: blksize = 0; /* try to dynamically determine from MSS */).

Two more comments:

1. When -l is set to less than 1008 bytes, net.c/udp_sendmsg_gso() prints something like msg: 4294967295 != 65390 (iperf3 continues to run but I am not sure if this is not existing iperf3 bug).

2. I got the following compilation warning:

net.c: In function ‘udp_sendmsg_gso’:
net.c:569:15: warning: assignment discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
  569 |  iov.iov_base = data;

[malladisankar]

Can someone please update if there is any update on these commits? Are these available in master branch?

[atorgue]

Hi,
No update on this PR ? No other way to test GSO NIC feature using iperf ?

[jkasten2]

I had to run the following to get the new iperf3 --gsro feature to be built; automake and then autoheader. This updated configure file correctly then I was able to run ./configure and then make as normal.

Tested with a Intel 8500t CPU on Ubuntu server 24.10 with an rtl8126 NIC and I see almost a 4 times drop in CPU usage when sending UDP packets. From ~100% usage on 1 core, to ~25% on 1 core. Nice work to the authors of this PR!

However I am also seeing two bugs so far:

1. Running iperf3 -c 192.168.17.127 --gsro -u results in an error "Floating point exception (core dumped)".
   • Work around is to add -l
2. Probably related, but if anything less than -l 508 is used I get this error:

msg: 4294967295 != 65403

Unexpected error (22)

iperf3 didn't crash, and sends a few packets then results show 0/0.

Odd that someone else noted above that 1008 vs mine 508 is the lowest for -l here.

Edit: The -l low limit is probably due to UDP_MAX_SEGMENTS in the linux kernel. It was recently changed from 64 to 128, which is why I seen a lower number.

Edit2: Ya that was it, this is the code I changed on top of this PR:

diff --git a/src/iperf_client_api.c b/src/iperf_client_api.c
index 8f51c25..6993a9c 100644
--- a/src/iperf_client_api.c
+++ b/src/iperf_client_api.c
@@ -483,6 +483,11 @@ iperf_connect(struct iperf_test *test)
         test->settings->gso_dg_size = test->settings->blksize;
         /* use the multiple of datagram size for the best efficiency. */
         test->settings->gso_bf_size = (test->settings->gso_bf_size / test->settings->gso_dg_size) * test->settings->gso_dg_size;
+
+        /* Avoid UDP_MAX_SEGMENTS defined in the linux kernel */
+        if (test->settings->gso_bf_size / test->settings->gso_dg_size > 64) {
+            test->settings->gso_bf_size = test->settings->gso_dg_size * 64;
+        }
     }
 #endif

Results for UDP Sending:

root@ubuntu-test:~/iperf3-upd-gso# ./src/iperf3 -c 192.168.17.127 --gsro -u -l 320 -b 0 -V
iperf 3.16
Linux ubuntu-test 6.11.0-19-generic #19-Ubuntu SMP PREEMPT_DYNAMIC Wed Feb 12 21:43:43 UTC 2025 x86_64
Control connection MSS 1460
Time: Wed, 12 Mar 2025 06:50:29 GMT
Connecting to host 192.168.17.127, port 5201
      Cookie: <REDACTED>
GSO (320)
GRO
[  5] local 192.168.17.131 port 52396 connected to 192.168.17.127 port 5201
Starting Test: protocol: UDP, 1 streams, 320 byte blocks, omitting 0 seconds, 10 second test, tos 0
[ ID] Interval           Transfer     Bitrate         Total Datagrams
[  5]   0.00-1.00   sec   140 MBytes  1.17 Gbits/sec  457216  
[  5]   1.00-2.00   sec   247 MBytes  2.07 Gbits/sec  809216  
[  5]   2.00-3.00   sec   247 MBytes  2.07 Gbits/sec  807808  
[  5]   3.00-4.00   sec   247 MBytes  2.07 Gbits/sec  808448  
[  5]   4.00-5.00   sec   247 MBytes  2.07 Gbits/sec  809792  
[  5]   5.00-6.00   sec   244 MBytes  2.05 Gbits/sec  800256  
[  5]   6.00-7.00   sec   246 MBytes  2.07 Gbits/sec  806976  
[  5]   7.00-8.00   sec   247 MBytes  2.07 Gbits/sec  809600  
[  5]   8.00-9.00   sec   246 MBytes  2.07 Gbits/sec  806848  
[  5]   9.00-10.00  sec   246 MBytes  2.07 Gbits/sec  806720  
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.00  sec  2.30 GBytes  1.98 Gbits/sec  0.000 ms  0/7722880 (0%)  sender
[  5]   0.00-10.21  sec  1.34 GBytes  1.13 Gbits/sec  0.002 ms  3234368/7722756 (42%)  receiver
CPU Utilization: local/sender 34.5% (2.5%u/31.9%s), remote/receiver 53.1% (8.7%u/44.4%s)

iperf Done.

For me at least, anything less than 320 byte packets drops the pps down like a rock.

[gegles]

Any progress on this? Could we get this feature merged at some point? Thx!

[gegles]

Please, take a look at #1925, hopefully this can become the definitive PR for this feature

[bmah888]

Thanks for the PR! This has been superceded by PR #1925, which was just merged.