socat CI: run the test suite as parallel shards via parallel-make-check.py#10771
Merged
Conversation
|
retest this please |
…mands
Let any command ride the build/check pool, not just wolfSSL builds:
build false skips configure/make/check (config is just prepare+run)
netns true runs each command under 'bwrap --unshare-net --cap-add
CAP_NET_ADMIN' (its own network namespace) so parallel network
tests can't collide on ports and can configure that namespace
shards fan a config out into N instances, each with $SHARD (1..N) and
$SHARDS=N in its env and its own build-<name>-<k> dir, so a
command can split its work N ways (the pool load-balances them)
Error out, rather than silently degrade, on two misconfigurations that
otherwise surface as confusing test failures: netns requested but bwrap
missing (commands would share the host namespace and collide on ports),
and config-name collisions after shard fan-out (two jobs would share a
build dir and race).
The socat suite is sleep-bound and slow run serially. Drive it through parallel-make-check.py as ~6 shards per CPU, 2 running per CPU at once: each shard runs a round-robin slice of the tests in its own bwrap network namespace (so parallel shards don't collide on ports) and its own build-dir copy. The work is almost all waiting, so the oversubscription just overlaps the waits. Install bubblewrap so the netns isolation actually happens (without it the runner silently shares one namespace and the shards collide). Each fresh netns is IPv4-loopback only, so re-create IPv6 loopback (CAP_NET_ADMIN) for the ::1 / dual-stack tests, and add non-loopback placeholders (fc00::1, 192.0.2.1) so glibc's AI_ADDRCONFIG still returns both families - without them socat's getaddrinfo fails on numeric non-loopback addresses, e.g. the multicast tests. Relax the AppArmor unprivileged-userns restriction so the bwrap netns + CAP_NET_ADMIN work on ubuntu-24.04.
julek-wolfssl
force-pushed
the
socat-parallel-shards
branch
from
8e2b302 to
f2fa741
Compare
netns needs bwrap; without it commands silently share the host network namespace and parallel network tests collide on ports. Skip the check for --list (it inspects configs, runs nothing), hard-fail on CI so a missing- bubblewrap misconfig can't silently degrade, and locally just warn and fall back to the shared namespace.
dgarske
approved these changes
Jun 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The socat suite runs ~590 tests sequentially in a single job and is
sleep-bound: a handful of tests sit in fixed waits (INTRANETRIPPER alone sleeps
~140s at
-t 1.0) that dominate the ~10 min runtime.This generalizes the shared parallel runner
(
.github/scripts/parallel-make-check.py) so any command can ride its workerpool, not just wolfSSL build configs, and uses it to shard the socat tests
across a single runner.
parallel-make-check.py— three additive config keysDefaults keep existing build configs behaving exactly as before:
build: false— skip configure/make/check; run only theprepare/runcommands, so an arbitrary command can use the pool.
netns: true— run each command underbwrap --unshare-net(its ownnetwork namespace) so parallel network tests can't collide on ports. Needs
bubblewrap; warns and falls back to the shared namespace if
bwrapismissing.
shards: N— fan a config out into N instances, each with$SHARD(1..N) and
$SHARDS=Nin its env and its ownbuild-<name>-<k>dir. Thepool (
--threads) bounds how many run at once, soN> threadsload-balances dynamically. Composes with the existing
--shardCI split.socat.ymlOne config (
build:false,netns:true,shards:12) runs a round-robin sliceof
test.shper shard (seq $SHARD $SHARDS 999), each in its own networknamespace and its own copy of the build dir (their generated certs/temp files
would otherwise race).
--no-fail-fastruns every shard so all unexpectedfailures are reported, as the unsharded run did. The job timeout drops from 30
to 15 min.