Adaptive FHSS: prove restoration on air, and draw it in the RF primer

README.md

@@ -29,9 +29,13 @@ long-range digital video links.
   channel — can change frame by frame. That turns one dongle into an
   adaptive-link engine: unequal error protection for video layers, live
   power control, per-packet frequency hopping.
-- **Frequency hopping at FHSS speed.** A channel hop costs ~0.5–2.5 ms
-  depending on chip — fast enough to hop on every packet
-  ([how](docs/frequency-hopping.md)).
+- **Frequency hopping at FHSS speed, and a hopset that adapts.** A channel hop
+  costs ~0.5–2.5 ms depending on chip — fast enough to hop on every packet
+  ([how](docs/frequency-hopping.md)) — in a keyed order an observer cannot
+  predict. The link also learns: a channel that stops delivering is dropped
+  from the schedule by authenticated agreement between the two ends, and
+  revisited later by keyed probes in case it recovers
+  ([how](docs/fhss.md)).
 - **Narrowband modes the kernel can't do.** 5 and 10 MHz channels on every
   supported generation — including the decade-old RTL8812AU and RTL8814AU the
   vendor never gave narrowband — half/quarter the bandwidth, more range from
@@ -239,7 +243,12 @@ one `IRtlDevice` interface covers all four generations.
 - [FHSS](docs/fhss.md) — the anti-jam design article: keyed SipHash hop
   schedules, slot-locked lockstep RX, and
   [jammer resilience](docs/jammer-resilience.md) — measured delivery against
-  parked and following jammers, and where a follower breaks.
+  parked and following jammers, and where a follower breaks. It carries on
+  into the adaptive half: how the two ends agree on a change to the hopset
+  without either trusting the other, why the endpoint that must decode is the
+  one that decides, why a transmitter may argue only that a move leaves *it*
+  worse off rather than that it disagrees, and what stops an adversary who can
+  make channels look bad from herding the link onto one it then jams.
 - [Narrowband](docs/narrowband.md) — 5/10 MHz channels across all three
   generations: the baseband re-clock, the per-chip register machinery, and the
   walls (RF re-latch edges, per-die clock coupling, the 5 MHz/5 GHz CFO limit).

docs/rf-primer.md

@@ -3,7 +3,7 @@
 devourer talks to a Wi-Fi radio at a very low level — subcarriers, constellations,
 gain control, the transmit and receive chains. If you're new to that machinery,
 the terms in the other docs (per-tone SNR, EVM, CCA, AGC, occupied bandwidth) can
-feel like jargon. This page is a picture book: ten short animations, each
+feel like jargon. This page is a picture book: twelve short animations, each
 built in the DEVOURER live-monitor style, that show what the machinery actually
 looks like — from a single subcarrier all the way to a hopping, diversity-combined,
 bandwidth-hopping link. Read it top to bottom and the rest of the docs will click.
@@ -120,6 +120,55 @@ escapes. Done per-packet (`DEVOURER_HOP_*`), hopping doubles as a
 frequency-diversity interleaver for the outer FEC: losses are spread thin across
 frequencies instead of wiping out a run of packets on one.
 
+![Adaptive hopset — the jammed channel stops being scheduled](img/hopset_adapt.gif)
+
+Bounding the damage is not the same as avoiding it. Most interference is
+*furniture* — a fixed AP, a neighbouring video link, a microwave, a radar. It
+isn't trying to win; it's just sitting there, and it will still be sitting there
+in an hour. A hop set that keeps visiting its channel keeps paying for it, every
+round, forever. The **receiver** is the endpoint that has to decode, so its
+per-channel delivery is the authoritative evidence (energy
+readings only *classify* an impairment; a quiet channel with a dead link and a
+noisy one that still delivers are opposite decisions). It **proposes**; it never
+acts. The transmitter owns the schedule, and answers with an authenticated commit
+naming a **future** slot, repeated until that slot arrives — so both ends swap
+schedules together and a lost frame can't split them apart. Because the hop order
+is **keyed**, dropping a channel re-keys the permutation rather than leaving a
+readable gap in a pattern. And exclusion is not a ratchet: keyed **probes** go
+back and look — the round, the position and the channel all derived from the key,
+so recovery isn't itself a periodic target — and a channel that recovered comes
+back. Measured against a narrowband interferer parked on one member of a
+four-channel set: delivery 0.72 → 0.83, with the channel restored on its own once
+the interferer stopped.
+
+![Anti-herding — the schedule refuses to shrink past its floor](img/hopset_herding.gif)
+
+Now take the furniture away and put something *adversarial* there instead, and
+the picture above stops being the story — the one that replaces it ends with the
+link still bleeding, and that ending is the point. Two measurements say why.
+Against a **blind** parked jammer, a keyed hop order and a plainly sequential one
+deliver identically — secrecy buys nothing against something that isn't looking;
+it only earns its keep against an adversary that *reacts*. And against a jammer that
+**follows** — moving onto a still-active channel after every exclusion — the link
+does not get better. It gets *walked down*. And notice that every one of those
+exclusion requests is individually correct: the receiver really is losing that
+channel, and dropping it really would help, for about a dozen dwells. Granted
+without limit, that reasonable-looking sequence ends with the link parked on one
+frequency of the attacker's choosing, which is exactly where an adversary wants
+it. So an adaptive exclusion loop is itself an attack surface, and the shipped
+one is built timid on purpose: one channel per update, a mandatory gap between
+updates (a *refused* proposal spends it too, so bouncing proposals can't flood
+the control path), nothing acted on when the whole band degrades together, and a
+hard **floor** on how many channels stay active. When that floor is reached the
+next proposal is refused, however well-evidenced it is, and the link spends the
+rest of the flight bleeding across the channels it has left rather than being
+herded onto one. On the four-channel bench set the floor binds after a single
+exclusion, and that's the measured result: exclusion depth one, active set held
+at three, no collapse. It is a pass rather than a win, and the difference matters
+— the advantage here isn't that the link learns, it's that it learns without
+becoming teachable by an adversary. The protocol, the policy and the numbers are
+in [`fhss.md`](fhss.md).
+
 ## 9. Trading robustness for throughput in time — bandwidth TDMA
 
 ![Bandwidth TDMA — two stations flipping together under a shared clock](img/tdma_schedule.gif)

tests/hopset_adaptive_jammer.sh

@@ -356,7 +356,12 @@ else
 fi
 ADAPT_DELIV=$(delivery "$OUT/rx_adapt.log")
 
-if [ "$MODE" = "parked" ] || [ "$MODE" = "sense" ]; then
+if [ "$MODE" = "parked" ] || [ "$MODE" = "sense" ] || [ "$MODE" = "failsafe" ]; then
+    # The failsafe needs this phase as much as the receiver-driven modes do:
+    # an autonomous exclusion that can never be undone is a link that shrinks
+    # permanently the first time something transient sits on a channel. With
+    # the receiver still mute, the only evidence that can bring the channel
+    # back is the transmitter's own keyed probes.
     echo "== phase C: jammer off — probes must restore the channel, ${RESTORE_S}s =="
     kill_jammer
     : > "$OUT/rx_restore_mark"
@@ -477,6 +482,28 @@ elif [ "$MODE" = "failsafe" ]; then
         echo "   rx: $(gm "$OUT/rx_adapt.log" | tr '\n' ' ')"
         fails=$((fails+1))
     fi
+    # An autonomous exclusion must be undoable by the transmitter's own
+    # evidence, or the link ratchets down permanently. With the receiver still
+    # mute through phase C, a restore here can only have come from the keyed
+    # probes the transmitter aired on the excluded channel.
+    require "$OUT/tx_adapt.log" '"ev":"hopset.probe"' "TX aired keyed probes"
+    if grep -F '"origin":"failsafe"' "$OUT/tx_adapt.log" 2>/dev/null |
+       grep -qF '"kind":"restore"'; then
+        echo "PASS: the transmitter restored the channel from its own probes"
+    else
+        echo "FAIL: no autonomous restore after the jammer stopped"
+        fails=$((fails+1))
+    fi
+    # ...and the receiver has to have followed that too.
+    last_rx=$(grep -F '"ev":"hopset.activate"' "$OUT/rx_adapt.log" 2>/dev/null |
+              tail -1 | sed -n 's/.*"mask":"0x\([0-9a-f]*\)".*/\1/p')
+    nch=$(( $(tr -cd ',' <<<"$CHANNELS" | wc -c) + 1 ))
+    full=$(printf '%x' $(( (1 << nch) - 1 )))
+    [ "${last_rx:-x}" = "$full" ] &&
+        echo "PASS: receiver ended on the full hopset again (0x$full)" ||
+        { echo "FAIL: receiver ended on 0x${last_rx:-none}, expected 0x$full"
+          fails=$((fails+1)); }
+
     # Every committed change moved exactly one channel: the autonomous path is
     # bound by the same structural limits a proposal is.
     if gm "$OUT/tx_adapt.log" | awk '{print $2}' |

tools/hopset_adapt_gif.py

@@ -0,0 +1,248 @@
+#!/usr/bin/env python3
+"""Animated adaptive hopset — 'the band heals', in the DEVOURER live-monitor
+style. The stationary half of a pair: hopset_herding_gif.py draws the same eight
+channels and the same opening act, and ends the other way.
+
+    tools/hopset_adapt_gif.py -o docs/img/hopset_adapt.gif
+
+The hop set and the interferer's channel are the ones from hop_pattern.gif, so
+the three figures of the section read as one story. A stationary occupant sits
+on ch 52: the receiver scores the dwells it
+loses and asks for the channel to be dropped, the transmitter commits the new
+schedule at a named future slot, and the channel stops being visited. Then the
+occupant goes away — and nothing would ever know, except that keyed recovery
+probes go back and look anyway. Three clean probes and the channel is back in
+the set.
+
+Deliberately scoped to a *stationary* occupant, which is what most interference
+is (a fixed AP, another link, a microwave) and the only case where exclusion is
+an unambiguous win. An interferer that follows the exclusions has a different
+ending; that is the companion figure, not this one. Needs Pillow.
+"""
+from __future__ import annotations
+
+import argparse
+import os
+import random
+import sys
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from monitor_style import (AMBER, CYAN, DIM, GRID, INK, OK, WARN, chrome, font,
+                           new_frame, save_gif)
+
+CHANS = [36, 40, 44, 48, 52, 56, 60, 64]   # the hop set of the section's first figure
+JAM = 4                    # the occupant sits on ch 52, where it sat there too
+
+NT = 64
+COMMIT_T = 16              # ch 52 excluded here
+STOP_T = 30                # the occupant goes away
+PROBES = (36, 44, 52)      # keyed recovery probes revisit the excluded channel
+RESTORE_T = 58             # three clean probes, and it is back in the set
+
+
+def schedule(rnd):
+    sched, last = [], -1
+    for t in range(NT):
+        if t in PROBES:
+            sched.append(JAM)
+            last = JAM
+            continue
+        excluded = COMMIT_T <= t < RESTORE_T
+        pool = [c for c in range(len(CHANS))
+                if c != last and not (excluded and c == JAM)]
+        c = rnd.choice(pool)
+        sched.append(c)
+        last = c
+    return sched
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument("-o", "--out", default="hopset_adapt.gif")
+    ap.add_argument("--hold", type=int, default=16)
+    ap.add_argument("--ms", type=int, default=110)
+    args = ap.parse_args()
+
+    sched = schedule(random.Random(0x265))
+    for t in (3, 9):   # make the opening act unmistakable
+        if sched[t - 1] != JAM and sched[t + 1] != JAM:
+            sched[t] = JAM
+
+    nC = len(CHANS)
+    padL, padT, padB = 60, 96, 62
+    cellw, gh = 11.0, 300
+    gw = int(NT * cellw)
+    panelW = 230
+    W = padL + gw + 24 + panelW
+    H = padT + gh + padB
+    cellh = gh / nC
+
+    def cell_xy(ti, ci):
+        return padL + ti * cellw, padT + (nC - 1 - ci) * cellh
+
+    def tx(t):
+        return padL + t * cellw
+
+    jam_y = cell_xy(0, JAM)[1]
+
+    imgs = []
+    for fi in range(NT + args.hold):
+        cursor = min(fi, NT - 1)
+        seen = min(cursor + 1, NT)
+        excluded = cursor >= COMMIT_T
+        stopped = cursor >= STOP_T
+        restored = cursor >= RESTORE_T
+        img, d = new_frame(W, H)
+        chrome(d, W, H, "ADAPTIVE HOPSET",
+               "a stationary occupant on ch 52 — the receiver scores what it "
+               "decodes, the transmitter commits, the channel stops being "
+               "visited", fi)
+
+        d.rectangle([padL, padT, padL + gw, padT + gh], outline=(0, 70, 80))
+        for ci, ch in enumerate(CHANS):
+            _, y = cell_xy(0, ci)
+            d.text((padL - 34, y + cellh / 2 - 7), f"{ch}", font=font(11),
+                   fill=WARN if (ci == JAM and not stopped) else DIM)
+            d.line([padL, y, padL + gw, y], fill=GRID)
+
+        # the occupant's span, and the excluded span laid over it
+        d.rectangle([padL, jam_y, tx(min(seen, STOP_T)), jam_y + cellh],
+                    fill=(40, 12, 12))
+        if excluded:
+            xr = tx(min(seen, RESTORE_T))
+            d.rectangle([tx(COMMIT_T), jam_y, xr, jam_y + cellh], fill=(16, 19, 26))
+            hx = tx(COMMIT_T) + 6
+            while hx < xr:
+                d.line([hx, jam_y + 6, hx - 6, jam_y + cellh - 6], fill=(30, 38, 50))
+                hx += 12
+            if cursor > COMMIT_T + 5:
+                lx, ly = tx(COMMIT_T) + 16, jam_y + cellh / 2 - 7
+                d.rectangle([lx - 5, ly - 3, lx + 92, ly + 15], fill=(16, 19, 26))
+                d.text((lx, ly), "not scheduled", font=font(11), fill=DIM)
+
+        # the trace
+        lost = 0
+        for ti in range(seen):
+            ci = sched[ti]
+            x, y = cell_xy(ti, ci)
+            probe = ti in PROBES
+            hit = ci == JAM and ti < STOP_T and not probe
+            if hit:
+                lost += 1
+            fade = 0.45 + 0.55 * (ti / max(1, cursor)) if ti < cursor else 1.0
+            col = AMBER if probe else (WARN if hit else CYAN)
+            d.rectangle([x + 2, y + 8, x + cellw - 2, y + cellh - 8],
+                        fill=tuple(int(c * fade) for c in col))
+            if hit:
+                d.line([x + 2, y + 8, x + cellw - 2, y + cellh - 8], fill=INK)
+                d.line([x + 2, y + cellh - 8, x + cellw - 2, y + 8], fill=INK)
+            if probe:  # has to read against the hatched, excluded row
+                d.rectangle([x + 1, y + 7, x + cellw - 1, y + cellh - 7],
+                            outline=AMBER, width=2)
+                d.line([x + cellw / 2, y + cellh - 6, x + cellw / 2,
+                        padT + gh + 4], fill=(90, 78, 30))
+            if ti == cursor and fi < NT:
+                d.rectangle([x + 1, y + 7, x + cellw - 1, y + cellh - 7],
+                            outline=INK, width=2)
+            if ti:
+                x0, y0 = cell_xy(ti - 1, sched[ti - 1])
+                d.line([x0 + cellw / 2, y0 + cellh / 2,
+                        x + cellw / 2, y + cellh / 2], fill=(30, 44, 60))
+
+        # the moments, marked on the axis
+        if excluded:
+            d.line([tx(COMMIT_T), padT, tx(COMMIT_T), padT + gh], fill=AMBER, width=2)
+            d.text((tx(COMMIT_T) - 30, 76), "exclude ch 52", font=font(11), fill=AMBER)
+        if stopped:
+            mx = tx(STOP_T)
+            yy = padT
+            while yy < padT + gh:
+                d.line([mx, yy, mx, yy + 5], fill=DIM)
+                yy += 10
+            d.text((mx - 34, 76), "occupant stops", font=font(11), fill=DIM)
+        if restored:
+            d.line([tx(RESTORE_T), padT, tx(RESTORE_T), padT + gh], fill=OK, width=2)
+            d.text((tx(RESTORE_T) - 52, 76), "ch 52 restored", font=font(11), fill=OK)
+
+        d.text((padL, padT + gh + 8), "time →   (each cell = one dwell)",
+               font=font(11), fill=DIM)
+        if cursor >= PROBES[0]:
+            d.text((tx(PROBES[0]) - 12, padT + gh + 8), "↑ keyed recovery probes",
+                   font=font(11), fill=AMBER)
+
+        by = padT + gh + 30
+
+        def bracket(t0, t1, label, col):
+            x0, x1 = tx(t0), tx(t1)
+            d.line([x0, by, x1, by], fill=col)
+            d.line([x0, by - 4, x0, by], fill=col)
+            d.line([x1, by - 4, x1, by], fill=col)
+            d.text(((x0 + x1) / 2 - 3.2 * len(label), by + 5), label,
+                   font=font(11), fill=col)
+
+        bracket(0, COMMIT_T, "JAMMED", WARN if not excluded else DIM)
+        if excluded:
+            bracket(COMMIT_T, min(seen, RESTORE_T), "ADAPTED",
+                    OK if not restored else DIM)
+        if restored:
+            bracket(RESTORE_T, seen, "WHOLE AGAIN", OK)
+
+        # readout
+        x0 = padL + gw + 22
+        d.text((x0, padT - 2), "LIVE READOUT", font=font(12), fill=CYAN)
+        y = padT + 24
+
+        def line(lbl, val, c=INK):
+            nonlocal y
+            d.text((x0, y), lbl, font=font(11), fill=DIM)
+            d.text((x0 + 100, y - 3), val, font=font(15, True), fill=c)
+            y += 30
+
+        gen = 3 if restored else (2 if excluded else 1)
+        out = excluded and not restored
+        mask = "".join("0" if (out and ci == JAM) else "1" for ci in range(nC))
+        done = sum(1 for p in PROBES if cursor >= p)
+        line("generation", f"{gen}", AMBER if excluded else INK)
+        line("hop set", mask, OK)
+        line("active", f"{nC - 1 if out else nC}/{nC}", AMBER if out else OK)
+        line("probes", f"{done}/{len(PROBES)}" if excluded else "—",
+             AMBER if 0 < done < len(PROBES) else (OK if done else DIM))
+        line("dwells lost", f"{lost}", WARN if lost else OK)
+
+        if restored:
+            st, sc = "RESTORED", OK
+        elif stopped:
+            st, sc = "PROBING", AMBER
+        elif excluded:
+            st, sc = "ADAPTED", OK
+        else:
+            st, sc = "LEARNING", WARN
+        d.rectangle([x0, y, x0 + 200, y + 30], outline=sc, width=2)
+        d.ellipse([x0 + 8, y + 10, x0 + 18, y + 20], fill=sc)
+        d.text((x0 + 28, y + 6), st, font=font(14, True), fill=sc)
+        y += 44
+
+        if st == "LEARNING":
+            cap = ("ch 52 keeps losing", "dwells — the receiver",
+                   "asks for it to go.")
+        elif st == "ADAPTED":
+            cap = ("ch 52 is out, committed", "at a named slot so both",
+                   "ends swap together.")
+        elif st == "PROBING":
+            cap = ("the occupant left, and", "nothing would ever know —",
+                   "so keyed probes go back", "and look anyway.")
+        else:
+            cap = ("three clean probes and", "ch 52 is back in the set.",
+                   "exclusion is evidence,", "not a one-way ratchet.")
+        for ln in cap:
+            d.text((x0, y), ln, font=font(11), fill=INK)
+            y += 15
+
+        imgs.append(img)
+
+    save_gif(imgs, args.out, ms=args.ms, colors=48)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())