Skip to content

test(coverage): run dead + DB_REGISTER groups (lock_deadlock 0.7->66%, env_register 0->55%) #76

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gburd merged 1 commit into from
Jul 29, 2026
Merged

test(coverage): run dead + DB_REGISTER groups (lock_deadlock 0.7->66%, env_register 0->55%) #76

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 24 additions & 2 deletions test/coverage/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ Knobs (all optional env vars):
| `COV_TIMEOUT`| `2400` | seconds ceiling for the test run |
| `COV_REP` | `0` | set `1` to also run the replication (rep/repmgr) tests |
| `COV_XA_UPG` | `1` | run the XA + on-disk-upgrade drivers (fast, non-hanging) |
| `COV_DEAD_REG` | `1` | run the deadlock-detector + DB_REGISTER multi-process tests |

Set `COV_REP=1` to add the replication suite (biggest cold surface: rep/ +
repmgr/ ~= 12.4k lines at 0.8%). It moves them to ~56% line / ~42% branch. Those
Expand Down Expand Up @@ -131,6 +132,26 @@ Tcl tests now drive the whole surface (both in the default `COV_TESTS`):
binding exposes only `-clear`, not `-all`); `repmgr_stat.c` needs a live
repmgr transport (`repmgr009`+, `COV_REP=1`) for its send-path counters.

`COV_DEAD_REG` (on by default) runs the two multi-process suites that reach the
deadlock detector and the process-registry crash/recovery path. Like `COV_REP`,
they run driver-per-test with a per-test timeout and orphan-worker cleanup
(they reset `TESTDIR` and each spawns child `tclsh` via `wrap.tcl`, so a hung
worker must not wedge the whole run):

- **Deadlock detector** — the `dead` group (`dead001`–`dead006`) spawns
`ddscript.tcl` workers that grab locks in a ring/clump cycle; the detector
(`src/lock/lock_deadlock.c`, `__lock_detect`) then picks a victim. `dead002`/
`dead003` set `-lock_detect` so detection runs **in-process** in each worker
(which flushes its own `.gcda`); `dead001`/`dead004`/`dead005` use the
standalone `db_deadlock` utility. Proc counts are trimmed to `{2 4}` (`{4}`
for `dead005`) — the full `{2 4 10}` matrix adds only minutes, no new lines.
Lift: `lock_deadlock.c` **0.7%→~66%**.
- **DB_REGISTER** — `env012` (with `env007`) uses `envscript.tcl` to open the
env with `DB_REGISTER`, "crash" a process (kill without close), then reopen
with `-recover`/`-failchk` so the survivor detects the dead slot
(`src/env/env_register.c`, `__envreg_register`/`__envreg_isalive`) and runs
recovery. Lift: `env_register.c` **0%→~55%**.

Outputs land in `build_unix/` (gitignored):

- `coverage-summary.txt` — the line/branch/function totals
Expand Down Expand Up @@ -265,11 +286,12 @@ why they lag the single-process access-method tests:
| 0.0 | 0.0 | 701 | 556 | `repmgr/repmgr_msg.c` | replication manager |
| 0.0 | 0.0 | 556 | 656 | `rep/rep_elect.c` | replication elections |
| 0.0 | 0.0 | 541 | 460 | `rep/rep_automsg.c` | replication |
| 0.0 | 0.0 | 457 | 674 | `lock/lock_deadlock.c` | deadlock detection |
| 0.0 | 0.0 | 431 | 470 | `sequence/sequence.c` | sequences |
| 0.0 | 0.0 | 394 | 398 | `xa/xa.c` | XA transactions |
| 5.4 | 2.0 | 445 | 306 | `qam/qam_files.c` | queue extent files |
| 30.1 | 17.4 | 1397 | 1702 | `btree/bt_compact.c` | btree compaction (was 0%) |
| 54.7 | 43.1 | 223 | 174 | `env/env_register.c` | DB_REGISTER / failchk (was 0%, COV_DEAD_REG) |
| 57.1 | ~ | 394 | 398 | `xa/xa.c` | XA transactions (was 0%, COV_XA_UPG) |
| 65.9 | 43.3 | 457 | 674 | `lock/lock_deadlock.c` | deadlock detection (was 0.7%, COV_DEAD_REG) |

Next-highest-leverage additions (future work): a **replication smoke test**
(needs the multi-process rep harness — `rep0NN` / `repmgr0NN` exist but each
Expand Down
49 changes: 49 additions & 0 deletions test/coverage/run_coverage.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,55 @@ if [ "${COV_XA_UPG:-1}" = 1 ]; then
echo " .gcda files after xa/upg: $(find . -name '*.gcda' | wc -l)"
fi

# --- optional deadlock-detector + DB_REGISTER drivers (COV_DEAD_REG=1) -------
# The deadlock DETECTOR (lock/lock_deadlock.c, __lock_detect) and the
# process-registration + failchk-on-crash path (env/env_register.c,
# __envreg_register/isalive) are both multi-process: the `dead` group and
# env012 spawn worker tclsh via wrap.tcl (ddscript.tcl / envscript.tcl) that
# contend for locks in a cycle (detector picks a victim) or open the env with
# DB_REGISTER, crash, and let a survivor recover. They cannot run in the main
# single-tclsh COV_TESTS loop (they reset TESTDIR and a hung worker would wedge
# the whole run), so -- like COV_REP -- they run driver-per-test with a
# per-test timeout and orphan-worker cleanup. Lifts lock_deadlock.c 0.7%->~66%
# and env_register.c 0%->~55%. proc counts are trimmed (default {2 4}) so each
# test finishes well inside the timeout; the full {2 4 10} matrix adds no new
# coverage, only minutes. Set COV_DEAD_REG=0 to skip.
if [ "${COV_DEAD_REG:-1}" = 1 ]; then
echo "== run deadlock + DB_REGISTER drivers (COV_DEAD_REG=1) =="
: "${COV_DEAD_REG_TIMEOUT:=300}"
# Each entry is "name:tcl-call"; trimmed proc counts keep each run bounded.
# A bash array so the spaces inside a call ({2 4}) survive word-splitting.
cov_dead_reg_tests=(
"dead001:dead001 {2 4}"
"dead002:dead002 {2 4}"
"dead003:dead003 {2 4}"
"dead004:dead004"
"dead005:dead005 {4}"
"dead006:dead006 {2 4}"
"env007:env007"
"env012:env012"
)
dregtcl="$bld/.cov-deadreg-one.tcl"
for spec in "${cov_dead_reg_tests[@]}"; do
name="${spec%%:*}"; call="${spec#*:}"
printf 'source ../test/tcl/test.tcl\nif {[catch {%s} r]} { puts "FAIL %s: $r"; exit 3 }\nputs "PASS %s"\n' \
"$call" "$name" "$name" > "$dregtcl"
timeout "$COV_DEAD_REG_TIMEOUT" "$TCLBIN" "$dregtcl" >/tmp/cov-dreg-$name.log 2>&1
rc=$?
# kill orphan workers a hung/killed test may have left behind
pkill -f 'wrap.tcl' 2>/dev/null || true
pkill -f 'ddscript' 2>/dev/null || true
pkill -f 'envscript' 2>/dev/null || true
pkill -f 'db_deadlock' 2>/dev/null || true
if [ $rc -eq 124 ]; then echo "HANG $name"
elif [ $rc -eq 0 ] && grep -q "^PASS $name" /tmp/cov-dreg-$name.log; then echo "PASS $name"
else echo "FAIL $name (rc=$rc)"; fi
find TESTDIR -mindepth 1 -delete 2>/dev/null || true
done
rm -f "$dregtcl"
echo " .gcda files after dead/register: $(find . -name '*.gcda' | wc -l)"
fi

# --- aggregate ---------------------------------------------------------------
echo "== capture (lcov) =="
# NOTE: capture from .libs (not .) -- libtool double-compiles, and only the
Expand Down
Loading