Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions test/coverage/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,44 @@ state) -- states not reachable through the public API without fault injection.
(`run_secmethod` / `run_secenv`, which run a full access-method test *twice*
under encryption, add only ~1-2 pp over sec001+sec002 -- not worth ~doubling
the subset runtime, so they are left out.)
## Lock configuration + region growth + the varint codec (`lock007`, `test143:btree`)

Three cold files, all added to the default `COV_TESTS`:

- **`lock/lock_method.c`** (was ~22%): the `DB_ENV` lock-subsystem config
setters/getters. Cold because the rest of the suite runs with default lock
sizing and detection. **`lock007`** (COV group `lock`) sets every knob
before open — `set_lk_max_locks`/`lockers`/`objects`, `set_lk_partitions`,
`set_lk_tablesize`, the `DB_MEM_LOCK`/`LOCKER`/`LOCKOBJECT` init counts, and
`set_lk_detect` for all nine deadlock-detection policies (plus a rejected
bogus policy) — then reads them back through the getters. Lift:
**lock_method.c 10%→~38%** (isolated subset; both the `ENV_ILLEGAL_AFTER_OPEN`
set path and the `LOCKING_ON` region get path are hit).
- **`lock/lock_alloc.incl`** (was ~27%): the lock-region object/locker/lock
allocator, `#include`d into `lock_region.c`. Cold because default runs never
exhaust the initial free lists. `lock007`'s Part c allocates 200 lockers ×
40 distinct read-lock objects, exhausting the free lists and forcing the
region-growth loop (`__env_alloc` + free-list refill). Lift:
**lock_alloc.incl 14%→~82%**.
- **`common/db_compint.c`** (was ~24%): the compressed-integer (varint) codec
used by btree compression (`bt_compress.c`). Cold in the subset because no
default test opens a `-compress` btree. **`test143:btree`** (COV group
`test`) stores records whose *data sizes* span the codec's 1-byte, 2-byte
and 3-byte size classes (1 B … 100 KB) into a `-compress` btree and reads
them back, driving `__db_compress_int` on write and `__db_decompress_int32`
on read, plus the key prefix/suffix length compression. Lift:
**db_compint.c 0%→~16%** in isolation.

**Codec ceiling / PBT tier.** db_compint cannot reach 100% from Tcl: btree
compression only ever marshals **32-bit** lengths, so the 64-bit
`__db_decompress_int` (~130 of the file's 324 lines) and the 4–9 byte size
classes are unreachable from *any* Tcl workload. Those paths are exhaustively
property-tested by [`test/pbt/pbt_compint.c`](../pbt/pbt_compint.c) — a
separate property-based-testing tier that round-trips random 64-bit values
through `__db_compress_int`/`__db_decompress_int` across every size class,
and is **not** part of this Tcl coverage subset (hence db_compint shows
"cold" in the ranking even though the codec is well covered overall). The
tcl-reachable ceiling is ~24%.

Outputs land in `build_unix/` (gitignored):

Expand Down
24 changes: 23 additions & 1 deletion test/coverage/run_coverage.sh
Original file line number Diff line number Diff line change
Expand Up @@ -83,9 +83,31 @@ bld="$root/build_unix"
# mt19937db.c 0->~96, hmac.c 10->~91, rijndael-alg-fst.c 19->~84,
# crypto.c 51->~78, aes_method.c 28->~44, rijndael-api-fst.c 6->~30 (capped:
# BDB only uses AES MODE_CBC, so the ECB/CFB1/pad* halves are dead code here).
# lock007 + test143:btree light up three cold lock/codec files:
# lock/lock_method.c -- the DB_ENV lock-config setters/getters. Cold
# because the rest of the suite runs with default lock sizing. lock007
# sets every knob (set_lk_max_locks/lockers/objects, set_lk_partitions,
# set_lk_tablesize, the DB_MEM_LOCK/LOCKER/LOCKOBJECT init counts, and
# set_lk_detect for all nine deadlock-detection policies) and reads them
# back through the getters (10->~38%).
# lock/lock_alloc.incl -- the lock-region object/locker/lock allocator
# (included into lock_region.c). Cold because default runs never exhaust
# the initial free lists. lock007's many-locker workload (200 lockers x
# 40 distinct objects) forces the region-growth loop (14->~82%).
# common/db_compint.c -- the compressed-integer (varint) codec used by
# btree compression (bt_compress.c). Cold because no default-subset test
# opens a -compress btree. test143:btree stores records whose data sizes
# span the codec's 1/2/3-byte size classes and reads them back, driving
# __db_compress_int / __db_decompress_int32 (0->~16%, the full
# tcl-reachable ceiling). NOTE: the 64-bit __db_decompress_int and the
# 4-9 byte size classes are unreachable from any Tcl workload (btree
# compression only ever marshals 32-bit lengths); they are exhaustively
# property-tested by test/pbt/pbt_compint.c, a separate tier not in this
# subset -- so db_compint's ceiling here is ~24%, not 100%.
: "${COV_TESTS:=lock001: txn001: ssi001: ssi002: recd001:btree \
recd002:btree recd016:btree env007: \
btree/test001 btree/test111 \
lock007: \
btree/test001 btree/test111 test143:btree \
hash/test001 hash/test006 hash/test010 hash/test025 hash/test077 \
queue/test001 queue/test007 queue/test025 \
recno/test001 recno/test006 recno/test024 recno/test025 \
Expand Down
134 changes: 134 additions & 0 deletions test/tcl/lock007.tcl
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# See the file LICENSE for redistribution information.
#
# Copyright (c) 2024 Oracle and/or its affiliates. All rights reserved.
#
# $Id$
#
# TEST lock007
# TEST Lock subsystem configuration coverage.
# TEST Exercises every DB_ENV lock configuration setter (set_lk_max_locks,
# TEST set_lk_max_lockers, set_lk_max_objects, set_lk_partitions,
# TEST set_lk_tablesize, the DB_MEM_LOCK/LOCKER/LOCKOBJECT init counts, and
# TEST set_lk_detect for every deadlock-detection policy) and reads the
# TEST values back through the corresponding getters. Then runs a workload
# TEST that allocates many locks/lockers/objects on distinct objects so the
# TEST lock region grows past its initial free lists (src/lock/lock_alloc.incl).
proc lock007 { } {
source ./include.tcl

puts "Lock007: Lock configuration setters/getters + region growth"

env_cleanup $testdir

# Part a: set every lock config knob before open, then open a locking
# env and read the values back. The setters store into the DB_ENV
# (ENV_ILLEGAL_AFTER_OPEN path); the getters read the live region
# (LOCKING_ON path) after open, so both halves of each get_/set_ are hit.
puts "\tLock007.a: set every lock config knob, read back via getters"
set maxlocks 5000
set maxlockers 2000
set maxobjects 5000
set partitions 3
set tablesize 509
set initlocks 1500
set initlockers 600
set initobjects 1500

set eflags "-create -lock -home $testdir -mode 0644 \
-lock_max_locks $maxlocks -lock_max_lockers $maxlockers \
-lock_max_objects $maxobjects -lock_partitions $partitions \
-lock_tablesize $tablesize -lock_locks $initlocks \
-lock_lockers $initlockers -lock_objects $initobjects \
-lock_detect default"
set env [eval {berkdb_env} $eflags]
error_check_good env [is_valid_env $env] TRUE

error_check_good get_max_locks \
[$env get_lk_max_locks] $maxlocks
error_check_good get_max_lockers \
[$env get_lk_max_lockers] $maxlockers
error_check_good get_max_objects \
[$env get_lk_max_objects] $maxobjects
error_check_good get_partitions \
[$env get_lk_partitions] $partitions
# get_lk_detect: DB_LOCK_DEFAULT was requested; with no detector
# running the region stays at the requested (or NORUN) value. Just
# confirm the getter succeeds and returns a valid policy string.
set det [$env get_lk_detect]
error_check_bad get_detect [string length $det] 0

error_check_good env_close_a [$env close] 0

# Part b: validate the set_lk_detect policy switch -- every documented
# policy must be accepted, and a bogus one must be rejected. Do this
# on a fresh lock env for each policy (set on an already-open locking
# env exercises the LOCKING_ON region path in __lock_set_lk_detect).
puts "\tLock007.b: set_lk_detect accepts every policy, rejects garbage"
foreach pol {default expire maxlocks maxwrites minlocks minwrites \
oldest youngest random} {
env_cleanup $testdir
set e [berkdb_env -create -lock -home $testdir \
-lock_detect $pol]
error_check_good detect_$pol [is_valid_env $e] TRUE
error_check_good detect_close_$pol [$e close] 0
}
# A bogus policy string is rejected by the Tcl binding before it ever
# reaches set_lk_detect.
env_cleanup $testdir
set ret [catch {berkdb_env -create -lock -home $testdir \
-lock_detect boguspolicy} res]
error_check_good detect_bogus_rejected $ret 1

# Part c: allocate many locks/lockers/objects to grow the region.
# Each locker takes read locks on many distinct objects; with hundreds
# of lockers and thousands of objects this exhausts the initial free
# lists and drives the region-growth loop in lock_alloc.incl.
puts "\tLock007.c: many-locker workload to grow the lock region"
env_cleanup $testdir
set nlockers 200
set nobjs 40
set env [berkdb_env -create -lock -home $testdir \
-lock_max_locks 40000 -lock_max_lockers 4000 \
-lock_max_objects 40000 -lock_partitions 4]
error_check_good env_c [is_valid_env $env] TRUE

set lockers {}
set locks {}
for {set i 0} {$i < $nlockers} {incr i} {
set locker [$env lock_id]
lappend lockers $locker
for {set j 0} {$j < $nobjs} {incr j} {
# Distinct object per (locker,obj); read locks never
# conflict so nothing blocks, but each is a fresh
# lock+object allocation.
set obj "obj_${i}_${j}"
set lockp [$env lock_get read $locker $obj]
error_check_good lock_get_c [is_substr $lockp $env] 1
lappend locks $lockp
}
}

# Confirm we actually allocated a lot of locks.
set nlocks_now [lock007_stat $env "Current number of locks"]
error_check_good grew_locks [expr {$nlocks_now >= $nlockers * $nobjs}] 1

# Release everything.
foreach lockp $locks {
error_check_good lock_put [$lockp put] 0
}
foreach locker $lockers {
error_check_good free_id [$env lock_id_free $locker] 0
}
error_check_good env_close_c [$env close] 0
}

# Pull a single integer statistic out of "$env lock_stat" by its label.
proc lock007_stat { env label } {
set stat [$env lock_stat]
foreach pair $stat {
if { [is_substr [lindex $pair 0] $label] != 0 } {
return [lindex $pair 1]
}
}
return -1
}
89 changes: 89 additions & 0 deletions test/tcl/test143.tcl
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# See the file LICENSE for redistribution information.
#
# Copyright (c) 2024 Oracle and/or its affiliates. All rights reserved.
#
# $Id$
#
# TEST test143
# TEST Compressed-integer codec coverage (src/common/db_compint.c).
# TEST Stores records into a -compress btree whose data sizes span the
# TEST compressed-integer size classes the codec uses to marshal record
# TEST lengths: 1-byte (size <= 127), 2-byte (size <= 16511) and 3-byte
# TEST (size <= ~2M). __bam_compress_marshal_data() runs __db_compress_int
# TEST over data->size on write and __db_decompress_int32 on read, so a
# TEST round-trip over these sizes drives the codec's 1/2/3-byte encode and
# TEST decode paths. Keys share long common prefixes so the prefix/suffix
# TEST length compression is exercised too. (The full 4-9 byte size classes
# TEST are unit-tested exhaustively by test/pbt/pbt_compint.c, a separate
# TEST property-based tier not run under the Tcl coverage subset.)
proc test143 { method {tnum "143"} args } {
source ./include.tcl

# Compression is btree-only.
if { [is_btree $method] == 0 } {
puts "Test$tnum skipping for method $method (btree only)."
return
}

set args [convert_args $method $args]
set omethod [convert_method $method]

# Work in an env so we get a page/mpool big enough for overflow data.
env_cleanup $testdir
set env [berkdb_env -create -home $testdir -mode 0644 -cachesize {0 4194304 1}]
error_check_good env [is_valid_env $env] TRUE

set testfile test$tnum.db
puts "Test$tnum: $method compressed-integer codec (varying record sizes)"

set db [eval {berkdb_open -create -env $env -mode 0644} \
$args {-compress} $omethod $testfile]
error_check_good dbopen [is_valid_db $db] TRUE

# Data sizes chosen to land in each compressed-int size class for
# data->size: 1 (1-byte), 100 (1-byte boundary), 200 (2-byte),
# 16000 (2-byte near max), 20000 (3-byte), 100000 (3-byte).
set sizes {1 50 100 127 128 200 5000 16000 16511 16512 20000 100000}

puts "\tTest$tnum.a: put records spanning codec size classes"
set n 0
foreach sz $sizes {
# Long shared key prefix so key prefix/suffix lengths compress.
set key [format "commonkeyprefix_%08d" $n]
set data [repeat "x" $sz]
set ret [eval {$db put} {$key $data}]
error_check_good put_$n $ret 0
set expect($key) $data
incr n
}

puts "\tTest$tnum.b: read back and verify (decompress path)"
foreach key [array names expect] {
set ret [$db get $key]
error_check_good get_$key [llength $ret] 1
set pair [lindex $ret 0]
error_check_good key_$key [lindex $pair 0] $key
error_check_good len_$key \
[string length [lindex $pair 1]] [string length $expect($key)]
error_check_good data_$key [lindex $pair 1] $expect($key)
}

# Full cursor scan forces decompression of every compressed page.
puts "\tTest$tnum.c: cursor scan of all compressed records"
set dbc [$db cursor]
error_check_good cursor [is_valid_cursor $dbc $db] TRUE
set count 0
for {set ret [$dbc get -first]} {[llength $ret] > 0} \
{set ret [$dbc get -next]} {
set pair [lindex $ret 0]
set k [lindex $pair 0]
error_check_good scan_$k \
[string length [lindex $pair 1]] [string length $expect($k)]
incr count
}
error_check_good scan_count $count [llength $sizes]
error_check_good dbc_close [$dbc close] 0

error_check_good db_close [$db close] 0
error_check_good env_close [$env close] 0
}
5 changes: 3 additions & 2 deletions test/tcl/testparams.tcl
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ set test_names(fop) [list fop001 fop002 fop003 fop004 fop005 fop006 \
set test_names(init) [list rep029 rep030 rep031 rep033 rep037 rep038 rep039\
rep055 rep060 rep061 rep062 rep070 rep072 rep084 rep085 rep086 rep087 \
rep089 rep098]
set test_names(lock) [list lock001 lock002 lock003 lock004 lock005 lock006]
set test_names(lock) [list lock001 lock002 lock003 lock004 lock005 lock006 \
lock007]
set test_names(log) [list log001 log002 log003 log004 log005 log006 \
log007 log008 log009]
set test_names(logverify) [list logverify001 logverify002]
Expand Down Expand Up @@ -99,7 +100,7 @@ set test_names(test) [list test001 test002 test003 test004 test005 \
test109 test110 test111 test112 test113 test114 test115 test116 test117 \
test119 test120 test121 test122 test123 test124 test125 test126 test127 \
test128 test129 test130 test131 test132 test133 test134 test135 test136 \
test137 test138 test139 test140 test141 test142]
test137 test138 test139 test140 test141 test142 test143]

set test_names(txn) [list txn001 txn002 txn003 txn004 txn005 txn006 \
txn007 txn008 txn009 txn010 txn011 txn012 txn013 txn014]
Expand Down
Loading