Commit aa16aa2
authored
feat(restheart-mongo): keploy compat lane sample + Java line coverage gate (#134)
* feat(restheart-mongo): keploy compat lane sample (scaffold)
Mirrors the doccano-django sample shape: the sample owns
orchestration (compose / bootstrap / traffic / coverage), keploy
CI lanes consume it as a thin wrapper.
This is a SCAFFOLD — the full traffic loop driven by the existing
keploy/enterprise lane (`compat_trigger_record_traffic` in
.ci/scripts/restheart-linux.sh, ~600 lines covering CRUD on
/<db>/<coll> + GraphQL + files + ACL + users + bulk + aggregations)
needs to be ported into flow.sh::restheart_record_traffic in a
follow-up. The current loop is deliberately minimal (CRUD on a
seed collection) which is enough to prove the sample boots
end-to-end without keploy.
Layout:
Dockerfile — pin to softinstigate/restheart:9.2.1
docker-compose.yml — mongo:7 + restheart:9.2.1, env-driven
flow.sh — bootstrap | record-traffic | coverage | list-routes
keploy.yml.template — globalNoise for _etag/_oid/lastModified/Date
README.md — handoff + status notes
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
* feat(restheart-mongo): port full RESTHeart REST surface
Replace the minimal record-traffic stub with the complete loop that
the keploy compat lane needs to gate. flow.sh::restheart_record_traffic
now drives the full RESTHeart 9.x surface end-to-end against bare
RESTHeart, and restheart_list_routes enumerates every (method, route)
tuple it fires so coverage stays in lockstep.
Covered surfaces:
- CRUD on /<db>/<coll> + /<db>/<coll>/<docid> (HAL, _size, _meta,
_indexes, ETag conditional flow, writeMode insert/update/upsert,
$-operator PATCH variety)
- Aggregations via _meta.aggrs with avars variable interpolation
(scalars / arrays / nested / missing / malformed)
- Bulk writes (POST array body, filter PATCH, filter DELETE,
larger 25-doc batches, mixed valid/invalid)
- GraphQL apps (gql-apps registration, query / mutation / fragment /
alias / multi-op, BSON scalar coercion on outputs and inputs,
introspection, error paths)
- Files / GridFS (.files buckets, multipart upload, binary download
with Range requests, metadata fetch, delete)
- ACL rules (predicate evaluator across method / path-prefix /
qparams-* / bson-request-* / equals[%U,...] / in[%h,...]) plus
the mongo permission interceptors (readFilter, writeFilter,
projectResponse, mergeRequest, filterOperatorsBlacklist,
propertiesBlacklist, allowBulk*)
- Users (/users) with the userPwdHasher bcrypt interceptor; reader /
writer roles authenticating via Basic + Bearer; wrong-password deny
- Sessions / multi-doc transactions (/_sessions/<id>/_txns/<txnid>)
with commit and abort branches
- Auth services (/token form grants, JWT, Auth-Token, Digest,
OAuth metadata under /.well-known/oauth-*)
- Diagnostics (/ping, /metrics in json/prometheus/openmetrics, per-db
and per-coll, /health/db, OPTIONS preflight, gzip request encoding,
Accept-Encoding negotiation)
- MongoMountResolver (multiple databases, encoded collection names,
root /_size and /_meta, trailing-slash and double-slash variants)
restheart_bootstrap now PUTs every collection record-traffic touches.
README.md describes the sample as a complete keploy compat lane
sample and lists every surface it exercises.
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
* ci(restheart-mongo): add per-sample coverage gate workflow
Adds .github/workflows/restheart-mongo.yml plus the helper
.github/workflows/scripts/run-and-measure.sh, modeled on the
doccano-django sample's coverage gate.
* paths-scoped trigger: pull_request and push-to-main both filter
on `restheart-mongo/**` and `.github/workflows/restheart-mongo.yml`,
so changes to other samples in this repo do not trigger this
workflow (and vice versa).
* Three jobs: build-coverage (PR HEAD), release-coverage (PR base
with first-PR bootstrap escape hatch), and coverage-gate that
fails the PR if coverage drops more than COVERAGE_THRESHOLD
percentage points (default 1.0pp, override via repo variable
RESTHEART_COVERAGE_THRESHOLD).
* Helper script brings the sample up via its own
docker-compose.yml, waits for the RESTHeart listener (treating
both 200 and 401 as ready since `/` requires auth), runs
flow.sh bootstrap → record-traffic → coverage, and emits the
parsed percentage onto $GITHUB_OUTPUT for the gate job.
* Isolated from the enterprise lane: the enterprise PR pipeline
(.woodpecker/restheart-linux.yml) calls `flow.sh coverage` only
informationally and does not gate on it. The gate lives only
here, on the sample repo, so coverage regressions surface on
PRs that touch this sample without coupling enterprise CI to
the route table.
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
* ci(restheart-mongo): dump container logs on app wait timeout
build-coverage on PR #134 hung 8 min when restheart never bound
on port 8080 (last_code=000). The helper script silently looped
through both the wait and flow.sh bootstrap timers, then the gate
job aborted without surfacing why restheart didn't start. Adding
an explicit fail-fast + docker logs dump after the 240s wait so a
future failure surfaces the restheart Java traceback (or the
mongo connection error, or whatever else).
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
* feat(restheart-mongo): real Java line coverage via JaCoCo overlay
Replaces the prior API-route-surface "coverage" (counting fired
routes / curated route table) with actual JaCoCo line coverage
of the RESTHeart 9.x JVM under traffic.
Architecture:
- `Dockerfile.coverage` is a multi-stage build: stage 1 (alpine)
fetches JaCoCo 0.8.13 (jacocoagent.jar + jacococli.jar), stage
2 layers them into the upstream restheart image (which is
distroless — no shell, no curl, so jars must be pulled in a
builder stage and COPY'd over).
- `docker-compose.coverage.yml` is an OVERLAY: applied via `-f
docker-compose.yml -f docker-compose.coverage.yml`. It sets
JAVA_TOOL_OPTIONS=-javaagent:.../jacocoagent.jar=output=tcpserver,...
so JaCoCo attaches at JVM start and listens on port 6300.
The base `Dockerfile` and `docker-compose.yml` are untouched,
so keploy/integrations and keploy/enterprise CI lanes consume
the base compose and pay zero JaCoCo cost (the agent rewrites
bytecode at class-load, adding ~5-10% per-call overhead that
would slow record/replay).
- `flow.sh::restheart_report_coverage` shells into a one-off
coverage container to dump execution data via JaCoCo TCP and
render an XML report against /opt/restheart/restheart.jar.
When called against the base image (no overlay) it prints
"INFO: ... uninstrumented" and exits 0 so enterprise lanes'
`flow.sh coverage || true` informational calls keep working.
Also fixes a pre-existing config bug in the base
docker-compose.yml's RHO env var: the override syntax uses ';'
as a key->value separator (the upstream image's default
RHO uses ';'); the prior YAML-folded version used ',' which
RESTHeart parsed as part of the connection-string value, leading
RESTHeart to ignore the override and bind /http-listener/host to
its localhost default — making the HTTP listener unreachable
from the host port mapping. The base compose now uses ';' AND
explicitly overrides /http-listener/host -> "0.0.0.0".
Removed:
- `restheart_list_routes` (curated route table denominator).
- `restheart_list_recorded_routes` (keploy-tests / fired-routes
reader).
- The legacy route-surface `restheart_report_coverage` body.
- `list-routes` subcommand.
Validated locally: helper produced `coverage=52.3` to
GITHUB_OUTPUT against a clean stack (1663/3182 lines covered
in restheart.jar; INSTRUCTION coverage 50.8%).
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
* ci(restheart-mongo): drop trailing prose from sticky comment
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
* docs(restheart-mongo): document coverage overlay; drop list-routes/FIRED_ROUTES refs
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
* fix(restheart-mongo): pin JWT key + correct globalNoise format for deterministic replay
Two record/replay determinism fixes the full keploy/enterprise compat
matrix needs to go green:
1) RHO `/jwtConfigProvider/key` pinned to a fixed string. Default is
`key: null` which makes RESTHeart auto-generate a random HS256 secret
per container start. Recorded JWT bearers carry an HS256 signature
over the payload using that secret, so a fresh-container replay phase
rejects the recorded bearer with 401 even though --freezeTime keeps
`exp` valid. Pinning the secret keeps the bearer signature verifiable
across record→replay container restarts.
2) keploy.yml.template rewritten to use NESTED globalNoise format
(`body: { field: [] }`) instead of flat dotted keys (`body.field: []`).
Keploy's matcher reads `globalNoise.global` as map[section][field]regex
and treats dotted keys as literal outer keys, never matching the body
section. Verified by walking pkg/matcher/http/match.go and the
employee-manager sample's commented example. Fields added:
header.{Date, Content-Length, Auth-Token},
body.{_etag, _oid, _id, lastModified, client_ip, latencyMs, access_token}.
Validated locally with all three matrix cells (record-stable-replay-pr,
record-pr-replay-pr, record-pr-replay-stable) — each reaches 296/296
PASSED with these two changes plus the lane-side --port and --freezeTime
flags already in keploy/enterprise#1889.
* fix(restheart-mongo): bound JVM heap to prevent OOM on shared CI runners
RESTHeart 9.x's default JVM uses MaxRAMPercentage=25, which on a
typical Woodpecker runner cgroup (~8GB) lands ~2GB heap. With the
keploy/enterprise compat matrix running 3 restheart cells concurrent
alongside parse-server / umami / doccano cells (12 lane-cells total
on one runner), aggregate memory pressure triggers cgroup OOM and
the kernel SIGKILLs lighter processes (bash, curl) mid-bootstrap.
Observed in keploy/enterprise pipeline 3721/26 (record-stable-replay-pr):
RESTHeart came up cleanly (`127.0.0.1:18070 is now reachable`), then
`bash flow.sh bootstrap 240` got SIGKILL'd (exit 137) before any
seed PUTs landed. Same shape across all 3 restheart cells in 3721.
Cap heap at -Xmx512m (start at -Xms128m). Local single-cell
record/replay peaks at ~280MB heap, so 512m is comfortable headroom
without bleeding into neighbours. With the bound, each cell's
footprint stays under ~700MB (heap + native + mongo); 3 restheart
cells fit alongside other lanes on a shared runner.
Validated locally: cell record-pr-replay-pr 296/0 PASSED with the
new JAVA_TOOL_OPTIONS in place. RESTHeart logs confirm
"Picked up JAVA_TOOL_OPTIONS: -Xms128m -Xmx512m ..." at startup.
* fix(restheart-mongo): prefix RESTHEART_ADMIN_AUTH with `Authorization:` at curl call sites
flow.sh's 214 admin-authenticated curl invocations passed the credential
as `-H "$RESTHEART_ADMIN_AUTH"` where RESTHEART_ADMIN_AUTH defaults to
`Basic YWRtaW46c2VjcmV0` (the credential value alone, no header name).
curl renders that as a header value with no name and the request
arrives at RESTHeart with no Authorization header, producing 401.
Effect on the lane:
- bootstrap's `PUT /<db>/<coll>` seed-collection writes all 401'd
silently (each is `|| true`'d), so the seed collections were
NEVER created. record-traffic ran against an empty mongo.
- record-traffic's 214 admin-authenticated calls all 401'd. The
recorded testcases are 256 × 401 (out of 296), 25 × 404, and
only 9 × 200 (the public /ping and OPTIONS preflights).
- Lane still passed-by-equality because both record and replay
fail identically (the 401 wire bytes match), but the lane was
not actually exercising RESTHeart's admin surface.
Fix: change the call site, not the env. `-H "$RESTHEART_ADMIN_AUTH"`
→ `-H "Authorization: $RESTHEART_ADMIN_AUTH"`. The env var stays a
plain credential ("Basic YWRtaW46c2VjcmV0"), the curl line constructs
the full HTTP header. 214 sites swapped via:
sed -i 's|-H "\$RESTHEART_ADMIN_AUTH"|-H "Authorization: $RESTHEART_ADMIN_AUTH"|g'
Bare-smoke validation (docker compose up + flow.sh bootstrap +
flow.sh record-traffic against vanilla restheart 9.2.1):
before: 256 × 401 25 × 404 9 × 200
after: 110 × 200 59 × 201 49 × 404 33 × 405 25 × 204
16 × 401 11 × 400 8 × 403 4 × 409 3 × 500
The 16 remaining 401s are the genuinely intentional ones — bogus
JWT bearer probe, malformed Digest, no-auth liveness, and similar.
The 8 × 403 are real ACL-deny coverage that was previously
unreachable behind the universal 401.
* fix(restheart-mongo): noise Etag/Location + disable ACL cache for time-freeze compat
After fixing the Authorization-header-prefix bug in flow.sh (parent
commit), 214 admin-auth calls that previously 401'd now actually
mutate state and reach RESTHeart's normal write/read path. That
exposed two new categories of record/replay divergence the all-401
recording had hidden:
1) Etag header: RESTHeart auto-stamps a hash on every doc / collection
response. body._etag is already in globalNoise, but the same hash
also appears as a response header (Etag) — needs to be noised
separately. Without it, ~85 of 306 tests fail on header.Etag diff.
2) Location header: POST /_sessions returns Location: /_sessions/<uuid>
where the UUID is server-generated. 2 tests fail without noising.
Both added to keploy.yml.template's globalNoise.global.header.
Separate fix on the compose side: disable mongoAclAuthorizer's cache
(`/mongoAclAuthorizer/cache-enabled->false` via RHO). Default is
cache-enabled with TTL=5_000ms, backed by Caffeine. Caffeine's
internal ticker uses System.nanoTime, which keploy's --freezeTime
LD_PRELOAD shim intercepts — so at replay the wall-clock-bound TTL
never expires. flow.sh's `sleep 6` between `POST /acl` and the
writer-permission test works at record (clock advances, cache reloads
the new rule) but is ineffective at replay (cache stuck on whatever
it loaded first). Result: writer gets 403 at replay despite recorded
200. Disabling the cache makes ACL rules read-through from mongo on
every request — small perf cost in the test environment but
eliminates the time-freeze × cache-TTL interaction.
Local cell record-pr-replay-pr verified 306/306 PASSED with both
fixes in place. Test count went 296 → 306 (10 additional tests
captured because admin operations now mutate state and unlock paths
that previously 404'd).
---------
Signed-off-by: Akash Kumar <meakash7902@gmail.com>1 parent f3fc534 commit aa16aa2
10 files changed
Lines changed: 1905 additions & 0 deletions
File tree
- .github/workflows
- scripts
- restheart-mongo
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
0 commit comments