Add test suite - #19
Open
SoundGoof wants to merge 6 commits into
Open
Conversation
Introduces the first tests this repo has had. The JS suites load the browser scripts into a vm sandbox with jQuery and Raphael stubbed, so dhmap.init genuinely runs and its private registries become observable without a browser. The Python suite drives ipplan2dhmap.py as a subprocess against a database built from a topology fixture, giving a topology -> database -> generator -> topology round trip. Tests characterize current behaviour rather than desired behaviour, so they pass on a clean checkout. Four known defects are pinned as such: the doubled renderSwitch offsets, the unguarded alert_hosts dereference, the crash on a Grid hall with no objects, and the AttributeError on a hall-less name without digits. localserver.py imported SimpleHTTPServer/SocketServer and so could not run on Python 3 at all despite its shebang; ported to http.server with the /analytics/ redirect semantics unchanged, plus --port and --analytics-port so tests can bind free ports. src/examples/ had not been touched since the initial commit and crashed against the current dhmap.init: the data was a flat array predating the hall/Grid rewrite, example.js passed booleans where status strings are expected, and index.html was missing the elements init looks up. Repaired and now covered by tests so it cannot rot again silently. A real ipplan database can be dropped into local/ to run the same assertions against real data; those tests skip when none is present, which is how CI runs.
Completes the suite with the two tiers that need real browser machinery, plus the backend they talk to. The fake backend presents two faces over a single scenario: the Prometheus HTTP API, using the metric names the real rules in scripts/prometheus already use, and the seven analytics endpoints dhmon.js fetches. Because both are rendered from the same data they cannot disagree, and a test holds them to that. It also means `make serve` gives a populated map with no monitoring system present, which is the first time this repo has been runnable end to end locally. The browser tiers run in a container rather than installing anything on the host: jsdom and Playwright live in a podman volume and the browsers come preinstalled in the image. CI uses the same image, so the two environments match. The E2E tier drives the real dev server and follows the /analytics redirect rather than stubbing at the network layer. That redirect is a development-only artifact - production reverse-proxies /analytics same-origin - so it makes the fetches cross-origin and preflighted, and the fake backend answers OPTIONS accordingly. Testing with realistic hall names surfaced a defect worth noting: hall element ids are built by string concatenation, so a hall named "Hall 1" yields id="menu_hall_Hall 1", which jQuery parses as a descendant selector and never matches. Hall colour roll-up therefore does nothing for every hall in production. Pinned as a known defect rather than fixed here. The suite was checked for teeth: inverting the CRITICAL ping threshold, pointing the page at a missing vendor file, and changing a colour out of rgb() form each fail the relevant tier.
Production apache is configured with `ProxyPass /analytics http://localhost:5000` (dhmon::analytics), so the browser sees a single origin. localserver.py issued a 302 to localhost:5000 instead, which made the same fetches cross-origin, and jQuery's X-Requested-With header turned them into preflighted ones. That divergence was only visible once the browser tier existed: it forced CORS handling into the fake backend to work around behaviour production never exhibits. Proxying instead removes the workaround - verified by stripping the CORS headers entirely and confirming the page still loads clean. Also surfaces a clear 502 when the backend is unreachable, rather than a redirect to a port with nothing behind it. The fake backend keeps its CORS headers so it still works when a page is pointed straight at port 5000, but nothing depends on them now.
The workflow pinned actions/checkout@v4, setup-node@v4, setup-python@v5 and upload-artifact@v4; all four are on v7. Moved to the moving v7 tags. Node moves to 24, the current LTS, which is also what the playwright image ships - so the two CI jobs and a local podman run now agree on the runtime rather than testing node 22 in one place and 24 in another. ipplan2dhmap.py runs on whatever python the server happens to ship, so the core job now runs a small matrix over 3.12 and 3.14 rather than a single version. Both legs were verified before committing: 3.14 with node 22 on the host and 3.12 with node 24 in the container, 55 + 49 passing on each. The playwright image is already on 1.62.0, which is current.
Picks up the merged vendor cleanup (PR #18) so this branch's diff is computed against current master.
The proxy relayed the response to the client from inside the try that catches OSError. BrokenPipeError is an OSError, so a browser that went away mid-response - a reload, a closed tab, or the 10s poll racing a navigation - was caught by the handler meant for an unreachable backend. It then answered 502 "analytics backend unreachable" down the socket that had just closed, which raised again and printed two stacked tracebacks per occurrence. Nothing failed as a result, but CI logs and local dev sessions filled with tracebacks that point at the backend when the backend is fine. Fetch upstream first, then relay, so writing to the client is outside that handler; and treat BrokenPipeError/ConnectionResetError as the ordinary event it is. The test hangs up mid-request and asserts stderr stays clean. It fails against the previous implementation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
dhmap's first tests: 118 across four tiers, plus a fake monitoring backend so the app runs locally without a real dhmon.
Nothing currently validates a change before it reaches
/var/www/html/dhmap. With a jQuery 3→4 upgrade on the table, that needs fixing first.make test-unit— JS logic in avmsandboxmake test-py— generator, dev server, fake backendmake test-dom— menu under jsdom + vendored jQuerymake test-e2e— full app in headless chromiumThe core tiers need only
python3andnode. The browser tiers run in a container — deps live in a podman volume, browsers ship in the image,node_modulesnever touches the host or the puppet-deployed tree. CI uses the same image.No source was restructured to make it testable: the scripts are loaded into a prepared
vmcontext with jQuery and Raphael stubbed, sodhmap.initreally runs.Bug found
Hall ids are built by concatenation, so
Hall 1givesid="menu_hall_Hall 1". jQuery reads that as a descendant selector and matches nothing — menu hall colour roll-up has never worked for any production hall. Only visible once fixtures used realistic names. Pinned, not fixed.Characterization, not correction
Tests pin current behaviour so they pass on a clean checkout. Known defects are labelled where pinned:
dhmap.js—renderSwitchmutates its argument andinitruns it twice, so centering offsets apply twice (width 5 → 8.4).dhmap.js:358— aGridhall with no objects → TypeError.dhmon.js:137—alert_hostsdereferenced outside the:116null-guard; a slowalerts.hostsfetch throws.ipplan2dhmap.py:44—.group(1)on a possibleNone; a name likeVIPraisesAttributeError.dhmenu.js:107— the hall id defect above.Repaired (all were broken)
localserver.pyimportedSimpleHTTPServer— Python 2 only, so it could not run at all. Ported tohttp.server, redirect semantics unchanged.src/examples/crashed against currentdhmap.init(pre-2017 data shape, booleans for status strings, missing DOM elements). Now covered by tests.make servenow brings the app up with fake SNMP data.Fake backend
One scenario, two faces that cannot disagree: Prometheus (
/api/v1/query,/metrics, using the real metric names fromscripts/prometheus) and the seven analytics endpointsdhmon.jsfetches. A test pins that the saturated fixture crosses both the dhmap threshold (>0.95) and the Prometheus rule's (>90%) — same formula, so map and alerts can't drift apart unnoticed.Data handling
Fixtures are committed, outputs never are —
data.json, temp DBs, reports andnode_modulesare gitignored, andgit statusis clean after a full run. Drop a real ipplan intolocal/(gitignored) and every command uses it;make servethen renders the real event map. Real-data assertions are structural only and never name devices. CI has no database and never needs one.Verification
rgb()form → 2 failures.make serverenders the real event map (39 objects, 24 devices) with fake SNMP.Review notes
whereami.htmlis atest.fixme(): no#menu_container, sodhmap.initthrows. Recorded, not repaired.localserver.pynow reverse-proxies/analyticsthe way production apache does (ProxyPass /analytics http://localhost:5000) rather than issuing a 302. The redirect made these fetches cross-origin and preflighted, which production never does; proxying removes that divergence. Verified by stripping the fake backend's CORS headers and confirming the page still loads clean.Follow-ups
dhmapmanifest'signorelist (node_modules,test,.github,Makefile,package*.json).