feat: enhanced dashboard with statistics (#25)#1075
Open
Wikid82 wants to merge 20 commits into
Open
Conversation
Adds RequestLog struct to record proxied HTTP requests for the enhanced dashboard statistics feature (issue #25). Includes BeforeCreate hook for UUID generation, compound (host_id, timestamp) indexes, and GDPR-safe pseudonymised client IP hashing. Registers model in AutoMigrate.
- Add stats_types.go with StatsPushData, HostStat, StatusStat, StatsPushMessage, and BroadcastHub interface (avoids import cycles) - Add StatsIngester: channel buffer=1000, batch flush at 100 entries or 500ms interval via CreateInBatches; atomic dropped-count tracking - Hash client IPs with SHA-256 (first 16 bytes, hex) for GDPR safety - Add LogWatcher.RegisterIngester() + fan-out in broadcast() - TDD: 6 tests covering count-flush, timer-flush, back-pressure, graceful Stop drain, IP hashing determinism, and fan-out wiring
Adds StatsService providing GetSummary (30s TTL cache), GetTopHosts, GetStatusDistribution, GetTrafficVolume, and GetCertExpiry with input validation allowlists. Extends stats_types.go with StatsSummary, TrafficBucket, and CertExpiry types.
Add typed API functions and interfaces for all 8 stats endpoints (summary, top-hosts, status-distribution, traffic-volume, cert-expiry, requests, health, and WebSocket hub) with full Vitest test coverage (33 tests).
Add six TanStack Query hooks (useStatsSummary, useTopHosts, useStatusDistribution, useTrafficVolume, useCertExpiry, useStatsHealth) with stable query keys and appropriate polling intervals. Add useStatsWebSocket hook that tracks live summary updates via the stats WebSocket and disables REST polling when connected. Full Vitest coverage for all hooks (22 tests). Also remove unicorn/no-array-for-each ESLint rule removed in unicorn v66.
Add 8 pure presentational components under frontend/src/components/stats/: - RequestCountWidget: 3-stat card for 24h/7d/30d request counts - TopHostsChart: horizontal bar chart (recharts BarChart) - StatusDistributionChart: donut chart with accessible HTML summary list - TrafficVolumeChart: line chart with KB/MB Y-axis formatting - CertExpiryList: accessible table with red/amber/green day-based color coding - ServiceHealthWidget: WebSocket live/offline indicator + dropped-event warning - PeriodSelector: controlled radio button group for 24h/7d/30d - BucketSelector: controlled radio button group for 1h/6h/1d All components are pure (no data fetching), strictly typed with no `any` types, and keyboard accessible. Includes 58 Vitest unit tests covering loading states, data rendering, color coding, and interaction callbacks.
Add a responsive Statistics section to the Dashboard page below the existing content. Uses useStatsWebSocket for live updates, useState for period/bucket controls, and the six stats hooks + eight stats components (RequestCountWidget, ServiceHealthWidget, CertExpiryList, TrafficVolumeChart, TopHostsChart, StatusDistributionChart, PeriodSelector, BucketSelector). Layout is mobile-first with single column on small screens, 2-col on sm/md, 3-col top row on lg. Adds dashboard.statistics and dashboard.trafficVolume i18n keys to all five locale files. Expands Dashboard tests from 3 to 12 cases.
- tests/stats.spec.ts: 12 E2E tests covering all 9 required scenarios (stats heading, period selector, bucket selector, request count widget, service health widget, cert expiry section, traffic/top-hosts/status distribution chart containers) plus accessibility radio-count assertion - backend/internal/api/handlers/stats_api_integration_test.go: adds TestStatsAPI_CertExpiry_366Days_Returns400 to cover the upper-bound validation (within_days > 365 returns HTTP 400); simplifies function signature to remove unused return value - backend/internal/services/stats_ingester_test.go: adds TestStatsIngester_RegisterHub and TestStatsIngester_ToRequestLog_InvalidTimestamp to cover the RegisterHub wiring path and the timestamp parse-error fallback All 12 E2E tests pass against the running E2E container at :8080. Backend unit tests pass (88.4% coverage, above 87% minimum). Frontend tests pass (87.86% statement coverage, above 85% minimum). GORM scan: 0 CRITICAL/HIGH findings.
Contributor
|
You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool. What Enabling Code Scanning Means:
For more information about GitHub Code Scanning, check out the documentation. |
Contributor
✅ Supply Chain Verification Results✅ PASSED 📦 SBOM Summary
🔍 Vulnerability Scan
📎 Artifacts
Generated by Supply Chain Verification workflow • View Details |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Adds targeted tests to cover all previously uncovered patch lines: Backend: - stats_ws_hub_test.go (new): full hub coverage — constructor, non-blocking broadcast, ctx cancel exit, client broadcast, slow-client drop, client unregister, StatsWS upgrade-error path, StatsWS nil-hub close - stats_handler_test.go: error-path 500s for all six handlers, non-integer within_days → 400, invalid limit param silently ignored - stats_ingester_test.go: Stop flushes batches > batchSize; Run drains big batch on ctx cancel (covers batchSize branch in drain loop) - stats_service_test.go: GetTrafficVolume 6h and 1d buckets; GetSummary DB error Frontend: - StatusDistributionChart: extended recharts mock calls Pie label/Tooltip content; adds 1xx test to cover statusClass "other" return - TrafficVolumeChart: mock calls YAxis tickFormatter with MB/KB/B values and Tooltip content to cover formatBytes branches - TopHostsChart: mock calls Tooltip content including hostname ?? label fallback - CertExpiryList: adds undefined-data test to cover (data ?? []) branch - useStatsWebSocket: adds non-stats_update message test for the else branch
…/show controls Adds ELI5 info tooltips to all 6 dashboard stats widgets, a color-coded legend for the Top Hosts chart, and a per-widget visibility toggle persisted in localStorage so users can hide widgets they don't need.
Adding the Dashboard "Customize" button (which also carries aria-expanded) shifted DOM order and caused the WebKit navigation E2E test to target it instead of the sidebar, since the sidebar's collapsible accordion buttons never actually exposed aria-expanded. Add the missing attribute to the real sidebar toggles and scope the test to the sidebar so it tests what it claims to.
Wikid82
added
enhancement
GetTopHosts only selected host_id and a count, never the hostname, so every entry in the Top Hosts legend and tooltip rendered with a blank or duplicate label. With every chart category collapsing to the same empty string, Recharts merged the bars and hovering any bar showed the same (highest) data point. Join proxy_hosts by UUID to populate the real hostname, falling back to host_id if the host has since been deleted.
…king startup PRAGMA quick_check was running synchronously in Connect() and could take well over a minute on larger databases (observed 93.5s), despite being intended as a non-blocking, warn-only check.
The previous fix moved PRAGMA quick_check to a goroutine, but the main pool is capped at one connection (SQLite single-writer constraint), so the background check still held that connection for the full scan and blocked AutoMigrate behind it. Open a dedicated connection for the check so it no longer serializes against the rest of startup.
GetTopHosts joined proxy_hosts.domain_names, showing raw domains in the legend/tooltip instead of the user-assigned host Name. Switch the join to proxy_hosts.name to match what users configured.
Optional lookups (e.g. caddy.keepalive_idle/keepalive_count settings) return ErrRecordNotFound when unset, which call sites already handle via `if err == nil` fallbacks. GORM's default logger still logged these as errors on every startup. Configure IgnoreRecordNotFoundError so only real query errors and slow queries are logged.
RequestLog.HostID stores the raw Host header seen by the proxy (a domain), not the ProxyHost UUID, so the previous join on proxy_hosts.uuid = request_logs.host_id never matched and silently fell back to showing the domain. Build a domain -> name lookup from proxy_hosts.domain_names (which can hold several comma-separated domains per host) and resolve hostnames against that instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #25
Implements a comprehensive statistics dashboard with real-time updates, charts, and service health indicators.
LogWatcherfan-out; batches writes to SQLite (100 entries or 500 ms, whichever comes first). No duplicate log parsing./api/stats/(all JWT-protected). WebSocket pushes live summary updates; REST polling automatically disabled when WS is connected.unicorn/no-array-for-eachbreakage from dep bump8d527834.Test plan
go test ./... -count=1— all backend tests passscan-gorm-security.sh --check— 0 CRITICAL/HIGHlefthook run pre-commit— 0 issues (golangci-lint, go-vet, semgrep)http://localhost:8080) — 12/12 passnpm run type-check— zero errorsnpm run build— cleango build ./...— clean