feat(gr26): persist CAN frames and trace signals back to source#139
Open
BK1031 wants to merge 8 commits into
Open
feat(gr26): persist CAN frames and trace signals back to source#139BK1031 wants to merge 8 commits into
BK1031 wants to merge 8 commits into
Conversation
Adds two gr26-local tables. gr26_can stores the raw CAN frame (vehicle, node, can id, post-MQTT-header bytes, timestamp) with a composite unique key on (vehicle_id, node_id, timestamp). gr26_can_signal is a 1-N join keyed on signal_id that maps each persisted signal back to the frame it was decoded from. Both tables are gr26-only so the schema can change year-to-year without touching the shared signal table or mapache-go. HandleMessage now upserts the gr26_can row first (returning the stored ulid), persists the decoded signals, then bulk-inserts the join rows. The signal upsert in CreateSignals stops overwriting id on conflict and uses Returning so signals[i].ID reflects the actually-stored value across retransmits — the join table assumes signal ids are stable. WS publish moves to after the DB insert so subscribers see the persistent id rather than the freshly-generated-then-discarded one.
✅ Deploy Preview for gr-mapache ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Move CreateCAN ahead of the messageStruct lookup and FillFromBytes step so unknown can ids and decode failures still produce a gr26_can row. The frame ends up in the database with no signals or join rows, which is the right shape for "what bytes did we receive that we couldn't parse" debugging — same dataset answers both "trace this signal" and "why did this signal never appear." Length and upload-key checks still bail before any persistence: a frame shorter than the MQTT envelope can't be sliced safely, and an invalid upload key means the data isn't trusted to belong to the vehicle.
Adds a nullable jsonb metadata column on gr26_can. HandleMessage now
attempts the decode first, then writes the frame in a single insert
with the outcome stamped in:
- decode succeeded -> metadata is null
- unknown can id -> {"status":"unknown_can_id"}
- decode error -> {"status":"decode_error","error":"..."}
Single write per frame instead of insert-then-update, and the upsert's
DoUpdates picks up the new column so retransmits with different
outcomes overwrite the old metadata.
Queries: "frames we couldn't decode" is now a one-liner — WHERE
metadata IS NOT NULL — without the LEFT JOIN against gr26_can_signal.
Pairs the categorical status field with a human-readable note. Renames
the prior decode_error 'error' field to 'note' so both states share a
schema:
{"status":"unknown_can_id","note":"no decoder registered for can id 0x999"}
{"status":"decode_error","note":"<err.Error()>"}
Successful decodes now write {"status":"ok"} into gr26_can.metadata
instead of leaving it null. Every frame has a populated metadata field,
so the decode-outcome filter is metadata->>'status' across the board.
Returns the stored CAN frame plus three things derived on demand:
- signals: every persisted signal currently linked to this frame via
gr26_can_signal, fetched in a single join.
- fields: the gr26 decoder is re-run over the stored bytes so the
response carries per-field name, byte offset, size, sign, endian,
the byte slice that contributed, the raw decoded integer, and the
full signal names produced (with the node prefix). Returns nil
fields if the can id has no decoder or the bytes don't fit — both
cases already captured in can.metadata.
- bytes: hex-encoded so the dashboard can render the hex grid
without re-decoding the JSON's default base64.
Route is registered under /gr26/messages/:id and rincon's existing
/gr26/** wildcard already exposes it through kerbecs, so no gateway
change.
Adds a SignalEvent wire type on gr26 that embeds mapache.Signal and adds
can_message_id. The hub's channel and Publish now carry SignalEvent so
WS subscribers can go from a streamed signal back to the source frame.
WebSocket publishing moves out of CreateSignal/CreateSignals and into
HandleMessage, which fans out an event per signal once the join rows
have landed.
The dashboard's debug page picks up can_message_id from the WS payload,
makes rows that have one clickable, and opens a MessageTraceDialog that
fetches /gr26/messages/:id and renders:
- the frame's metadata + decode status,
- the bytes as a per-byte hex grid colored by which field owns each
byte,
- the field list with offset/size/sign/endian/raw value/signal names,
- the persisted signals from this frame (the clicked one highlighted).
Rows without can_message_id (signals from a vehicle whose service
doesn't yet publish it) just stay non-clickable.
Reverts the SignalEvent wrapper and gr26-local Hub channel type. The live WebSocket again emits exactly mapache.Signal — no extra fields, no gr26-local wire shape. Drops gr26/model/signal.go, hub channel and Publish go back to mapache.Signal, WS handler and pending coalesce map follow. Adds GET /gr26/signals/:id which joins gr26_can_signal -> gr26_can and returns the same trace shape as /gr26/messages/:id. Both handlers share respondWithCAN; only the lookup func differs. Dashboard captures the streamed signal.id (already on mapache.Signal) and uses it for the trace fetch. MessageTraceDialog now takes signalId + vehicleType and hits /<vehicleType>/signals/:id, so the same component works for any service that exposes that route.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
gr26_cantable stores each decoded CAN frame (vehicle, node, can id, post-MQTT-header bytes, timestamp) with a composite unique key on(vehicle_id, node_id, timestamp).gr26_can_signaltable is a 1-N join keyed onsignal_idthat maps each persisted signal back to the frame it came from.HandleMessagenow upserts thegr26_canrow first, persists signals, then bulk-inserts the join rows.idon conflict and usesclause.Returningsosignals[i].IDreflects the actually-stored value across retransmits — required because the join table assumes signal ids are stable.signaltable andmapache-goare untouched so this can change year-to-year without affecting other services.