Description
MetaEnvelope updates are silently dropped after the first ingest: a given subscriber receives each envelope exactly once, ever, regardless of how many times the envelope's payload is subsequently updated. This breaks all cross-platform update sync for every consumer and is verified in production — a User envelope updated multiple times shows exactly 1 entry in GET /api/me/deliveries.
Root Cause
Three components combine to create the silent drop:
database/entities/Packet.ts — Packet is upserted keyed on MetaEnvelope id. Re-ingesting an updated envelope reuses the same Packet row, preserving the original packetId.
database/entities/Delivery.ts — Delivery carries @Unique(["subscriptionId", "packetId"]), so only one delivery row can ever exist per (subscription × packet) pair.
services/IngestService.ts — IngestService.ingest inserts new deliveries with .orIgnore(). When re-ingesting an updated envelope the unique constraint is violated and the insert is silently discarded — no delivery is queued, no error is raised.
Result: first ingest → delivery queued ✅. Every subsequent update → same packetId → duplicate unique key → .orIgnore() swallows it → no delivery ❌.
Steps to Reproduce
- Subscribe to a User (or any) MetaEnvelope via the awareness-service API.
- Ingest the envelope for the first time — confirm 1 delivery appears in
GET /api/me/deliveries. ✅
- Update the envelope's payload and trigger re-ingest (same envelope id, new data).
- Call
GET /api/me/deliveries again — still shows exactly 1 delivery; the update is silently dropped. ❌
- Repeat step 3–4 any number of times; delivery count never increments.
Expected Behavior
Every ingest of a MetaEnvelope whose payload has materially changed should queue a new delivery for each active subscriber, so consumers receive all updates in order and cross-platform sync is maintained.
Proposed Fix
Add a contentHash column to Delivery and adjust the uniqueness constraint:
|
Current |
Proposed |
Delivery unique key |
(subscriptionId, packetId) |
(subscriptionId, packetId, contentHash) |
contentHash value |
— |
sha256(stableStringify(payload.data)) |
IngestService insert |
.orIgnore() |
.orIgnore() (unchanged) |
- Retries of the same payload still hit the unique key →
.orIgnore() correctly deduplicates. ✅
- Re-ingests of an updated payload produce a different
contentHash → unique key does not collide → delivery is queued. ✅
- No change to the retry / dead-letter flow.
Requires a database migration to add the nullable-then-backfilled contentHash column and drop/recreate the unique index.
Supporting Media
N/A — reproduce via GET /api/me/deliveries on any prod User envelope that has been updated more than once.
Desktop
Backend service — not applicable.
Additional Context
⚠️ This is NOT a duplicate of fix/max-retries-deadletter-queue. That branch addresses retry caps and dead-letter status transitions only; it does not touch the delivery dedup key and will not fix this issue. Do not close as a dup.
Affected files (at minimum):
database/entities/Packet.ts
database/entities/Delivery.ts
services/IngestService.ts
- A new migration under
database/migrations/
Description
MetaEnvelope updates are silently dropped after the first ingest: a given subscriber receives each envelope exactly once, ever, regardless of how many times the envelope's payload is subsequently updated. This breaks all cross-platform update sync for every consumer and is verified in production — a User envelope updated multiple times shows exactly 1 entry in
GET /api/me/deliveries.Root Cause
Three components combine to create the silent drop:
database/entities/Packet.ts—Packetis upserted keyed onMetaEnvelopeid. Re-ingesting an updated envelope reuses the samePacketrow, preserving the originalpacketId.database/entities/Delivery.ts—Deliverycarries@Unique(["subscriptionId", "packetId"]), so only one delivery row can ever exist per (subscription × packet) pair.services/IngestService.ts—IngestService.ingestinserts new deliveries with.orIgnore(). When re-ingesting an updated envelope the unique constraint is violated and the insert is silently discarded — no delivery is queued, no error is raised.Result: first ingest → delivery queued ✅. Every subsequent update → same
packetId→ duplicate unique key →.orIgnore()swallows it → no delivery ❌.Steps to Reproduce
GET /api/me/deliveries. ✅GET /api/me/deliveriesagain — still shows exactly 1 delivery; the update is silently dropped. ❌Expected Behavior
Every ingest of a MetaEnvelope whose payload has materially changed should queue a new delivery for each active subscriber, so consumers receive all updates in order and cross-platform sync is maintained.
Proposed Fix
Add a
contentHashcolumn toDeliveryand adjust the uniqueness constraint:Deliveryunique key(subscriptionId, packetId)(subscriptionId, packetId, contentHash)contentHashvaluesha256(stableStringify(payload.data))IngestServiceinsert.orIgnore().orIgnore()(unchanged).orIgnore()correctly deduplicates. ✅contentHash→ unique key does not collide → delivery is queued. ✅Requires a database migration to add the nullable-then-backfilled
contentHashcolumn and drop/recreate the unique index.Supporting Media
N/A — reproduce via
GET /api/me/deliverieson any prod User envelope that has been updated more than once.Desktop
Backend service — not applicable.
Additional Context
Affected files (at minimum):
database/entities/Packet.tsdatabase/entities/Delivery.tsservices/IngestService.tsdatabase/migrations/