Skip to content

MetaEnvelope updates never re-delivered — Delivery dedup key collides on every update #984

@plansombl

Description

@plansombl

Description

MetaEnvelope updates are silently dropped after the first ingest: a given subscriber receives each envelope exactly once, ever, regardless of how many times the envelope's payload is subsequently updated. This breaks all cross-platform update sync for every consumer and is verified in production — a User envelope updated multiple times shows exactly 1 entry in GET /api/me/deliveries.

Root Cause

Three components combine to create the silent drop:

  1. database/entities/Packet.tsPacket is upserted keyed on MetaEnvelope id. Re-ingesting an updated envelope reuses the same Packet row, preserving the original packetId.
  2. database/entities/Delivery.tsDelivery carries @Unique(["subscriptionId", "packetId"]), so only one delivery row can ever exist per (subscription × packet) pair.
  3. services/IngestService.tsIngestService.ingest inserts new deliveries with .orIgnore(). When re-ingesting an updated envelope the unique constraint is violated and the insert is silently discarded — no delivery is queued, no error is raised.

Result: first ingest → delivery queued ✅. Every subsequent update → same packetId → duplicate unique key → .orIgnore() swallows it → no delivery ❌.

Steps to Reproduce

  1. Subscribe to a User (or any) MetaEnvelope via the awareness-service API.
  2. Ingest the envelope for the first time — confirm 1 delivery appears in GET /api/me/deliveries. ✅
  3. Update the envelope's payload and trigger re-ingest (same envelope id, new data).
  4. Call GET /api/me/deliveries again — still shows exactly 1 delivery; the update is silently dropped. ❌
  5. Repeat step 3–4 any number of times; delivery count never increments.

Expected Behavior

Every ingest of a MetaEnvelope whose payload has materially changed should queue a new delivery for each active subscriber, so consumers receive all updates in order and cross-platform sync is maintained.

Proposed Fix

Add a contentHash column to Delivery and adjust the uniqueness constraint:

Current Proposed
Delivery unique key (subscriptionId, packetId) (subscriptionId, packetId, contentHash)
contentHash value sha256(stableStringify(payload.data))
IngestService insert .orIgnore() .orIgnore() (unchanged)
  • Retries of the same payload still hit the unique key → .orIgnore() correctly deduplicates. ✅
  • Re-ingests of an updated payload produce a different contentHash → unique key does not collide → delivery is queued. ✅
  • No change to the retry / dead-letter flow.

Requires a database migration to add the nullable-then-backfilled contentHash column and drop/recreate the unique index.

Supporting Media

N/A — reproduce via GET /api/me/deliveries on any prod User envelope that has been updated more than once.

Desktop

Backend service — not applicable.

Additional Context

⚠️ This is NOT a duplicate of fix/max-retries-deadletter-queue. That branch addresses retry caps and dead-letter status transitions only; it does not touch the delivery dedup key and will not fix this issue. Do not close as a dup.

Affected files (at minimum):

  • database/entities/Packet.ts
  • database/entities/Delivery.ts
  • services/IngestService.ts
  • A new migration under database/migrations/

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingpriority - 1Really important bug. App is broken, major functionality is unusable

Type

No type
No fields configured for issues without a type.

Projects

Status

Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions