Feature parity: extensions needed to replace magic-indexer for certified-app

[holkexyz]

Feature parity: extensions needed to replace magic-indexer for certified-app

We're evaluating moving certified-app from magic-indexer onto upstream hyperindex (filing here because issues are disabled on hyperindex-v2, and v2 forks from this repo via GainForest/hyperindex). The three repos share the same architecture (Go AT Protocol AppView, Tap/Jetstream ingestion, lexicon-driven GraphQL schema), but magic-indexer carries a set of consumer-facing extensions that certified-app depends on and that upstream hyperindex / hyperindex-v2 do not. This issue catalogs them so they can be upstreamed.

What works out-of-the-box on hyperindex-v2

Once the app.certified.* and org.hypercerts.* lexicons are registered, the generic schema surface already handles:

• totalCount on every connection root (used for /welcome network-counts strip)
• where: { type: { eq } }, where: { subject: { eq } }, where: { did: { in } } (used for project / follower / badge-definition lookups)
• first / after / edges{cursor,node} / pageInfo

So appCertifiedActorProfile, appCertifiedActorOrganization, orgHypercertsCollection, appCertifiedGraphFollow, and appCertifiedBadgeDefinition are essentially free.

What's missing

1. Activity-feed connection args on orgHypercertsClaimActivity

The certified-app feed (src/app/api/indexer/route.ts, op Activities) calls this query against magic-indexer:

query Activities(
  $first: Int!, $after: String,
  $labels: [String!],
  $excludeLabels: [String!],
  $authors: [String!],
  $search: String
) {
  orgHypercertsClaimActivity(
    first: $first, after: $after,
    labels: $labels, excludeLabels: $excludeLabels,
    authors: $authors, search: $search
  ) { totalCount edges{cursor node{...}} pageInfo{...} }
}

labels / excludeLabels / authors / search do not exist on hyperindex-v2's auto-generated connection roots. We also use where: { contributor: { eq: $did } } for the per-user "contributed" feed (op ContributedActivities), which is not auto-derived from the lexicon since contributor is a sub-field on a union variant.

Recommended implementation:

• Add a ConnectionArgRegistry keyed by lexicon NSID. Each entry provides the GraphQL arg → SQL clause mapping.
• For org.hypercerts.claim.activity, register:
  • labels: [String!] → JOIN against the labels table (see #3) with IN (...) and EXISTS (...)
  • excludeLabels: [String!] → NOT EXISTS (... IN (...))
  • authors: [String!] → did IN (...) (already implicit in v2's where.did.in, but elevate as a first-class connection arg so the API matches the upstream Hyperindex precedent)
  • search: String → to_tsquery against a GIN index across title, shortDescription, description, and the workScope.scope JSON path (see #4)
  • where.contributor → recursive descent into the contributor array, with the same did/identity tri-form handling magic-indexer documents in its contributor filter (DID values only; non-DID handles silently skipped; MaxArrayContributorScan cap)

These arg registrations should live in internal/graphql/query/connection.go, alongside the existing where/sortBy plumbing.

2. Denormalised fields on appCertifiedBadgeAward

The endorsement detail card (src/hooks/use-received-endorsements.ts, op ReceivedEndorsements) needs the issuer's actor profile and the recipient's response inlined on each award node:

appCertifiedBadgeAward(...) {
  edges { node {
    uri cid did createdAt note badge
    issuer { did handle displayName description avatarCid pds }
    response { state weight createdAt }
  }}
}

Without this we re-introduce a /api/resolve-did fan-out per award row on first paint, which is exactly what magic-indexer issue #96 closed.

Recommended implementation:

• Port internal/graphql/schema/derived_fields.go from magic-indexer (it has no equivalent in v2).
• Register two derived fields for app.certified.badge.award:
  • issuer — joins the award's did to app.certified.actor.profile (with app.bsky.actor.profile ingestion as a fallback once enabled)
  • response — joins the award URI to the latest app.certified.badge.response record by the award's subject, ordered by sort_at DESC NULLS LAST (matches magic-indexer issue add Vercel deploy button to README.md #26 so reset → accept resolves correctly)
• Sort order on the appCertifiedBadgeAward connection itself should default to sort_at DESC NULLS LAST so resets land before originals.

3. Inline labels on every record

Every record-bearing connection in magic-indexer exposes a flat labels: [String!] array — the current set of active labeller values for that record. This replaces a separate label query.

hyperindex-v2 only exposes externalLabels (different shape: {src, uri, cid, val, cts}, scoped to ATProto labelers).

Recommended implementation:

• Add a labels resolver to the connection-node builder when the lexicon has any registered labeller.
• Batch-load via DataLoader keyed by record URI; resolve from the existing labels table.
• Keep externalLabels as the structured form (useful when the consumer needs the labeller DID), but materialize labels as the flat array consumers like certified-app expect.

4. Full-text search arg

where.title.contains (per-field, 3-char min) is not enough — the certified-app feed search box queries across title, shortDescription, description, and the JSON workScope.scope field as a single AND-ed phrase.

Recommended implementation:

• Generate a single tsvector column per record type (built at ingestion time from the configured set of text fields), GIN-indexed.
• Expose search: String as a connection arg on each record connection where the lexicon has registered a search profile.
• Implicit-AND between whitespace-separated terms (matches magic-indexer behavior).

5. Authenticated notifications subsystem

certified-app's notification dropdown (src/app/api/notifications/route.ts) talks to a separate /notifications/graphql endpoint on magic-indexer, authenticated by AT Protocol service-auth JWT (the acting DID is taken from the JWT iss, not a variable). Operations: notifications, unreadNotificationCount, updateNotificationsSeen.

This is the largest missing piece — it's an authenticated, write-capable GraphQL endpoint with a service-auth JWT verification layer and persistent "seen at" state per actor.

Recommended implementation:

• Add internal/notifications/ subsystem with: ingestion side that materializes notification rows from interaction records (follows, badge awards, replies, etc.), and read side that queries them per recipient DID.
• Service-auth verification middleware: validate the JWT iss (acting DID), aud (this indexer's DID), lxm (= com.hypergoat.notification.query for reads / com.hypergoat.notification.markSeen for the mutation), exp ~60s window, jti replay cache.
• Schema: notifications(first, after) connection, unreadNotificationCount { count more }, updateNotificationsSeen(seenAt: String) mutation.

Smaller items

• appCertifiedTempGraphEndorsement — legacy temp endorsement records (pre-badge-migration). certified-app still reads from this for the compat window; can drop once unused.
• MaxArrayContributorScan cost cap — magic-indexer skips contributor lookups on records with too many contributors. v2 should adopt this or similar to avoid pathological queries.

Strategic ask

If hyperindex-v2 is the org's strategic direction, the path of least pain is to upstream items 1, 2, 3, and 4 here and either fold notifications (item 5) into v2 or carve it out as a small companion service. That clears the way for certified-app — and any other consumer — to migrate off the magic-indexer fork.

Happy to send PRs for any/all of these once an approach is agreed.

cc maintainers