graph/db: cross-version graph Store by ellemouton · Pull Request #10656 · lightningnetwork/lnd

ellemouton · 2026-03-23T12:06:11Z

Summary

Make the graph Store interface cross-version so that query methods work
across gossip v1 and v2, enabling callers to interact with the graph without
needing to know which gossip version announced a given node or channel.

This is part of the Gossip 1.75 epic.

Key changes

Version-aware horizon queries: NodeUpdatesInHorizon and
ChanUpdatesInHorizon now take a gossip version and a typed range
(NodeUpdateRange/ChanUpdateRange) that enforces v1=timestamps vs
v2=block-heights at the type level. Full SQL implementations for both
versions.
Cross-version iteration: ForEachNode and ForEachChannel iterate
across all gossip versions, yielding each unique node/channel exactly once
(highest version preferred).
PreferHighest fetch helpers: FetchChannelEdgesByIDPreferHighest,
FetchChannelEdgesByOutpointPreferHighest return the highest available
gossip version for a channel without the caller specifying one.
GetVersions queries: GetVersionsBySCID and GetVersionsByOutpoint
report which gossip versions exist for a given channel.
Version-aware FilterKnownChanIDs: Now takes a gossip version parameter.
Zombie-revival logic moved from ChannelGraph into VersionedGraph so
callers through that type don't need to pass a version explicitly.
Prefer-highest node traversal: ForEachNodeDirectedChannel and
FetchNodeFeatures on ChannelGraph iterate across all versions (v2
first), so pathfinding prefers v2-announced channels and features.

gemini-code-assist · 2026-03-23T12:06:23Z

Warning

Gemini encountered an error creating the summary. You can try again by commenting /gemini summary.

bhandras

LGTM 🥇

graph/db/graph_test.go

graph/db/sql_store.go

graph/db/graph.go

graph/db/sql_store.go

bitromortac

Nice work, cool to see that both versions are used as a source 🎉 (have a few nits that optionally could be fixed)

graph/db/graph_test.go

graph/db/options.go

graph/db/interfaces.go

graph/db/kv_store.go

bitromortac · 2026-03-26T13:41:34Z

graph/db/graph_test.go

+	// GetVersions for a non-existent SCID should return empty.
+	versions, err = store.GetVersionsBySCID(ctx, 999999)
+	require.NoError(t, err)
+	require.Empty(t, versions)


we could add a test for the v2 channel

graph/db/benchmark_test.go

bitromortac · 2026-03-26T13:49:11Z

graph/db/sql_store.go

+					if !exists {
+						entry = &nodeEntry{}
+						nodesByPub[pub] = entry
+						order = append(order, pub)


Memory-wise this is not a huge regression because the sql db anyhow fetches all the rows at once?

Addressed by replacing the whole-graph materialization with paginated preferred-row traversal. The SQL side now pages over distinct logical keys and streams one preferred node/channel at a time.

graph/db/sql_store.go

bitromortac · 2026-03-26T14:08:27Z

graph/db/graph_test.go

+	require.False(t, features.IsEmpty(),
+		"both-version node should have features")


nit: we could make the test more strict by changing one of the feature vectors and compare against it

Done. The test now assigns distinct v1/v2 feature vectors and asserts that the v2 feature vector is the one returned

ellemouton · 2026-03-27T11:31:38Z

Made two main follow-up changes after review:

Added new SQL indexes for the new horizon-sync paths.
These support the v2 block-height node horizon scan and the policy-driven channel horizon queries, so large SQL graph DBs do not fall back to broad scans/sorts when serving horizon updates.
Removed the large-DB memory regressions in the new cross-version traversal paths.
ForEachNode / ForEachChannel no longer materialize the full merged v1/v2 graph in memory first; the SQL backend now pages over distinct logical keys and streams the preferred row directly.
ForEachNodeDirectedChannel was similarly changed so the no-cache path no longer buffers the full directed adjacency set before invoking the callback.

That keeps the prefer-highest cross-version behavior, but avoids whole-graph collection in memory on large SQL-backed networks.

Add version-aware range types for channel and node update horizon queries. V1 gossip uses unix timestamps for ordering while v2 uses block heights, so each range type validates that the correct bound type is provided for the requested gossip version. These types will be used in follow-up commits to version the NodeUpdatesInHorizon and ChanUpdatesInHorizon Store methods.

Replace the (startTime, endTime time.Time) parameters on NodeUpdatesInHorizon and ChanUpdatesInHorizon with (v GossipVersion, r NodeUpdateRange/ChanUpdateRange). The range types enforce version-correct bounds at the type level: v1 uses unix timestamps, v2 uses block heights. The KV store rejects non-v1 versions since it only stores v1 data. The SQL store handles v1 queries; v2 block-height range queries are added in the following commits. VersionedGraph wrappers supply the version from the embedded field, so callers only pass the range.

Add the SQL indexes needed by the new v2 horizon sync paths: - graph_nodes(version, block_height, pub_key) - graph_channel_policies(version, block_height, channel_id) - graph_channel_policies(version, last_update, channel_id) These support the block-height node horizon scan and the policy-driven channel horizon scans without degrading to broad table scans. The index changes are added directly to migration 9 because no released version contains that migration yet, so it is still safe to edit it in place.

Add SQL queries for v2 node and channel horizon lookups that use block heights instead of unix timestamps. For channels, also reshape the horizon query so it starts from matching policy rows and then hydrates the candidate channels, which keeps the work proportional to the matching update set rather than the full graph. Regenerate sqlc for the new query surface.

Hook the new v2 block-height horizon queries into the SQL store's NodeUpdatesInHorizon and ChanUpdatesInHorizon implementations. This replaces the temporary "not yet implemented" SQL v2 horizon paths, adds the block-height pagination cursor handling needed for channels, and covers the end-to-end behavior with SQL backend tests.

Add a gossip version parameter to FilterKnownChanIDs on the Store interface so that it can filter channel IDs for the correct gossip version. The KV store rejects non-v1 versions. Move the zombie-revival logic from ChannelGraph into VersionedGraph so that FilterKnownChanIDs is only exposed on VersionedGraph (which supplies the version from its embedded field). This means callers no longer need to pass a version explicitly.

Add four new Store interface methods: - FetchChannelEdgesByIDPreferHighest: version-agnostic channel lookup that returns the highest available gossip version. - FetchChannelEdgesByOutpointPreferHighest: same but keyed by outpoint. - GetVersionsBySCID: returns which gossip versions exist for a SCID. - GetVersionsByOutpoint: same but keyed by outpoint. The KV store implementations delegate to v1 since that's the only version it supports. The SQL store implementations iterate over versions from highest to lowest (for PreferHighest) or lowest to highest (for GetVersions). These will be used in follow-up commits to make ChannelGraph callers version-agnostic.

Add dedicated SQL pagination queries for cross-version graph traversal so SQL backends can stream one logical node or channel at a time without loading and merging both gossip versions in memory. ListPreferredNodesPaginated first pages over distinct pubkeys across the v1 and v2 node tables. For each pubkey it then picks exactly one stored row in this order: v2 with a signature, v1 with a signature, bare v2, then bare v1. That keeps traversal on the highest usable version while still falling back when the preferred row is missing auth data. ListPreferredChannelsWithPoliciesPaginated does the same for channel SCIDs. It first pages over distinct SCIDs across both channel versions, then selects one channel row per SCID in this order: v2 with at least one matching-version policy, v1 with at least one matching-version policy, bare v2, then bare v1. This avoids a higher-version channel without policy data hiding a lower-version channel that is actually usable for traversal. Regenerate sqlc for the new query surface.

Change the graph traversal entry points to return one logical node or channel across all stored gossip versions instead of forcing callers to iterate a single version at a time. For SQL backends, ForEachNode now uses the prefer-highest node query and returns one row per pubkey, preferring the highest announced version and falling back to the highest-version shell node when no signed announcement exists. ForEachChannel now uses the prefer-highest channel query and returns one row per SCID, preferring the highest version with matching policy data and otherwise falling back to the highest stored channel row. Update the ChannelGraph and VersionedGraph wrappers to use the new version-agnostic store methods, keep the KV implementation on its v1-only behavior, and cover the SQL no-cache traversal paths with dedicated prefer-highest tests.

…rappers Switch FetchChannelEdgesByID and FetchChannelEdgesByOutpoint on ChannelGraph to use the PreferHighest store variants so that callers get the highest available gossip version without needing to specify one. Also add GetVersionsBySCID and GetVersionsByOutpoint convenience wrappers on ChannelGraph that delegate to the corresponding Store methods.

Update ForEachNodeDirectedChannel and FetchNodeFeatures on ChannelGraph to iterate across all gossip versions (highest first) instead of hardcoding v1. This ensures that channels announced via v2 are preferred over v1 and that v2 features are used when available.

changed quite a bit

ellemouton force-pushed the g175-graph-db-cross-version branch from 5337e61 to cbe9436 Compare March 23, 2026 12:07

ellemouton self-assigned this Mar 23, 2026

ellemouton force-pushed the g175-graph-db-cross-version branch 4 times, most recently from f6937d1 to fe7a97b Compare March 23, 2026 15:16

ellemouton requested review from bhandras and bitromortac March 23, 2026 15:32

ellemouton mentioned this pull request Mar 23, 2026

graph/db: v2 model and store support #10657

Open

bhandras previously approved these changes Mar 24, 2026

View reviewed changes

ellemouton force-pushed the g175-graph-db-cross-version branch 2 times, most recently from 6a07ff9 to d455707 Compare March 24, 2026 13:30

ellemouton added this to the v0.22.0 milestone Mar 25, 2026

bitromortac reviewed Mar 26, 2026

View reviewed changes

ellemouton force-pushed the g175-graph-db-cross-version branch from d455707 to 4620278 Compare March 27, 2026 11:23

ellemouton added 6 commits March 27, 2026 13:50

ellemouton force-pushed the g175-graph-db-cross-version branch from 4620278 to c3c0e14 Compare March 27, 2026 12:13

ellemouton added 6 commits March 27, 2026 14:19

docs: add release note for cross-version graph Store

1ef529b

ellemouton force-pushed the g175-graph-db-cross-version branch from c3c0e14 to 1ef529b Compare March 27, 2026 12:24

ellemouton mentioned this pull request Mar 30, 2026

graph/db: version horizon queries for v1/v2 gossip #10691

Merged

ellemouton marked this pull request as draft March 30, 2026 08:57

saubyk added this to lnd v0.22 Mar 30, 2026

github-project-automation bot moved this to Backlog in lnd v0.22 Mar 30, 2026

saubyk moved this from Backlog to In progress in lnd v0.22 Mar 30, 2026

		require.False(t, features.IsEmpty(),
		"both-version node should have features")

Conversation

ellemouton commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Key changes

Uh oh!

gemini-code-assist bot commented Mar 23, 2026

Uh oh!

bhandras left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bitromortac left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bitromortac Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bitromortac Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

ellemouton Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bitromortac Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

ellemouton Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

ellemouton commented Mar 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ellemouton commented Mar 23, 2026 •

edited

Loading