graph/db: v2 model and store support#10657
graph/db: v2 model and store support#10657ellemouton wants to merge 16 commits intolightningnetwork:elle-g175-graph-db-cross-versionfrom
Conversation
Add version-aware range types for channel and node update horizon queries. V1 gossip uses unix timestamps for ordering while v2 uses block heights, so each range type validates that the correct bound type is provided for the requested gossip version. These types will be used in follow-up commits to version the NodeUpdatesInHorizon and ChanUpdatesInHorizon Store methods.
Replace the (startTime, endTime time.Time) parameters on NodeUpdatesInHorizon and ChanUpdatesInHorizon with (v GossipVersion, r NodeUpdateRange/ChanUpdateRange). The range types enforce version-correct bounds at the type level: v1 uses unix timestamps, v2 uses block heights. The KV store rejects non-v1 versions since it only stores v1 data. The SQL store handles v1 queries; v2 block-height range queries are added in the following commits. VersionedGraph wrappers supply the version from the embedded field, so callers only pass the range.
Add GetNodesByBlockHeightRange and GetChannelsByPolicyBlockRange SQL queries for v2 gossip horizon lookups which use block heights instead of unix timestamps. Hardcode version=1 in GetNodesByLastUpdateRange and GetChannelsByPolicyLastUpdateRange since only v1 gossip uses unix-timestamp-based ordering. Implement the v2 gossip cases in the SQL store's NodeUpdatesInHorizon and ChanUpdatesInHorizon using the new queries, replacing the placeholder TODO stubs. Add extractMaxBlockHeight helper for v2 channel pagination cursor tracking. Also add a TestV2HorizonQueries integration test that exercises both node and channel horizon lookups with block-height ranges on the SQL backend.
Add a gossip version parameter to FilterKnownChanIDs on the Store interface so that it can filter channel IDs for the correct gossip version. The KV store rejects non-v1 versions. Move the zombie-revival logic from ChannelGraph into VersionedGraph so that FilterKnownChanIDs is only exposed on VersionedGraph (which supplies the version from its embedded field). This means callers no longer need to pass a version explicitly.
Add four new Store interface methods: - FetchChannelEdgesByIDPreferHighest: version-agnostic channel lookup that returns the highest available gossip version. - FetchChannelEdgesByOutpointPreferHighest: same but keyed by outpoint. - GetVersionsBySCID: returns which gossip versions exist for a SCID. - GetVersionsByOutpoint: same but keyed by outpoint. The KV store implementations delegate to v1 since that's the only version it supports. The SQL store implementations iterate over versions from highest to lowest (for PreferHighest) or lowest to highest (for GetVersions). These will be used in follow-up commits to make ChannelGraph callers version-agnostic.
Remove the gossip version parameter from ForEachNode and ForEachChannel on the Store interface. Both methods now iterate across all gossip versions, yielding each unique node/channel exactly once. A versionsMask (uint32) is passed to the callback where bit 0 indicates a v1 entry exists and bit 1 indicates v2. The KV store always passes versionsMask=1 since it only stores v1 data. The SQL store collects entries across both versions, deduping by pub key (nodes) or channel ID (channels), preferring the highest version's data when both exist. VersionedGraph wrappers discard the versionsMask so that existing callers through that type don't need to change their callbacks.
…rappers Switch FetchChannelEdgesByID and FetchChannelEdgesByOutpoint on ChannelGraph to use the PreferHighest store variants so that callers get the highest available gossip version without needing to specify one. Also add GetVersionsBySCID and GetVersionsByOutpoint convenience wrappers on ChannelGraph that delegate to the corresponding Store methods.
Update ForEachNodeDirectedChannel and FetchNodeFeatures on ChannelGraph to iterate across all gossip versions (highest first) instead of hardcoding v1. This ensures that channels announced via v2 are preferred over v1 and that v2 features are used when available.
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly advances the graph database's support for gossip v2 by integrating new wire conversion utilities, enhancing versioned graph wrappers, and introducing specialized SQL queries for efficient v2 node traversal. These changes ensure better compatibility and handling of versioned gossip messages, laying crucial groundwork for the upcoming Gossip 1.75 epic. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
ellemouton
force-pushed
the
g175-graph-db-models
branch
from
e610b68 to
170379c
Compare
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces comprehensive support for Gossip V2 within the graph database. It includes new models and conversion helpers for node announcements, channel authentication proofs, edge information, and edge policies. The database layer has been updated with new SQL queries to support versioned graph traversal, prioritizing Gossip V2 data where available. Existing code has been refactored to use new helper methods for improved abstraction and consistency across gossip versions. A review comment highlighted an inconsistency in SQL alias naming for policy fields, suggesting a change from policy_2_ to policy2_ for better consistency.
| cp2.message_flags AS policy_2_message_flags, | ||
| cp2.channel_flags AS policy_2_channel_flags, |
There was a problem hiding this comment.
The aliases for policy 2 fields (policy_2_message_flags, policy_2_channel_flags) use an underscore between 'policy' and '2'. This is inconsistent with some other queries in this file (e.g., GetChannelsByIDs uses policy2_message_flags) and also with the aliases for policy 1 in this same query (policy1_...).
For consistency, it would be better to use policy2_... for all policy 2 aliases.
This also applies to the GetChannelByOutpointPreferHighestVersionWithPolicies query on lines 1264-1265.
cp2.message_flags AS policy2_message_flags,
cp2.channel_flags AS policy2_channel_flags,Update NodeFromWireAnnouncement to accept the lnwire.NodeAnnouncement interface instead of only *NodeAnnouncement1, and return an error for unsupported types. Add a NodeAnnouncement() method that returns the version-appropriate wire message from a Node model. Also add NodeTimestamp() which returns the version-appropriate ordering timestamp (unix time for v1, block height for v2).
Add ChannelAuthProofFromWireAnnouncement and ChannelAuthProofFromAnnounceSignatures for constructing auth proofs from wire messages. Add ChannelEdgeInfoFromWireAnnouncement for building a ChannelEdgeInfo from a ChannelAnnouncement interface. Also add ChannelEdgePolicyFromWireUpdate for constructing a ChannelEdgePolicy from a ChannelUpdate interface, and update the KV and SQL stores to use the new ChannelEdgeInfo.ChanProofBytes() accessor.
Add ToNodeAnnouncement which serializes a Node model back to its version-appropriate lnwire.NodeAnnouncement wire message. This enables round-tripping between the internal model and wire format. Also add V2NodeAddrs/SetV2NodeAddrs helpers for extracting and setting the typed address fields on v2 node announcements, and comprehensive tests for node wire round-tripping.
Add MarkEdgeZombie and MarkEdgeLive convenience methods to VersionedGraph that supply the gossip version from the embedded field, matching the pattern used by other VersionedGraph wrappers.
Add SQL queries for version-aware node traversal used by the sqlNodeTraverser: ListChannelsForNodeV2, GetV2NodesByPubKeys, ListChannelsWithPoliciesForCachePaginatedV2, and ListChannelsPaginatedV2. Update the KV store's ForEachNodeDirectedChannel to return ErrVersionNotSupportedForKVDB for non-v1 instead of silently returning no results.
Filter out non-v3 onion addresses when setting v2 node addresses, since v2 gossip only supports v3 onion addresses. Also update the SQL store to handle v2 node address filtering during persistence and retrieval.
ellemouton
force-pushed
the
g175-graph-db-models
branch
from
170379c to
c718ee9
Compare
ellemouton
force-pushed
the
elle-g175-graph-db-cross-version
branch
from
fe7a97b to
d455707
Compare
|
@ellemouton, remember to re-request review from reviewers when ready |
3 participants
Summary
Complete the graph database's v2 gossip support by adding wire conversion
helpers, versioned graph wrappers, SQL queries for v2 node traversal, and
v3-only onion address filtering.
Depends on: #10656 (graph/db: cross-version graph Store)
Part of the Gossip 1.75 epic.
Key changes
Node model generalization:
NodeFromWireAnnouncementnow accepts thelnwire.NodeAnnouncementinterface (v1 and v2). AddNodeAnnouncement()for version-appropriate wire round-tripping,
NodeTimestamp()forversion-aware ordering, and
ToNodeAnnouncement()for full serializationback to wire format including v2 address helpers.
Channel model helpers:
ChannelAuthProofFromWireAnnouncement,ChannelAuthProofFromAnnounceSignatures,ChannelEdgeInfoFromWireAnnouncement, andChannelEdgePolicyFromWireUpdatefor constructing internal models fromversioned wire messages.
VersionedGraph zombie wrappers:
MarkEdgeZombieandMarkEdgeLiveconvenience methods that supply the gossip version from the embedded field.
SQL v2 node traversal: Add
ListChannelsForNodeV2,GetV2NodesByPubKeys,ListChannelsWithPoliciesForCachePaginatedV2, andListChannelsPaginatedV2queries for version-aware node traversal.V3-only onion addresses: Filter out non-v3 onion addresses for v2
gossip, since the v2 protocol only supports v3 onion services.