-
Notifications
You must be signed in to change notification settings - Fork 855
Subgraph support for GLVs #3428
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
GumpacG
wants to merge
4
commits into
apache:master
Choose a base branch
from
GumpacG:subgraph
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
225 changes: 225 additions & 0 deletions
225
gremlin-dotnet/src/Gremlin.Net/Structure/IO/GraphBinary4/Types/GraphSerializer.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,225 @@ | ||
| #region License | ||
|
|
||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one | ||
| * or more contributor license agreements. See the NOTICE file | ||
| * distributed with this work for additional information | ||
| * regarding copyright ownership. The ASF licenses this file | ||
| * to you under the Apache License, Version 2.0 (the | ||
| * "License"); you may not use this file except in compliance | ||
| * with the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, | ||
| * software distributed under the License is distributed on an | ||
| * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| * KIND, either express or implied. See the License for the | ||
| * specific language governing permissions and limitations | ||
| * under the License. | ||
| */ | ||
|
|
||
| #endregion | ||
|
|
||
| using System; | ||
| using System.Collections; | ||
| using System.Collections.Generic; | ||
| using System.IO; | ||
| using System.Linq; | ||
| using System.Threading; | ||
| using System.Threading.Tasks; | ||
|
|
||
| namespace Gremlin.Net.Structure.IO.GraphBinary4.Types | ||
| { | ||
| /// <summary> | ||
| /// A <see cref="Graph"/> serializer for GraphBinary. The wire format is a count-prefixed | ||
| /// list of vertices (each with their vertex properties and meta-properties), followed by a | ||
| /// count-prefixed list of edges (each with their properties). Vertex/edge labels are written | ||
| /// as a single-element list, parent placeholders are written as <c>null</c>. | ||
| /// </summary> | ||
| public class GraphSerializer : SimpleTypeSerializer<Graph> | ||
| { | ||
| /// <summary> | ||
| /// Initializes a new instance of the <see cref="GraphSerializer" /> class. | ||
| /// </summary> | ||
| public GraphSerializer() : base(DataType.Graph) | ||
| { | ||
| } | ||
|
|
||
| /// <inheritdoc /> | ||
| protected override async Task WriteValueAsync(Graph value, Stream stream, GraphBinaryWriter writer, | ||
| CancellationToken cancellationToken = default) | ||
| { | ||
| await writer.WriteNonNullableValueAsync(value.Vertices.Count, stream, cancellationToken) | ||
| .ConfigureAwait(false); | ||
| foreach (var vertex in value.Vertices.Values) | ||
| { | ||
| await WriteVertexAsync(vertex, stream, writer, cancellationToken).ConfigureAwait(false); | ||
| } | ||
|
|
||
| await writer.WriteNonNullableValueAsync(value.Edges.Count, stream, cancellationToken) | ||
| .ConfigureAwait(false); | ||
| foreach (var edge in value.Edges.Values) | ||
| { | ||
| await WriteEdgeAsync(edge, stream, writer, cancellationToken).ConfigureAwait(false); | ||
| } | ||
| } | ||
|
|
||
| private static async Task WriteVertexAsync(Vertex vertex, Stream stream, GraphBinaryWriter writer, | ||
| CancellationToken cancellationToken) | ||
| { | ||
| await writer.WriteAsync(vertex.Id, stream, cancellationToken).ConfigureAwait(false); | ||
| await writer.WriteNonNullableValueAsync(new List<string> { vertex.Label }, stream, cancellationToken) | ||
| .ConfigureAwait(false); | ||
|
|
||
| var vertexProperties = AsList<VertexProperty>(vertex.Properties); | ||
| await writer.WriteNonNullableValueAsync(vertexProperties.Count, stream, cancellationToken) | ||
| .ConfigureAwait(false); | ||
| foreach (var vp in vertexProperties) | ||
| { | ||
| await writer.WriteAsync(vp.Id, stream, cancellationToken).ConfigureAwait(false); | ||
| await writer.WriteNonNullableValueAsync(new List<string> { vp.Label }, stream, cancellationToken) | ||
| .ConfigureAwait(false); | ||
| await writer.WriteAsync((object?)vp.Value, stream, cancellationToken).ConfigureAwait(false); | ||
|
|
||
| // placeholder for the parent vertex | ||
| await writer.WriteAsync(null, stream, cancellationToken).ConfigureAwait(false); | ||
|
|
||
| var metaProperties = AsList<Property>(vp.Properties); | ||
| await writer.WriteNonNullableValueAsync(metaProperties, stream, cancellationToken) | ||
| .ConfigureAwait(false); | ||
| } | ||
| } | ||
|
|
||
| private static async Task WriteEdgeAsync(Edge edge, Stream stream, GraphBinaryWriter writer, | ||
| CancellationToken cancellationToken) | ||
| { | ||
| await writer.WriteAsync(edge.Id, stream, cancellationToken).ConfigureAwait(false); | ||
| await writer.WriteNonNullableValueAsync(new List<string> { edge.Label }, stream, cancellationToken) | ||
| .ConfigureAwait(false); | ||
|
|
||
| await writer.WriteAsync(edge.InV.Id, stream, cancellationToken).ConfigureAwait(false); | ||
| // placeholder for the in-vertex label (always null in this context) | ||
| await writer.WriteAsync(null, stream, cancellationToken).ConfigureAwait(false); | ||
|
|
||
| await writer.WriteAsync(edge.OutV.Id, stream, cancellationToken).ConfigureAwait(false); | ||
| // placeholder for the out-vertex label (always null in this context) | ||
| await writer.WriteAsync(null, stream, cancellationToken).ConfigureAwait(false); | ||
|
|
||
| // placeholder for the parent (never present) | ||
| await writer.WriteAsync(null, stream, cancellationToken).ConfigureAwait(false); | ||
|
|
||
| var edgeProperties = AsList<Property>(edge.Properties); | ||
| await writer.WriteNonNullableValueAsync(edgeProperties, stream, cancellationToken) | ||
| .ConfigureAwait(false); | ||
| } | ||
|
|
||
| /// <inheritdoc /> | ||
| protected override async Task<Graph> ReadValueAsync(Stream stream, GraphBinaryReader reader, | ||
| CancellationToken cancellationToken = default) | ||
| { | ||
| var graph = new Graph(); | ||
|
|
||
| var vertexCount = | ||
| (int)await reader.ReadNonNullableValueAsync<int>(stream, cancellationToken).ConfigureAwait(false); | ||
| for (var i = 0; i < vertexCount; i++) | ||
| { | ||
| var vertex = await ReadVertexAsync(stream, reader, cancellationToken).ConfigureAwait(false); | ||
| if (vertex.Id != null) | ||
| { | ||
| graph.Vertices[vertex.Id] = vertex; | ||
| } | ||
| } | ||
|
|
||
| var edgeCount = | ||
| (int)await reader.ReadNonNullableValueAsync<int>(stream, cancellationToken).ConfigureAwait(false); | ||
| for (var i = 0; i < edgeCount; i++) | ||
| { | ||
| var edge = await ReadEdgeAsync(graph, stream, reader, cancellationToken).ConfigureAwait(false); | ||
| if (edge.Id != null) | ||
| { | ||
| graph.Edges[edge.Id] = edge; | ||
| } | ||
| } | ||
|
|
||
| return graph; | ||
| } | ||
|
|
||
| private static async Task<Vertex> ReadVertexAsync(Stream stream, GraphBinaryReader reader, | ||
| CancellationToken cancellationToken) | ||
| { | ||
| var vId = await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); | ||
| var vLabelList = (List<string?>)await reader | ||
| .ReadNonNullableValueAsync<List<string?>>(stream, cancellationToken).ConfigureAwait(false); | ||
| var vLabel = vLabelList.Count > 0 ? vLabelList[0] ?? "" : ""; | ||
|
|
||
| var vpCount = (int)await reader.ReadNonNullableValueAsync<int>(stream, cancellationToken) | ||
| .ConfigureAwait(false); | ||
| var vertexProperties = new List<VertexProperty>(vpCount); | ||
| for (var j = 0; j < vpCount; j++) | ||
| { | ||
| var vpId = await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); | ||
| var vpLabelList = (List<string?>)await reader | ||
| .ReadNonNullableValueAsync<List<string?>>(stream, cancellationToken).ConfigureAwait(false); | ||
| var vpLabel = vpLabelList.Count > 0 ? vpLabelList[0] ?? "" : ""; | ||
| var vpValue = await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); | ||
|
|
||
| // discard the parent vertex placeholder | ||
| await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); | ||
|
|
||
| var metaProps = await reader.ReadNonNullableValueAsync<List<object?>>(stream, cancellationToken) | ||
| .ConfigureAwait(false); | ||
| var metaPropsArray = (metaProps as List<object>)?.ToArray() ?? Array.Empty<object>(); | ||
|
|
||
| vertexProperties.Add(new VertexProperty(vpId, vpLabel, vpValue, null, metaPropsArray)); | ||
| } | ||
|
|
||
| return new Vertex(vId, vLabel, vertexProperties.Cast<object>().ToArray()); | ||
| } | ||
|
|
||
| private static async Task<Edge> ReadEdgeAsync(Graph graph, Stream stream, GraphBinaryReader reader, | ||
| CancellationToken cancellationToken) | ||
| { | ||
| var eId = await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); | ||
| var eLabelList = (List<string?>)await reader | ||
| .ReadNonNullableValueAsync<List<string?>>(stream, cancellationToken).ConfigureAwait(false); | ||
| var eLabel = eLabelList.Count > 0 ? eLabelList[0] ?? "" : ""; | ||
|
|
||
| var inVId = await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); | ||
| // discard the in-vertex label placeholder (always null on the wire) | ||
| await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); | ||
| var outVId = await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); | ||
| // discard the out-vertex label placeholder (always null on the wire) | ||
| await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); | ||
| // discard the parent placeholder | ||
| await reader.ReadAsync(stream, cancellationToken).ConfigureAwait(false); | ||
|
|
||
| var edgeProps = await reader.ReadNonNullableValueAsync<List<object?>>(stream, cancellationToken) | ||
| .ConfigureAwait(false); | ||
| var edgePropsArray = (edgeProps as List<object>)?.ToArray() ?? Array.Empty<object>(); | ||
|
|
||
| var inVertex = ResolveVertex(graph, inVId); | ||
| var outVertex = ResolveVertex(graph, outVId); | ||
|
|
||
| return new Edge(eId, outVertex, eLabel, inVertex, edgePropsArray); | ||
| } | ||
|
|
||
| private static Vertex ResolveVertex(Graph graph, object? vertexId) | ||
| { | ||
| if (vertexId != null && graph.Vertices.TryGetValue(vertexId, out var existing)) | ||
| { | ||
| return existing; | ||
| } | ||
| return new Vertex(vertexId, ""); | ||
| } | ||
|
|
||
| private static List<T> AsList<T>(IEnumerable? source) | ||
| { | ||
| if (source == null) | ||
| { | ||
| return new List<T>(); | ||
| } | ||
| return source.Cast<T>().ToList(); | ||
| } | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we need to mention python here too? it's essentially all the GLVs being announced for the first time. when i did python i don't think i added any docs because i knew all these other ones had to land too.
i also think this doesn't read like Upgrade Documentation. It should be doing more to explain the relevance of this feature and less about technical details and minutiae.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've changed it to mention that all GLVs support subgraphs and took out the technical details. Thanks!