feat(produce): schema-aware Produce Record backend (UX-1292)#2461
Open
c-julin wants to merge 2 commits into
Open
feat(produce): schema-aware Produce Record backend (UX-1292)#2461c-julin wants to merge 2 commits into
c-julin wants to merge 2 commits into
Conversation
Adds schema-introspection + sample-generator infrastructure for the schema-aware Produce Record flow: - proto: new GenerateSchemaSample RPC; extended PublishMessagePayloadOptions with optional schema_id / index_path so the wire format can address nested Protobuf messages. - backend/pkg/schemasample: schema-type-aware sample JSON generator (Avro / Protobuf / JSON Schema), unit-tested. - backend/pkg/console: GetSchemaRegistrySubjectDetails now embeds per-version MessageTypes for Protobuf entries (descriptor walk via cachedSchemaClient), so the produce UI can render a typed message-type picker without a second RPC. /schema-registry/schemas endpoint + new GetAllSchemas service method. Per-version message-type resolution extracted to populateProtoMessageTypes helper to keep the parent function under the cyclop ceiling. - backend/pkg/serde: Protobuf+schemaId payloads now dispatch through the schema-registry serde path so registered subjects resolve correctly.
Contributor
|
The latest Buf updates on your PR. Results from workflow Buf CI / validate (pull_request).
|
5 tasks
Addresses adversarial code review findings on PR #2461: - JSON tags on MessageTypeInfo so the REST subject-details response uses fullyQualifiedName/indexPath camelCase. Without tags Go encoded them as FullyQualifiedName/IndexPath, breaking the frontend message-type picker. - avroSamplePrimitiveOrRef no longer double-marks the visited flag. The duplicate-flag bug caused the first reference to a non-recursive named record to render as null instead of expanding the record's fields. - walkMessageTypes skips synthetic map-entry descriptors so map<K,V> field types don't appear as selectable message types in the produce UI. - handleGetAllSchemas clamps Limit to 1000 (and defaults to 1000 when unset) so an unbounded request can't pull MB-sized payloads. - Regression test TestAvro_NonRecursiveRecordReference locks the visited-flag fix.
r-vasquez
approved these changes
May 22, 2026
| if schemas[i].Type != sr.TypeProtobuf { | ||
| continue | ||
| } | ||
| i := i |
Contributor
There was a problem hiding this comment.
Do we still need this on the latest go? doesn't go fix ./... removes it?
| func walkMessageTypes(msgs protoreflect.MessageDescriptors, prefix []int32, out *[]proto.MessageTypeInfo) { | ||
| for i := 0; i < msgs.Len(); i++ { | ||
| md := msgs.Get(i) | ||
| path := append(append([]int32(nil), prefix...), int32(i)) |
Contributor
There was a problem hiding this comment.
Suggested change
| path := append(append([]int32(nil), prefix...), int32(i)) | |
| path := append(slices.Clone(prefix), int32(i)) |
Nit (I believe) you can use slices.Clone
Comment on lines
+28
to
+29
| // Avro/JSON-Schema literal type names. Extracted as constants so goconst doesn't | ||
| // flag the repeated occurrences and to make typos compile-time errors. |
Contributor
There was a problem hiding this comment.
Suggested change
| // Avro/JSON-Schema literal type names. Extracted as constants so goconst doesn't | |
| // flag the repeated occurrences and to make typos compile-time errors. | |
| // Avro/JSON-Schema literal type names. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Backend half of the schema-aware Produce Record flow. Frontend lands in a stacked PR on top of this branch.
GenerateSchemaSampleRPC; extendedPublishMessagePayloadOptionswithschema_id+index_pathso the wire format can address nested Protobuf messages.backend/pkg/schemasample— schema-type-aware sample JSON generator (Avro / Protobuf / JSON Schema), unit-tested.backend/pkg/console:GetSchemaRegistrySubjectDetailsembeds per-versionMessageTypesfor Protobuf entries (descriptor walk viacachedSchemaClient), so the produce UI can render a typed message-type picker without a second RPC./schema-registry/schemasendpoint +GetAllSchemasservice method.populateProtoMessageTypesto keep the parent function under the cyclop ceiling.backend/pkg/serde— Protobuf+schemaId payloads now dispatch through the schema-registry serde path so registered subjects resolve correctly.Test plan
task backend:lint— 0 issues locally.task proto:generateregenerates cleanly.