[fix][fn] Allow retainKeyOrdering on Go functions - #26421
Merged
david-streamlio merged 3 commits intoAug 26, 2026
Conversation
### Motivation apache#26414 taught the Go runtime to honour retainKeyOrdering: resolveSubscriptionType in pulsar-function-go/pf/instance.go selects a KeyShared subscription for it, the same way python_instance.py does. Nothing can reach that code on the cluster path. doGolangChecks still refuses retainKeyOrdering outright ("Retain Key Orderering not yet supported in Go function"), and validateNonJavaFunction is called by the worker REST API (FunctionsImpl), so cluster submission is exactly what the guard blocks. LocalRunner never calls it, which is why the KeyShared branch is reachable under localrun and nowhere else. Unlike the Python dead letter case, convert() carries retainKeyOrdering into FunctionDetails unconditionally, so this guard is the only thing in the way. ### Modifications Drop the guard from doGolangChecks. The combinations that would genuinely be contradictory are already rejected in doCommonChecks, for every runtime: retainKeyOrdering with EFFECTIVELY_ONCE ("retain Key ordering cannot be set"), and retainKeyOrdering together with retainOrdering ("Only one of retain ordering or retain key ordering can be set"). doGolangChecks refuses EFFECTIVELY_ONCE for Go before either is reached, so nothing needs repeating here. The Go client needs no extra configuration for KeyShared: ConsumerOptions leaves KeySharedPolicy nil, toProtoKeySharedMeta(nil) returns nil, and the broker then applies its default auto-split hash range. The maxMessageRetries guard stays: the Go runtime does not honour retryDetails yet, and a test pins that so this change cannot be widened by accident. ### Verifying this change Five tests in FunctionConfigUtilsTest: Go accepts retainKeyOrdering and converts to a KEY_SHARED subscription; accepts retainOrdering and converts to FAILOVER; rejects both ordering modes together; rejects retainKeyOrdering with EFFECTIVELY_ONCE; and still refuses message retries. Confirmed the tests are not vacuous: restoring the guard fails testGoFunctionAcceptsRetainKeyOrdering.
…in-key-ordering The @option descriptions in CmdFunctions carry a runtime marker that the docs sync parses into the "Support" column of the published pulsar-admin CLI reference, and that `functions create --help` prints verbatim. Both flags were marked #Java. That has been wrong for Python since long before apache#26414: python_instance.py selects Failover for retainOrdering and KeyShared for retainKeyOrdering. apache#26414 added the same to the Go runtime, and the preceding commit makes retainKeyOrdering reachable for a Go function submitted to a cluster, so both now read #Java, Python, Go.
freeznet
approved these changes
Aug 26, 2026
freeznet
left a comment
Contributor
There was a problem hiding this comment.
The validation and conversion paths are consistent with the Java and Python function behavior. I left one non-blocking simplification suggestion inline.
The active constraints are already stated in doCommonChecks, which rejects retainKeyOrdering with EFFECTIVELY_ONCE and retainKeyOrdering together with retainOrdering, and both are covered by tests. Keeping doGolangChecks limited to active validation avoids a second description of the rules, and drops references to specific Go and Python implementation files that would go stale.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Master Issue: #26404
Follow-up to #26414, which closed #26405.
Motivation
#26414 taught the Go runtime to honour
retainOrderingandretainKeyOrdering:resolveSubscriptionTypeinpulsar-function-go/pf/instance.goselectsFailoverfor the first andKeySharedfor the second, matchingpython_instance.py.Half of it cannot be reached.
doGolangChecksstill refusesretainKeyOrderingoutright:validateNonJavaFunctionhas a single caller, the worker REST API (FunctionsImpl), so cluster submission is exactly what the guard blocks.LocalRunnernever calls it, which is why theKeySharedbranch is reachable underlocalrunand nowhere else. As master stands,pulsar-admin functions create --go ... --retain-key-orderingfails at creation, and the code #26414 added for it is dead on the cluster path.retainOrderingis unaffected — it was never gated — so that half of #26414 works today.This is the same shape of gap as the Python dead letter queue in #26400, with one difference:
convertcarriesretainKeyOrderingintoFunctionDetailsunconditionally (L297), so this guard is the only thing in the way.#26405 was auto-closed when #26414 merged. Its body only described the runtime gap, so the validation half was never in its scope; happy to reopen it instead of tracking this here if that is preferred.
Modifications
Drop the guard from
doGolangChecks.The combinations that would genuinely be contradictory are already rejected in
doCommonChecks, for every runtime:retainKeyOrderingwithEFFECTIVELY_ONCE— "When effectively once processing guarantee is specified, retain Key ordering cannot be set"retainKeyOrderingtogether withretainOrdering— "Only one of retain ordering or retain key ordering can be set"doGolangChecksrefusesEFFECTIVELY_ONCEfor Go before either is reached, so nothing needs repeating for Go specifically. ThemaxMessageRetriesguard stays — the Go runtime does not honourretryDetailsyet (#26406) — and a test pins it so this change cannot be widened by accident.The Go client needs no extra configuration for
KeyShared:ConsumerOptionsleavesKeySharedPolicynil,toProtoKeySharedMeta(nil)returns nil, and the broker then applies its default auto-split hash range.Correct the CLI runtime markers. The
@Optiondescriptions inCmdFunctionscarry a marker that the docs sync parses into the Support column of the published pulsar-admin CLI reference, and thatfunctions create --helpprints verbatim.--retain-orderingand--retain-key-orderingwere both#Java, which has been wrong for Python since long before #26414 —python_instance.pyhas always selectedFailoverandKeySharedfor them. Both now read#Java, Python, Go.Verifying this change
This change added tests and can be verified as follows:
FunctionConfigUtilsTest: Go acceptsretainKeyOrderingand converts to aKEY_SHAREDsubscription; acceptsretainOrderingand converts toFAILOVER; rejects both ordering modes together; rejectsretainKeyOrderingwithEFFECTIVELY_ONCE; and still refuses message retries.FunctionConfigUtilsTest: 42/42 pass.testGoFunctionAcceptsRetainKeyOrderingand nothing else.Does this pull request potentially affect one of the following parts:
This is relaxing only. A configuration that was accepted before is still accepted; one input that was rejected —
retainKeyOrderingon a Go function — is now accepted, and behaves as the runtime already implements. Java and Python functions are untouched.Documentation
doc-requireddoc-not-neededThe
FunctionConfigtable indocs/functions-cli.mddocumentsretainOrderingandretainKeyOrderingwith no runtime qualification. With this change that is accurate for the first time — all three runtimes honour both — so the table needs no edit. The#Java, Python, Gomarkers above keep the generated CLI reference in step.