[fix][fn] Honour producerSpec batching configuration in the Go function runtime

[david-streamlio]

Fixes #26391

Motivation

PIP-401 (#23860) made producer batching configurable for Pulsar Functions. The setting travels as ProducerConfig.batchingConfig → ProducerSpec.batchingSpec in FunctionDetails, and the Java runtime applies it in ProducerBuilderFactory.

The Go runtime never reads it. getProducer hardcodes BatchingMaxPublishDelay: time.Millisecond * 10 and never sets DisableBatching, so batchingSpec arrives in the instance — the generated bindings in pb/Function.pb.go already carry it — and is silently dropped.

Two user-visible consequences:

• Every Go function is pinned to a 10ms publish-latency floor. Below roughly one message per 10ms per instance, that inverts the point of batching: every message opens a batch, waits the full linger for companions that never arrive, and flushes alone. At 27 msg/s per instance, messages arrive every ~37ms, so each one pays a flat 10ms tax while every batch still contains exactly one message.
• A function configured with batchingConfig.enabled: false still batches, because DisableBatching is never set.

maxPendingMessages was ignored as well.

This is the Go counterpart of #26390, fixed for Python in #26392.

Modifications

• Add pf/producerConfig.go with producerOptionsFromSpec(), translating a ProducerSpec into pulsar.ProducerOptions.
• Use it in getProducer(), which serves both the sink producer and the producers behind context.NewOutputMessage(), so one call site covers both paths.

The translation follows the same rules as the Java runtime:

• Unset or non-positive spec fields are left at their zero value so the client default applies.
• A nil spec, or a spec with no BatchingSpec, keeps batching enabled with a 10ms delay, matching BatchingUtils.convertFromSpec(nil).
• BatchingSpec.BatchBuilder overrides ProducerSpec.BatchBuilder, matching the order ProducerBuilderFactory applies them in.

The existing compression and batchBuilder handling moves into the same helper, so getProducer no longer mixes configuration translation with producer creation. That accounts for the deletions in instance.go.

No proto regeneration is needed: pb/Function.pb.go already contains BatchingSpec and ProducerSpec.GetBatchingSpec().

Out of scope, deliberately:

• DisableBlockIfQueueFull stays false (the producer blocks). The Java runtime hardcodes blockIfQueueFull(true) too and exposes no configuration for it, so making it configurable means adding a new ProducerSpec field — a config-surface change that belongs in its own PR.
• RoundRobinRouterBatchingPartitionSwitchFrequency and MaxPendingMessagesAcrossPartitions have no equivalent in the Go client and are ignored.

Documentation

There is no pulsar-function-go README to extend, and adding one solely for this would be out of proportion. The documentation here is the go doc comment on producerOptionsFromSpec, which carries the full ProducerSpec → pulsar.ProducerOptions mapping table and the three parity rules above.

The user-facing gap is in apache/pulsar-site and I have not addressed it there:

• The Pulsar Functions producer/batching configuration reference documents batchingConfig without saying which runtimes honour it. That silence is what made both [Functions] Python function runtime hardcodes producer batching config, ignoring producerSpec.batchingSpec #26390 and [Functions] Go function runtime hardcodes producer batching config, ignoring producerSpec.batchingSpec #26391 hard to spot — a user configures the setting and gets no signal that their function is ignoring it.
• With this PR and [fix][fn] Honour producerSpec batching configuration in the Python function runtime #26392 both merged, batchingConfig is honoured by all three runtimes, which makes a single accurate docs update possible for the first time. I raised the same point on [fix][fn] Honour producerSpec batching configuration in the Python function runtime #26392 and suggested waiting for this PR rather than writing a note that would go stale; if reviewers agree, one pulsar-site PR can cover Java, Python and Go together.

Marking doc-required for that reason. Happy to open the pulsar-site PR once both of these land.

Verifying this change

• Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

• 14 unit tests in pf/producerConfig_test.go: 10 covering the spec-to-options translation directly, and 4 driving getProducer through a fake pulsar.Client that captures the options.
• One of those drives a context.NewOutputMessage() topic rather than the sink topic, covering the second producer path.
• Explicit backwards-compatibility cases assert that a function with no producerSpec still gets DisableBatching: false, BatchingMaxPublishDelay: 10ms, CompressionType: LZ4 and the default batcher.
• Neutralising the helper and reverting the getProducer wiring fails 12 of the 14, confirming the tests pin the new behaviour rather than passing vacuously.
• go build ./... and go test ./pf/... pass (full package, not only the new tests).
• golangci-lint run -c ./golangci.yml ./pf reports 0 issues, using v2.12.2 — the version ci-go-functions.yaml installs.
• ./gradlew quickCheck passes; both new files carry the ASF header.

go vet reports three pre-existing lock-copy warnings (instance.go:312, instance.go:650, instanceConf.go:133). They are untouched by this change and CI does not run go vet.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

No box checked: no configuration surface is added or changed, and defaults are unchanged. This makes the Go runtime honour configuration that already exists and is already accepted from users today.

[david-streamlio]

Same note as #26392, which this mirrors for the Go runtime: the doc-required label here is stale.

apache/pulsar-site#1215 ("Document producerConfig.batchingConfig for Pulsar Functions") merged on 25 Aug and covers this surface, so I have relabelled to doc-complete. Happy to revert if a reviewer disagrees.

This one is green and has had no review activity since 20 Aug.

[freeznet]

Verified the ProducerSpec/BatchingSpec mapping against FunctionConfigUtils, BatchingUtils, ProducerBuilderFactory, and pulsar-client-go v0.20.0. getProducer covers both output paths, preserves the existing defaults, and applies the supported batching and pending-message settings. The full Go build and test suite pass locally.