fix(arrow/scalar): implement Release()/Retain() on *scalar.Extension#851
Open
zeroshade wants to merge 1 commit into
Open
fix(arrow/scalar): implement Release()/Retain() on *scalar.Extension#851zeroshade wants to merge 1 commit into
zeroshade wants to merge 1 commit into
Conversation
*scalar.Extension implemented neither Release() nor Retain(), so compute.ScalarDatum.Release() -- which only forwards to values that satisfy the unexported releasable interface -- never released an extension scalar's storage, leaking any buffers it held. Delegate to the storage scalar when it is Releasable, mirroring *scalar.Binary and the nested scalar types. TestLiteralToDatumIntervalYearToMonth previously used DefaultAllocator to dodge this leak; switch it to a CheckedAllocator with AssertSize(0) to lock in the fix.
lidavidm
approved these changes
Jun 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
Closes #827.
*scalar.Extensionimplementsvalue(),equals(),Validate(),ValidateFull(),CastTo(), andString(), but neitherRelease()norRetain().compute.ScalarDatum.Release()only forwardsRelease()when the wrapped scalar satisfies the unexportedreleasableinterface (arrow/compute/datum.go):Because
*scalar.Extensiondoesn't satisfy that interface, the type assertion fails and the inner storage scalar is never released — even though the storage scalar (e.g.*scalar.FixedSizeListbacking the interval extension types) carries buffer-bearing arrays. Any code path that produces an extension scalar and releases it through aScalarDatumleaks those buffers. Concretely this affects the*types.IntervalYearToMonthType/*types.IntervalDayTypepaths inarrow/compute/exprs.*scalar.Extensionwas the lone outlier among buffer-holding scalar types:*scalar.Binary,*scalar.List,*scalar.Struct,*scalar.Dictionary,*scalar.SparseUnion,*scalar.DenseUnion, and*scalar.RunEndEncodedall already delegateRetain/Releaseto their inner storage.What changes are included in this PR?
*scalar.Extensionnow implementsRetain()andRelease(), delegating to the storage scalar when it satisfies theReleasableinterface (arrow/scalar/scalar.go). This mirrors*scalar.Binaryand the nested scalar types:This is purely additive:
Releasable(e.g. primitive numeric storage) are no-ops, matching today's behavior.*scalar.FixedSizeListfor the interval extension types) are now properly released.Retain()half restores symmetry so a caller taking ownership of an extension scalar can bump the storage refcount.TestLiteralToDatumIntervalYearToMonthis switched tomemory.NewCheckedAllocatorwithdefer mem.AssertSize(t, 0)(arrow/compute/exprs/exec_internal_test.go). That test previously usedmemory.DefaultAllocatorspecifically to dodge this leak and carried a comment deferring the fix to this issue; the workaround comment is removed.Are these changes tested?
Yes.
TestLiteralToDatumIntervalYearToMonthnow asserts zero outstanding allocations via a checked allocator. Verified that it fails without the scalar change (invalid memory size exp=0, got=384, with leak traces pointing at the extension scalar's storage) and passes with it. The fullarrow/scalar/...andarrow/compute/exprs/...suites pass, andgo vetis clean.Are there any user-facing changes?
No breaking changes. Adding
Retain()/Release()methods to an exported type is non-breaking in Go. The only behavioral change is the intended one: buffers held by an extension scalar's storage are now released when the scalar (or aScalarDatumwrapping it) is released, instead of leaking.