[WIP-Don't review] Templatized scratch buffers for topology/rasterization ops

[sifakis]

⚠️ [WIP — please don't review yet] This is a draft to share direction and run CI early. The API and commit history are still in flux.

Summary

Make the transient scratch buffer type a (defaulted) template parameter of the CUDA topology/rasterization operators, so a downstream can inject its own buffer type for nanoVDB's internal scratch allocations without forking any nanoVDB headers.

Today these operators hardcode nanovdb::cuda::DeviceBuffer (i.e. cudaMallocAsync/cudaFreeAsync) for all of their internal scratch. A consumer that manages GPU memory through a different allocator — e.g. PyTorch's caching allocator in fvdb-core — currently has to fork the headers to redirect those allocations, otherwise the two allocator pools fragment each other and large workloads hit OOM. (See fvdb-core #655 for the workaround this is meant to replace.)

With this change the consumer instead instantiates the operator with its own ScratchBufferT, no patching required.

Design

• New ScratchBufferT template parameter on the operators, used for all device-only scratch.
  • Defaulted to nanovdb::cuda::DeviceBuffer on the public operators, so existing call sites and behavior are unchanged (byte-identical under the default).
  • Independent from the output buffer type. The grid the caller receives is still controlled by the existing getHandle<BufferT>() parameter; scratch is a separate axis. So you can mix, e.g. ScratchBufferT = DeviceBuffer with output BufferT = UnifiedBuffer, or vice-versa.
• The shared internal helper TopologyBuilder takes ScratchBufferT as a non-defaulted parameter (it's an implementation detail; the default belongs only on the public operators). This way an operator that forgets to forward the type fails to compile rather than silently falling back to DeviceBuffer.
• A scratch buffer type only needs a small interface: create(bytes, pool*, device, stream), deviceData(), and clear(stream).

What's covered

• tools/cuda/TopologyBuilder.cuh — the shared scratch (masks, offsets, parents, count buffers).
• tools/cuda/{DilateGrid,PruneGrid,MergeGrids,CoarsenGrid,RefineGrid}.cuh — thread the parameter through to TopologyBuilder.
• tools/cuda/MeshToGrid.cuh — its own scratch (transformed triangles, box/triangle pairs, sort/scan temporaries, etc.), plus its internal PruneGrid.

Supporting changes:

• Lifted TopologyBuilder::Data and MeshToGrid::BoxTrianglePair out to namespace scope (they don't depend on the scratch type), so the device functors that reference them see one consistent type across instantiations.
• cuda::UnifiedBuffer: added a stream-accepting clear(cudaStream_t) overload so it satisfies the scratch interface.
• MeshToGrid::getHandleAndUDF now actually honors its SidecarBufferT parameter (it previously hardcoded DeviceBuffer for the UDF sidecar allocation).

Dual-use buffers (intentionally left as DeviceBuffer for now)

Two buffers are host-populated and then uploaded (TopologyBuilder::mProcessedRoot, mData). These remain DeviceBuffer in this pass. The clean follow-up is to split each into a host buffer + a device ScratchBufferT half with an explicit copy between them; the parameter introduced here is already sufficient for that device half.

Testing

• The CUDA topology unit tests (DilateInjectPrune, RefineCoarsen, MergeGrids, MeshToGrid_UnitTetrahedron) were refactored into helpers templated on every buffer axis (scratch / grid-output / sidecar-output), following the existing testVoxelBlockManager<BuildT, BufferT> pattern, and are instantiated for both all-DeviceBuffer and all-UnifiedBuffer. All pass; results are identical between the two buffer types.
• Full TestNanoVDB.cu CUDA suite is green except one pre-existing, unrelated failure (UnifiedBuffer_IO, a missing test-data fixture).
• The ex_dilate / ex_refine / ex_coarsen / ex_mesh_to_grid CUDA examples build and run against OpenVDB references with correct results.

Notes for later (not for review yet)

• A working design note (TEMPLATIZED_BUFFERS_PLAN.md) is currently committed at the repo root and will be removed before this is opened for real review.
• Commit history is intentionally milestone-by-milestone and may be squashed/rebased.

🤖 Generated with Claude Code