-
-
Notifications
You must be signed in to change notification settings - Fork 387
Description
the numcodecs.fixescaleoffset codec is flawed for Zarr v3 and should not be used, because it has not been adapted properly to the Zarr v3 data model. To fix this we developed two new codecs in this PR zarr-developers/zarr-extensions#43
scale_offsetis an array->array codec that applies a scale and offset to the values of an input array. Here's the spec: https://github.com/d-v-b/zarr-extensions/tree/feat/scale-offset/codecs/scale_offsetcast_valueis an array -> array codec that changes the data type of the input array to a new data type by mapping the input values to output values via a casting procedure. Here's the spec: https://github.com/d-v-b/zarr-extensions/tree/feat/scale-offset/codecs/cast_value
we should implement these codecs here or externally and direct users away from numcodecs.fixedscaleoffset
one concern about cast_value is that it contains some logic that might be hard to express in performant numpy code. We may have to choose between a memory-and-compute inefficient pure numpy solution vs a performant compiled solution.
if you or someone you know used the scale-offset transformation with zarr data, either via a codec or via a metadata convention a la CF + xarray, please have a look at the linked PR, the specs, and keep an eye on the developments related to this feature in zarr python.