Skip to content

geotiff: uncompressed tiled writer pre-allocates a buffer it never uses #1736

@brendancol

Description

@brendancol

Describe the bug

xrspatial/geotiff/_writer.py:653 allocates a contiguous bytearray plus a memoryview at the top of the uncompressed-tile branch:

tile_bytes = tw * th * bytes_per_sample * samples
total_buf = bytearray(n_tiles * tile_bytes)
mv = memoryview(total_buf)

The inner loop then builds each tile separately via tile_arr.tobytes() and appends to a Python list. total_buf and mv are never read.

For an uncompressed write of a large raster, this is n_tiles * tw * th * bytes_per_sample * samples of dead allocation held alongside the actual tile list. Peak memory roughly doubles vs the compressed branch.

Expected behavior

Remove the unused total_buf and mv allocations.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingoomOut-of-memory risk with large datasets

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions