Skip mask IFDs when resolving overview_level (#1504)#1518
Merged
brendancol merged 2 commits intoxarray-contrib:mainfrom May 8, 2026
Merged
Skip mask IFDs when resolving overview_level (#1504)#1518brendancol merged 2 commits intoxarray-contrib:mainfrom
brendancol merged 2 commits intoxarray-contrib:mainfrom
Conversation
open_geotiff(path, overview_level=N) was indexing the IFD list directly. When a TIFF puts a transparency-mask IFD (NewSubfileType bit 2) between the full-res IFD and the overviews, as some GDAL COGs do, overview_level=1 returned the 1-bit mask instead of the first overview. The reader now filters out IFDs whose NewSubfileType has bit 2 set before indexing, matching what GDAL and rasterio do. Out-of-range overview_level raises ValueError with the actual count of non-mask IFDs (and any masks) instead of silently clamping to the last IFD. - IFD.subfile_type and IFD.is_mask properties on parsed IFDs - select_overview_ifd helper in _header.py - routed the four overview_level call sites (CPU read, COG-over-HTTP, GPU read, _read_geo_info) through the helper
There was a problem hiding this comment.
Pull request overview
This PR fixes open_geotiff(..., overview_level=N) selecting the wrong IFD when GeoTIFF/COG files interleave transparency-mask IFDs (NewSubfileType bit 2) with overviews, by routing overview selection through a helper that skips mask IFDs and raises on out-of-range requests.
Changes:
- Added
IFD.subfile_type/IFD.is_maskhelpers and a centralizedselect_overview_ifd()overview resolver. - Updated CPU, COG-over-HTTP, GPU, and geo-metadata read paths to use the shared overview-selection logic.
- Added regression tests covering mask-interleaving behavior, out-of-range errors, and normal COG behavior.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
xrspatial/geotiff/_header.py |
Adds NewSubfileType helpers and select_overview_ifd() to skip mask IFDs and validate overview_level. |
xrspatial/geotiff/_reader.py |
Routes local and HTTP COG read paths through select_overview_ifd() instead of indexing/clamping raw IFDs. |
xrspatial/geotiff/__init__.py |
Routes GPU read and _read_geo_info through select_overview_ifd() for consistent behavior. |
xrspatial/geotiff/tests/test_overview_filter.py |
Adds regression/unit tests for mask-interleaved IFD chains and out-of-range handling. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…orskip tifffile
- ``select_overview_ifd`` previously kept any non-mask IFD, which let
multi-page-document IFDs (NewSubfileType bit 1) through. Tighten the
filter to keep only the full-res IFD (subfile_type=0) plus genuine
overview IFDs (bit 0 set, bit 2 clear). Pages and any other future
flag combinations are now excluded so ``overview_level`` indexes the
pyramid only. Diagnostic text updated to "pyramid IFDs" / "non-pyramid
IFD".
- Test module had ``import tifffile`` at the top; switched to
``pytest.importorskip("tifffile")`` so the file doesn't fail at
collection in environments without tifffile.
- Removed unused ``IFD`` import from the test module.
- Added two new test cases: page-IFD (subfile_type=2) is filtered, and
overview-of-mask (subfile_type=5) is filtered (mask bit dominates).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1504.
open_geotiff(path, overview_level=N)was indexing the IFD list directly. When a TIFF puts a transparency-mask IFD (NewSubfileType bit 2) between the full-res IFD and the overviews, as some GDAL COGs do,overview_level=1returned the 1-bit mask instead of the first overview.The reader now filters out IFDs whose NewSubfileType has bit 2 set before indexing, matching what GDAL and rasterio do. An out-of-range
overview_levelraisesValueErrorwith the actual count of non-mask IFDs (and any mask IFDs present), instead of silently clamping to the last IFD.Changes:
IFD.subfile_typeandIFD.is_maskproperties on the parsed IFDselect_overview_ifdhelper in_header.pyoverview_levelcall sites (local CPU read, COG-over-HTTP, GPU read,_read_geo_info) through the helperTests in
xrspatial/geotiff/tests/test_overview_filter.pywrite a 3-IFD TIFF with a mask wedged between the full-res IFD and an overview, then check thatopen_geotiff(..., overview_level=1)returns the overview rather than the mask. The ValueError path and a normal 4-IFD COG (no masks) are covered too.