Skip to content

Map unsupported fsspec mtimes to NotImplementedException. - #588

Open
agrawal-priyank wants to merge 1 commit into
duckdb:mainfrom
agrawal-priyank:fix/fsspec-last-modified-not-implemented
Open

Map unsupported fsspec mtimes to NotImplementedException.#588
agrawal-priyank wants to merge 1 commit into
duckdb:mainfrom
agrawal-priyank:fix/fsspec-last-modified-not-implemented

Conversation

@agrawal-priyank

Copy link
Copy Markdown

Summary

  • Translate Python NotImplementedError and gcsfs KeyError('mtime') in PythonFilesystem::GetLastModifiedTime() into DuckDB NotImplementedException, so read_blob(...).last_modified is NULL instead of an untyped query failure.
  • Leave unrelated filesystem errors unchanged.
  • Add a custom fsspec regression test covering unsupported mtime, the gcsfs KeyError, real I/O errors, and a successful timestamp.

Fixes #584

Test plan

  • uv run pytest tests/fast/api/test_fsspec.py -v
  • uv run pytest tests/fast/test_filesystem.py -v
  • CI on this PR

DuckDB already turns NOT_IMPLEMENTED last-modified lookups into NULL; the Python adapter was leaking untyped exceptions (including gcsfs KeyError('mtime')), which aborted callers such as DuckLake CHECKPOINT.
@agrawal-priyank
agrawal-priyank force-pushed the fix/fsspec-last-modified-not-implemented branch from 169ffe2 to 62fc5e7 Compare August 16, 2026 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Translate unsupported fsspec modification times into a typed DuckDB error

1 participant