What happens?
When a filesystem registered through Connection.register_filesystem() cannot provide a modification time, querying the last_modified column of read_blob surfaces the raw Python exception instead of returning NULL.
One concrete case is gcsfs: GCSFileSystem.modified() indexes info(path)["mtime"], while some GCS object metadata (and synthesized directory entries) has no mtime. The resulting KeyError: 'mtime' currently reaches a DuckDB query as an untyped error and can abort callers such as DuckLake CHECKPOINT.
This is the Python/fsspec half of:
Why the error loses its type
PythonFilesystem::GetLastModifiedTime() currently calls filesystem.modified() directly. A Python exception crosses the nanobind boundary as std::exception; when DuckDB later constructs ErrorData(ex), it has ExceptionType::INVALID, so extensions cannot distinguish unsupported modification-time metadata from unrelated I/O failures without inspecting the message.
DuckDB core already has the desired downstream behavior: DirectFileReader converts ExceptionType::NOT_IMPLEMENTED failures for optional file metadata columns to NULL. An older_than predicate then naturally retains only files whose age is unknown.
Suggested direction
Translate the Python filesystem capability error at the adapter boundary in PythonFilesystem::GetLastModifiedTime():
- Catch
nb::python_error.
- Map Python
NotImplementedError to DuckDB NotImplementedException.
- Consider a narrowly scoped compatibility mapping for the current gcsfs
KeyError('mtime'), or coordinate with gcsfs so missing mtime is reported as NotImplementedError.
- Re-throw every unrelated Python exception unchanged.
Please add a regression test with a small custom fsspec filesystem whose modified() is unsupported, asserting that read_blob(...).last_modified is NULL while other filesystem errors still surface.
This keeps provider-specific exception classification in the Python filesystem adapter instead of requiring downstream extensions to accumulate message-matching helpers.
What happens?
When a filesystem registered through
Connection.register_filesystem()cannot provide a modification time, querying thelast_modifiedcolumn ofread_blobsurfaces the raw Python exception instead of returningNULL.One concrete case is
gcsfs:GCSFileSystem.modified()indexesinfo(path)["mtime"], while some GCS object metadata (and synthesized directory entries) has nomtime. The resultingKeyError: 'mtime'currently reaches a DuckDB query as an untyped error and can abort callers such as DuckLakeCHECKPOINT.This is the Python/fsspec half of:
CHECKPOINTfails withKeyError: 'mtime'when using GCS-buckets ducklake#1042Why the error loses its type
PythonFilesystem::GetLastModifiedTime()currently callsfilesystem.modified()directly. A Python exception crosses the nanobind boundary asstd::exception; when DuckDB later constructsErrorData(ex), it hasExceptionType::INVALID, so extensions cannot distinguish unsupported modification-time metadata from unrelated I/O failures without inspecting the message.DuckDB core already has the desired downstream behavior:
DirectFileReaderconvertsExceptionType::NOT_IMPLEMENTEDfailures for optional file metadata columns toNULL. Anolder_thanpredicate then naturally retains only files whose age is unknown.Suggested direction
Translate the Python filesystem capability error at the adapter boundary in
PythonFilesystem::GetLastModifiedTime():nb::python_error.NotImplementedErrorto DuckDBNotImplementedException.KeyError('mtime'), or coordinate with gcsfs so missingmtimeis reported asNotImplementedError.Please add a regression test with a small custom fsspec filesystem whose
modified()is unsupported, asserting thatread_blob(...).last_modifiedisNULLwhile other filesystem errors still surface.This keeps provider-specific exception classification in the Python filesystem adapter instead of requiring downstream extensions to accumulate message-matching helpers.