We want hive partitioning with filter pushdown. This means at the point of issuing a scan we need to filter out files which don't match column filters.
We don't want to parse Duckdb query, filter on hive columns and do some special handling because this is hacky.
This means we need a specific data source which filters files during scan().
We want hive partitioning with filter pushdown. This means at the point of issuing a scan we need to filter out files which don't match column filters.
We don't want to parse Duckdb query, filter on hive columns and do some special handling because this is hacky.
This means we need a specific data source which filters files during scan().