[#10669] feat(iceberg-rest): Add pagination support for list endpoints#10671
[#10669] feat(iceberg-rest): Add pagination support for list endpoints#10671laserninja wants to merge 1 commit intoapache:mainfrom
Conversation
|
|
Thanks for checking, @roryqi! 1. Backend considerations & how other Iceberg REST catalogs handle pagination: The Iceberg Catalog Java API ( This is consistent with how other Iceberg REST catalogs handle it:
Since Gravitino's 2. Will resolve the conflicts. |
What changes were proposed in this pull request?
Add
pageTokenandpageSizequery parameter support to the Iceberg REST server'slistNamespaces,listTables, andlistViewsendpoints, threading them through the full call chain (REST → Event/Hook Dispatchers → Executors → CatalogWrapper).Pagination is implemented as offset-based in-memory pagination in
IcebergCatalogWrapper. WhenpageSizeis provided, results are sliced andnextPageTokenis populated with the offset of the next batch. When no pagination parameters are provided, existing behavior (return all results) is preserved.Why are the changes needed?
The Iceberg REST spec defines
pageTokenandpageSizequery parameters on list endpoints, but Gravitino's Iceberg REST server did not accept or handle them. Clients with large catalogs cannot paginate results — everything was returned in a single response.Fix: #10669
Does this PR introduce any user-facing change?
Yes. The following REST endpoints now accept optional
pageTokenandpageSizequery parameters:GET /v1/{prefix}/namespacesGET /v1/{prefix}/namespaces/{namespace}/tablesGET /v1/{prefix}/namespaces/{namespace}/viewsHow was this patch tested?
testListNamespaceWithPaginationinTestIcebergNamespaceOperationstestListTablesWithPaginationinTestIcebergTableOperationstestListViewsWithPaginationinTestIcebergViewOperationsTestIcebergViewHookDispatcherandTestIcebergViewOperationExecutor./gradlew :iceberg:iceberg-rest-server:test -PskipITs)