- Add upgrade script
pg_incremental--1.4--1.5.sqlso existing installs receive thepg_cronguard in_drop_extension_trigger(the trigger function lives outside the extension, so fixingpg_incremental--1.0.sqlalone does not update it). The migration drops and recreates the function and event trigger and removes them from extension membership again. - Add
max_batches_per_runto file list pipelines (incremental.create_file_list_pipelineandincremental.file_list_pipelines). Default-1means a singleexecute_pipelinerun processes all unprocessed paths in that invocation; a positive value limits how many batch iterations run per call (one file per iteration when not batched, one array batch when batched).
- Use
lake_file.listas the default file list function, fallback to crunchy
- Add a
max_batch_sizeargument to sequence pipelines to limit the number of sequence IDs processed per execution - Improves handling of large batch uploads by allowing incremental processing in manageable chunks
- Adds an incremental.skip_file function to use for erroneuous files in file pipelines
- Removes the hard dependency on pg_cron at CREATE EXTENSION time
- Fixes bug that could cause batched file list pipelines to crash
- Add a
max_batch_sizeargument to file list pipelines - Improve performance of batched file list pipelines
- Adjust the default schedule of file list pipelines to every 15 minutes
- Fixes a bug that prevented file list pipelines from being refreshed
- Fixes a bug that prevented insert..select pipelines (#2)
- Fixes a bug that caused file list pipelines to repeat first file
- Add batched mode for file list pipeline
- Add incremental.default_file_list_function setting
- PostgreSQL 15 support
- Initial release