Skip to content

Latest commit

 

History

History
6 lines (3 loc) · 595 Bytes

File metadata and controls

6 lines (3 loc) · 595 Bytes

This sample shows how to use a variable in a scenario to build a partitioned dataset when the partitions are not all present.

Normally, when you build a range of partitions for a dataset, say "2018-01-01/2018-01-04", the build fails if one of the dates in the range is missing. The solution is to pass the explicit list of partitions to build, which implies retrieving this list first and passing it as comma-separated.

The flow in the project has input -> input_partitioned (by date) -> output (partitioned by date). The ouput dataset is build by the scenario on all available partitions.