BigQuery temporary table creation fails if table_tmp uses a GCS path with hierarchy

[mayurbarge]

Sling version:

1.5.3

What is the Operating System?

Mac

Do you have a CLI Pro/Platform subscription?

No

Description of the issue

We have a pipeline which loads data from SFTP -> BigQuery.
We need to keep the temporary files in staging GCS location, and have them arranged in a tree like bucket in GCS.
Like
gs://mybucket/sling_temp/google_dataset/2025-01-01/google_ads/file.csv
gs://mybucket/sling_temp/google_dataset/2025-01-01/google_search/file.csv

If I set
'target_options': {'table_tmp': 'google_dataset/2025-01-01/google_ads/file.csv'}}
It fails while creating temporary tables in BigQuery as forward slashes, dashes are not allowed in the BigQuery.

There is currently no provision to pass GCS path as an additional parameter, same table_tmp config is used in staging and bigquery causing a conflict.

This is not supported currently.

Replication Configuration

No special conf is necessary. paths are not supported.

Log Output

NA

[mayurbarge]

If I instead follow another approach with below configuration.

'target_options': {'table_tmp': 'google_temp_files.Search_2025-12-21'}}} and 'target_options': {'table_tmp': 'google_temp_files.Ads_2025-12-21'}}}

I will have a dataset google_temp_files which will hold all files. (At least one level in BigQuery)
But it will create flat files in GCS
gs://mybucket/sling_temp/google_temp_files.Ads_2025-12-21.csv.gz
gs://mybucket/sling_temp/google_temp_files.Search_2025-12-21.csv.gz

Staging GCS and target BigQuery temporary files location being same, an organized GCS and BigQuery trees is hard to get.