PrefectDbtRunner not working when running in parallel tasks

[sol-joseph]

Bug summary

We're using prefect_dask.DaskTaskRunner to run tasks in parallel. We have a flow that runs multiple tasks in parallel, each of which call PrefectDbtRunner. This previously worked fine when using DbtCoreOperation.

Now, they're interfering with each other e.g. They fail because of deps not running correctly as it's trying to import packages that have nothing to do with it, but are in the other one. When run in series they all work fine.

The also all seem to fail as soon as one of them gets a bug.

Version info

Version:              3.6.5
API version:          0.8.4
Python version:       3.10.19
Git commit:           857c60b5
Built:                Thu, Dec 04, 2025 08:01 PM
OS/Arch:              linux/x86_64
Profile:              ephemeral
Server type:          cloud
Pydantic version:     2.12.5
Server:
  Database:           sqlite
  SQLite version:     3.46.1
Integrations:
  prefect-dbt:        0.7.14.dev71+gadf77ef0a
  prefect-dask:       0.3.6
  prefect-shell:      0.3.1

Additional context

No response

[desertaxle]

Hey @sol-joseph! Do you have any logs that you can share from the failures that you're seeing when running PrefectDbtRunner instances in parallel? Also, what method are you using to run tasks in parallel?

[sol-joseph]

We're using DaskTaskRunner() as our task runner.

The failures vary depending on when exactly they all get kicked off, but on one run for example I get:

Encountered an error:
expected str, bytes or os.PathLike object, not NoneType

After the logs show it doing stuff completely unrelated to this project like:

Updating lock file in file path: /prefect/worker/dbt3/rollup/update_frequency_blend_ratios_e8787e41748e468fac62cf2cc1cce5c6_tmp/package-lock.yml

update_frequency_blend_ratios is a totally separate task and should not be running at all in this one

[desertaxle]

Could you provide a short example script that reliably shows the issue? Since the failures vary, we'll need to be able to reproduce the issue in order to fix it.

[sol-joseph]

The issue occurs every time for me, the only reason the failures vary is because they don't kick off at the exact same instance every time