Prefect worker as a thread with in the app process

[navya-sammeta]

Prefect needs the worker to execute the task. It's additional orchestration, and deployment complexity. Is it possibel to run the workers within another app with a thread pool instantiated?

Basically, Imagine, Django/fast-api invoking a background task, but the worker itself is included within the django app with it's own thread pool. So, One of the django nodes/pods will pick the task and execute it.

[zzstoatzz]

hi @navya-sammeta - have you checked out .serve? That sounds similar to what you're describing

from prefect import flow

@flow
def foo(): ...

if __name__ == "__main__":
  foo.serve(cron='* * * * *') # this will listen for scheduled runs of this flow and execute them

[navya-sammeta]

deployment for the flow and starts a long-running process that monitors for work from the Prefect serve

As per documentation, it seemed to create a new process instead of thread.