Skip to content

Scheduled functions: Concurrent execution by not awaiting on current run. - #5736

Open
Shubham8287 wants to merge 3 commits into
shub/fix-scheduled-timingfrom
shub/fix-scheduled-func
Open

Scheduled functions: Concurrent execution by not awaiting on current run.#5736
Shubham8287 wants to merge 3 commits into
shub/fix-scheduled-timingfrom
shub/fix-scheduled-func

Conversation

@Shubham8287

@Shubham8287 Shubham8287 commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Description of Changes

handle_queued only finishes after function associated with current item returns. It causes head-of-line blocking and cause scheduled reducers to get delayed behind slow procedures.

My intention was to fix only reducer behind procedure case but it turns out a general solution is easier to implement.

handle_queue now do not wait for function to finish but only adds a future in ScheduledActor::active_calls, which we poll later in ScheduledActor::run. Concurrency is bounded at ModuleHost level similar to normal procedures /reducers

Some implicit behavior changes:

  1. Scheduled functions are drained from SchedulerActor’s delay queue earlier and submitted to the database executor queue sooner.

  2. SchedulerActor::close may now take longer, because it waits for all already-dispatched scheduled calls
    in db executor, not just the single call that was previously awaited inline.

  3. (Major one) For already-expired scheduled functions, execution order is not guaranteed to match scheduled_at
    order. For example, if items scheduled for T- 5ms, T are all expired by the time the scheduler
    observes them, there is no guarantee that the T-5ms item is submitted before the T item though time observed from inside the reducer will still be monotonic.

API and ABI breaking changes

NA

Expected complexity level and risk

3, there could be unexpected implications related to module cleanup, resource exhaustion, etc.

Testing

Old tests has been modified for new behaviour.

@Shubham8287
Shubham8287 requested a review from gefjon August 13, 2026 13:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant