From b7c5427cb9f561d7c83cca93ea4a372e545c1271 Mon Sep 17 00:00:00 2001 From: Dinesh Saini Date: Thu, 30 Apr 2026 12:14:12 +0000 Subject: [PATCH] Avoid reloading job rows after dispatch and schedule In `Job.dispatch_all` and `Job.schedule_all` (the hot path of `enqueue_all` / `ActiveJob.perform_all_later`), the post-insert step rebuilt the returned set with: where(id: .where(job_id: jobs.map(&:id)).pluck(:job_id)) which always issues two statements per execution table -- a `pluck` on the execution table and a follow-up `SELECT ... FROM solid_queue_jobs WHERE id IN (...)` -- to re-read rows we already hold in memory. The `Job` instances passed in are the ones just returned by `create_all_from_active_jobs`, so they are persisted and have ids. Filter `jobs` in memory against the plucked execution ids instead. Per `enqueue_all` batch: * dispatch_all: 4 queries -> 2 (drops the two job-table reloads) * schedule_all: 2 queries -> 1 (drops the job-table reload) The avoided `SELECT` is the most expensive of the four: it scans the wide `solid_queue_jobs` row including the serialized `arguments` payload, so the saving is bytes-over-the-wire and not just a round trip. Semantic equivalence: * Each job is inserted into at most one of ready_executions / blocked_executions / scheduled_executions in this code path, so the in-memory filter selects exactly the same job ids the prior `where(id: ...)` would have returned. * Both call sites (`prepare_all_for_execution` which concatenates with `+`, and `Execution::Dispatching#dispatch_jobs` which calls `.map(&:id)`) already coerce the result to an Array and do not depend on it being an `ActiveRecord::Relation` or on row order. * `jobs` is the same collection that was just queried back in `create_all_from_active_jobs`, so attribute freshness is unchanged versus the previous reload. Verified against the existing `job_test`, `ready_execution_test`, `dispatcher_test`, and `concurrency_controls_test` suites on SQLite (55 runs, 0 failures). --- app/models/solid_queue/job/executable.rb | 14 +++++--------- app/models/solid_queue/job/schedulable.rb | 3 ++- 2 files changed, 7 insertions(+), 10 deletions(-) diff --git a/app/models/solid_queue/job/executable.rb b/app/models/solid_queue/job/executable.rb index b0a4cb939..d47442bb6 100644 --- a/app/models/solid_queue/job/executable.rb +++ b/app/models/solid_queue/job/executable.rb @@ -41,15 +41,11 @@ def dispatch_all_one_by_one(jobs) end def successfully_dispatched(jobs) - dispatched_and_ready(jobs) + dispatched_and_blocked(jobs) - end - - def dispatched_and_ready(jobs) - where(id: ReadyExecution.where(job_id: jobs.map(&:id)).pluck(:job_id)) - end - - def dispatched_and_blocked(jobs) - where(id: BlockedExecution.where(job_id: jobs.map(&:id)).pluck(:job_id)) + job_ids = jobs.map(&:id) + dispatched_ids = ReadyExecution.where(job_id: job_ids).pluck(:job_id) + + BlockedExecution.where(job_id: job_ids).pluck(:job_id) + dispatched_ids = dispatched_ids.to_set + jobs.select { |job| dispatched_ids.include?(job.id) } end end diff --git a/app/models/solid_queue/job/schedulable.rb b/app/models/solid_queue/job/schedulable.rb index 8ce2a4a93..32708de69 100644 --- a/app/models/solid_queue/job/schedulable.rb +++ b/app/models/solid_queue/job/schedulable.rb @@ -23,7 +23,8 @@ def schedule_all_at_once(jobs) end def successfully_scheduled(jobs) - where(id: ScheduledExecution.where(job_id: jobs.map(&:id)).pluck(:job_id)) + scheduled_ids = ScheduledExecution.where(job_id: jobs.map(&:id)).pluck(:job_id).to_set + jobs.select { |job| scheduled_ids.include?(job.id) } end end