Batch Support - #142

Open

jpcamara wants to merge 69 commits into

jpcamara:batch-poc

Commits on Jan 17, 2026

Batch job POC
jpcamara
committed
94e4371
Use ActiveSupport::IsolatedExecutionState to honor user isolation level setting
jpcamara
committed
8b694ac
Ability to retrieve batch from a job
jpcamara
committed
07e0552
Allow batch jobs to be instances
jpcamara
committed
6cce814
Use text so the jobs store properly on mysql
jpcamara
committed
2c6a519
Handle on_failure and on_success
jpcamara
committed
5bffda3
Allow enqueueing into a batch instance
jpcamara
committed
7769a53
Block enqueueing if the batch is finished
jpcamara
committed
1ec4bff
Migration to allow nesting batches
jpcamara
committed
503b888
Expanded batch readme
jpcamara
committed
a36d13d
Force an initial batch check
jpcamara
committed
09ce612
Initial batch lifecycle tests
jpcamara
committed
84cc97a
Add job batches to queue_schema.rb as well
jpcamara
committed
c37cde4
Refactor internals and api namespace of batches

jpcamara
and
mhenrixon
committed
cb51f62
Move away from a batch_processed_at to batch_execution model
jpcamara
committed
fd0dc75
Reduce complexity of batches implementation
jpcamara
committed
8ba3ece
Test updates
jpcamara
committed
7da00ed
Create batch executions alongside ready and scheduled executions
jpcamara
committed
b94ce64
Leftover from previous implementation
jpcamara
committed
fa1a702
Move batch completion checks to job
jpcamara
committed
c02f4ad
Support rails versions that don't have after_all_transactions_commit
jpcamara
committed
e28d20e
Remove support for nested batches for now
jpcamara
committed
bd85d74
Fix starting batch in rails 7.1
jpcamara
committed
0595283
Helper status method
jpcamara
committed
18e016a
Remove parent/child batch relationship, which simplifies the logic
jpcamara
committed
5a19cf5
Performance improvements
jpcamara
committed
093e06a
We no longer need to keep jobs
jpcamara
committed
b096939
Removing pending_jobs column
jpcamara
committed
071b024
Update doc to reflect current feature state
jpcamara
committed
479a16d
We always save the batch first now, so we don't need to upsert
jpcamara
committed
d9ee1d6
Rubocop
jpcamara
committed
62bc2ac
Accidental claude.md
jpcamara
committed
5516be4
Allow omitting a block, which will just enqueue an empty job
jpcamara
committed
28f8dc0
Switch batch_id to active_job_batch_id
jpcamara
committed
acf1767
Make it so metadata is more ergonomic to include
jpcamara
committed
ddd4c13
Bad query field
jpcamara
committed
92087c4
Update metadata interface
jpcamara
committed
db19c14
Give more breathing room for CI test runs
jpcamara
committed
fcae152
Simplify code for how callbacks are serialized/deserialized
jpcamara
committed
9834e7e
.presence || nil is redundant
jpcamara
committed
46a353e
Move class << self to match rest of codebase
jpcamara
committed
7ea178c

Commits on Jan 21, 2026

Add description to batches
jpcamara
committed
eb54d47

Commits on Jan 24, 2026

Switch callback job batch interface
jpcamara
committed
e0b6e38

Commits on Apr 18, 2026

Merge branch 'main' into batch-poc
jpcamara
authored
3a725e2

Commits on Jul 29, 2026

Merge branch 'main' into batch-poc
jpcamara
authored
1ae3b8e
Fix batch completion and start lifecycle races

jpcamara
and
claude
committed
1b72c02
Fix batch progress accounting

jpcamara
and
claude
committed
abc6577
Capture batch membership at enqueue time

jpcamara
and
claude
committed
5fdee19
Keep batch tracking consistent across dispatch, failure and discard

jpcamara
and
claude
committed
700678e
Sweep stalled batches from the dispatcher's maintenance timer

jpcamara
and
claude
committed
1c1bb97
Pin batch invariants with race and regression tests

jpcamara
and
claude
committed
259d877
Document batch behavior, maintenance and upgrades

jpcamara
and
claude
committed
064053e
Keep the empty job and callbacks on Solid Queue in mixed-adapter apps

jpcamara
and
claude
committed
b5ca56d
Give the empty-batches lifecycle test the same time as its siblings

jpcamara
and
claude
committed
ef26096
Repair tracking rows leaked by swallowed removal errors

jpcamara
and
claude
committed
75f1a0a
Capture batch membership at build time too, and sweep without delay

jpcamara
and
claude
committed
8788fa4
Run the Active Job enqueue callback chain for batch callbacks

jpcamara
and
claude
committed
cbfa930
Clarify batch membership timing
jpcamara
committed
b469e77
Simplify batch membership docs
jpcamara
committed
9374e86
Clarify batch enqueue timing
jpcamara
committed
6a17dd0
Test deferred enqueue batch capture
jpcamara
committed
a9313ac
Clarify batch invariant comments
jpcamara
committed
6344e21

Commits on Aug 2, 2026

Merge pull request #3 from jpcamara/batch-review-fixes
jpcamara
authored
a2c772f

Commits on Aug 3, 2026

Align the batches README section with the house style

jpcamara
and
claude
committed
ae396b1

Commits on Aug 17, 2026

Merge remote-tracking branch 'origin/main' into batch-poc
rosa
committed
f0b5623
Count logical jobs in batch counters instead of attempts

rosa
and
claude
committed
b8701bd
Ship the batches schema as an optional migration until 2.0

rosa
and
claude
committed
b2459df
Document why the batch completion re-check is database-proof

rosa
and
claude
committed
6c9b2da
Extract batch sweeping into a Sweepable concern

rosa
and
claude
committed
d82a23e