Skip to content

fix(#699): move batch state from transient to engine_data#710

Merged
chubes4 merged 4 commits intomainfrom
fix/699-batch-state-engine-data
Mar 8, 2026
Merged

fix(#699): move batch state from transient to engine_data#710
chubes4 merged 4 commits intomainfrom
fix/699-batch-state-engine-data

Conversation

@chubes4
Copy link
Member

@chubes4 chubes4 commented Mar 8, 2026

Summary

  • Moves batch fan-out state (data packets, engine snapshot, offset) from a WordPress transient to the parent job's engine_data column
  • Batch state now survives deploys, cache flushes, and Redis restarts — the root cause of 22 batch_state_missing failures on 2026-03-07

What changed

fanOut() — stores batch_state key in engine_data alongside existing batch metadata. No more set_transient().

processChunk() — reads batch_state from engine_data, updates offset there between chunks, unset()s the key when all chunks are processed (or on cancellation). No more get_transient()/delete_transient().

Teststest_fanout_stores_transienttest_fanout_stores_batch_state_in_engine_data, zero-children test seeds engine_data instead of transient, new test_batch_state_cleaned_up_after_all_chunks_processed test.

No backward compat for in-flight transient batches — any batch mid-flight during a deploy was already dead (that's the bug this fixes).

Closes #699

@github-actions
Copy link

github-actions bot commented Mar 8, 2026

Homeboy Results — data-machine

Lint

Tooling versions

  • Homeboy CLI: homeboy 0.71.0
  • Extension: wordpress from https://github.com/Extra-Chill/homeboy-extensions
  • Extension revision: unknown
  • Action: Extra-Chill/homeboy-action@v1

ℹ️ PR test scope resolved to full for compatibility with installed Homeboy CLI

lint (changed files only)

  • PHPStan: PHPSTAN SUMMARY: 202 errors at level 5

Test

Tooling versions

  • Homeboy CLI: homeboy 0.71.0
  • Extension: wordpress from https://github.com/Extra-Chill/homeboy-extensions
  • Extension revision: unknown
  • Action: Extra-Chill/homeboy-action@v1

⚡ PR test scope resolved to changed

test

  • PHPCS: LINT SUMMARY: 0 errors, 5 warnings
  • Fixable: 5 | Files with issues: 2 of 362
Top violations
  Generic.Formatting.MultipleStatementAlignment.NotSameWarning     5
- PHPStan: PHPSTAN SUMMARY: 202 errors at level 5 - OK (155 tests, 1244 assertions)

Audit

Tooling versions

  • Homeboy CLI: homeboy 0.71.0
  • Extension: wordpress from https://github.com/Extra-Chill/homeboy-extensions
  • Extension revision: unknown
  • Action: Extra-Chill/homeboy-action@v1

ℹ️ PR test scope resolved to full for compatibility with installed Homeboy CLI

audit (changed files only)

  • Actionable audit summary:
  • Alignment score: 0.693
  • Severity counts: info: 2, unknown: 73, warning: 2
  • Drift increased: no
  • Outliers in current run: 73
  • Parsed outlier entries: 73
  • Top actionable findings:
    1. inc/Abilities/Engine/PipelineBatchScheduler.php — naming_mismatch — Helper-like name does not match convention suffix 'Ability': PipelineBatchScheduler
    2. tests/Unit/Engine/PipelineBatchSchedulerTest.php — missing_method — Missing method: tear_down
    3. tests/Unit/Engine/PipelineBatchSchedulerTest.php — missing_interface — Missing interface: WP_UnitTestCase
    4. tests/Unit/Engine/PipelineBatchSchedulerTest.php — unreferenced_export — Public function 'set_up_before_class' is not referenced by any other file
    5. tests/Unit/Abilities/SystemAbilitiesTest.php — outlier — (outlier)

Homeboy Action v1

chubes4 added 3 commits March 7, 2026 21:16
Tests were using hardcoded flow_id=1/pipeline_id=1 which don't exist
in the test DB. When the engine triggers FlowFiles::get_file_context()
on child job creation, it throws because the flow doesn't exist.

Fix: create real pipeline + flow in setUp and use those IDs.
@chubes4 chubes4 merged commit 9bcdd21 into main Mar 8, 2026
3 checks passed
@chubes4 chubes4 deleted the fix/699-batch-state-engine-data branch March 8, 2026 03:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Batch fan-out: move batch state from transient to engine_data

1 participant