Skip to content

[Fix-18540][Master] Reset the runtime state when recreating a failed task instance - #18541

Open
SEPURI-SAI-KRISHNA wants to merge 1 commit into
apache:devfrom
SEPURI-SAI-KRISHNA:Fix-18540
Open

[Fix-18540][Master] Reset the runtime state when recreating a failed task instance#18541
SEPURI-SAI-KRISHNA wants to merge 1 commit into
apache:devfrom
SEPURI-SAI-KRISHNA:Fix-18540

Conversation

@SEPURI-SAI-KRISHNA

Copy link
Copy Markdown

Was this PR generated or assisted by AI?

YES. The missing field resets were found and the fix and unit tests were drafted with AI
assistance (Claude Code); the behaviour, the comparison against the sibling factories and
the final code were reviewed and verified by me.

Purpose of the pull request

Closes #18540.

FailedRecoverTaskInstanceFactory recreates a FAILURE / KILL task instance when a
workflow is recovered with "Recover failed tasks". It clones the old instance, clears part
of the runtime state and inserts it as a new row — but it never resets retryTimes,
startTime, endTime, pid or alertFlag, all of which cloneTaskInstance copies.

Because a task that failed after exhausting its retries has retryTimes == maxRetryTimes,
and TaskExecution#isTaskInstanceCanRetry() is retryTimes < maxRetryTimes, the
recreated instance starts with a fully consumed retry budget and is never retried on the
recovery run, however many retries the task definition asks for.

The two sibling factories already do this correctly — FirstRunTaskInstanceFactory sets
retryTimes(0), startTime(null), endTime(null), alertFlag(NO), and
RetryTaskInstanceFactory sets startTime(null), endTime(null), pid(0) while
deliberately incrementing retryTimes. This change brings the recover factory in line
with them.

The stale startTime / endTime are user visible too: until the task actually starts, the
new instance is shown with the previous attempt's timestamps, and
TaskInstanceServiceImpl#queryTaskListPaging derives its duration from them.

Brief change log

  • FailedRecoverTaskInstanceFactory#createTaskInstance: reset retryTimes to 0 and
    clear startTime, endTime, pid and alertFlag on the recreated task instance.
  • Added FailedRecoverTaskInstanceFactoryTest.

Verify this pull request

This change added tests and can be verified as follows:

  • Added FailedRecoverTaskInstanceFactoryTest with three cases: the retry budget is
    restored, the runtime state of the failed attempt is cleared, and the origin instance is
    still marked invalid while the new one is inserted.
./mvnw -pl dolphinscheduler-master -am clean test \
    -Dtest=FailedRecoverTaskInstanceFactoryTest \
    -Dsurefire.failIfNoSpecifiedTests=false

Verified locally:

  • The new tests fail on dev and pass with this change. On dev the recreated instance
    reports retryTimes = 3 where 0 is expected, and keeps the startTime of the failed
    attempt where null is expected.
  • The full dolphinscheduler-master suite passes: 98 tests, 0 failures, 0 errors.
  • ./mvnw -pl dolphinscheduler-master spotless:check passes.

Pull Request Notice

Pull Request Notice

If your pull request contains incompatible change, you should also add it to docs/docs/en/guide/upgrade/incompatible.md

🤖 Generated with Claude Code

…task instance

- Reset retryTimes so a recovered task gets its whole retry budget back
  - Clear startTime/endTime/pid/alertFlag inherited from the failed attempt
  - Add FailedRecoverTaskInstanceFactoryTest

  Closes apache#18540

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@SbloodyS SbloodyS added bug Something isn't working first time contributor First-time contributor labels Aug 8, 2026
@SbloodyS SbloodyS added this to the 3.5.0 milestone Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend bug Something isn't working first time contributor First-time contributor test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] [Master] Recovered failed task keeps the exhausted retryTimes and never retries

2 participants