Skip to content

Introduce local timeout for activity heartbeats#2804

Open
Sushisource wants to merge 4 commits intotemporalio:masterfrom
Sushisource:activity-heartbeat-timeout-local
Open

Introduce local timeout for activity heartbeats#2804
Sushisource wants to merge 4 commits intotemporalio:masterfrom
Sushisource:activity-heartbeat-timeout-local

Conversation

@Sushisource
Copy link
Member

What was changed

See title

Why?

User request, matches Core behavior (though only for heartbeat timeouts, not overall activity timeouts)

Checklist

  1. Closes Keep track of current heartbeat timeout while retrying failures in doHeartBeatLocked #2775

  2. How was this tested:
    Added test

  3. Any docs updates needed?

@Sushisource Sushisource requested a review from a team as a code owner March 12, 2026 00:05
private static final long DEFAULT_LOCAL_HEARTBEAT_TIMEOUT_BUFFER_MILLIS = 5000;

static long getLocalHeartbeatTimeoutBufferMillis() {
String envVal = System.getenv("TEMPORAL_ACTIVITY_TIMEOUT_DELAY");
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did this b/c I didn't want a worker option for a setting I expect basically no one will ever care about adjusting. If we feel otherwise, can promote.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could use a Java property here, what we did for some other debug configs, but this does work as well

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh that's a good call let's do that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh does no test use this setting? I thought you needed this to make tests faster. If no test needs it I would just make it a constant.

this.localHeartbeatTimeoutBufferMillis = localHeartbeatTimeoutBufferMillis;
if (this.heartbeatTimeoutMillis > 0) {
this.heartbeatTimeoutFuture =
heartbeatExecutor.schedule(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason you did and executor and a future instead of just keeping track of the last time a heartbeat succeeded and throwing an exception if it exceeded the timeout?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Throw from where? The heartbeating method itself? Good point. I did this because I was thinking we'd proactively cancel the activity a-la what Core does, but, that turns out not to work here without more refactoring.

Can certainly change it to not bother w/ a future, or could do the more elaborate thing. Former is probably sufficient and is easier.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Keep track of current heartbeat timeout while retrying failures in doHeartBeatLocked

2 participants