Skip to content

ShardedDaemonProcess: throttle keep-alive messages and limit to subset of nodes#2734

Open
pjfanning wants to merge 5 commits intoapache:mainfrom
pjfanning:copilot/copy-changes-from-pr-31837
Open

ShardedDaemonProcess: throttle keep-alive messages and limit to subset of nodes#2734
pjfanning wants to merge 5 commits intoapache:mainfrom
pjfanning:copilot/copy-changes-from-pr-31837

Conversation

@pjfanning
Copy link
Member

Ports akka/akka-core#31837. Keep-alive pings were sent from every node on a fixed timer, causing thundering herd issues and unnecessary load (e.g. waking all projection instances simultaneously).

Part of #2730

Settings (ShardedDaemonProcessSettings)

  • Added keepAliveFromNumberOfNodes: Int (default: 3) — only the first N sorted cluster members send keep-alives, making selection deterministic across nodes
  • Added keepAliveThrottleInterval: FiniteDuration (default: 100 ms) — inter-message delay when sending keep-alives
  • Both settings exposed via Scala/Java with* APIs

KeepAlivePinger behavior

  • Identities sorted for deterministic ordering across nodes
  • If node is already MemberStatus.Up at actor start, fires Tick immediately instead of waiting for SelfUp
  • isActive() check gates whether this node participates in keep-alive at all
  • Keep-alive sending is now async via Source.throttle + pipeToSelf, so the next timer fires only after the current batch completes

Config (reference.conf)

pekko.cluster.sharded-daemon-process {
  keep-alive-from-number-of-nodes = 3
  keep-alive-throttle-interval = 100 ms
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants