Add ZoneAwareLB settings for MeshConfig and DestinationRule by sschepens · Pull Request #3733 · istio/api

sschepens · 2026-07-02T18:28:42Z

This is needed to implement Zone Aware Routing support in Istio.

I'm adding a zone_aware_lb_setting in DestinationRule to allow configuring enabling Zone Aware LB,it's mutually exclusive with locality_lb_setting and it's settings are very similar, with the following differences:

distribute is not actually needed here because Envoy distributes traffic automatically
failoverPriority cannot specify region, zone or subZone topology labels because we always partition by region and zone and subZone don't make sense with Zone Aware because envoy automatically shifts traffic between zones.

I'm also adding a mesh-wide zone_aware_lb_setting to enable zone aware routing globally. For practical reasons, there is no way to simply reject this setting being enabled together with locality_lb_setting because locality_lb_setting is actually enabled by default and validation occurs with the defaults merged.

Issue: istio/istio#35102 (comment)
RFC: https://docs.google.com/document/d/1nPVSTIy_s0s9cRBS7DNwHZkDXhHO1qW4x9F9W7y_gyo/edit?tab=t.0#heading=h.xw1gqgyqs5b

istio-policy-bot · 2026-07-02T18:28:46Z

😊 Welcome @sschepens! This is either your first contribution to the Istio api repo, or it's been
a while since you've been here.

You can learn more about the Istio working groups, Code of Conduct, and contribution guidelines
by referring to Contributing to Istio.

Thanks for contributing!

Courtesy of your friendly welcome wagon.

sschepens · 2026-07-02T18:29:25Z

@ramaraochavali @keithmattix @howardjohn could you take a look at this please?

ramaraochavali

Added some comments. @hzxuzhonghu can you PTAL as well?

ramaraochavali · 2026-07-09T07:10:56Z

+//
+// - The downstream proxy is configured with  `ISTIO_META_ENABLE_SELF_DISCOVERY=true`
+//    (set via `proxyMetadata`).
+// - The upstream cluster has at least `minClusterSize` hosts (default 6). Below


What is the significance of minClusterSize and the default value 6? Does it mean if a service has two pods per AZ - we can not use zone aware load balancing?

If it has two AZs and two pods per AZ, then by default it wouldn't do ZoneAwareRouting, but this is configurable of course, this is just Envoy's default.

There isn't much documentation on why the default is 6. My understanding is that this is based on statistical analysis, since Zone Aware Routing computes per-zone percentages based on the ratio of local-zone hosts to total hosts and deciding which fraction of traffic can stay local vs what must be sent cross-zone to keep hosts evenly loaded. When you have very few hosts the traffic can be skewed by rounding and end up being not very accurate. Also with very little hosts, when a host appears/disappears it changes the percentages dramatically which make the traffic distribution unstable, you could end up overloading individual hosts.

ramaraochavali · 2026-07-09T07:12:57Z

+//   this threshold zone-aware load balancing is disabled entirely, regardless of other
+//   settings.
+// - The upstream cluster spans more than one zone.
+// - There is at least one healthy endpoint in the local zone.


So if all endpoints in the local zone are unhealthy would the traffic spill over to other zones as per the zone aware load balancing calculations or is it completely disabled? Which load balancing is used if it is disabled?

I can clarify this, if all endpoints in the local zone are unhealthy traffic spills to other zones, basically envoy "fallbacks" to doing the selected LB algorithm (ie ROUND_ROBIN, etc) across all the endpoints of priority 0. Just as if you were to configure locality weigthed lb without specifying locality weights.

ramaraochavali · 2026-07-09T07:15:25Z

+  // apply. This is useful when failing over traffic across regions would not
+  // improve service health or may need to be restricted for other reasons
+  // like regulatory controls.
+  message Failover {


Should we call this FailoverRegion or call from and to as from_region and to_region to be explicit?

We could, but my intention was to leave this the same as in LocalityLbSetting, in both cases you can only specify regions, I'd prefer to maintain the consistency.

ramaraochavali · 2026-07-09T07:24:09Z

+  // Minimum number of hosts in the upstream cluster required for zone-aware
+  // load balancing to be enabled. If the total number of upstream hosts falls below
+  // this value, zone-aware load balancing is disabled for this cluster regardless of
+  // other settings. This guards against zone-skew issues that can occur with


This is confusing. Is it total number of hosts in upstream cluster or is it per zone? How is zone-skew identified based on upstream cluster hosts count?

It's the total number of hosts across all zones in priority 0 (same region).

Regarding the skew, I have clarified this in the first comment.

do you think we should clarify this further in the documentation?

sschepens · 2026-07-13T18:44:27Z

@ramaraochavali I improved the docs a bit. Do you think we should further explain why minClusterSize is needed?

ramaraochavali · 2026-07-14T11:25:17Z

Do you think we should further explain why minClusterSize is needed?

I think "why" it is needed is clear. But what value should be configured by user is not clear. How will I know what to set? How to tune it - are there any envoy stats that help us to tune it?

sschepens · 2026-07-14T12:26:22Z

I think "why" it is needed is clear. But what value should be configured by user is not clear. How will I know what to set? How to tune it - are there any envoy stats that help us to tune it?

You could just check per-host and per-az traffic to see if the traffic is unbalanced. Envoy also provides per-zone stats which adds stats a per-zone stat to upstream cluster, you can use this to correctly verify that you're doing Zone Aware Routing and to know how much traffic is being kept local vs remote.

ramaraochavali · 2026-07-15T10:52:41Z

ok. Looks like lb_zone_cluster_too_small metric tells us if zone aware routing is disabled because of this cluster size. And we can tune based on the metrics you shared above. Do you if it automatically turns off zone aware routing if the size is less and turns it back on when the endpoints increase say for example because of HPA?

Should we just keep this out of our API? have you seen any one having need to tweak this?

sschepens · 2026-07-15T12:12:17Z

ok. Looks like lb_zone_cluster_too_small metric tells us if zone aware routing is disabled because of this cluster size. And we can tune based on the metrics you shared above. Do you if it automatically turns off zone aware routing if the size is less and turns it back on when the endpoints increase say for example because of HPA?

Yes, it turns off and on automatically when the cluster size changes (for example HPA).

Should we just keep this out of our API? have you seen any one having need to tweak this?

I'd prefer we kept it, we had some cases in which we had to configure this for some services, it's really the only knob Zone Aware Routing has, the other knobs are just failover configurations which are not strictly part of Zone Aware Routing.

sschepens · 2026-07-15T14:57:37Z

@ramaraochavali I'm gonna remove the release note check on this since it will be added with the implementation PR

sschepens · 2026-07-15T14:59:27Z

/retest

keithmattix · 2026-07-15T15:17:51Z

-  // However, this requires outlierDetection to actually take effect for a particular
+  // Only one of locality_lb_setting and zone_aware_lb_setting should be enabled;
+  // if both are set, zone_aware_lb_setting takes precedence and this setting is ignored.
+  // Failover settings, require outlierDetection to actually take effect for a particular


Suggested change

// Failover settings, require outlierDetection to actually take effect for a particular

// Failover settings require outlierDetection to actually take effect for a particular

sschepens requested a review from a team as a code owner July 2, 2026 18:28

istio-testing added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jul 2, 2026

This was referenced Jul 2, 2026

lb: implement zone aware load balancer istio/istio#60765

Open

proposal: support zone aware routing istio/istio#35102

Open

add zone aware lb settings

97dd888

sschepens force-pushed the zone-aware-routing branch from d14020f to 97dd888 Compare July 8, 2026 14:23

ramaraochavali reviewed Jul 9, 2026

View reviewed changes

istio-testing added the needs-rebase Indicates a PR needs to be rebased before being merged label Jul 11, 2026

Merge remote-tracking branch 'upstream/master' into zone-aware-routing

ff807f8

istio-testing removed the needs-rebase Indicates a PR needs to be rebased before being merged label Jul 13, 2026

improve docs

8d2f811

sschepens force-pushed the zone-aware-routing branch from 3f85b5e to 8d2f811 Compare July 13, 2026 18:45

ramaraochavali approved these changes Jul 15, 2026

View reviewed changes

sschepens added the release-notes-none Indicates a PR that does not require release notes. label Jul 15, 2026

keithmattix approved these changes Jul 15, 2026

View reviewed changes

istio-testing merged commit c9bb2b8 into istio:master Jul 15, 2026
5 checks passed

	// Failover settings, require outlierDetection to actually take effect for a particular
	// Failover settings require outlierDetection to actually take effect for a particular

Uh oh!

Conversation

sschepens commented Jul 2, 2026

Uh oh!

istio-policy-bot commented Jul 2, 2026

Uh oh!

sschepens commented Jul 2, 2026

Uh oh!

ramaraochavali left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sschepens commented Jul 13, 2026

Uh oh!

ramaraochavali commented Jul 14, 2026

Uh oh!

sschepens commented Jul 14, 2026

Uh oh!

ramaraochavali commented Jul 15, 2026

Uh oh!

sschepens commented Jul 15, 2026

Uh oh!

sschepens commented Jul 15, 2026

Uh oh!

sschepens commented Jul 15, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants