Target Health and Zonal Distribution: Which Zones Receive Traffic

A load balancer's health checks do not only decide which targets receive requests. On a Network Load Balancer they decide which Availability Zones exist as far as DNS is concerned, and that mechanism has a terminal state most people have never looked at: when nothing is healthy anywhere, the load balancer starts sending traffic to everything.

This guide is about where traffic actually lands, which is a different question from whether the load balancer is up.

Cross-zone is off by default on one and mandatory on the other#

The single most consequential difference between the two load balancer types, and it is a default rather than a feature:

The target group setting wins. If cross-zone is enabled on the load balancer and disabled on the target group, traffic to that target group does not cross zones. One exception: a target group of type alb always inherits from the load balancer and cannot override.

For a Classic Load Balancer the default depends on how you created it — disabled via the API or CLI, enabled via the console — which is worth knowing only because it means you cannot infer the setting from the era of the deployment.

What the distribution actually looks like#

AWS's own worked example is the clearest way to see why this matters. Two Availability Zones, two targets in zone A, eight in zone B. Route 53 hands out load balancer node addresses evenly, so each node receives 50% of client traffic.

With cross-zone off, each node can only reach targets in its own zone, so zone A's two targets split the same 50% that zone B's eight targets split. A four-to-one imbalance in instance count becomes a four-to-one imbalance in load per instance.

Every target is drawn, so the four-to-one difference in count is the picture rather than a figure in a table. Route 53 hands out the two node addresses evenly, so each column receives half the clients regardless of what is behind it — and with cross-zone off a node can only reach its own column. Two targets splitting half the traffic each get four times the load of eight targets splitting the other half.

This is the practical argument for symmetric capacity per zone rather than symmetric capacity in total. An autoscaling group that is "big enough" across the Region can still be badly overloaded in one zone.

Zonal DNS is the mechanism#

A Network Load Balancer has a DNS record and an IP address per enabled Availability Zone. Health does not just gate targets, it gates those records:

• Every enabled zone with at least one healthy target publishes its address.
• A zone whose target group has no healthy target has its address removed from DNS, so new connections stop being directed there at all. AWS's zonal health and health check pages both state this without qualification; the troubleshooting page adds "and cross-zone load balancing is disabled", which is the reading that makes mechanical sense — a node with cross-zone on can still reach healthy targets elsewhere, so there is nothing to withdraw for. Do not count on zone withdrawal as a failure signal while cross-zone is on.
• Zonal health is visible as the ZonalHealthStatus CloudWatch metric, which is the signal you want to alarm on rather than inferring zone withdrawal from a drop in traffic.

A zone can fail its zonal health check for more reasons than "no healthy targets" — the healthy count falling below a configured minimum, a zonal shift or auto-shift in progress, or AWS shifting traffic away from a zone it has detected problems in. Those last ones mean a zone can disappear from DNS without anything being wrong with your targets.

Fail-open makes total failure look like total health#

Here is the part worth internalising. If every target fails health checks in every enabled zone, the load balancer does not stop serving. It fails open: DNS publishes all zonal addresses again, and traffic is sent to all targets regardless of their health status.

A Network Load Balancer also fails open when the target group is empty.

So consider what an outside observer sees. A fully healthy load balancer publishes every zonal address. A completely broken one publishes every zonal address. Partial failure is the only state where DNS tells you something is wrong, because that is the only state where an address is missing.

While partial failure is in effect, a target that has gone unhealthy gets a TCP RST for packets on client connections associated with it, rather than a silent drop. That is a useful distinction when reading a client-side trace: a reset points at a target the load balancer has already given up on, a timeout points somewhere else.

Health check details that cause false failures#

Five behaviours that produce unhealthy targets for reasons unrelated to the application:

• A redirect is a failed health check. The check compares the response code to the Matcher, whose default is 200, and it does not follow a Location header. An application hardened to redirect HTTP to HTTPS answers the health check with 301, is marked Target.ResponseCodeMismatch, and goes out of service for doing what the security review asked. AWS's own worked example of that reason code in the health check logs is a 301. Lab 21 is this failure, and the design decision — exempt the health path, widen the matcher, or check HTTPS — is what it is about.

• TLS 1.3-only targets fail HTTPS health checks. A target group configured with HTTPS health checks requires the target to support an earlier version such as TLS 1.2. Hardening a target to TLS 1.3 only will take it out of service.

• The health check Host header is not the target. For HTTP and HTTPS health checks it contains the IP address of the load balancer node and the listener port, not the target's address and health check port. Virtual-host routing or a strict Host check on the target will reject it.

• A TLS listener generates extra connections. Adding one triggers a listener connectivity test, so you will see TCP connections from the load balancer to targets that carry no data packets. They are not an attack and not a misconfiguration.

• An empty Availability Zone counts as unhealthy. Not neutral — unhealthy.

Turning cross-zone off has prerequisites#

If you disable it for zonal isolation or latency, the constraints are real:

• Provision enough target capacity in every zone the load balancer is enabled in.
• Ensure all target groups participate in the same zones. A zone that is empty for one target group is an unhealthy zone.
• On an Application Load Balancer, target stickiness is not supported with cross-zone off, and neither are Lambda targets. Registering targets also requires the AvailabilityZone parameter, and specific zone values are only accepted when cross-zone is off.
• On an Application Load Balancer, avoid empty subnets entirely. An ALB publishes zonal addresses in DNS for empty subnets, and HTTP requests arriving at an empty zone return 503.

That last one is the trap in an otherwise reasonable change: adding a subnet in a new zone ahead of the capacity that will live there creates a zone that answers DNS and serves errors.

What to instrument#

• ZonalHealthStatus, per zone. This is the direct signal that a zone has left DNS, and it is more precise than watching request counts fall.
• Healthy target count per zone, with a threshold above one. The interesting alarm is not zero — zero in every zone triggers fail-open and the symptom changes shape. It is one, in a zone that should have several.
• Whether fail-open is currently active. All-zones-unhealthy is a state you want paged on, precisely because DNS looks normal during it.
• Target group emptiness. An empty target group fails open, which means a failed deployment that deregistered everything presents as a load balancer distributing traffic to nothing in particular.

Next#

The egress track picks up the other direction, including the per-AZ data processing charges that cross-zone load balancing contributes to.