"The ephemeral port range" means four different things

Write a network ACL rule for return traffic and you need an ephemeral port range. Ask what that range is and you get 32768-65535, and everyone nods, because it is the number people carry.

Here is the first surprise: 32768-65535 is not any operating system's range. It is the range AWS uses in its own example network ACL, and the same page immediately says you might want a different one. It has been copied out of a worked example into production rules for years.

The actual ranges depend on what is choosing the port, and AWS enumerates them on that page:

Six choosers, four distinct ranges. And note that the Linux figure tops out around 61000, not 65535 — net.ipv4.ip_local_port_range reads 32768 60999 on most hosts. So the number everyone writes is wider than Linux at the top and far too narrow at the bottom, which is exactly the wrong shape: the surplus buys nothing and the shortfall drops traffic.

The NAT Gateway figure is documented plainly: NAT gateways use ports 1024–65535. That is the range that matters here, because it is the one doing the choosing in front of your NACL.

What the gap produces#

Put a restrictive NACL on the subnet holding a NAT Gateway. Permit outbound 443. Permit inbound on "the ephemeral range", meaning 32768-65535.

Traffic from a private workload reaches the NAT Gateway, which rewrites the source to its own address and a port somewhere in 1024-65535. The reply comes back addressed to that port. If the port is 32768 or above, the inbound rule matches and the reply is admitted. If it is between 1024 and 32767, no rule matches, the reply hits the implicit deny, and the connection dies.

Because network ACLs are stateless, the reply is evaluated entirely on its own. There is no record that a request went out on that port and no reason to admit the answer.

So the outcome of any given connection is decided by a port number nobody chose and neither host can see. Run the same command twice and get two different results.

The top bar is not an operating system, it is AWS's worked example, and the four below it are the real ranges. The example rule and the AWS-managed range agree above 32768 and disagree below it, and the hatched span is every source port a NAT Gateway can pick that the rule was never written to admit. Roughly half of the gateway's range falls in it, which is why the same command succeeds and fails on alternate runs. The legacy Windows bar sits wholly inside the gap, which is the same defect with an older client instead of a gateway.

Why it survives for weeks#

The symptom is intermittent, and intermittent failures get absorbed. Retry logic covers most of them, success rates look acceptable, and nobody escalates. It usually surfaces when a retry meets a non-idempotent endpoint and produces a duplicate — a payment, a submission, a webhook — at which point the reliability problem has already become a correctness problem.

It is also invisible from both hosts. A NACL denial produces no ICMP and no log entry on either side: the sender sees a timeout and the receiver sees nothing at all. VPC Flow Logs on the subnet are the only place the REJECT appears, and only if they were enabled on the right subnet — logs on the workload's interface show the request leaving and no reply, which tells you nothing about where the reply died.

Recognising the shape#

Intermittent network failures come in a small number of shapes, and identifying which one you have shortens the search enormously:

Per-connection variation. Some connections fail, retries succeed, no time correlation. Something chosen freshly per connection differs — a port, a flow hash, a target selection.

Per-destination variation. Some destinations always fail. Routing, or a rule specific to that prefix.

Time-correlated variation. Failures cluster. Capacity, throttling, or something on a schedule.

Per-host variation. Some sources always fail. Placement, a subnet, a security group.

Port range mismatches are firmly in the first category, and in an AWS network the per-connection shortlist is short: an ephemeral or NAT source port, a flow hash selecting an Availability Zone or appliance, or a load balancer choosing a target.

The fix, and what it costs#

Widen the rule to 1024-65535. Then be honest about what that means: you have admitted essentially everything above the well-known ports on the inbound side, which leaves the NACL doing very little as a security control there.

AWS's own suggestion for clawing some of that back is to deny specific ports you care about and place those deny rules earlier in the table than the wide allow — network ACL rules are evaluated in number order, so a deny at rule 90 is consulted before an allow at rule 100. That genuinely helps for a known-bad port list. It does not get you back to a default-deny posture, because you are now enumerating what to block rather than what to permit, and that is a losing position to defend.

None of which is a flaw in the fix. It is the nature of stateless filtering in front of a device that does address translation. Which leads to the more useful conclusion:

Do not put restrictive network ACLs on subnets containing managed network devices. NAT Gateways, interface endpoint ENIs, NLB nodes, and Resolver endpoints all have port behaviour you do not control and AWS may change. A NACL in front of them is a bet on undocumented behaviour, and the payout is an intermittent failure.

Reserve NACLs for coarse, direction-agnostic boundaries — deny a CIDR outright, block a protocol entirely — and let security groups do per-flow work. Security groups are stateful, so they need no return-path rule at all, which removes the entire class of mistake.

The NLB variant#

The fourth row in the table matters for a different reason. With client IP preservation disabled, an NLB supports roughly 55,000 simultaneous connections to each unique target, defined as IP address and port, using source ports in the same 1024-65535 range. Exceed it and you get port allocation errors rather than a NACL drop.

It is the same underlying resource — source ports toward one endpoint — and the same reason "just disable client IP preservation" is a trade rather than a free fix. You gain simpler security group rules and you accept a per-target ceiling.

There is a lab for the NACL case if you want to watch connections succeed and fail at random and then find the boundary in flow logs: the egress that worked half the time. The NAT Gateway guide covers the port accounting and the separate 350-second idle timeout that produces a similar-looking failure for a completely different reason.