Skip to content
Networking7 min read

A 200 millisecond penalty is what a completely broken IPv6 path looks like

Happy Eyeballs means a dead address family costs you latency instead of errors. The fixed delay on dual-stack destinations is a specific enough fingerprint to diagnose from, and it can sit unnoticed indefinitely.

· Venerable Networks

Enable IPv6 on a VPC and you touch four things. Three of them announce themselves.

StepVisible where
Associate an IPv6 CIDR with the VPCVPC console, obvious
Assign an IPv6 CIDR to the subnetSubnet console, obvious
Auto-assign IPv6 addresses on launchSubnet setting, and visible on every instance
Add a ::/0 routeRoute table, IPv6 rows, easy to miss

Skip the fourth and every visible surface still says IPv6 is on. Instances have global addresses. ip -6 route shows a default route. The console shows IPv6 CIDRs on the VPC and the subnet.

And IPv6 does not work off the VPC.

Why it times out rather than failing fast

This detail matters for diagnosis. Instances receive an IPv6 default route from the VPC router by router advertisement, independently of what the VPC route table contains. So the guest kernel believes it has full IPv6 connectivity and sends the packets.

They are discarded at the VPC route table, where no ::/0 route matches. The symptom is a timeout, not Network is unreachable. If the guest had no route, you would get the immediate error and the cause would be obvious. Because it has one, the failure looks like something out on the network.

A related trap: associating an IPv6 CIDR creates a local route for that prefix automatically. So the route table genuinely does have an IPv6 row in it, which makes it look configured, and IPv6 works fine between instances in the VPC. Only leaving fails.

The IPv6 row that exists, and the one that does not
VPC, dual-stack Attached: Internet gateway. dual-stack subnet contains subnet , default route 0.0.0.0/0 → igw, holding Instance; peer subnet , default route no ::/0 anywhere, holding Peer instance.IPv6 internetVPC, dual-stackdual-stack subnetsubnetpublicInstancehas an IPv6 addresspeer subnetpublicno ::/0 anywhereRisk — IPv6 leaving the VPC matches nothingPeer instance0.0.0.0/0 → igwIPv6 local routeInternet gatewayIPv4 only here

Three of the four IPv6 steps announce themselves, and the fourth is a route. Associating an IPv6 CIDR creates its local route automatically, so the table has an IPv6 row and looks configured, and IPv6 between instances genuinely works. What is missing is ::/0 at an egress-only internet gateway, which costs nothing. The guest has a default route from router advertisement, so it sends the packets and the VPC discards them — a timeout rather than an unreachable, which is what makes it look like something out on the network.

The part that hides it

Most destinations are dual-stack, and clients implement Happy Eyeballs. curl resolves both families, starts the IPv6 attempt first because AAAA is preferred, waits out its happy-eyeballs timeout — 200 ms by default — then starts IPv4 in parallel and uses whichever connects first.

So instead of failing, every dual-stack destination gets slower by roughly a fixed amount:

curl -s -o /dev/null -w 'default: %{time_total}s\n' https://www.cloudflare.com/
# default: 0.267s
 
curl -4 -s -o /dev/null -w 'ipv4:    %{time_total}s\n' https://www.cloudflare.com/
# ipv4:    0.067s

That gap is the fingerprint. A fixed latency penalty of roughly the client's happy-eyeballs timeout, appearing on dual-stack destinations only, is a broken IPv6 path. The size depends on the client: libcurl defaults to 200 ms, while Firefox and Chrome default to 300 ms. RFC 8305 recommends 250 ms as the default "Connection Attempt Delay", and it is worth reading what the RFC says about that number — because IPv6 problems are expected to get rarer, the delays "SHOULD be increased with time as client software is updated". The tax on a broken IPv6 path is designed to grow, not shrink.

Only IPv6-only destinations fail outright. So a completely non-functional address family can sit behind a working one indefinitely, taxing every connection, until some dependency happens to publish AAAA records only.

Turn the correlation into a demonstration

A 200 ms gap is suggestive. Plenty of things cost 200 ms, and you will not convince anyone with a single number. Make the timer the variable instead:

curl --happy-eyeballs-timeout-ms 1    -s -o /dev/null -w 'he=1ms:    %{time_total}s\n' https://www.cloudflare.com/
# he=1ms:    0.067s
 
curl --happy-eyeballs-timeout-ms 1000 -s -o /dev/null -w 'he=1000ms: %{time_total}s\n' https://www.cloudflare.com/
# he=1000ms: 1.065s

At 1 ms the penalty disappears and the request matches forced IPv4. At 1000 ms it grows to almost exactly one second. The delay tracks the knob, so the delay is the knob — you are watching IPv6 be attempted and abandoned on a timer you control.

That is the whole diagnosis, from a client, without touching the network or having any access to it. It also distinguishes this from every other cause of a few hundred milliseconds, which a single comparison against -4 does not.

Localising it in two commands

The useful pair is inside versus outside the VPC:

# To another instance in the same VPC
ping6 -c 3 2600:1f16:16c6:c501:4cea:1644:1047:75bb
# 3 packets transmitted, 3 received, 0% packet loss
 
# Off the VPC
ping6 -c 3 -W 3 2606:4700:4700::1111
# 3 packets transmitted, 0 received, 100% packet loss

An address family that works locally and not remotely points at exactly one place, and it eliminates the interface and the addressing in one step.

One caveat on that first command, learned the hard way: the target's security group has to permit ICMPv6 inbound, and an "all traffic" egress rule does not give you that. If it does not, the in-VPC ping fails too, both halves of the pair look identical, and the result is worthless — you will read it as a total IPv6 failure and start looking at the interface. Confirm the rule exists before you trust the comparison, or use a TCP probe against a listener instead, since a permitted port is usually less ambiguous than ICMP:

ncat -6 --recv-only -w 5 "$PEER" 5000

--recv-only matters. Without it ncat exits before printing the reply and you get a clean exit code with no output, which reads like a failure and is not one.

Then read the route table's IPv6 rows specifically, because the IPv4 rows will look complete:

aws ec2 describe-route-tables --route-table-ids "$RTB" \
  --query 'RouteTables[0].Routes[].[DestinationCidrBlock,DestinationIpv6CidrBlock,GatewayId,State]' \
  --output table

The audit query

The interesting state is a route table with an IPv4 default and no IPv6 default, in a VPC that has IPv6:

aws ec2 describe-route-tables --filters "Name=vpc-id,Values=$VPC" \
  --query 'RouteTables[?
      Routes[?DestinationCidrBlock==`0.0.0.0/0`] &&
      !Routes[?DestinationIpv6CidrBlock==`::/0`]
    ].RouteTableId' --output text

Anything that prints is a subnet where IPv6 is assigned and unroutable.

The same asymmetry applies to network ACLs, which hold IPv4 and IPv6 rules as separate sets. A NACL hardened for IPv4 only will admit IPv6 traffic it was never intended to — which is the more alarming version of this problem, because it fails open rather than closed.

Egress-only internet gateways are free

For a private subnet the IPv6 equivalent of NAT is an egress-only internet gateway, and it costs nothing. No hourly charge, no data processing charge.

For this job no translation is needed at all. Every IPv6 address is globally routable, so the requirement is not address translation but preventing inbound connections, which is exactly what an egress-only internet gateway does.

Be precise about one thing here, because "there is no NAT for IPv6" is a tempting simplification and it is wrong. A NAT Gateway does support IPv6 traffic — it performs NAT64, which paired with DNS64 on the Route 53 Resolver lets IPv6 workloads reach IPv4 resources, whether those sit in another subnet, another VPC, on-premises, or on the internet. That is a genuinely different problem from IPv6 egress, and it is the one case where IPv6 does put you back on a metered NAT Gateway.

So the rule is about direction rather than address family: reaching IPv6 destinations over IPv6 is free, and reaching IPv4 destinations from IPv6 is not.

This is one of the genuinely nice properties of dual-stack, and it is worth saying plainly because teams reach for a NAT Gateway out of habit when they turn IPv6 on: the expensive component on the IPv4 side has a free counterpart on the IPv6 side. A workload that can prefer IPv6 for its egress is a workload paying less for it.

Monitor the family you enabled

Nothing in an account with this misconfiguration will alarm. Add a synthetic check that forces the family:

curl -6 -s -m 5 -o /dev/null https://ipv6.google.com/ \
  || echo "FAIL: IPv6 egress is broken"

Otherwise the only signal is a dependency that happens to be IPv6-only, arriving whenever it arrives.

There is a lab that deploys this exact state, including the latency symptom being dismissed as noise in the ticket: the half of the internet that was missing. The VPC guide covers how route evaluation works, which is where the two families being independent stops being surprising.