AWS
29 posts tagged AWS.
The one network limit that does not grow when you scale up
Every instance can send 1,024 packets per second to the VPC resolver, per network interface, and the number is the same on a t3.nano and a u-24tb1.metal. Throttled queries never reach anything that logs, so the symptom is intermittent DNS failure with no evidence anywhere — and scaling up, the instinctive fix, changes nothing.
Networking5 minYour 25 Gbps instance does 5 Gbps to the internet, and that is documented
EC2 instance bandwidth is one number on the spec sheet and at least three in practice. Traffic through an internet gateway is capped at 5 Gbps on anything under 32 vCPUs, and a single flow is capped at 5 Gbps nearly everywhere. Neither is a fault, and both get filed as one.
Networking6 minTen kinds of traffic VPC Flow Logs never record
A flow log with no REJECT lines is routinely read as proof that nothing was blocked. AWS documents ten categories of traffic that flow logs exclude by design, and several of them are precisely the traffic you would be looking for when something is wrong.
Networking6 minA regional NAT Gateway makes the most common NAT mistake impossible to express
The zonal NAT Gateway has a placement rule, a per-AZ route table requirement, and a single-zone dependency, and each one is a documented outage. Regional mode does not fix those bugs. It removes the configuration they live in.
Networking7 minus-east-1a is not the same place in your account and theirs
Availability Zone names are mapped per account, deliberately and randomly, in every Region that existed before November 2012. A PrivateLink consumer who builds in us-east-1a can be refused an endpoint for a service that is, by the provider's naming, in us-east-1a. The identifier that is the same everywhere is the AZ ID, and almost nothing is written in it.
Networking6 minA custom DNS server in your DHCP options turns off private DNS for every endpoint in the VPC
Private hosted zones, interface endpoint private DNS, and Route 53 Resolver rules all live in one place: the VPC resolver at .2. Point the DHCP option set at your own DNS servers and every instance stops asking it. Nothing is deleted, nothing is disabled, and every one of those features quietly stops applying.
Networking5 minClient VPN connected, authenticated, and reaching nothing
A Client VPN endpoint has its own route table and its own authorization rules, and a destination has to be in both before a connected user can reach it. Add one without the other and the tunnel is up, the client has an address, and every packet goes nowhere — with a different silence depending on which one you forgot.
Networking6 minThe Site-to-Site VPN tunnel that waits to be called
By default, AWS never initiates a Site-to-Site VPN tunnel. Your device has to. So a tunnel that only carries traffic AWS originates — monitoring, a scheduled job, a backup pull — comes up, idles, drops, and stays down until something on your side sends a packet. The fix is one setting that most people have never seen, and it requires IKEv2.
Networking6 minDirect Connect has a button that breaks it on purpose, and almost nobody presses it
StartBgpFailoverTest takes a virtual interface's BGP session down for a duration you choose, so you can watch traffic move to the backup path before an outage does it for you. It is the only honest way to know a hybrid failover works, it has a three-hour default most people do not expect, and it can be stopped at any time.
Networking6 minThe network failures that present as success
Most AWS networking incidents announce themselves with a timeout. A smaller set return a valid answer and do the wrong thing, and every diagnostic instinct that works on the first kind has nothing to grab on the second.
Networking5 minOn-premises cannot reach a NAT Gateway through a virtual private gateway
The same design works with a Transit Gateway and does not work with a VGW. It is one sentence in the documentation, and it decides whether centralised egress for on-premises traffic is possible at all.
Networking6 minPrivateLink traffic always arrives from the load balancer, whatever you configure
Client IP preservation has no effect on inbound PrivateLink traffic. If you turn it on and write client-IP security group rules, direct clients work and every PrivateLink consumer breaks — which is a difficult failure to see from the provider side.
Networking7 minYou can route around the local route, and that is how intra-VPC inspection works
"The local route always wins and cannot be overridden" is the version most people learn. It is wrong twice: a route more specific than the VPC CIDR wins by longest prefix match, and the local route's target can be replaced outright. The first is what makes inspecting traffic between two subnets possible.
Networking5 minBFD negotiates to the slower peer, so your failover is as fast as their config
You can set a 300 millisecond interval on your side and still get whatever the other end asked for. Intervals negotiate to the slower value and can be asymmetric, which means detection time is not a number you control alone.
Networking5 minA stale static route silently shadows the BGP route you are advertising
VPC route priority puts static routes above propagated ones at the same prefix. So a leftover static route wins over a healthy BGP advertisement, and the routing table looks correct because the route is there.
Networking5 minThe NAT Gateway drops idle connections after 350 seconds and sends a RST
It is not a silent timeout. The connection is actively reset, which means your application sees a connection error at a moment when nothing was wrong, and connection pools are the most common casualty.
Networking4 minA 200 millisecond penalty is what a completely broken IPv6 path looks like
Happy Eyeballs means a dead address family costs you latency instead of errors. The fixed delay on dual-stack destinations is a specific enough fingerprint to diagnose from, and it can sit unnoticed indefinitely.
Networking7 minProxy protocol is sent on health checks, where it carries nothing useful
Enable it on an NLB target group and the header is prepended to health check connections too. On those connections it contains no client information at all, so the feature adds nothing there and still marks every target unhealthy.
Networking5 minA Resolver rule outranks your private hosted zone, and the zone is never consulted
Two DNS sources can claim the same domain in one VPC. The Resolver rule wins at equal specificity and the hosted zone is never read — and nothing in the answer reliably tells you which source served it, so the check has to be a configuration check.
Networking6 minYou cannot borrow a peered VPC's NAT Gateway, but the reverse works
Routing a peered VPC's egress through your NAT Gateway is unsupported edge-to-edge routing. Sending NAT Gateway traffic through a peering to a peered destination is supported, and AWS documents a Return to Sender behaviour there that surprises people.
Networking5 minSecurity group referencing across a Transit Gateway needs two flags, not one
It has to be enabled on the transit gateway and on every VPC attachment. Enable it in one place and it does nothing, with no error. The limitation list is also longer than most people check.
Networking5 minAppliance mode has a prerequisite that silently undoes it
Enable appliance mode and forget route propagation on the associated Transit Gateway route table, and every flow falls back to hash-based Availability Zone selection. The attachment still reports appliance mode enabled.
Networking8 minAn interface endpoint with private DNS off is a bill with no benefit
It costs the same per hour as one that works, reaches available, passes every reachability check, and nothing routes to it. The Terraform default is false, which is most of why this ships.
Networking5 minVPC peering has no reverse path forwarding, and AWS documents what that costs
A hub peered to two VPCs that share a CIDR will deliver one of them's replies to the other. AWS states the behaviour outright, and the configuration is legal to build because the overlap guardrail is narrower than people think.
Networking5 minThe misconfiguration that does not appear in the API output
A subnet with no explicit route table association is returned by nothing. DescribeRouteTables says so in one line of its own reference, and it means "check the route tables" structurally cannot find the problem.
Networking4 min"The ephemeral port range" means four different things
Network ACLs get written with 32768-65535 because that is the number everyone knows. It is nobody's actual range — it is AWS's example. A NAT Gateway uses 1024-65535, and the gap between the two is an intermittent failure that retries hide for weeks.
Networking6 minOne Transit Gateway rule explains both an outage and a security hole
Route lookup happens against the route table associated with the attachment traffic arrived on. That single sentence produces a connection that should work and does not, and a connection that should be impossible and is not.
Networking5 minWhen the AWS docs contradict themselves: NLB client IP preservation
Two pages in the same Network Load Balancer user guide state opposite defaults for client IP preservation. The disagreement decides which security group rules are correct, and getting it wrong produces a target that passes health checks and refuses every client.
Networking6 minThree AWS networking limits that changed shape, not just value
A remembered quota is worse than no quota, because it feels like knowledge. Three AWS networking limits where the mechanism changed underneath the number, and the design decision each one now points at.
Networking8 min