Lab 18: The Deny Rule That May Not Be Read
Two identical servers in one subnet under one network ACL that denies your address. One sits behind a Global Accelerator with client IP preservation on. AWS documents two opposite answers about whether the deny applies to it, so this lab settles it in your account rather than telling you.
- Debugging time
- ~25 min
- Reading time
- 10 min
- Reported by
- Security Engineering
- Tier
- Professional
Confirm whether subnet network ACLs still apply once client IP preservation is enabled
Reported by Security Engineering
We are putting Global Accelerator in front of the public API and enabling client IP address preservation so the rate limiter sees real client addresses instead of accelerator addresses.
Review flagged a dependency. Two of our subnet network ACLs carry deny rules for specific source ranges, and those denies are the only place those ranges are blocked. If preservation changes whether network ACLs are evaluated, those rules stop doing anything and nobody finds out.
The documentation does not settle it. One page says network ACLs are not evaluated for this traffic. Another page in the same guide says preserved traffic still respects them. We need an answer for our own account before the change goes ahead, not a reading of the documentation.
What you are working with
One subnet, one Availability Zone, two identical HTTP servers. Every variable that could explain a difference between them is held constant except one: which of them is registered behind the accelerator.
One source address and two ways into one subnet. The edges are drawn from each entry point out to the client, which is the convention on these diagrams for anything outside the VPC — the requests you send travel the other way. Both servers share a subnet, a route table, a network ACL and a security group, and the security group permits port 80 from anywhere on purpose so that it cannot be what blocks anything and the network ACL is the only control in play. The control host is a second instance rather than the endpoint's own public address because AWS recommends against sending traffic directly to a resource that is also behind an accelerator — the paths can collide and add TCP delay, which would turn a clean timeout into something you cannot attribute.
The accelerator's endpoint is an EC2 instance, which is one of the two endpoint types where client IP address preservation is permanently on and cannot be disabled. That removes the main way this experiment could go wrong: there is no preservation setting to get accidentally left off, so you cannot end up measuring the unpreserved path and concluding the wrong thing.
Scope and constraints
- The deny rule is scoped to your address and port 80 only, so Session Manager, the accelerator's health checks, and the instances' own outbound traffic are unaffected. Health checks do not come from your laptop.
- Network ACLs are stateless, so one inbound deny is the whole test. A dropped request never reaches the server and there is nothing to return.
- Global Accelerator is a global service whose control plane lives in us-west-2. The Terraform uses a second provider alias for that reason; without it the apply fails on the accelerator rather than on anything you configured.
- Cost is about 7 cents an hour. Read the clean-up section before you start — this lab has a billing trap that the others do not.
Deploy
The deny rule has to name the address your requests actually come from, so pass it at apply time.
cd labs/lab-18-global-accelerator-nacl-bypass
terraform init
terraform apply -var="client_cidr=$(curl -s https://checkip.amazonaws.com)/32"Confirm it recorded the address you expect, and give the accelerator a minute — a new accelerator
takes a short while to go from IN_PROGRESS to DEPLOYED.
terraform output denied_source
aws globalaccelerator describe-accelerator \
--accelerator-arn "$(terraform output -raw accelerator_arn)" \
--region us-west-2 \
--query 'Accelerator.{Status:Status,Enabled:Enabled}'
# {
# "Status": "DEPLOYED",
# "Enabled": true
# }Confirm preservation is actually on
Before testing anything, establish that the endpoint really is preserving client addresses. If it is not, the rest of the lab measures nothing.
aws globalaccelerator describe-endpoint-group \
--endpoint-group-arn "$(terraform output -raw endpoint_group_arn)" \
--region us-west-2 \
--query 'EndpointGroup.EndpointDescriptions[].{Endpoint:EndpointId,Preserved:ClientIPPreservationEnabled,Health:HealthState}'Expect Preserved: true and Health: HEALTHY. If health is not yet HEALTHY, wait — the endpoint
has to pass a check before the accelerator will route to it, and a request sent early fails for that
reason rather than for the reason you are testing.
Establish that the deny rule works
The control host. Same subnet, same network ACL, reached the ordinary way through the internet gateway.
curl -sS --max-time 8 "$(terraform output -raw control_url)"
# curl: (28) Connection timed out after 8001 millisecondsA timeout rather than a refusal, which is what a network ACL deny looks like — the packet is dropped without an answer. This is the step that makes the rest of the lab meaningful: the rule is present, correctly scoped, and demonstrably dropping your traffic on this port in this subnet.
The decisive step
Same subnet. Same network ACL. Same deny rule, still in place. Same source address. The only difference is that this request arrives through the accelerator, with your address preserved.
curl -sS --max-time 8 "$(terraform output -raw accelerator_url)"Record what you get. There is no expected output printed here on purpose.
| What you see | What it means |
|---|---|
endpoint-behind-accelerator | The network ACL was not evaluated. The guidelines page is the accurate one, and a subnet deny is not a control on accelerated traffic. |
| A timeout | The network ACL was evaluated. The best-practices page is the accurate one, and subnet denies still apply. |
control-direct-over-igw | Something is wrong with the build — that banner should be unreachable through the accelerator. Check that the endpoint registered is the instance you think it is. |
Then read the source address as it arrives, which tells you what the server itself sees regardless of which result you got:
aws ssm start-session --target "$(terraform output -raw endpoint_instance_id)"
sudo tcpdump -ni any 'tcp port 80' -c 6Retry the accelerated request from your laptop while that runs. If preservation is working, the source on arrival is your own public address, not an AWS-owned one — which is the whole reason the question matters. A rate limiter or an audit log on this host is about to start recording real client addresses, and the subnet control that was protecting it may or may not still be running.
This debrief is part of Labs Pro
The root-cause analysis, packet-flow walkthrough, and Terraform remediation diff for this lab are available to Labs Pro members. One payment of $49, no subscription, and it covers every Pro lab now and later.
The brief, the reproduction steps, and the Terraform stay free — you can still solve this one yourself.
Already bought it? Sign in and it unlocks.