Lab 13: The Attachment That Joined The Wrong Route Table
A Transit Gateway route table isolates a regulated VPC, and the isolation is correctly built. A second VPC is attached without an explicit association and can send traffic straight into it. Nothing is broken, nothing alarms, and the segmentation on the diagram is not the segmentation in force.
- Debugging time
- ~30 min
- Reading time
- 12 min
- Reported by
- Cloud Security
- Tier
- Specialty
Audit finding: unsegmented VPC has a path into the regulated environment
Reported by Cloud Security
Reachability Analyzer says a host in the general-purpose VPC can reach the regulated VPC across the Transit Gateway. We do not believe it.
The regulated attachment is associated with a dedicated isolation route table. That table has no propagations. We built it specifically so the regulated environment could not talk to anything and nothing could talk to it, and we verified it at the time — from the regulated host, nothing is reachable. That still holds; I just re-checked.
Networking says the isolation table is correct and they are right. Security says the path exists and they are also right. Both of those cannot be true, so one of us is measuring the wrong thing.
Nothing is failing. No alarm has fired. This is an audit finding, not an outage.
What you are working with
One Transit Gateway, two VPCs, one Availability Zone.
The asymmetry is in the VPC route tables and it is deliberate. The general VPC carries a route for 10.110.0.0/16 at the transit gateway; the regulated VPC has no transit gateway route at all, so it can originate nothing across the hub. The regulated workload's security group admits 5000 from its own VPC CIDR only — which is a filter at the interface, not a boundary at the hub.
| Resource | Configuration |
|---|---|
| Transit Gateway | Default association and propagation not specified |
| Regulated VPC | 10.110.0.0/16, workload listening on 5000 |
| General VPC | 10.111.0.0/16 |
| Regulated attachment | Explicitly associated with tgw-rtb-isolation, no propagation |
tgw-rtb-isolation | No routes propagated into it |
| General attachment | No explicit association |
| General VPC route table | 10.110.0.0/16 → Transit Gateway |
sg-regulated | Port 5000 from 10.110.0.0/16 only; ICMP from 10.0.0.0/8 |
The row that matters is the one that says "no explicit association." Everything else in the table is a deliberate decision that someone made correctly.
- EC2
General host
10.111.1.x → 10.110.1.x
- RTB
General VPC route table
10.110.0.0/16 → transit gateway
- GW
TGW default route table
the general attachment is associated here
- VPCE
Regulated attachment ENI
traffic is delivered into 10.110.0.0/16
- DEST
Regulated workload
ICMP answered; TCP 5000 dropped by the security group
The isolation route table is never consulted on this path. It governs what the regulated attachment can send, not what can be sent to it.
Scope and constraints
- In scope: whether a path exists from the general VPC into the regulated VPC, and which route table decides it.
- Out of scope: the isolation route table's contents. It is correct and it does what it was built to do.
- Both teams' observations are accurate. Reconciling them is the exercise.
- The security group on the regulated host is deliberately narrow. It is not the fix — the question is whether the network boundary holds, not whether a second control catches what gets through.
Deploy the broken state
cd lab-13-tgw-default-route-table
terraform init
terraform apply
aws ssm start-session --target "$(terraform output -raw general_instance_id)"TARGET=$(terraform output -raw regulated_private_ip)Confirm the path exists
From the general VPC host:
ping -c 3 "$TARGET"
# 64 bytes from 10.110.1.62: icmp_seq=1 ttl=253 time=1.44 ms
# 3 packets transmitted, 3 received, 0% packet lossICMP crosses the Transit Gateway and is answered. Note ttl=253 — two hops consumed, which is what
traversing a Transit Gateway looks like.
The security researcher's finding is correct. Now the TCP attempt:
nc -vz -w 5 "$TARGET" 5000
# Ncat: Connection timed out.That timeout is the security group on the regulated host, not the network. Confirm the difference by watching arrival on the far side.
Watch the traffic arrive inside the regulated VPC
Open a session on the regulated host and capture:
aws ssm start-session --target "$(terraform output -raw regulated_instance_id)"
sudo tcpdump -ni any "host 10.111.1.0/24 or icmp" -c 6Then, from the general host, run the ping and the nc again. The regulated host sees:
09:41:12.118 IP 10.111.1.55 > 10.110.1.62: ICMP echo request, id 4210, seq 1
09:41:12.118 IP 10.110.1.62 > 10.111.1.55: ICMP echo reply, id 4210, seq 1
09:41:20.446 IP 10.111.1.55.39820 > 10.110.1.62.5000: Flags [S], seq 118..
09:41:21.449 IP 10.111.1.55.39820 > 10.110.1.62.5000: Flags [S], seq 118..The SYNs arrive. They are dropped at the interface by the security group, which is why the client sees a timeout, but the packets crossed the Transit Gateway and entered the regulated VPC. A host in the general VPC can address the regulated environment, which means it can port scan it and can deliver UDP payloads to anything with a permissive rule.
Confirm the isolation table really is correct
The networking team is not wrong either. From the regulated host:
nc -vz -w 5 10.111.1.55 5000
# Ncat: Connection timed out.
ping -c 2 10.111.1.55
# 2 packets transmitted, 0 received, 100% packet lossNothing. The regulated VPC can initiate nothing across the Transit Gateway, exactly as designed. And the table is genuinely empty:
aws ec2 search-transit-gateway-routes --region us-east-1 \
--transit-gateway-route-table-id "$(terraform output -raw isolation_route_table_id)" \
--filters "Name=state,Values=active,blackhole" \
--query 'Routes[].[DestinationCidrBlock,State]' --output table
# (empty)Both observations hold. The isolation works in one direction and was only ever capable of working in one direction.
Find which route table the general attachment is in
aws ec2 get-transit-gateway-route-table-associations --region us-east-1 \
--transit-gateway-route-table-id "$(terraform output -raw default_route_table_id)" \
--query 'Associations[].[TransitGatewayAttachmentId,ResourceType,State]' --output table
# --------------------------------------------------------
# | tgw-attach-0a1b2c3d | vpc | associated |
# --------------------------------------------------------Compare that against the general attachment's ID:
terraform output general_attachment_id
# tgw-attach-0a1b2c3dIt is in the default route table. Now look at what that table can reach:
aws ec2 search-transit-gateway-routes --region us-east-1 \
--transit-gateway-route-table-id "$(terraform output -raw default_route_table_id)" \
--filters "Name=state,Values=active" \
--query 'Routes[].[DestinationCidrBlock,Type,State]' --output table
# ------------------------------------------------------------
# | 10.110.0.0/16 | propagated | active |
# | 10.111.0.0/16 | propagated | active |
# ------------------------------------------------------------The regulated prefix is in there, propagated. Nobody added it.
Find why
aws ec2 describe-transit-gateways --region us-east-1 \
--transit-gateway-ids "$(terraform output -raw transit_gateway_id)" \
--query 'TransitGateways[0].Options.{Assoc:DefaultRouteTableAssociation,Prop:DefaultRouteTablePropagation}'
# {
# "Assoc": "enable",
# "Prop": "enable"
# }Both enabled. Neither was ever set in the Terraform.
This debrief is part of Labs Pro
The root-cause analysis, packet-flow walkthrough, and Terraform remediation diff for this lab are available to Labs Pro members. One payment of $49, no subscription, and it covers every Pro lab now and later.
The brief, the reproduction steps, and the Terraform stay free — you can still solve this one yourself.
Already bought it? Sign in and it unlocks.