Lab 07: The Tunnel That Was Up And Went Nowhere
A Site-to-Site VPN reports available, the tunnel reports UP, and the static route for the on-premises prefix is listed on the VPN connection. No traffic passes in either direction, and every page in the VPN console is green.
- Debugging time
- ~35 min
- Reading time
- 12 min
- Reported by
- Network Engineering
- Tier
- Specialty
New Site-to-Site VPN shows both endpoints healthy but carries no traffic
Reported by Network Engineering
The VPN to the branch site was built yesterday. Connection state is available, the tunnel is UP,
and the static route for 192.168.0.0/16 is configured on the connection — we can see it in the
console.
Nothing works. The branch cannot reach the application host in the VPC, and the application host cannot reach anything at the branch. It is not slow or intermittent; it is silent in both directions.
We have checked the customer gateway address, the pre-shared key, and the security groups. The
security group on the AWS side permits the entire branch prefix on all protocols, so it is not that.
The tunnel would not be UP if the IPsec configuration were wrong.
What you are working with
Two VPCs in one account. The second one stands in for the branch office: it runs a Libreswan host as the customer gateway, so the whole path is reproducible without hardware.
Read the two route tables against each other. The branch side sends 10.50.0.0/16 at the IPsec host's interface so the daemon can encapsulate it. The AWS side's table holds a default route for tooling and its implicit local route, and nothing else — the virtual private gateway is attached to the VPC and no route table on this diagram references it.
- EC2
Application host (AWS VPC)
10.50.1.x — replying to 192.168.1.x
- RTB
Subnet route table
local 10.50.0.0/16, default → internet gateway
Dropped — no route for 192.168.0.0/16 at all — the virtual private gateway is attached but nothing propagates its routes into this table
- GW
Virtual private gateway
attached, tunnel UP — never consulted
- DEST
Branch host
192.168.1.x
The tunnel's health is irrelevant to this hop. A route table decides whether traffic is ever handed to the gateway, and this one has no entry for the branch.
| Resource | Configuration |
|---|---|
| AWS VPC | 10.50.0.0/16, subnet 10.50.1.0/24 |
| Branch VPC (simulated) | 192.168.0.0/16, subnet 192.168.1.0/24 |
| Virtual private gateway | Attached to the AWS VPC |
| VPN connection | ipsec.1, static routes only, one tunnel configured |
| VPN connection route | 192.168.0.0/16 — present |
| AWS route table | local plus a default route to the internet gateway |
sg-aws | All protocols from 192.168.0.0/16 |
The branch side is a second VPC with an IPsec daemon on one instance. It is scaffolding, not the subject.
Scope and constraints
- In scope: why traffic does not pass while the connection and tunnel report healthy.
- Out of scope: the pre-shared key, the customer gateway address, IKE parameters, and both security groups. The security group on the AWS side deliberately permits everything from the branch prefix so it cannot be a second cause.
- Static routing, not BGP, so there is no BGP session to blame and no route advertisement to count.
- This lab does not need a working tunnel to diagnose. The evidence is all in the AWS API. If your tunnel is down, you can still complete the diagnosis and the fix; you just cannot watch traffic flow at the end.
Deploy the broken state
cd lab-07-vpn-route-propagation
terraform init
terraform apply
# Shell on the branch host
aws ssm start-session --target "$(terraform output -raw onprem_instance_id)"The rest of this lab is part of Labs Pro
The reproduction steps, the deployable Terraform, and the debrief with root cause and remediation diff for this lab are available to Labs Pro members. One payment of $49, no subscription, and it covers every Pro lab now and later.
The incident brief is open, so you can see what you would be solving before you decide.
Already bought it? Sign in and it unlocks.
The rest of this lab is part of Labs Pro
The reproduction steps, the deployable Terraform, and the debrief with root cause and remediation diff for this lab are available to Labs Pro members. One payment of $49, no subscription, and it covers every Pro lab now and later.
The incident brief is open, so you can see what you would be solving before you decide.
Already bought it? Sign in and it unlocks.