SpecialtyProSite-to-Site VPNAWS VPC

Lab 07: The Tunnel That Was Up And Went Nowhere

A Site-to-Site VPN reports available, the tunnel reports UP, and the static route for the on-premises prefix is listed on the VPN connection. No traffic passes in either direction, and every page in the VPN console is green.

Debugging time: ~35 min
Reading time: 11 min
Reported by: Network Engineering
Tier: Specialty

INC-1402SEV-3InvestigatingOpened 2026-09-18 08:05 UTC

New Site-to-Site VPN shows both endpoints healthy but carries no traffic

Reported by Network Engineering

The VPN to the branch site was built yesterday. Connection state is available, the tunnel is UP, and the static route for 192.168.0.0/16 is configured on the connection — we can see it in the console.

Nothing works. The branch cannot reach the application host in the VPC, and the application host cannot reach anything at the branch. It is not slow or intermittent; it is silent in both directions.

We have checked the customer gateway address, the pre-shared key, and the security groups. The security group on the AWS side permits the entire branch prefix on all protocols, so it is not that. The tunnel would not be UP if the IPsec configuration were wrong.

What you are working with

EC2
Application host (AWS VPC)
10.50.1.x — replying to 192.168.1.x
RTB
Subnet route table
local 10.50.0.0/16, default → internet gateway
Dropped — no route for 192.168.0.0/16 at all — the virtual private gateway is attached but nothing propagates its routes into this table
GW
Virtual private gateway
attached, tunnel UP — never consulted
DEST
Branch host
192.168.1.x

The tunnel's health is irrelevant to this hop. A route table decides whether traffic is ever handed to the gateway, and this one has no entry for the branch.

| Resource | Configuration | | --- | --- | | AWS VPC | 10.50.0.0/16, subnet 10.50.1.0/24 | | Branch VPC (simulated) | 192.168.0.0/16, subnet 192.168.1.0/24 | | Virtual private gateway | Attached to the AWS VPC | | VPN connection | ipsec.1, static routes only, one tunnel configured | | VPN connection route | 192.168.0.0/16 — present | | AWS route table | local plus a default route to the internet gateway | | sg-aws | All protocols from 192.168.0.0/16 |

The branch side is a second VPC with an IPsec daemon on one instance. It is scaffolding, not the subject.

Scope and constraints

In scope: why traffic does not pass while the connection and tunnel report healthy.
Out of scope: the pre-shared key, the customer gateway address, IKE parameters, and both security groups. The security group on the AWS side deliberately permits everything from the branch prefix so it cannot be a second cause.
Static routing, not BGP, so there is no BGP session to blame and no route advertisement to count.
This lab does not need a working tunnel to diagnose. The evidence is all in the AWS API. If your tunnel is down, you can still complete the diagnosis and the fix; you just cannot watch traffic flow at the end.

Preflight

The branch instance installs libreswan from the Amazon Linux 2023 repositories. That is the only part of this lab that depends on the guest operating system rather than on AWS, so if the tunnel does not establish, check it first:

sudo systemctl status ipsec
sudo ipsec status
sudo journalctl -u ipsec --no-pager | tail -30

Allow three to five minutes after apply. IKE negotiation, the daemon starting, and the AWS side marking the tunnel UP are not instant.

Deploy the broken state

cd lab-07-vpn-route-propagation
terraform init
terraform apply
 
# Shell on the branch host
aws ssm start-session --target "$(terraform output -raw onprem_instance_id)"

AWS_HOST=$(terraform output -raw aws_private_ip)
VPN=$(terraform output -raw vpn_connection_id)
RTB=$(terraform output -raw aws_route_table_id)

Confirm the failure

From the branch host:

nc -vz -w 8 "$AWS_HOST" 8080
# Ncat: Connection timed out.

And from the AWS host, in the other direction:

aws ssm start-session --target "$(terraform output -raw aws_instance_id)"
 
nc -vz -w 8 192.168.1.20 8080
# Ncat: Connection timed out.

Silent both ways. A one-directional failure would point at return routing or a stateless filter; a bidirectional silence means the traffic is not being carried at all.

Confirm the VPN really is healthy

This is the part the ticket already did, and it is worth doing again properly, because the answer is genuinely "yes, it is healthy."

aws ec2 describe-vpn-connections --vpn-connection-ids "$VPN" \
  --query 'VpnConnections[0].{State:State,Type:Type,Static:Options.StaticRoutesOnly}'
# {
#     "State": "available",
#     "Type": "ipsec.1",
#     "Static": true
# }
 
aws ec2 describe-vpn-connections --vpn-connection-ids "$VPN" \
  --query 'VpnConnections[0].VgwTelemetry[].{Outside:OutsideIpAddress,Status:Status}' \
  --output table
# ----------------------------------------
# |         DescribeVpnConnections       |
# +------------------+-------------------+
# |  Outside         |  Status           |
# +------------------+-------------------+
# |  52.x.x.x        |  UP               |
# |  34.x.x.x        |  DOWN             |
# +------------------+-------------------+

One tunnel UP is all this lab configures, and one UP tunnel carries traffic. The DOWN second tunnel is expected here and is not the fault — see the debrief on why that would still be a finding in production.

Confirm the route exists on the connection

aws ec2 describe-vpn-connections --vpn-connection-ids "$VPN" \
  --query 'VpnConnections[0].Routes' --output table
# --------------------------------------------------
# |             DescribeVpnConnections             |
# +----------------------+-------------+-----------+
# | DestinationCidrBlock | Source      | State     |
# +----------------------+-------------+-----------+
# | 192.168.0.0/16       | Static      | available |
# +----------------------+-------------+-----------+

available. The prefix is declared on the VPN connection and AWS considers it usable. Every check the ticket describes passes.

Read the route table the traffic actually uses

aws ec2 describe-route-tables --route-table-ids "$RTB" \
  --query 'RouteTables[0].Routes[].{Dest:DestinationCidrBlock,Target:GatewayId,State:State,Origin:Origin}' \
  --output table
# ------------------------------------------------------------------------
# |                          DescribeRouteTables                         |
# +----------------+---------------+----------+--------------------------+
# | Dest           | Target        | State    | Origin                   |
# +----------------+---------------+----------+--------------------------+
# | 10.50.0.0/16   | local         | active   | CreateRouteTable         |
# | 0.0.0.0/0      | igw-0a1b2c3d  | active   | CreateRoute              |
# +----------------+---------------+----------+--------------------------+

There is no 192.168.0.0/16. The route exists on the VPN connection and does not exist in the route table that the subnet actually consults.

Confirm that propagation is the reason, rather than a route that failed to install:

aws ec2 describe-route-tables --route-table-ids "$RTB" \
  --query 'RouteTables[0].PropagatingVgws'
# []

Empty. No virtual private gateway is propagating into this table.

New Site-to-Site VPN shows both endpoints healthy but carries no traffic

What you are working with#

Scope and constraints#

Deploy the broken state#

Confirm the failure#

Confirm the VPN really is healthy#

Confirm the route exists on the connection#

Read the route table the traffic actually uses#

This debrief is part of Labs Pro