Skip to content
Networking4 min read

One Transit Gateway rule explains both an outage and a security hole

Route lookup happens against the route table associated with the attachment traffic arrived on. That single sentence produces a connection that should work and does not, and a connection that should be impossible and is not.

· Venerable Networks

Most Transit Gateway confusion traces back to one sentence, and it is worth learning as a rule rather than rediscovering it as a symptom:

Route lookup happens against the route table associated with the attachment the traffic arrived on.

Not the destination's route table. Not a global table. The ingress attachment's.

Two consequences follow, and they look like completely unrelated classes of problem. One is an outage. The other is a segmentation bypass that nothing alarms on.

Consequence one: a request arrives and the reply does not

Two spoke VPCs, both attached to a Transit Gateway, both correctly associated with the same spoke route table. Spoke A can reach shared services. Spoke B times out.

Every association is correct. The attachments are all available. The check that most onboarding runbooks perform — is this attachment associated with the right route table — passes for both spokes.

What differs is propagation. Spoke B's prefix was never propagated into the shared services route table. So when the shared service answers, the lookup happens against its attachment's table, and that table has no route back to spoke B.

The request works and the reply has nowhere to go. Every component is healthy, and the failure is a hole in a table nobody was looking at, because the table you naturally inspect is the one belonging to the thing that is failing.

The general form: association and propagation are independent operations, and a working path needs both, in both directions. Association decides which table governs traffic leaving an attachment. Propagation decides which tables learn how to reach it. Doing one and not the other produces exactly this asymmetry.

Consequence two: isolation that only works one way

Now the same rule with the sign flipped.

A regulated VPC is attached and associated with a dedicated isolation route table with nothing propagated into it. From the regulated host, nothing is reachable. The isolation is real, it was verified, and it holds.

Then another VPC is attached without an explicit route table association. Because DefaultRouteTableAssociation and DefaultRouteTablePropagation are both enabled by default, that attachment joins the default route table — where every attachment's prefixes were propagated automatically, including the regulated VPC's.

That VPC can now send traffic into the regulated environment. The isolation table is never consulted on that path, because the traffic did not arrive on the regulated attachment.

Both teams end up correct and in disagreement. Networking tested the direction their table controls and found it closed. Security tested the direction nobody's table controlled and found it open.

Why the mental model has to change

The instinct is to treat a Transit Gateway route table as a firewall attached to a VPC. It is not. It is an outbound policy for its members, and it says nothing about traffic coming the other way.

Which means isolation is not a property you attach to the thing being isolated. It is a property you establish from every other attachment's route table. To isolate a VPC you have to be able to say that no other table contains a route to it — and that is a statement about N tables, not one.

That reframing changes what you check. "Is this VPC isolated?" is unanswerable by looking at the VPC's own table. The answerable version is "which tables contain a route to this prefix, and who is associated with those tables?"

Checking it

This is enumerable, and testing cannot substitute for enumeration. A test proves a path exists; it cannot prove no path exists, because you would have to test every pair. The route tables are the authoritative answer and there are few enough to read exhaustively:

for rtb in $(aws ec2 describe-transit-gateway-route-tables \
    --query 'TransitGatewayRouteTables[].TransitGatewayRouteTableId' --output text); do
  echo "== $rtb"
  echo "  members:"
  aws ec2 get-transit-gateway-route-table-associations \
    --transit-gateway-route-table-id "$rtb" \
    --query 'Associations[].TransitGatewayAttachmentId' --output text
  echo "  can reach:"
  aws ec2 search-transit-gateway-routes --transit-gateway-route-table-id "$rtb" \
    --filters "Name=state,Values=active" \
    --query 'Routes[].DestinationCidrBlock' --output text
done

For each table that gives you "who is in here" and "where can they go", which is the full reachability matrix. A segmentation design is correct only if that output matches the diagram, and it belongs in CI rather than in an annual review.

Reachability Analyzer is the other tool worth using here, because it reasons over configuration instead of sending packets. That means it finds paths that exist but nobody has exercised — which is precisely the category the second failure above falls into.

The defaults deserve a decision

On any Transit Gateway used for segmentation, set both defaults to disable:

resource "aws_ec2_transit_gateway" "hub" {
  default_route_table_association = "disable"
  default_route_table_propagation = "disable"
}

With them enabled, the failure mode of forgetting to associate an attachment is full connectivity, discovered by an auditor months later. With them disabled, the failure mode is an attachment that can reach nothing, discovered by the team that just onboarded, within the hour.

Both are failures. Only one of them is a finding.

Both halves of this are labs if you want to watch them happen: the propagation gap for the outage, and the default route table for the exposure. The Transit Gateway routing guide covers the association and propagation model in full.