Cloud WAN as a Hybrid Endpoint: A Third Termination Point, and the First Ceiling You Share

The previous lesson framed hybrid connectivity around one under-discussed decision: whether you terminate on a virtual private gateway or a Transit Gateway. There is a third option, and it is the one where the numbers stop being yours. A core network edge terminates a Site-to-Site VPN or a Direct Connect gateway with the same per-attachment quotas a Transit Gateway gives you — and then adds a ceiling shared across the whole network, removes a prefix control you were relying on, and inverts the ASN rule you just learned.

This lesson is about Cloud WAN as a place to land hybrid connectivity. What the policy document contains, field by field, is on the Cloud WAN reference sheet; the sheet is the better place to look up a section name or a default.

A core network edge is the third endpoint#

The per-attachment ceilings are the Transit Gateway's. That is the first thing to internalize, because it means migrating from a Transit Gateway does not buy you headroom.

Cloud WAN figures from the Cloud WAN quotas page; the VPN columns are unchanged from the previous lesson. Both Cloud WAN route quotas are adjustable, but only by asking your Solutions Architect or Technical Account Manager — there is no Service Quotas console entry to raise them yourself.

Three structural limits are worth knowing before you draw anything, because none of them are adjustable at all: one core network per global network, one edge per Region per core network, and a soft ceiling of 40 segments per core network. You cannot put two edges in one Region to split a blast radius, and you cannot run two core networks side by side in one global network to stage a migration.

The ceiling you share with every other segment#

Every ceiling in this track so far has been scoped to one object. 100 prefixes per BGP session. 1,000 routes per VPN attachment. 10,000 per Direct Connect gateway. You could reason about your own headroom by looking at your own resources.

This changes who can cause your incident. A team that adds a Region to the policy instantiates every existing segment at that edge. A team that relaxes require-attachment-acceptance lets attachments join a segment when a tag changes, which the reference sheet covers under defaults that decide reachability. Either action grows the route count in a network you share, and neither shows up as a change to anything you own.

The instrumentation consequence is in what to instrument below, and it is the one metric on this page that has no per-team equivalent.

Direct Connect loses its prefix filter#

The Direct Connect sheet makes the case that allowed prefixes mean opposite things depending on the gateway at the far end of the association — a filter over a virtual private gateway, static origination over a Transit Gateway. A core network association is a third case, and it is the one nobody expects.

AWS states it plainly: the Cloud WAN attachment does not support allowed prefixes lists, and all prefixes in the associated segment are advertised to the Direct Connect gateway. Whatever ends up in that segment — by attachment policy, by a tag edit, by another account's VPC joining — leaves over your circuit.

The association is also far narrower than the Transit Gateway equivalent. A Direct Connect gateway associates with one core network and one segment of it, and while that association exists the gateway cannot be associated to other resources at all. Compare the fan-out the sheet documents for the Transit Gateway path — 6 Transit Gateways per Direct Connect gateway, 30 virtual interfaces — and the shape of the design changes: one circuit reaches one segment, and reaching a second segment means a second Direct Connect gateway.

One segment rather than the two association models side by side, because the table above already separates those and the thing worth drawing is what the missing field costs you. Every attachment in the segment is advertised over the circuit, and the segment's membership is not fixed — the third attachment is marked because an attachment policy can place it here when a tag changes, in another account, and its prefixes then leave over your Direct Connect without anyone touching the Direct Connect configuration. With a Transit Gateway association, allowed prefixes is where you would stop that.

Two smaller details that follow from the same page. The attachment uses the transit virtual interface type, so the private-VIF path is not an option here — which also means the transit VIF figures on the Direct Connect sheet still govern the inbound direction. And the inbound quota and the outbound quota are different numbers measured on different objects: on-premises to AWS is the transit VIF's allocation, while AWS to on-premises is 5,000 outbound routes per Direct Connect gateway attachment. Checking headroom on one tells you nothing about the other, which is the same trap the sheet flags for VIFs and associations.

The ASN rule inverts#

The Direct Connect sheet's ASN collisions section gives a rule that is easy to carry forward incorrectly: a Transit Gateway and a Direct Connect gateway must have different ASNs, and because 64512 is the default for both, a pair of defaults fails the association outright. The mental model that leaves you with is "make sure these two numbers are not equal."

That is a harder rule to satisfy by accident than it looks. A core network's asn-ranges has to sit inside the 16-bit private span 64512-65534 or the 32-bit span the reference sheet lists, and the default Direct Connect gateway ASN — 64512 — is the first number of the 16-bit one. Take the obvious range and leave the gateway at its default and the association is invalid before you have configured anything. Pick the core network's ranges and the gateway's ASN together.

There is a third ASN rule in the same family, on the reference sheet: an edge's ASN cannot be changed once assigned, and a Transit Gateway carrying the same ASN as an edge cannot peer with it. Three rules that all sound like "the ASNs must differ" and each constrain something else.

AS_PATH survives the core network#

Routes crossing a core network keep their AS_PATH — the attribute is retained across the core network, the Direct Connect gateway, and the virtual interface. That is what you want, and it is not a given for something that re-originates routes.

It is also not the same as AS_PATH being decisive. The previous lesson's warning about prepending silently doing nothing still applies on the Direct Connect side: without local preference communities, outbound selection falls to the Direct Connect location's relative distance to the originating Region, which is evaluated before AS_PATH. Cloud WAN preserving your prepends does not move them up the evaluation order. Local preference is still the control that works.

MTU, and the attachments that cannot discover it#

A core network carries 8,500 bytes between VPCs, including Transit Gateway peering and tunnel-less Connect VPC attachments, and 1,500 bytes over VPN. Packets larger than 8,500 arriving at the core network are dropped rather than fragmented.

Two behaviours make this worth a section rather than a table row.

Path MTU Discovery does not work where you need it most. Cloud WAN supports PMTUD for traffic ingressing on VPC attachments only — not Connect, not Site-to-Site VPN, not Direct Connect, and not peering attachments. Every hybrid attachment type is on the unsupported list. So the usual mechanism for discovering a smaller path MTU is unavailable on exactly the paths that have one.

MSS clamping is enforced for all packets, which is the mitigation and the reason this is survivable. The core network rewrites the maximum segment size on the way through, so TCP negotiates something that fits without needing the ICMP messages PMTUD depends on. That covers TCP. It does not cover anything that is not TCP, which is where a hybrid path carrying UDP-encapsulated traffic or a tunnelling protocol can still silently black-hole large packets.

Set against the Direct Connect MTU table, where a transit VIF runs at 1,500 or 8,500 and the lower value wins when two sources advertise the same route, the practical posture is: assume 1,500 on any hybrid path unless you have verified 8,500 end to end, and do not rely on discovery to tell you which one you got.

Bandwidth, and the Connect path#

The previous lesson's ceiling still holds — a single VPN tunnel caps near 1.25 Gbps and a single flow cannot exceed it, because the hash pins it to one tunnel. Cloud WAN does not change that.

What it adds is the Connect attachment, which is the SD-WAN and third-party-appliance path:

ECMP scales horizontally across multiple Connect peers or multiple Connect attachments — but not between the BGP peerings of a single Connect peer. That is the distinction that catches people building redundancy: two BGP sessions on one peer give you failover, not aggregate throughput.

A Connect attachment also will not function at all until inside-cidr-blocks is set on the core network configuration, which the reference sheet covers under the object model. It is a field that is optional in the schema and mandatory in practice.

What to instrument#

Everything from the previous lesson's list carries over — tunnel state per tunnel, BGP session state, advertised route count trended against its ceiling. Three additions specific to landing hybrid connectivity on a core network:

• Total routes in the core network, against 10,000. This is the metric with no per-team equivalent, and the one your own resources cannot tell you about. Trend it and alarm well below the ceiling, because the growth comes from policy changes in accounts you may not watch.
• Core network policy version changes. A tag edit can move an attachment between segments when acceptance is not required, so the policy document is a change surface that behaves like a routing change. Treat a new policy version the way you would treat a route table edit.
• CloudWatch Network Monitor on the Direct Connect gateway attachment, which supports latency and packet loss metrics for this attachment type. Note that the Network Health Indicator is not supported here, so do not build a dashboard expecting it.

Where this leaves you#

The through-line holds, with the objects changed: AWS networking failures are usually about a relationship between two objects rather than one broken object. What Cloud WAN moves is where the second object lives. A segment is healthy and the policy says it does not share with the one you are trying to reach. A Direct Connect gateway is available and associated to the wrong segment, and there is no prefix filter to limit the damage. An ASN is valid and sits inside a range that makes the association impossible.

The difference from the rest of this track is that the second object is now a declarative document somebody else can edit, and a tag can be the thing that edits it.

Where to practice#

There is no Cloud WAN lab, and the reason is cost rather than reluctance: the reference sheet works out that a two-Region core network with two VPC attachments runs to roughly four times the most expensive lab on this site, billed per hour from the moment the policy names a Region. The sheet exists in place of that lab, and it is written to be read against the console rather than a teardown script.

The mechanism, though, is the same one two existing labs break. Lab 07 is a Site-to-Site VPN that is available, whose tunnel is UP, and whose route goes nowhere because propagation was never enabled on the table the subnet consults — the Cloud WAN equivalent is a segment that does not share. Lab 03 is the Transit Gateway version of the same defect, and since a core network edge inherits much of a Transit Gateway's behaviour, it is the closest thing to rehearsing this failure for the price of a few cents.

For the policy model itself — segments, sharing, attachment policies, service insertion — go to the Cloud WAN reference sheet. For how route tables behave on the Transit Gateway that a core network edge resembles, the Transit Gateway guide covers segmentation built from absence, which is the model Cloud WAN replaces with a document.