Security Groups and Network ACLs: Quotas, Weights, Tracking, and Ports

Two filters, applied in sequence to every packet that reaches a subnet. The network ACL decides at the subnet edge, the security group decides at the network interface, and the two disagree about almost everything: what a rule can say, how rules are evaluated, whether the reply is implied, and how much a rule costs against its quota. This sheet is the numbers and the exceptions, each from the page that states it.

The two controls, side by side#

From the VPC infrastructure security documentation, the comparison AWS publishes:

The same page states which is primary: "Use security groups as the primary mechanism for controlling network access to your VPCs. When necessary, use network ACLs to provide stateless, coarse-grain network control." A network ACL's documented roles are "a secondary control (for example, to deny a specific subset of traffic)" and "defense-in-depth in case an instance is ever launched without the correct security group."

Two consequences of the table that are not spelled out in it. A security group cannot express "deny", so the only way to block a specific source on an instance is to not allow it, and the only place a deny rule exists is the network ACL. And because a security group evaluates all rules, rule order is meaningless there and decisive in a network ACL.

Quotas#

From Amazon VPC quotas. Defaults, per Region unless stated.

RegionPer account, per Region

Security groupsPer RegionApplies to individual account VPCs and shared VPCs. Above 5,000, AWS recommends paginating describe calls.

counted here: 2,500 default, adjustable

Rules per security groupPer group, per direction, per address family60 inbound and 60 outbound; 60 IPv4 and 60 IPv6 within each. Rules × groups-per-interface must not exceed 1,000.

counted here: 60 default, adjustable

Security groups per network interfacePer interfaceThe product with rules-per-group is capped at 1,000. Raising one may require lowering the other.

counted here: 5 default, up to 16

Network ACLsPer VPCOne network ACL associates with one or more subnets.

counted here: 200 default, adjustable

Rules per network ACLPer ACL, per direction20 inbound and 20 outbound. Maximum 40 each, "but network performance might be impacted."

counted here: 20 default, up to 40

Customer-managed prefix listsPer RegionAdjustable only by support case, not through the Service Quotas console. Other accounts’ references to a list you share count against your quota.

counted here: 100 lists · 1,000 entries each · 10,000 references per resource type

Three details from the quota page that decide whether a request will be refused:

• The 60 is per direction and per address family. "For an account with the default quota of 60 rules, a security group can have 60 inbound rules for IPv4 traffic and 60 inbound rules for IPv6 traffic." A dual-stack group has four independent budgets of 60.
• The product cap. "This quota multiplied by the quota for security groups per network interface cannot exceed 1,000." Five groups of 60 is 300. Sixteen groups of 60 is 960. Sixteen groups of 100 is refused.
• Network ACL rules have a hard ceiling. 40 per direction, and AWS warns performance may suffer before you get there. There is no 1,000-rule network ACL.

What a rule costs#

The quota counts entries, not lines. From the security group rules documentation:

The customer-managed case is the one that surprises: a prefix list created with max-entries 20 and holding three CIDRs costs 20 rules. The weight is fixed when the list is created and does not follow the contents.

The AWS-managed weights, from AWS-managed prefix lists:

AWS's own gloss on the first row: "The default quota is 60 rules, leaving room for only 5 additional rules in a security group." A dual-stack origin referencing both CloudFront lists is at 110 before any other rule, over the default on its own. The same page notes the route table side: at a default of 50 routes, "you must request a quota increase before you can add the prefix list to a route table." Lab 22 is a group refusing its fifth visible rule for this reason, and the CloudFront post is what the 55 buys.

Stateful, and the exceptions#

A security group's statefulness is implemented by connection tracking, and the connection tracking documentation defines two kinds of flow that are not tracked the way the word "stateful" implies.

Untracked flows. A flow is not tracked when the rules are wide enough that tracking is unnecessary:

If a security group rule permits TCP or UDP flows for all traffic (0.0.0.0/0 or ::/0) and there is a corresponding rule in the other direction that permits all response traffic (0.0.0.0/0 or ::/0) for any port (0-65535), then that flow of traffic is not tracked, unless it is part of an automatically tracked connection.

The consequence is in the next sentence: "An untracked flow of traffic is immediately interrupted if the rule that enables the flow is removed or modified." A tracked connection survives the removal of the rule that admitted it; an untracked one does not. Two servers with the same open sessions behave differently when the same rule is deleted, and the difference is whether the inbound source was 0.0.0.0/0 or something narrower. Lab 23 is that experiment.

Automatically tracked connections. The opposite exception, where tracking happens regardless of rule width:

Connections made through the following are automatically tracked, even if the security group configuration does not otherwise require tracking: Egress-only internet gateways, Global Accelerator accelerators, NAT gateways, Network Firewall firewall endpoints, Network Load Balancers, AWS PrivateLink (interface VPC endpoints), AWS Lambda (Hyperplane elastic network interfaces), DynamoDB gateway endpoints – Each connection to DynamoDB consumes 2 conntrack entries.

Everything on that list consumes tracking capacity even behind an open rule, and DynamoDB consumes double.

The idle timeout, and the Nitro v6 change. Tracked connections expire when idle, and the default depends on the instance generation:

TCP established timeout: … Min: 60 seconds. Max: 432000 seconds. Default: 350 seconds for Nitro v6 instance types (excluding P6e-GB200); 432000 seconds for all other instance types.

Five days on most instances, under six minutes on Nitro v6. AWS's advice on the same page: "test your applications on Nitrov6 generation instances with the reduced 350 second default connection tracking timeout before deploying to production", and for long-lived connections, "configure TCP keep alives to be sent at intervals of less than 5 minutes." The 350 is the same number as the Network Load Balancer's default idle timeout, and the NLB documentation warns that if the load balancer's timeout exceeds the target interface's tracking timeout, the target drops the entry while the load balancer still considers the connection live.

Tracking has a ceiling. "Amazon EC2 defines the maximum number of connections that can be tracked per instance. After the maximum is reached, any packets that are sent or received are dropped." The metric is conntrack_allowance_available, and when it reaches zero the symptom is packet loss with every rule correct.

Stateless, and the ephemeral range#

A network ACL implies nothing. From the custom network ACL documentation: "For every rule that you add, there must be an inbound or outbound rule that allows response traffic." The response arrives on the port the client chose, and the client decides the range:

Many Linux kernels (including the Amazon Linux kernel) use ports 32768-61000. Requests originating from Elastic Load Balancing use ports 1024-65535. Windows operating systems through Windows Server 2003 use ports 1025-5000. Windows Server 2008 and later versions use ports 49152-65535. A NAT gateway uses ports 1024-65535. AWS Lambda functions use ports 1024-65535.

The range that fails most often is the second-to-last. An inbound network ACL rule allowing 32768-65535 because "that is the Linux ephemeral range" is correct for replies to the instance and wrong for replies to the NAT Gateway in the same subnet, which uses ports from 1024. The connection succeeds when the NAT Gateway picked a port above 32768 and fails when it did not. Lab 11 is that failure; Lab 02 is the simpler one, where the return rule is missing altogether.

AWS's practical guidance, same page: "to cover the different types of clients that might initiate traffic to public-facing instances in your VPC, you can open ephemeral ports 1024-65535", and then deny specific ports within it, with the denies "earlier in the table than the allow rules that open the wide range."

Two other network ACL facts worth having on the same page:

• Rule numbers are the evaluation order, and the first match wins. From the network ACL rules documentation: "As soon as a rule matches traffic, it's applied regardless of any higher-numbered rule that might contradict it." The * rule is the implicit deny at the end and cannot be deleted.
• IPv4 and IPv6 are evaluated separately. "None of the rules for IPv4 traffic apply to IPv6 traffic." A dual-stack subnet needs its ephemeral-range rule twice.

Defaults#

What a VPC ships with, from the default network ACL and default security group pages:

Two traps in the defaults. The default security group's inbound rule admits only members of the same group, so two instances in it reach each other on every port and nothing else reaches either; AWS's own example makes the point that "the default rules do not allow the instances to receive traffic from the internet gateway or the NAT gateway." And the default network ACL's IPv6 rules are added only when the IPv6 CIDR is associated, with a caveat: "if you've modified the inbound rules of a default network ACL, we do not add the rule that allows all inbound IPv6 traffic when you associate an IPv6 block with the VPC." A VPC that had its default ACL edited before going dual-stack has no IPv6 allow at all.

Where a rule reference silently admits nothing#

A security group rule whose source is another security group admits "the private IP addresses from the network interfaces for the EC2 instances associated with the referenced security group." Three documented cases where that set is empty or unreachable:

• Through a middlebox. "If you reference the security group of the other instance as the source, this does not allow traffic to flow between the instances" when the path between them goes through an appliance. Use the private IP or subnet CIDR instead.
• Stale. The referenced group was deleted, or the peering, transit gateway or VPC association that made it reachable was removed. The rule stays, admits nothing, and re-arms if the path returns. The stale rules post has the four conditions and the two commands.
• Across a transit gateway without both flags. Referencing requires security group referencing enabled on the transit gateway and on each attachment; inbound rules only. The two-flags post covers the limitation list.

What bypasses both#

Some paths do not consult the controls at all, or consult only one:

• Global Accelerator with client IP preservation. AWS publishes two opposite statements about whether the subnet's network ACL is evaluated for this traffic. Lab 18 deploys a deny and measures the answer in your account rather than asserting one here.
• CloudFront VPC origins. "Inbound NACL rules are not evaluated for traffic from CloudFront to your VPC origin. However, outbound NACL rules are evaluated on the return path."
• Link-local addresses. Traffic to the reserved 169.254.0.0/16 services (metadata, the .2 resolver's link-local alias) is handled by the host: "Security groups and network ACLs do not filter this traffic."