Lab 22: The Security Group With Four Rules That Was Full
A web tier security group with four inbound rules refuses a fifth. The console shows four. The quota is sixty. The error names neither the rule that is in the way nor the number it is counting, and the change that filled the group was the one the review recommended.
- Debugging time
- ~20 min
- Reading time
- 14 min
- Reported by
- Web Platform
- Tier
- Associate
Cannot add an inbound rule to the web security group; API says the rule limit is reached but the group has four rules
Reported by Web Platform
I need to add one inbound rule to sg-web — HTTPS from a partner's address ranges, which we keep in
a prefix list. Terraform refuses it:
Error: creating VPC Security Group Ingress Rule: operation error EC2:
AuthorizeSecurityGroupIngress, https response error StatusCode: 400,
api error RulesPerSecurityGroupLimitExceeded: The maximum number of rules
per security group has been reached.The group has four inbound rules. I have counted them in the console and with
describe-security-group-rules. The quota for rules per security group is sixty, I checked Service
Quotas and we are on the default. Nothing is near any limit.
I tried a plain CIDR rule instead of the prefix list to see if it was something about prefix lists.
A /16 works. A rule referencing the corporate prefix list, which has three entries in it, is
refused with the same error. So I can add some rules and not others, to a group with four rules, and
the error says the group is full.
What you are working with
One VPC, one security group, two customer-managed prefix lists, one AWS-managed one. No instances.
Four inbound rules, drawn as the four things the console shows. Three of them are a CIDR each. The first references the AWS-managed CloudFront prefix list, and how much that one rule counts for is the entire lab. The two customer-managed lists are drawn at their maximum size rather than their current size, for a reason the Reproduction will make clear.
| Resource | Configuration |
|---|---|
| Security group | sg-web, four inbound rules, one outbound |
| Rule 1 | TCP 443 from the AWS-managed com.amazonaws.global.cloudfront.origin-facing prefix list |
| Rules 2–4 | TCP 443 from 10.52.0.0/16; TCP 22 from 10.50.8.0/24; ICMP echo from 10.50.0.0/16 |
| Corporate prefix list | 3 entries, max_entries 8, referenced by nothing |
| Partner prefix list | 3 entries, max_entries 200, referenced by nothing |
| Quota | Inbound rules per security group: 60 (default, unchanged) |
Nothing is deployed that bills by the hour. The failure is an API refusal, and it is observable with no running resources at all.
Scope and constraints
- In scope: why a group showing four rules refuses a fifth, and why some fifth rules are accepted while others are refused.
- Out of scope: the VPC, the quota value itself, and the contents of the prefix lists. All correct.
- The reporter is right that the group shows four rules and that the quota is sixty. Both are true, and the group is still full. Work out what "full" is measuring.
- The reporter's experiment — a
/16is accepted, a three-entry prefix list is refused — is the most useful observation in the ticket. It tells you the unit being counted is not "rules". - The fix is a sizing decision, and there are two places to make it.
Deploy the broken state
cd lab-22-security-group-prefix-list-quota
terraform init
terraform applyUnder a minute. There are no instances to wait for.
SG=$(terraform output -raw security_group_id)
CF_PL=$(terraform output -raw cloudfront_prefix_list_id)
CORP_PL=$(terraform output -raw corporate_prefix_list_id)
PARTNER_PL=$(terraform output -raw partner_prefix_list_id)Confirm what the ticket says
Four inbound rules:
aws ec2 describe-security-group-rules \
--filters "Name=group-id,Values=$SG" \
--query 'SecurityGroupRules[?IsEgress==`false`].[IpProtocol,FromPort,ToPort,CidrIpv4,PrefixListId]' \
--output table
# -------------------------------------------------------------
# | tcp | 443 | 443 | None | pl-3b927c5a |
# | tcp | 443 | 443 | 10.52.0.0/16 | None |
# | tcp | 22 | 22 | 10.50.8.0/24 | None |
# | icmp | 8 | -1 | 10.50.0.0/16 | None |
# -------------------------------------------------------------And the quota, which is the default:
aws service-quotas get-service-quota --service-code vpc --quota-code L-0EA8095F \
--query 'Quota.[QuotaName,Value]' --output text
# Inbound or outbound rules per security group 60.0Four rules, quota of sixty. Both facts confirmed.
Reproduce the refusal
Try the rule the ticket wanted — HTTPS from the partner prefix list:
aws ec2 authorize-security-group-ingress --group-id "$SG" \
--ip-permissions "IpProtocol=tcp,FromPort=443,ToPort=443,PrefixListIds=[{PrefixListId=$PARTNER_PL}]"
# An error occurred (RulesPerSecurityGroupLimitExceeded) when calling the
# AuthorizeSecurityGroupIngress operation: The maximum number of rules per
# security group has been reached.Refused. Now the reporter's experiment — the corporate list, which has only three entries:
aws ec2 authorize-security-group-ingress --group-id "$SG" \
--ip-permissions "IpProtocol=tcp,FromPort=443,ToPort=443,PrefixListIds=[{PrefixListId=$CORP_PL}]"
# An error occurred (RulesPerSecurityGroupLimitExceeded) ...Also refused. And a plain CIDR:
aws ec2 authorize-security-group-ingress --group-id "$SG" \
--ip-permissions "IpProtocol=tcp,FromPort=443,ToPort=443,IpRanges=[{CidrIp=10.53.0.0/16}]"
# {
# "Return": true,
# "SecurityGroupRules": [ ... ]
# }Accepted. The group now has five rules and still refuses the three-entry prefix list. Remove the test rule before continuing so the arithmetic below matches:
aws ec2 revoke-security-group-ingress --group-id "$SG" \
--ip-permissions "IpProtocol=tcp,FromPort=443,ToPort=443,IpRanges=[{CidrIp=10.53.0.0/16}]"So the quota is not counting rules. A one-CIDR rule costs something small and fits. A three-entry prefix list costs something larger and does not. The question is what each kind of rule costs.
Ask the prefix lists what they weigh
A customer-managed prefix list has two sizes: how many entries it holds, and how many it is allowed to hold.
aws ec2 describe-managed-prefix-lists --prefix-list-ids "$CORP_PL" "$PARTNER_PL" \
--query 'PrefixLists[].[PrefixListName,MaxEntries]' --output table
# ----------------------------------------------
# | vn-lab-22-corporate-ranges | 8 |
# | vn-lab-22-partner-ranges | 200 |
# ----------------------------------------------Three entries each, but maximums of 8 and 200. Now the AWS-managed one the first rule references:
aws ec2 describe-managed-prefix-lists --prefix-list-ids "$CF_PL" \
--query 'PrefixLists[0].[PrefixListName,MaxEntries]' --output text
# com.amazonaws.global.cloudfront.origin-facing 55Fifty-five. If a rule referencing a prefix list counts as that list's maximum rather than as one rule, the first rule alone is 55 of the 60, and the arithmetic of everything the reporter saw falls out:
| Rule | Counts as | Running total |
|---|---|---|
| 443 from CloudFront prefix list | 55 | 55 |
443 from 10.52.0.0/16 | 1 | 56 |
22 from 10.50.8.0/24 | 1 | 57 |
ICMP from 10.50.0.0/16 | 1 | 58 |
| a fifth rule from one CIDR | 1 | 59 — fits |
| a fifth rule from the corporate list | 8 | 66 — refused |
| a fifth rule from the partner list | 200 | 258 — refused |
Four visible rules. Fifty-eight of sixty consumed. Two left.
This debrief is part of Labs Pro
The root-cause analysis, packet-flow walkthrough, and Terraform remediation diff for this lab are available to Labs Pro members. One payment of $49, no subscription, and it covers every Pro lab now and later.
The brief, the reproduction steps, and the Terraform stay free — you can still solve this one yourself.
Already bought it? Sign in and it unlocks.