AWS Auto Scaling Groups

Auto Scaling Groups

An Auto Scaling group keeps a target number of EC2 instances running, replaces ones that fail, and spreads them across availability zones. Even with no scaling policy attached, that self-healing is worth having, which is why a group with min, max and desired all set to the same number is a perfectly sensible production configuration.

Launch templates, not launch configurations

Launch configurations are deprecated and cannot be created for accounts that never used them. They were immutable, supported no versioning, and could not express newer features. If you are maintaining any, migrating is overdue.

aws ec2 create-launch-template --launch-template-name web-lt \
  --version-description v1 \
  --launch-template-data '{
    "ImageId": "ami-0abcdef1234567890",
    "InstanceType": "t3.medium",
    "SecurityGroupIds": ["sg-0123456789abcdef0"],
    "IamInstanceProfile": {"Name": "web-instance-profile"},
    "MetadataOptions": {"HttpTokens": "required"},
    "TagSpecifications": [{"ResourceType":"instance","Tags":[{"Key":"Name","Value":"web"}]}],
    "UserData": "IyEvYmluL2Jhc2gKZWNobyBoZWxsbwo="
  }'

"HttpTokens": "required" enforces IMDSv2, which closes the SSRF path to instance credentials. Make it a default in every template.

UserData must be base64 encoded, base64 -w0 userdata.sh on Linux, base64 -i userdata.sh on macOS.

aws autoscaling create-auto-scaling-group \
  --auto-scaling-group-name web-asg \
  --launch-template LaunchTemplateName=web-lt,Version='$Latest' \
  --min-size 2 --max-size 10 --desired-capacity 2 \
  --vpc-zone-identifier "subnet-0a1b2c3d,subnet-4e5f6a7b" \
  --target-group-arns "$TG_ARN" \
  --health-check-type ELB --health-check-grace-period 180 \
  --default-instance-warmup 120

$Latest or $Default

Version='$Latest' means new instances always use the newest template version. Convenient, and it means editing a template changes behaviour with no further action, which is not always what you want. $Default pins to a version you promote explicitly, making the template change and the rollout separate steps. For production, pin and promote.

Health checks

--health-check-type ELB --health-check-grace-period 180

This is the setting most worth getting right. The default, EC2, only checks whether the hypervisor thinks the instance is running, so an instance whose application has crashed stays in service indefinitely, receiving traffic and failing every request.

ELB makes the group use the load balancer's target health, so an instance failing its HTTP health check is replaced. If you have a load balancer, use it.

The grace period is how long after launch the group ignores health checks. Too short and you get a loop: the instance boots, is not ready, is marked unhealthy, is terminated, and a replacement starts the same cycle. Set it above your genuine boot-to-ready time, and watch out for a group that never stabilises after an AMI change made startup slower.

--default-instance-warmup is the newer, preferred setting: it covers both health checks and metric collection, so a warming instance does not drag the group's average CPU down and trigger further scaling.

Instance refresh

The supported way to roll a fleet onto a new AMI or instance type, without an external deployment tool:

aws autoscaling start-instance-refresh \
  --auto-scaling-group-name web-asg \
  --preferences '{
    "MinHealthyPercentage": 90,
    "InstanceWarmup": 180,
    "SkipMatching": true,
    "AutoRollback": true,
    "MaxHealthyPercentage": 110
  }'

aws autoscaling describe-instance-refreshes \
  --auto-scaling-group-name web-asg \
  --query 'InstanceRefreshes[0].[Status,PercentageComplete,StatusReason]'

[ "InProgress", 40, null ]

Three preferences earn their place. SkipMatching leaves instances already running the target configuration alone, which turns a re-run into a no-op. AutoRollback reverts to the previous template version if the refresh fails, rather than leaving the group half-migrated. MaxHealthyPercentage above 100 lets the group launch replacements before terminating, so capacity never dips. Worth it if MinHealthyPercentage: 90 would still take you below what you need.

Lifecycle hooks

Hooks pause an instance at launch or termination so something can act before it proceeds:

aws autoscaling put-lifecycle-hook \
  --auto-scaling-group-name web-asg \
  --lifecycle-hook-name drain-connections \
  --lifecycle-transition autoscaling:EC2_INSTANCE_TERMINATING \
  --heartbeat-timeout 300 --default-result CONTINUE \
  --notification-target-arn "$SNS_ARN" --role-arn "$ROLE_ARN"

Use them to deregister from service discovery, flush logs, or finish in-flight jobs on termination. The critical part is completing the hook when done:

aws autoscaling complete-lifecycle-action \
  --auto-scaling-group-name web-asg \
  --lifecycle-hook-name drain-connections \
  --lifecycle-action-result CONTINUE \
  --instance-id i-0123456789abcdef0

Forget that call and every instance sits in Terminating:Wait for the full heartbeat timeout, which, with --default-result ABANDON, can also strand a group mid-scale. Set CONTINUE as the default result so a broken hook degrades rather than blocks.

Termination policies

When scaling in, the group has to choose which instance to remove:

aws autoscaling update-auto-scaling-group \
  --auto-scaling-group-name web-asg \
  --termination-policies OldestLaunchTemplate Default

To keep a specific instance, one you are mid-investigation on, use scale-in protection:

aws autoscaling set-instance-protection \
  --auto-scaling-group-name web-asg \
  --instance-ids i-0123456789abcdef0 --protected-from-scale-in

Mixed instances and Spot

Combining on-demand and Spot across several instance types is where real savings live:

aws autoscaling update-auto-scaling-group \
  --auto-scaling-group-name web-asg \
  --mixed-instances-policy '{
    "LaunchTemplate": {
      "LaunchTemplateSpecification": {"LaunchTemplateName":"web-lt","Version":"$Default"},
      "Overrides": [
        {"InstanceType":"t3.medium"},{"InstanceType":"t3a.medium"},
        {"InstanceType":"t2.medium"},{"InstanceType":"m5.large"}
      ]
    },
    "InstancesDistribution": {
      "OnDemandBaseCapacity": 2,
      "OnDemandPercentageAboveBaseCapacity": 25,
      "SpotAllocationStrategy": "price-capacity-optimized"
    }
  }'

That keeps two on-demand instances as a floor, then fills 75% of anything above with Spot. List several instance types. Spot interruption risk falls sharply with more options, and a single-type Spot group is the configuration that gives Spot its bad reputation.

aws autoscaling update-auto-scaling-group \
  --auto-scaling-group-name web-asg --capacity-rebalance

Capacity rebalancing launches a replacement when AWS signals an upcoming Spot interruption, rather than waiting for the two-minute termination notice.

Common errors

ValidationError: You must use a valid fully-formed launch template

The template does not exist, the version is wrong, or the name has a typo. describe-launch-template-versions confirms.

Launching a new EC2 instance. Status Reason: Your requested instance type
(t3.medium) is not supported in your requested Availability Zone

The subnet's zone does not offer that type. Remove that subnet or add overrides with types that exist there.

Instances launching and terminating in a loop. Health check grace period shorter than boot time, almost always.

Group stuck at Terminating:Wait. A lifecycle hook that nothing is completing.

Quick reference

---

Key takeaways

• Launch configurations are deprecated. Use launch templates, and pin $Default in production.
• Set --health-check-type ELB. The EC2 default keeps a crashed application in service.
• A grace period shorter than boot time causes a launch-terminate loop.
• --default-instance-warmup stops a warming instance skewing the metrics that drive scaling.
• Instance refresh with AutoRollback and SkipMatching is the supported way to roll an AMI.
• Always call complete-lifecycle-action, and default hooks to CONTINUE.
• List several instance types in a mixed instances policy: it is what makes Spot reliable.
• describe-scaling-activities explains every launch and termination, including refusals.

---

Check your understanding

---

Frequently Asked Questions

What is the difference between a launch template and a launch configuration?

Launch configurations are the older, immutable, unversioned mechanism, and they are deprecated. AWS no longer allows new ones in accounts that never used them, and they cannot express newer features such as mixed instance types or IMDSv2 enforcement. Launch templates are versioned, so you can promote or roll back a specific version, and they work with instance refresh and mixed instances policies. Any remaining launch configuration should be migrated; there is no benefit to keeping one.

Why does my Auto Scaling group keep replacing healthy instances?

Usually a health check grace period shorter than the real boot-to-ready time, so instances are judged unhealthy before the application starts serving and terminated in a loop. It often appears after an AMI or application change makes startup slower. Check describe-scaling-activities, which states the reason for each termination. The other cause is a health check path that returns a redirect or 404, which fails even on a fully working instance.

Should health check type be EC2 or ELB?

ELB, whenever there is a load balancer. The EC2 default only considers the instance's own status checks, meaning a host whose application has crashed is still considered healthy and keeps receiving traffic while failing every request. ELB defers to the target group's health, so a failed HTTP check results in replacement. Pair it with a grace period longer than boot time, and a health check endpoint that does not depend on the database.

How do I roll out a new AMI to an Auto Scaling group?

Create a new launch template version with the AMI, then start-instance-refresh with MinHealthyPercentage, InstanceWarmup, SkipMatching and AutoRollback set. The group replaces instances gradually while keeping the healthy proportion above your floor, and reverts automatically if the refresh fails. Setting MaxHealthyPercentage above 100 lets replacements launch before old instances are terminated, so capacity never dips during the roll.

What are lifecycle hooks used for?

Pausing an instance at launch or termination so something can run first. Deregistering from service discovery, uploading final logs, waiting for in-flight jobs to finish, or completing configuration before the instance takes traffic. Whatever handles the hook must call complete-lifecycle-action when done, or the instance waits out the entire heartbeat timeout. Set --default-result CONTINUE so a failure in your handler delays the action rather than abandoning it.

How do I use Spot instances in an Auto Scaling group safely?

Use a mixed instances policy with OnDemandBaseCapacity set to the minimum you must always have, a percentage split above that, and the price-capacity-optimized allocation strategy. The single most important detail is listing several instance types in the overrides. Spot interruption risk falls sharply with more options, and a group pinned to one type is what gives Spot its unreliable reputation. Enable capacity rebalancing so replacements launch when AWS signals an upcoming interruption.