Auto Scaling and Building a Full Application

In part six we spread visitors across two servers and watched the site carry on when one failed. Two problems remain: a failed server stays failed until you step in, and the number of servers is fixed however busy or quiet things get.

Think of an online store: its night-time traffic is nothing like its traffic during a sale. Prepare servers for the peak and you pay for idle machines most of the time. Prepare fewer and the site falls over at the worst moment. The solution is an Auto Scaling group.

Auto Scaling groups (ASG) #

An Auto Scaling group manages a set of servers for you. Give it a recipe and three numbers, and it handles the rest:

It spreads servers across Availability Zones, registers them with the load balancer automatically, and replaces any that fail. The service itself is free; you pay only for the servers.

Launch templates #

To launch servers on its own, the group needs a recipe: the system image, instance type, security group, IAM role and User Data. Everything you chose by hand in part three is saved in a template.

This is where ready-made AMIs pay off: the faster a server comes up, the faster the group responds to load.

Health checks: don't miss this option #

By default, the group checks only the server itself. If the server is running but the application on it has stopped, it won't be replaced. Turn on ELB health checks so it relies on the load balancer's check, and the server gets replaced when the application itself fails.

When to grow and when to shrink #

The metric you choose matters. CPU is a good start, but for websites requests per server (RequestCountPerTarget) is often more honest, because it reflects visitors directly.

To give new servers time to affect the numbers, target tracking and step policies wait out an instance warmup period. If you haven't set one, they use the default cooldown of 300 seconds.

An Auto Scaling group scales servers. If your application also has ECS services, DynamoDB tables or Aurora replicas, the AWS Auto Scaling service can manage a scaling plan for all of them in one place.

Hands-on: watch the group grow and heal itself #

We'll build on the load balancer and target group from part six.

1. Template: in EC2, open Launch Templates, then Create launch template. Name it web-template, choose Amazon Linux 2023, t3.micro and the servers' security group, and paste the User Data from part three.
2. Group: from Auto Scaling Groups, create web-asg from this template, and pick two subnets in different Availability Zones.
3. Load balancer: choose Attach to an existing load balancer, select the target group, and turn on ELB health checks.
4. Numbers: Desired 1, Minimum 1, Maximum 3.
5. Policy: choose Target tracking on average CPU with a 40% target.

Sign in to the server through EC2 Instance Connect and generate CPU load. A t3.micro has two virtual CPUs, so we run two processes:

yes > /dev/null &
yes > /dev/null &

After a few minutes, see what happened from CloudShell:

# What the group did, and why
aws autoscaling describe-scaling-activities \
  --auto-scaling-group-name web-asg --max-items 5 \
  --query "Activities[].[StatusCode,Description]" --output table

# The current servers, their zones and health
aws autoscaling describe-auto-scaling-groups \
  --auto-scaling-group-names web-asg \
  --query "AutoScalingGroups[0].Instances[].[InstanceId,AvailabilityZone,HealthStatus]" \
  --output table

You'll see new servers come up and join the load balancer without you touching anything. Stop the load, and after about fifteen minutes the group shrinks on its own:

pkill yes

Now the fun part: delete a server yourself and watch what happens:

aws ec2 terminate-instances --instance-ids i-0123456789abcdef0

The group notices the count has dropped below what's wanted and launches a replacement within minutes. That's an application healing itself.

Putting the pieces together: a full application #

You now have all the pieces of the server tier. This is the common design for a fault-tolerant web application:

The key rule: a server is replaceable #

In this design, any server can vanish at any moment: scaled in by the group, replaced after a failure, or reclaimed by Spot. So nothing important stays on it. This is what's called a stateless application. Each kind of data has its place off the server:

We haven't reached some of these services yet; the coming sections cover them. What matters now is the idea: whenever you design something, ask "what happens if this server disappears right now?" If the answer is "nothing", you're on the right track.

The same thinking leads to immutable infrastructure: you don't update a running server. You build a new image, put it in a new launch template, and let instance refresh replace the group's servers gradually. Every server then matches what you tested, and going back means returning to the previous image.

Cost in this design #

• Cover the minimum with a Savings Plan, since it runs all the time.
• Keep the extras on On-Demand or Spot.
• Review your minimum regularly; many people set it too high.

Elastic Beanstalk: the shortcut #

Assembling all this by hand is work. Elastic Beanstalk does it for you: upload your code and choose the language, and it creates the load balancer, Auto Scaling group, servers and monitoring. The service itself is free; you pay for the resources it creates.

• Two common environments: one low-cost server for development, or several servers behind a load balancer for production.
• Two kinds of tier: Web Server for direct requests, Worker for processing jobs from a queue.
• Ordinary, visible resources: you can see them in your account and change them; it isn't a closed box.

It suits you if you want to focus on the code of a conventional application. If you have special requirements or many systems, you'll usually build it yourself or move to the containers covered in part twenty.

Summary #

• An ASG keeps the server count between a minimum and maximum, replaces failures and works with the load balancer.
• A launch template is the server's recipe, and a ready-made AMI speeds up the response.
• Turn on ELB health checks so a server is replaced when the application itself fails.
• Start with a target tracking policy, and pick a metric that truly reflects load.
• Keep servers stateless: anything important lives off them.

Test yourself #

Before you move on, try these questions to check you've got this part. Each one is a practical scenario, and the explanation appears once you answer.

That completes the high availability section. The question left is where the data lives once it's off the server. That's the next section's subject, starting in part eight with the managed databases RDS and Aurora.