The Ultimate Guide to EC2 Instances

Capacity Pool Overview

A capacity pool is one given instance type + size in an availability zone. Below is an example containing 36 capacity pools across 2 availability zones in Region A: 17 of the pools are unavailable at the moment of inspection in Region A. Let’s say you wanted r5.24xl and, as you can see, it’s not available in either zone. You might be tempted to back out and just pay for an on-demand instance, but wait. Staying flexible is the key to being successful with Spot Instances.

The Importance of Being Flexible

Get the most out of Spot by staying flexible in these three ways:

Instance flexibility

If you have specific vCPU and memory requirements, use combinations of instance sizes, generations, or even other families to meet your needs. Spot Advisor makes this easier by publishing likelihood of service interruption by type, size, and region.

Time flexibility

If the workload can wait until the r5.24xl capacity pool becomes available again, consider simply waiting it out. Instance Hibernation is a great tool to leverage in these scenarios because it helps you resume work between interruptions without having to start over or migrate workloads.

Region flexibility

Each Availability Zone in every region has its own capacity pools. With the right combination, you can still achieve savings compared to on-demand instances.

Most of the legwork around finding the lowest-priced capacity pools and spot instances can be done for you using AWS Spot Fleet. Use the Instance Weighting feature to compile a list of instance types/sizes compatible with your applications and give them each a score;
Spot Fleet then delivers the needed resources upon deployment and replaces them when interruptions occur.

Common Questions

— How many capacity pools should I use?

There isn’t a perfect answer, but generally it’s recommended you stay between 4 to 21 capacity pools. Your main goal here is to distribute your workload across enough capacity pools to handle the occasional interruption--but not so many that your fleet is constantly exposed to them.

— Should I be afraid of Spot Instance interruptions?

You won’t have to be as long as you are prepared for it. Before your instance is interrupted, AWS sends a 2-minute interruption notice. Since January of 2018, these notifications can be leveraged as a CloudWatch event to minimize the impact of interruptions significantly. Just check out this AWS walkthrough utilizing CloudWatch Events to automatically deregister Spot Instances from an Elastic Load Balancing Application Load Balancer. And don’t forget about Instance Hibernation, which gives you the option to resume workloads that were interrupted.

With a combination of multiple capacity pools and scripts that orchestrate your workloads around interruptions, you can feel much more confident in running on Spot Instances.

Use Cases

Now that you have a solid understanding of how Spot Instances work, you’ve probably put together that most use cases should have the following in common:

AWS divides their Spot use cases based on time sensitivity

Time Sensitive Use Cases	Description
Web Services	Experiments
APIs	Development
Big Data	Testing
Grid Computing	One-time Queries
Production sequencing	Modal training

Most of the above use cases could utilize one of the following Spot use strategies:

Acceleration

Think 1,000 nodes for 1 hour versus 100 nodes for 10 hours. Reducing your workload hours by multiplying your vCPUs ensures faster delivery with no worry of interruptions.

Diversification

Make interruptions irrelevant by clustering different instance types/families and distributing your workloads among them.

Hibernation

Resume workloads when resources are available.

Spot instances can be intimidating at first due to their risk of interruptions, but today you’ve learned several ways to mitigate that risk. Now go get those epic savings!

8. EC2 Reserved Instances: An
Overview

In the EC2 Pricing Options section, we gave you a quick summary of the benefits and pitfalls of choosing a Reserved instance. Now, let’s look at Reserved Instances (RI) in more detail.

First, let’s tackle a common misconception about RIs. People generally tend to think of a Reserved Instance as one particular instance that you deploy for the duration of the contract term. That is not the case; RIs are simply a 1 or 3 year coupon that applies to any instance usage which matches what you paid for in the reservation (e.g. m5.2xl in us-east-1b running Linux). At the end of the month, any instance usage matching that criteria is billed at the discounted rate.

The price of a Reserved Instance is determined by three main factors: instance attributes, term commitment, and payment method. Here’s a breakdown:

Instance Attributes

Instance Type, Availability Zone, Tenancy, Platform

Term Commitment

1 year or 3 years

Payment Method

Upfront, Partial Upfront, or No Upfront

Deeper savings can be achieved through a combination of longer term commitments and upfront payment. That being said, who knows what their workload patterns will look like in 3 years? In reality, you run the risk of getting stuck with RIs that you no longer need. Sure, you can hawk them off on AWS’s RI Marketplace, but there’s no guarantee they’ll sell--and when they do, it might be for less than your initial contract. That’s money wasted.

Fortunately, you can prevent this by using a little strategy. But before we look at some strategies, let’s talk about the RI Classes that determine which approach to take.

Standard and Convertible Reserved Instances

To complicate pricing even more, there are two Reserved Instance Classes:
Standard RIs and Convertible RIs.

In some ways, these classes are pretty similar. All RIs, regardless of their class and terms, can apply their usage across the Availability Zones of their AWS region. They both can reserve capacity when assigned to a specific Availability Zone. And they both can be shared between multiple accounts when part of a consolidated billing family.

Here’s where they difference:

Differences	Standard	Convertible
1 Year Savings	40%	31%
3 Year Savings	60%	54%
Change Instance Family	No	Yes
Change Payment Option	No	Yes
Sellable on RI Marketplace	Yes	Price Reductions
Price Reductions	No	Yes

So which one’s better? Neither; you’ll likely want to use both for different needs. For the sake of this article, let’s look at Standard RI.

Standard RI

A 1 Year Standard RI with no upfront cost is the most typical scenario for reserved instances; it has the least risk with guaranteed savings against On-Demand.

One possible strategy using this scenario would be to select one or two long-term applications in your infrastructure that run 24/7 and migrate them to Reserved Instances that have the capacity to handle mild workload increases. Then evaluate your reservation needs quarterly, running on-demand instances in the interem for any increase in resource needs.

The absolute safest approach would be to transition only a certain percentage of your core workloads to Standard Reserved Instances. (Remember, EC2 pricing options are not all-or-nothing. Use a mix.) Something between 20-45% would still give you significant savings while allowing for future business needs to scale up or down.

Size flexibility is an important and practically useful benefit which should not be underestimated and which was introduced in the March of 2017. As long as you buy your reservation for a region (vs. an Availability Zone) then your reservation could apply to any reservation of that type but of any size using a normalization model described in this article.

Instance Size	Normalization Factor
nano	0.25
micro	0.5
small	1
medium	2
large	4
xlarge	8
2xlarge	16
4xlarge	32
8xlarge	64
10xlarge	80
16xlarge	128
32xlarge	256

A couple of common mistakes to avoid

A reservation subscription does not equate to availability of capacity in a given region when you require to launch an EC2, unless you purchase your reservation for only a specific Availability Zone. There lies a common mistake: If you buy your reservation for a specific AZ, the downside is that your subscription would not only not work for instances launched in any other AZ, but also not work for instances launched of a different size. So you are trading off capacity reservation for flexibility. You must watch your reservation coverage using tools provided in the AWS console to be sure that your usage is as expected.

In environments where the number of running instances vary during the day (peak vs. non-peak hours), it is easy to over-buy reservations if you don’t consider the hourly distribution of usage during the course of a day. The concept is a tricky, so it’s worth reading this blog if you scale your environment during the day.

9. Auto Scaling your EC2s

Auto Scaling is a powerful tool which automates the sizing of your infrastructure based on
your chosen throughput demands (such as increased traffic or workloads). Using Auto Scaling with EC2s ensures stable user experiences without overpaying for unused resources.

Here’s what you’ll need to leverage Auto Scaling:

• An AMI that contains your applications
• Launch Configurations to spin cloned EC2s
• An Auto Scaling Group to maintain your EC2s
• A Load Balancer to distribute incoming traffic within your ASG

Setup Basics

1. Launch Configurations

Launch Configurations are the blueprints needed to spin up additional instances in an Auto Scaling Group. They contain important information such as the AMI, security groups, EBS volumes, user data, and many more specifics for the instances being launched.

2. Auto Scaling Groups

Auto Scaling Groups contain a collection of EC2 instances primed to scale together based on metric thresholds that you have set in CloudWatch. This group can be configured with a minimum size, maximum size, and target capacity. Once the criteria has been met to trigger your CloudWatch alarm, the group scales in an attempt to manage your resource usage and mitigate over or under provisioning. When done correctly, this will greatly increase the efficiency of your total cost of operations and high availability of your application.

While going through the ASG configuration, you will need to select your Network (VPC) and Subnets. The VPC will determine the network space that the ASG will deploy instances in while the subnets will determine which Availability Zone your ASG has the ability to deploy your instances in. We recommend providing at least two different Availability Zones (subnets) for each Auto Scaling Group.

Lastly, you’ll want to attach an Elastic Load Balancer to your Auto Scaling Group.

3. Elastic Load Balancers

There are three Load Balancer types available in AWS: Application Load Balancers, Elastic Load Balancers (now referred to as classic or CLBs) and Network Load Balancers. Load balancers serve as a single point of contact for clients and distributes traffic to maintain application availability.

Application Load Balancers

Route traffic to specific resource types based on URL path and complex rules.
For example, pointing a specific URL to a specific micro-service.

Classic Load Balancers

Route traffic between clients and backend servers based on TCP port.

Network Load Balancers

Maintain a set of static IP’s (based on the AZ’s you attach) and route traffic to a static IP address or Target Group

For the purposes of this article, let’s take a look at a comparison between Classic ELBs
and ALBs:

Elastic Load Balancer (CLB)

Using an Elastic Load Balancer to serve up https://www.metricly.com/ will always send traffic to the same backend servers. This means that the backend server is responsible for routing requests based on headers. Without having some web proxy in place (apache, nginx, etc) a user hitting https://www.metricly.com/billing would get the same results as a user hitting https://www.metricly.com/blog

Application Load Balancer

Application Load Balancers can be set to do the proxying for you. Users hitting https://www.metricly.com/billing can be sent to one application target group, while https://www.metricly.com/blog can be sent to a completely different target group. This is extremely helpful in mitigating management of multiple load balancers by simply applying routing rules based on host and or header path.

Feature Comparison

Once you’ve picked your load balancer (most likely an ALB), it’s time to build some policies.

Feature	ELB	ALB
Cross-Zone Load Balancing	Manual enablement required	Automatically scales
Health Checks	Basic auto detection	Customizable with detailed HTTP error codes (200-399)
Cloudwatch	Basic	Added metrics
EC2 Classic VPC Support	Yes	No
Dynamic Port Mapping	No, fixed only.	Yes,via EC2 Container service
Access Logs	Yes	Yes + HTTP/S and Websocket requests
Path-Based Routing	No	Yes
Deletion Protection	No	Yes
Backend Auth	No	Yes

Auto Scaling Policies

Policies inform how and when your Auto Scaling Group resizes. Step Scaling and Track Scaling are the most commonly used methods, but we’re going to cover them all.

Simple Scaling versus Step Scaling

Simple Scaling is the classic scaling option AWS first published. It works by listening to a health check on your Auto Scaling Group and provisions more instances when the alarm fires. Once triggered, a cooldown period goes into effect to prevent overcompensation for multiple alarms in a given timeframe.

Unlike Simple Scaling, Step Scaling does not have a cooldown period. These policies evaluate all alarms that fire.

Adjustment Types

The following adjustment types are used in both Simple and Step Scaling policies:

ChangeInCapacity

Increases or decreases capacity by a set value.

ExactCapacity

Changes the ASG capacity to the exact value given.

PercentChangeInCapacity

Increases or decreases capacity of the ASG based on a set % of the whole group.

Target Track Scaling

Target Track Scaling is the newest and simplest way to manage your Auto Scaling policies. When creating the scaling policy, simply choose between creating a standard or custom metric and input a preferred value for it. The scaling policy then creates CloudWatch alarms tailored to maintaining your targeted metric and value. When the alarms fire, it scales back up (or down) to match your target.

Custom Scaling

A good custom metric must encapsulate the state of instance utilization across your entire ASG — otherwise, it scales poorly.