Spot Instances Best Practices

EC2 Spot Instances is an AWS resource that has enabled the usage of instances in a cost-effective manner. They work on the principle of supply-demand, with the spot price fluctuating as the need and capacity availability change. These unused EC2 instances are provided by AWS at almost a 90% discount on On-Demand Prices, making it a primary option to run flexible jobs.

Spot Instances are an ideal choice to run flexible workloads, stateless web apps, and non-production applications, and for developing and testing servers. Incidental tasks such as big data analytics, web services, and parallel computations can be performed using Spot Instances. Spot Instances are recommended for jobs that are not sensitive and can handle downtime, as their availability is dependent on certain factors. This eBook lists certain best practices and to-do’s that will allow you to leverage the benefits of spot instances to the fullest.

1. Deregister ELB on termination notice

What? While using spot instances, it is advised to take advantage of the Spot Instance Interruption Notice, which is given 2 minutes prior to the action through CloudWatch Events. This full 2 minutes of connection draining can be utilized to detach the Elastic Load Balancer, which will prevent traffic from coming to that particular spot instance.

How? AWS suggests the integration and usage of AWS Lambda and SNS to automate this process. CloudWatch sends the notice to Lambda, which will do the heavy lifting and detach the Application Load Balancer from the Instance. In order to facilitate this, certain features of Spot Instances need to be enabled i.e, ELB Integration & Tagging the Spot Fleet Instances. The tagging is used to deregister targets within the ELB.

In order to match with the two minute notice period, the ELB target deregistration delay can be set to two minutes. So on deregistration, the target is put into connection draining mode for 2 minutes.

2. Hibernate EC2 Spot Instances

What? In order to minimize the impact of interruptions, EC2 allows you to hibernate Spot Instances. This essentially puts the instance on hold, until EC2 finds a replacement instance when capacity becomes available.
When a request is made for a spot instance, it can be a ‘persistent’ request. So when the instance is terminated, the request remains open and EC2 launches a replacement instance, of the same pool. For such persistent requests, Spot Instances that are EBS backed (not Instance Store) can be put into hibernation mode, until a replacement spot instance is found. In the case of a Spot Fleet, the request type needs to be ‘maintain’.

How? When a spot instance is hibernated, the EBS volumes and RAM is preserved. While the instance is in hibernation, you are charged only for the EBS volumes, allowing you to optimize costs. Hibernation starts immediately after the interruption notice is received, without awaiting the 2 minute period. Only the Spot Service can start the instance after it has been hibernated when the capacity is available and your maximum price exceeds the spot price.

3. Leveraging Spot Fleet

Spot Fleet can be used to launch multiple SIs based on type, zones, and launch specifications to fulfill the target capacity that you specify. It also launches replacement instances that have been interrupted, to maintain target capacity; as long as capacity is available and the maximum price exceeds the spot price.

The allocation strategy can be either of these:
-‘lowestprice’ - SIs launched from pools with lowest cost
-’diversified’ - SIs launched from all pools
-‘instancepoolstousecount’ - A combination of the first two strategies

3.1 Instance Type Flexibility (High Availability)

What? The prices of spot instances in a particular zone fluctuate independently, so maintaining flexibility in instance types can allow you to leverage higher compute capacity at the same price. High price fluctuations can be negated by diversifying the instance type. Requesting multiple instances from different pools will improve application efficiency.

How? Launching Spot Instances through a Spot Fleet, and picking the ‘diversify’ specification permits you to use instances of different types from zones. The Fleet will pick up instances from different pools, as per what fits your request and according to your allocation strategy.

3.2 Instance Cost Optimization (Lowest Price)

What? Cost Optimization can be maintained by selecting the ‘lowestprice’ specification, which deploys the cheapest combination of spot instances. A maximum amount per hour that you are willing to pay for the Fleet, can also be set. The Fleet will deploy instances until the limit is met, even if the target capacity is not met.

3.3 Using On-Demand in a Spot Fleet (High Availability)

What? Using On-Demand as the primary capacity in a Spot Fleet request enables you to always have instance capacity.

How? You need to specify the total target capacity, and how many of those need to be On-Demand. The Fleet launches the On-Demand first, and the remaining are fulfilled from spot instances as the default purchasing option in a Spot Fleet is Spot Instances. For On-Demand instances, the default purchase option is ‘lowest price’, but the strategy can be changed to ‘prioritized’ which overrides the default.

4. Instance Weighting

What? While using the Spot Fleet, instance weighting allows you to define the compute/capacity units that each instance type would contribute to your application, assign weights and use ‘price per unit hour’ as the costing parameter, rather than ‘price per instance hour’. It essentially simplifies the pricing process, using instance weight to calculate the prices.

How?/Why? Usually, when instances are launched from multiple pools, each configuration has a different instance type and weight. You can attribute weights and adjust your maximum price for each instance.
The attribution of weights is based on the units in which you have defined the target capacity; the amount that the instance contributes to the total target capacity. The usual practice is to use the RAM or vCPU units as weights, assuming you define your target capacity in the same metric.
Based on the weights, the unit price per hour is calculated, and Spot Fleet chooses which launch configuration to deploy, depending on your allocation strategy.

5. Using Spot Blocks

What? In order to run certain uninterrupted jobs, you can make use of spot blocks, in which you can block the duration for the spot instance that you request. For jobs like continuous integration, encoding & rendering, and batch processing, that require a fixed duration to be completed, Spot Blocks is an ideal feature.

How? Spot Blocks run for a duration ranging from 1 to 6 hours, as per your specification and are charged at a fixed hourly rate for the entire duration. This rate is usually slightly higher than normal spot rates, but it is guaranteed to run for the entire duration, without interruption, except in rare cases. The instance is terminated upon the completion of the period, after the provision of a 2-minute termination notice.

6. Using Persistent Request instead of One-time Request

What? A Persistent Request remains open until you cancel the request or it expires. The request is renewed even after an instance is terminated. When consumers want a large amount of computing done, but at low prices; persistent requests are useful. It allows users to consistently and steadily work through the job, if the request remains open.

How? When a spot instance is stopped, or terminated or hibernated due to low capacity or price changes, a persistent request automatically launches another spot instance request, and finds a replacement instance of the same specification, when capacity becomes available again.

7. Monitoring Price History of Spot Instances & Measuring Savings

What? Understanding and tracking the price fluctuations overtime for different instances can be beneficial for users to make predictions and determine actions in what-if situations and the right pricing levels for various workloads. As a monitoring measure, the Spot Requests page shows you all the instances, and fleets that have been launched and the savings made from it, as compared to On-Demand prices for the same instance types.

How? Amazon EC2, through the AWS Console and APIs, equips you with a history of the Spot Price for each instance type in each Region. The prices go back to 90 days, on a per pool basis. Finding the instance pools which have minimal volatility and interruptions will improve your operations. Apart from this, the Spot Instance Advisor, an AWS tool can be used to determine which instance type to request for, in a specific region, based on the price fluctuations and the interruption rates.

Usually, through price history, it is observed that older generation instances tend to fluctuate less than new instances, and the interruptions are also contained. Picking such instance types will improve the availability of your instances. Learn more about AWS Scheduler in https://www.totalcloud.io/resource-scheduling