Accurately predicting the future will give you a massive advantage in almost anything, especially running a business. From knowing when runs on your business will happen and being prepared for it to foreseeing when you’ll get the most out of offering a discount, there’s little that an accurate forecast can’t help with.
That’s where performing an AWS Forecast comes in.
By using forecasting with your AWS accounts, including their own tool Amazon Forecast, you can take advantage of Amazon’s powerful machine learning systems to produce time-series forecasts that are pinpoint accurate. In this post we’ll teach you everything you need to know to get started with AWS forecasting and even give you a few tips on how to optimize your initial attempts.
We’ll cover:
- What is forecasting?
- Is an AWS forecast possible?
- What is Amazon Forecast?
- How does Amazon Forecast work?
- When is Amazon Forecast useful?
- Pros and cons of Amazon Forecast
- How to get the most out of your Amazon Forecast account
Let’s get started.
What is forecasting?
Forecasting is the practice of taking a data set and using it to predict what will happen in the future, particularly when the situation is subject to change. Whether you’re looking at air currents and cold fronts to forecast the weather or market and sales data to predict how much inventory you’ll need for the holiday rush, it’s all forecasting.
There are many different types of data you can use to forecast and the same data set can be interpreted in many different ways. Using the wrong forecasting system for a situation will yield entirely inaccurate results, which can be more damaging than not knowing anything at all.
In other words, it’s vital to know what you’re forecasting, why you need to forecast, and what forecasting method you’ll use. There are three to choose from:
- Qualitative forecasting
- Time-series forecasting
- Causal forecasting
Qualitative forecasting is based on data that isn’t quantifiable - it isn’t typically denoted numerically - and is used when there isn’t much available historical data. This includes data such as expert opinions on the viability of a new and unique company or product. There isn’t any historical data to show what typical numbers are for said company or product, so instead you have to rely on things like market experts and investors.
Qualitative forecasting is thus a great way to handle situations where you don’t have historical data to interpret trends and cycles by which you can project the future.
Time-series forecasting is entirely based on historical data, and uses it in the context of the time scale it occurred over to extrapolate trends and cycles to predict what the future will look like. This model primarily assumes that the time (be it the time of day, month, year, etc) is the cause of the trend or cycle rather than there also being other contributing factors. This is both a strength and a weakness, which we’ll touch on later in this post.
For example, if you were to examine market trends in order to predict a seasonal rush on your store (using your own or more general historical data), this would be a time-series forecast.
Causal forecasting is a much more complex field which tries to examine the ultimate cause behind a trend or cycle rather than simply allocating it to the time frame. The effort involved and complexity of the systems to create a causal forecast are immense to say the least so, while it can bring unique insight into a topic, it’s not practical to carry out this type of forecasting in every situation.
Is an AWS forecast possible?
With the groundwork out of the way we can get into the real reason you’re reading this post; performing an AWS forecast.
It’s entirely possible to forecast the different elements of your AWS account, from what your bills will look like through to usage, reliability and uptime, and even your ROI. The extent to which you can do this depends on what you’re trying to do and what tool or method you’re using to accomplish it.
One of the main tools for performing an AWS forecast is Amazon Forecast, but it’s not the best thing to use if you’re just looking to forecast your monthly bills.
While it’s possible to estimate projected costs, the mathematical forecasting offered by Amazon Forecast isn’t just about predicting your AWS bill based on regular usage. Forecasting is best applied to projecting models based on changes that you’re thinking of making.
For example, you could forecast how much return you’ll get out of a new EC2 instance, or what will happen if you run more of your business on the cloud. We’ll share some examples of how to conduct an AWS forecast by applying Amazon Forecast to your usage plans in this article.
If you just want to know what your AWS bill will be for your general operations, and how much you’ll be charged at the end of the current month, check out our articles on cloud cost models, AWS Billing console, EC2 instance pricing, AWS Lambda pricing, AWS Redshift pricing, and our full guide to AWS budgeting for the full details.
However, in the sake of thoroughness, let’s cover your alternate options quickly here.
Performing an AWS forecast without Amazon Forecast
If you want to forecast something simple such as your AWS bill, there are a few steps you’ll need to take. These are:
- Gather your AWS bills and CURs
- Isolate and categorize usage spend
- Analyze what will change and what will stay the same
- Calculate your base costs, scaling spend, and compounded step changes
- Calculate your monthly projected raw spend
First you need to gather your previous (and current) AWS bills and Cost and Usage Reports (CURs). These will tell you everything about your current AWS account, even if you have to dig into the meat of your CUR for some of the important data.
Speaking of which, the next step is to isolate and categorize your usage spend via your CURs. This will show you your line item charges (something that your standard bill doesn’t highlight), giving you more data to use in your forecasting efforts.
Next, analyze your bill and CUR for what is likely to change and what will stay the same. For example, a recurring monthly cost for EC2 instance hosting is unlikely to change if there are no plans to increase resource requirements or take on new projects. This will give you an idea of how much of your old bills can be ported into your AWS forecast without much editing.
Calculating base costs is easy, as you’ll now know which costs are going to run forwards from previous bills. Be careful to account for and subtract any discounts or sign-up bonuses that were included in your older bills.
Your scaling costs are found by taking all of the costs associated with scaling (eg, anything influenced by the number of customers you have), dividing them by the number of customers you have, then multiplying that number by the number of customers you’re projected to have in the following month. This will obviously depend on your projected customer growth rate.
Compounded step changes are a little more complex.
Compare the historical usage values of your resources from the past twelve months to the year before that and you’ll have an idea of what your future resource utilization could look like. Combine this with your Resource Health Threshold (what level of resource utilization you’re comfortable with), and you should be able to see which months you’ll need to spend extra to account for the higher resource utilization.
Add together your base costs, scaling costs, and compounded step changes and you’ll have your forecasted monthly projected raw spend!
This process is a great way to get a rough idea of what your bill will look like, but let’s say that you don’t have the time or expertise to do all of that analysis and calculation. That’s where tools like Amazon Forecast come into play.
What is Amazon Forecast?
Amazon Forecast is a system that produces time-series forecasts based on data you submit and machine learning models. It’s powered by Amazon’s own machine learning algorithms and doesn’t require you to have any of the necessary technology yourself to run it, making it a great entry point to time-series forecasting.
Better still, you don’t need to have any machine learning experience in order to use it!
How does Amazon Forecast work?
In order to use Amazon Forecast you first need to submit any relevant data to what you want to forecast. This will always require historical data, but you can include supplemental data too in order to get a more accurate forecast.
Amazon Forecast then takes this data and runs it through its machine learning systems and algorithms to decide what data is important, analyze it, and develop a forecasting model that best suits the situation. This model is called a “predictor”.
Note that your predictor can be trained in a specific way if you want to have more control over what kind of algorithm they use. Specifically, you can choose from a prebuilt algorithm or allow the system to pick the best one for you.
Finally, it applies the predictor to your data and generates a forecast based on your needs. All you have to do is take those learnings and apply them to your situation.
No tinkering with machine learning is required, but the whole thing can still be difficult to track when you first see it.
So let’s run through an example.
Let’s say that you’re a SAAS company offering event management software. You want to know what resources you’ll need to expand in order to meet the demand of your growing business and cut costs when you don’t need those resources. After all, your customers aren’t planning or running events 24/7 - there are consistent peaks during the year but these alone don’t account for and show your growth rate.
You submit your historical user and event data to Amazon Forecast, supplementing it with your growth rate.
Amazon will take that data, take note of important points such as the consistent cycles and demand spikes along with your growth, then will develop a model to accurately forecast what your upcoming year will look like. This could include when your peak demand will look like, when it will occur (barring any surprises) and how your user count will be affected.
That’s really all there is to it! You don’t need to have any experience with machine learning in order to take advantage of Amazon’s powerful forecasting systems. All you need is an Amazon Forecast account and the historical and relevant data to what you want to forecast.
When is Amazon Forecast useful?
As you’ve probably guessed, Amazon Forecast is invaluable when you need to forecast elements of your business that have been running for a while, but you’re not confident of the future.
So, the perfect situation for Amazon Forecast would be a company that has enough historical data to extrapolate meaning but that doesn’t want to shell out the money to get an in-house forecasting expert or the technology required to create their own forecasting models.
AltexSoft offers some great real-world examples of situations where time-series forecasting has been key, and these can very much be applied to Amazon Forecast too.
So, it could be the case that we’ve already covered that you need to forecast what you’ll need to meet demands based on historical data and market trends. If you purchase more stock or hire more people than needed you’ll only lose money, after all.
Alternatively, you could be in Fareboom’s shoes and need to predict how the market prices of what you’re offering will change. If what you’re offering is volatile (such as tickets for flights) then it can pay dividends to provide customers with a notice of whether your prices are likely to rise or fall. That way you’ll have more people who aren’t convinced buying tickets while they’re relatively cheap, and helping to keep demand more consistent during your slower periods (when things are generally cheaper).
Pros and cons of Amazon Forecast
To help you decide whether or not Amazon Forecast is right for you, let’s go over some of the pros and cons of using it in general (and versus other forecasting options).
You don’t need prior experience
The primary and biggest advantage of using Amazon Forecast over traditional forecasting methods is that you don’t need to have any prior experience with forecasting or machine learning in order to get accurate results.
Amazon’s systems will create the most accurate forecasting model they can specifically for your needs, so there’s literally nothing you need to do once you’ve submitted the data. This means that the barrier to entry is incredibly low, and you don’t have to have any training or expertise in order to get the same insights into your data.
There are no technology requirements
Similarly to not needing machine learning experience to use their services, you don’t need to own any of the necessary technology to analyze your datasets or generate your forecast. Again, this massively reduces the barrier to entry for forecasting, letting much smaller companies with limited budgets take advantage of forecasting to help build a successful business.
The lack of onsite technology requirements also means that there is a lower security risk.
Your data is centralized and secure
As mentioned above, letting Amazon take on forecasting work for you drastically reduces your security risks, because your data isn’t being analyzed onsite.
The only way for data to leak on your end would be to actively share your data sets or forecast once it’s been generated. However, if they’re secure, the only route for a security breach is through Amazon’s servers.
Good luck getting through their security measures.
Amazon Forecast is 50% more accurate than time-series forecasting alone
Not only is it secure, but Amazon Forecast is also 1.5x as accurate as traditional time-series forecasting (by their own measures). This gives it a huge advantage as accuracy is one of the (if not the) biggest weaknesses of forecasting in general.
Remember, none of your results can ever be 100% certain. However, using forecasting to predict what you will need (or how to pivot) to get ahead of the game could cause massive damage if your forecasting turns out to be inaccurate.
Again, Amazon Forecast solves part of the problem via its powerful machine learning and tried-and-tested algorithms, but what about the other sources of inaccuracy? Chief among them being…
Forecasting is only as accurate as the data you provide
It doesn’t matter how good Amazon Forecast’s algorithms are; if you provide bad data you’re going to get an inaccurate forecast. An inaccurate forecast will result in money and resources being lost or wasted if you use it in any meaningful way.
While this is a limit of any and all forecasting, it’s still worth noting as a drawback because it’s something that’s entirely within your control. Ultimately it’s up to you to have consistent and accurate data gathering measures and a secure repository for historical and relevant data.
Amazon Forecast doesn’t account for causes
Another core drawback is that Amazon Forecast is only concerned with time-series forecasting. This means that any and all conclusions drawn are going to be based on the time scale of the data that’s used.
Time-series forecasting is great for things that are due to trends and cycles. Seasonal surges in popularity for certain items and services are great examples of this.
However, it completely ignores any kind of causal analysis. This means that if there’s a more important underlying cause than the time, date or season for your data to be the way that it is, Amazon will be inaccurate to some degree.
For example, your data may indicate that your customers spike around July 4th every year, but neglect to mention that you run a discount at the time to celebrate the holiday. Therefore, Amazon could take away that the customer hike is due solely to the holiday celebrations rather than your discount being the thing that brings in business, thus throwing off future predictions (especially if you decide to remove the discount as a result of the forecast).
How to get the most out of your Amazon Forecast account
As stated above, the ideal use case for Amazon Forecast is one where you have the historical data to fuel their machine learning models and you know that the other potential causes for any trends and cycles have already been accounted for. At the very least you need to be aware of other causes and have assessed them as being less impactful than time.
Otherwise, the best way to start out with Amazon Forecast is to try it out on something small which has a relatively low number of data points required to create a forecast. Since Amazon Forecast primarily charges based on the forecasted data points, getting to grips with the system on a smaller scale will drastically reduce your initial bills while you do so.
Unfortunately, like much of AWS, it’s easy for the Amazon Forecast bill to run away from you. Getting surprised with an extra several hundred (or thousand) extra dollars on your bill because you’ve tweaked your forecasts to run further into the future is a terrible position to be in.
That’s why we’d recommend using our AWS Spend Transparency Software to tie your AWS bills together in one easy-to-use place.
Whether you’re fretting about how much a change in your forecasting will hike up your bill, or you’re simply tired of trying to keep track of all of your AWS accounts separately, we’ve got you covered with our easy-to-use software.
