How much money are you happy to spend on a mistake? $10? Maybe $20?
What about $72,000 of surprise cloud costs?
Cloud cost management won’t necessarily prevent you from making these mistakes, but in a world where cloud computing is increasingly prevalent, it’s your best line of defense against an entirely new kind of vulnerability.
From spending money on resources you’re not using, to losing out on massive savings and falling foul of common pitfalls, this post (and the next few we publish) is going to tackle that problem for you.
Today we’ll be covering:
- What is cloud cost management?
- Why you should care about cloud cost management
- Cloud cost management strategies
- Supercharging your AWS cloud cost management
Let’s get started.
What is cloud cost management?
Cloud cost management is a set of strategies, tools, and techniques which allow organizations to measure, track, manage, and optimize the costs associated with their cloud technology. You may have heard of cloud cost optimization, which is the same thing under a different name.
Here at Aimably, we’ve written more times than we can count about the dangers of letting your cloud costs get out of hand. It’s incredibly easy to forget to cancel redundant resource commitments, to be utilizing tech that isn’t the best choice for your needs, and to generally be paying far more than you expect. That doesn’t even account for the times when someone makes a mistake and some of your costs skyrocket as a result!
That’s where cloud cost management comes in.
The core idea is to have a set of measures in place to make sure that you’re only using the tech and resources that are required to run your business. You need to have checks and balances to keep everything under control and, most importantly, to know when things are costing more than the value they generate.
It’s a balancing act, and it’s one that isn’t helped by the general obfuscation of information that many cloud services have when it comes to finances. Heck, if you want to stay on top of your AWS bills you’ll almost certainly need some third-party software to help, because their only native way to display that information (Cost and Usage Reports) is terrible at presenting it. Or, at least, terrible at showing it in a way most of us can understand it.
Why you should care about cloud cost management
Let’s say that you have a cloud setup that’s been working well for your business. It’s small-scale, but with some AWS EC2 instances powering your app, things are looking bright for the future. You’ve particularly benefited from using T-class instances.
Then the time comes to expand your operations as part of your planned growth.
More traffic than usual is headed through your instances, which is fine because they’re built to handle them. However, you’re not expecting your AWS bill to come back at the end of the month at several times the amount of your previous (consistent) bills.
This is a very simple example which is easily explained by the fact that T instances work on the basis of mostly low usage with occasional spikes. You build up “credits” while the instances are at low capacity, which are then used up whenever usage spikes above 10% of the maximum load. Once you run out of credits, you either start paying a lot more to keep using the instances at the same level, or you’re throttled back down below the threshold.
Remember, that’s only one example. You could forget to close down an instance that you phase out and continue to pay for it. What if you neglect to account for additional charges when data is transferred between locations?
Or maybe there’s a small recursion loop with no breaks in the code for your latest product idea that costs $72,000 in a few hours to run a “small” test.
The recursion fault in particular is a nasty pitfall, as it’s so easy to fall into and can have devastating results on your costs. It’s so prevalent that AWS Lambda has an official term for it - the “recursive runaway”!
There are any number of reasons that your cloud costs could creep beyond your control, or suddenly spike without your intention. That’s why it’s vital to have a solid cloud cost management policy in place - to monitor your cloud costs, know what they are, what they should be, and how they’re going to change in the future.
What your organization gets out of doing this is simple. You’ll have lower cloud costs, be able to more accurately predict what future bills should look like, know how taking certain actions will affect your costs (eg, expanding or migrating operations), and be able to present and justify your expenses much more easily to the rest of your team.
Accidents will still happen, sure, but even in the worst-case scenarios you will at least be able to tell that something bad is happening long before you would if you had no management methods in place.
Cloud cost management strategies
Let’s get into some core strategies for cloud cost management. Our next few posts will go into more detail on the solutions and software available, but here we’re going to cover the more broad strokes of how to form the core of your own cloud cost management efforts.
Cloud cost management strategy #1: Check your pricing plans
The first thing you need to do is to check the pricing plan for every cloud tool and service that you’re using and understand the options available to you if you want to switch. It’s all too easy to get locked into thinking that the standard On-Demand pricing plan is best for your needs, but that can leave you paying far more than you need to.
For example, many AWS services have some form of savings plan or discount that you might be eligible for. From AWS Credits to allow you (effectively) free usage to more accessible EC2 instance savings plans, there are a wealth of options available to you that it would be insane to ignore.
Sticking with EC2 instances, how do you know that your current method of paying for them is best? Reserved Instances and Savings Plans are a fantastic way to save money on resources that you know you’ll need in the long term, Spot Instances are a steal but depend on market demand for the instances you want to use, and Dedicated Hosts can save you a lot of money if you have some spare software licenses to use on them.
Cloud cost management strategy #2: Interpret your CURs
We’ve already said that AWS Cost and Usage Reports (CURs) are terrible. They’re the only way to see the data behind your usage and bills, but they can be an unintelligible mess of seemingly random information.
If you aren’t already experienced with understanding CURs, you won’t be able to learn anything from them.
Unfortunately, you don’t have the luxury of not understanding your CURs, as they’re the only native method for seeing your billing information in the detail that you need to fully understand and manage your cloud costs.
You have a couple of ways to fix this.
First, you could hire a specialist to solely focus on your AWS financials. They would be able to interpret your reports and tell you if something is looking suspicious or wrong, but training them with your systems will take time and it’s a lot of money to spend on just understanding your AWS costs.
Second, you or a team member could teach yourselves how to understand the CUR. This is a long, arduous process that will see you make mistakes and thus lose money while you make them.
Third, you can use third-party software to simplify the task and present your billing data in a reasonable way. More on that in the last section of this post!
Cloud cost management strategy #3: Review your instances
There’s no point in paying for a Ferrari when all you need is a car to take you 10 minutes away to the store. Similarly, you shouldn’t be paying through the nose for instances that go way beyond the capabilities that you need.
Take the time to review the instances that you’re running, what their specs are, and what they’re actually being used for. This is where interpreting your CURs comes in handy - you need to know how much your instances are actually getting used in order to make a decision on whether they’re generating enough value to warrant keeping them open.
You should also review what other options are available for the instances you’ve got. If you have a low-priority workload that isn’t time-sensitive, why not move it over to a Batch job on a Spot Instance and slash your costs as a result? Even if you don’t downgrade the type of instance for a cheaper one, there are plenty of opportunities to simply change the pricing plan that you’re on for that instance.
Cloud cost management strategy #4: Eliminate wasted resources
Another part of analyzing your CURs is that you’ll be able to see whether you’re paying for resources that aren’t getting used. It’s incredibly easy to run an instance for a specific purpose only to forget about it once the initial task is complete, leaving it running up your bill each month without actually generating any value in return.
If you’re paying for resources that you aren’t using or don’t need, scrap them and save the cost. You can always take out another instance if you need it later on.
Cloud cost management strategy #5: Use automatic scaling
Automatic scaling is a fantastic feature that’s all too easy to miss in the haze of other features and services. By letting your cloud infrastructure scale to meet your current demands, you get rid of the need to micromanage your accounts in order to avoid massively bloated costs.
The principle of auto-scaling is that your accounts will scale up when there is need to (high workloads, etc) and back down again when things are calmer. Not only does this let you dynamically adjust your resources without having to constantly tweak them to fit your needs, but it means that you don’t always have to plan for the maximum possible utilization of your systems. Doing so is expensive, and isn’t necessary most of the time.
Cloud cost management strategy #6: Create a budget
You don’t need us to tell you this, but forecasting and budgeting are absolutely necessary to effectively manage your cloud costs. Without a budget you won’t know how much money you have left to spend, how much you’ll need to reduce costs, or even whether your current operations should be scaled at all.
The trick with this isn’t necessarily the budget amount itself. It’s important to set the amount in agreement with the rest of your organization so that everyone is on the same page and you can’t get in trouble for spending money that you thought was there for you.
No, the true power of a budget is in the unity that it brings to your entire team and their cloud operations.
With a budget you have the context to judge whether you can afford a better instance type to improve performance, whether you should be operating out of multiple Availability Zones to improve user experience or just the one to limit costs, and much, much more.
If you need some guidance on where to start, here’s our take on building the ultimate AWS budget.
Cloud cost management strategy #7: Schedule instance shutdowns
We’ve mentioned eliminating wasted resources already, but what about when those resources are ones that you know you’ll be using soon? Surely it makes more sense to keep them running and retain their setup to make them easier to use when you need them?
Well, that’s why you can schedule your instances to shut down whenever you don’t need them. That way you’re not paying for constant uptime when there’s no value to be had from doing that.
It all depends on what your instances are designed to do and when you’re going to need them to run as a result. For example, if they’re related to in-house operations and your team is entirely onsite, you can probably schedule your instances to shut down in the evenings and over weekends. That should hugely reduce your costs and even make it easier to understand your cost and usage data (because the only data will be from when your team is active).
Supercharging your AWS cloud cost management
Cloud cost management is a complex topic that we could talk about for hours. That’s why we’re going to be producing a short series of posts all about the different elements of cloud cost management. We’ll be looking at everything from the solutions and software that’s available through to the specifics of applying cloud cost management principles to your AWS account.
If you’ve found this post useful, be sure to check back as they’re released! This section of the post will also be updated as they’re published to provide an easy way to find them.
Finally, if you need some help understanding your CUR, try out our AWS Invoice Management Software. We take your CUR data and present it in a more easily understandable dashboard, complete with full cost and usage data, making it easy to spot discrepancies between invoices and actual usage, and report AWS spending accurately according to business purpose.
