Do you remember Zune?

Exactly - not many people do, and for good reason.

Microsoft’s failed iPod competitor saw them lose hundreds of millions due mainly to one simple fact.

They didn’t get their market research right.

McKinsey has written about how successful companies are twice as likely to do market research, and the market research industry is worth roughly $73.4 billion. In other words, data-driven research is a fantastic way to give your company the edge.

That’s why AWS Data Exchange is a must if you’re considering doing your own market research.

In this post, I’ll cover:

• The pitfalls of traditional data-driven research
• What is AWS Data Exchange?
• Why use AWS Data Exchange?
• AWS Data Exchange use cases
• Breaking down the finances of AWS Data Exchange
• How to manage your AWS (and Data Exchange) costs

Let’s get started.

The pitfalls of traditional data-driven research

Traditionally, if you wanted to know something about your customer base, market trends, or any other data-driven topic, you had two options.

Go searching for existing data, or gather it yourself.

Let’s tackle gathering data yourself first. It’s horrible.

First you need to define what you need to know. This will usually involve several meetings of your team leaders which would be much better spent by everyone actually carrying out their tasks. Next you need to know both how to gather the data you need and have or find out how to get access to what you need to get started.

Here’s where self-gathered data tends to fall apart for anything but looking at your customer base outside of any other context.

The issue is that getting anyone to give you the time to answer some questions (especially for free) is incredibly hard, if not nigh on impossible.

For example, I remember having to stand outside a supermarket and try to get shoppers to fill out a questionnaire for a school project. It was only around 3 questions long, and I had the advantage of physically talking to my audience.

Nobody even slowed down to say “I’m not interested”.

You’ll be constantly fighting this resistance if you try to gather your own data, even if you try to survey your own customers. The simple fact is that it’s incredibly difficult to gather any kind of substantive data without experience of doing so and dedicated resources.

If you go searching for existing data you stand a slightly better chance of finding what you need, but there are plenty of caveats that still make it an undesirable choice.

Sure, data for specifics of industry trends and consumer actions is already out there, but good luck trying to get access to it yourself without hundreds of hours searching for the right repository, and then having to jump through (usually expensive) hoops to get access to it.

That’s where AWS Data Exchange can help.

What is AWS Data Exchange?

The AWS Data Exchange is a service from Amazon that allows you to access hundreds of data collections from across the world, spanning a vast array of industries and disciplines. Many of their data repositories are even free to use!

Using the Amazon Marketplace, you can search for the data that you need, be it consumer, technology, healthcare or finance-related. This makes it infinitely easier to find what you want to know.

Think of it as browsing a library as opposed to trawling through your crumpled receipts to find the one you need.

It’s a central location where you can search for data with specific characteristics, from companies that have larger datasets than you would ever be able to produce in-house. By using AWS Data Exchange, you’ll be able to choose from the same datasets as massive companies such as Accenture, Frost & Sullivan, and Goldman Sachs.

Going back to my questionnaire example, instead of having to rely on what scant answers you receive from busy shoppers who just want to go about their day, you could be using cutting-edge data gathered from hundreds of thousands of consumers.

The secondary effect of using AWS Data Exchange is that you can use Amazon’s various data analysis tools to easily find the information you need within said datasets. Speaking of which…

Why use AWS Data Exchange?

As I’ve already stated, the main benefits to using AWS Data Exchange are that you get access to a much wider spread of data than you’d be able to gather or research by yourself. This is true both for the amount of data (eg, the number of consumers surveyed) and the types of data.

Unless you already have a dedicated research department, there’s no way that you’ll have access to the same variety of data either.

For a start, gathering the data you need yourself (or with the help of your team) takes an inordinate amount of time, money, and expertise to make sure that you don’t corrupt your results.

It takes months (if not years) of dedicated research to gain any significant knowledge from your customers, competitors, financial markets, or pretty much any other vertical. That isn’t even accounting for the money you’ll spend getting that research done or the time you could instead spend taking action on the available data.

Then there’s the time you save analyzing the data you access!

AWS Data Exchange lets you take the data you’ve subscribed to and push it into Amazon’s analytics services, meaning you’ll be saving a huge amount of time and effort even after you’ve accessed the data you want. This includes:

• Amazon Athena for ad-hoc query
• Amazon EMR for big data processing
• AWS Glue for data integration and ETL
• AWS Lake Formation for building data lakes
• Amazon Redshift for data warehousing
• Amazon S3 to store the data for future transformation, processing, analysis data or machine learning model building

However, there are a few downsides (or at least things worth knowing) to AWS Data Exchange.

The primary downside is that you’ll be spending more on AWS. We’ll cover this in more detail in a later section of this post, but for now know that it’s not as simple as paying for a subscription to AWS Data Exchange.

In typical AWS fashion, the costs incurred aren’t immediately obvious (or easy to manage).

Then there’s the fact that, no matter how much easier it is to find, access, and analyze the data, you still have to do all three.

AWS Data Exchange doesn’t completely remove the need to understand what data you’ll need to access to learn what you need to know. Nor does it make immediately obvious which data sets you’ll need to subscribe to in order to get that information, and while AWS’ analytics suite helps to analyze it, you also still need to know how to do so.

In other words, you can’t sign up for the exchange and get your results with no prior knowledge or experience.

AWS Data Exchange use cases

So you’re interested in AWS Data Exchange, but aren’t totally sold on the idea of it. That’s fine - it can be a lot to wrap your head around.

To make things easier to understand the benefits of it, let’s dive into some examples of how AWS Data Exchange can be used in real life.

#1: Head of Marketing at an HR SaaS Startup

As the Head of Marketing, you need to know how best to target and reach your ideal customers. You know that your HR product (let’s say it’s focused on employee onboarding) can help millions of people unhappy with the way that onboarding is currently handled, but they need to know about your brilliant solution to their woes.

You need data.

Unfortunately, being a startup, you don’t have the resources to undertake a massive consumer survey. You could run a few A/B tests but with your limited customer base you won’t be able to accurately infer general consumer trends or how to reach those that haven’t heard of you already.

So you go to AWS Data Exchange.

You browse the marketplace to find data related to customers of other HR apps, marketing trends, successful campaigns in your sphere (HR apps, SaaS apps, and/or both), and general consumer spending on HR apps.

You use Amazon’s analytics suite to analyze the data you find and find out what features your target customers most value, what platforms they use most frequently, and what the most successful conversion method is for them.

Marketing is back in business.

#2: CEO of a leading financial firm

Finance is a sector entirely built on data, with a great portion of it dedicated to the risks associated with predicting trends. As the CEO of a financial firm, you need to know everything you can about the market volatility, various risk factors (of different investment avenues, of insuring certain customers, etc), compensation benchmarks, mortgage stability, and so on.

If you’re not swimming in cutting-edge data, your firm is destined to fail.

Hence why you turn to AWS Data Exchange.

The Amazon Marketplace offers you the same datasets that firms such as Goldman Sachs use to power their market research, meaning you have the same knowledge at your fingertips as the other world leaders in finance.

This lets you plan your trading strategies and policies, and form a solid (and competitive) risk portfolio to measure against any actions you take or new clients you take on in your insurance section.

Breaking down the finances of AWS Data Exchange

AWS Data Exchange is a fantastic tool in your arsenal, there’s no doubt about that. However, as I mentioned earlier, the main downside of using it comes with the financial implications of using it.

You see, the actual price tag for AWS Data Exchange doesn’t stop at a subscription to the service. You also need to pay for:

• The databases that store the data for the entire time that you need access to it (which could be the entire lifespan of your company)
• Access to the system
• API calls to update the data set
• The data ingress from the exchange to your databases (and egress if you’re selling or sharing results)
• A BI tool to detect the insights in the data and pay the ongoing subscriptions for the life of that tool

If you’ve ever tried to actively manage your AWS costs before (through FinOps or another method), you know just how hard adding all of this to the pile can be.

While you can research this information using Cost Explorer, AWS doesn’t natively come with a pre-configured dashboard to show you what you’re spending on a particular project, how much of that is going on each service you’re using, or how much extra it will cost to expand your services.

That’s why, before you consider using AWS Data Exchange, you need to level with your engineering team. You all need to understand:

• What services your team needs access to
• What benefits these services will bring
• How much these services will cost
• What the total budget for the project should be, and how much of that can be spent on each component that will be individually billed by AWS

Agreeing on and setting out the answers to these points will reduce your problems to just one; the difficulty of managing your AWS cost and making sure that you don’t go over budget.

That’s where Aimably comes in.

How to manage your AWS (and Data Exchange) costs

Aimably is to your AWS spend what AWS Data Exchange is to your data-driven research; it’s a tool that brings all of the information for your cloud computing costs into one easy-to-use dashboard.

This lets you know exactly how much you’re spending in total, what percentage of that figure is being spent on different elements of AWS, and much, much more.

For example, Aimably also offers consulting services to help you stay within budget when it comes to cloud computing. Aimably’s tools do everything from sending you daily, weekly, and monthly trend data for your AWS accounts (Aimably Pulse) to suggesting ways to cut waste spending without impacting performance (Aimably Reduce).

AWS costs are a minefield even before you start using Data Exchange. Try out Aimably today to get rid of your cost headaches while getting access to more data than you could dream of through AWS Data Exchange!

‍