What you’ll learn
- Why you should do Amazon AB testing
- How to use Manage Your Experiment
- What you should split test in your listings
Manage Your Experiments (MYE) is a key feature of the Amazon marketplace that sellers can use to AB split test to determine the type of content that will perform well for their audience on each ASIN.
Manage Your Experiments lets you create two different product pages for a single product, test the two product pages to determine what will attract your audience, and leverage the valuable insights to drive more sales.
Manage Your Experiments can run a split test or A/B test; thus, allowing you to take away the guesswork and use shopper behavior to create enticing content that will generate sales. Today, Manage Your Experiments gives Amazon sellers and vendors the ability to change product main image, product titles, and A+ content.
With Manage Your Experiments activated, your audience would be divided into two groups, with each group assigned to see the content of either version A or B. Amazon is then responsible for observing each version’s performance within 8 to 10 weeks. Amazon then offers weekly results and also suggests a better-performing version at the end of the test.
A+ content, product titles, and product images on the product page can be edited within Manage Your Experiments. Remember, product images and titles are what help shoppers decide which product to buy. A split test example is to tweak A+ content and see which one converts best.
How does Manage Your Experiments work?
MYE split tests your Amazon listings based on sales over a period of 4-8 weeks.
Half of your audience will receive a version of your product title, image, and A+ content at random, while the remaining half will receive the second version. Both versions of your product detail page will populate on the search results and product pages and also override your previous content, for better or worst.
The recommended duration for you to set this test is 10 weeks. Within the 10 weeks duration, but 4 weeks is fine for most. With these results, you can see which AB testing version resonates well with your audience.
Manage Your Experiments displays sample size, units sold per unique visitor, conversion rates, and metrics for sales. It also displays valuable data for one-year impact supposing you run the test for a year.
However, Manage Your Experiments is not for every brand registered seller.
Who is eligible to use this tool?
Not every seller on Amazon is eligible to use the Manage Your Experiments Tool.
Those eligible are sellers and vendors who are brand registered on Amazon and have high traffic ASINs. You need enough traffic and sales in order to get statistically relevant results.
Please note that Manage Your Experiments will display whether or not your ASIN is eligible in your portfolio. You may consider using Amazon advertising to gain more traffic if you’re not eligible to use the Manage Your Experiments Tool to run split tests.
Types of content available for split testing
As of today, three types of content are available for Amazon AB testing and splitting content to see which one wins. This content includes:
- Product Titles – Make your product titles highly informative.
- Product Images – Use stunning images to capture your audience’s attention.
- A+ Content – Develop clear content to describe your product’s features and bullet points.
How to use Manage Your Experiments
The process of setting up an Amazon AB testing experiment is simple. Follow the steps below to get started. Here’s Amazon’s official getting started page.
Step 1
Click the “Create a New Experiment” button to begin and choose the experiment type you want to run from the drop-down menu.
Step 2
Choose a reference ASIN or an eligible ASIN that you intend to run the experiment on.
Step 3
Enter the experiment details like start dates, duration, hypothesis, and name.
- Duration and start dates: The recommended duration is between 8-10 weeks. With this duration, you are sure to gather enough data. However, you may decide at any point to end the experiment.
- Hypothesis: This section covers the relevant questions that you expect the experiment to answer. The answers from the hypothesis can help shape your experience when setting up a new experiment in the future.
- Name: Giving your experiment a name helps to keep things organized.
Step 4
Here, you will need to select the content for your experiment. Look at the different boxes on your screen and enter the proposed images or titles appropriately. Please note that for Manage Your Experiments test, while you can use different images and titles for variational child ASINs, you still need to submit the content for the parent ASIN.
After entering the proposed images or titles, proceed to create new content for version B. To do this, click on the “start by duplicating Version A” link. You can then make any changes to the content you wish to test.
Step 5
This is the last stage of the process. In this step, you are required to submit your experiment for Amazon to validate. Once Amazon approves your AB split testing experiment, you can then sit back and observe how the result pans out. If the validation process fails, you need to review the content of your experiment to ensure that it complies with Amazon regulations.
How to interpret the results of your experiments
For every experiment performed using Amazon’s Manage Your Experiments tool, results are usually available in different forms and updated once every week until it gets to the last week of the experiment. You can assess the results of your experiment through the dashboard. All you need to do is to click on the name of the experiment, from there, you can view the performance of each version of your content.
The results will also clearly show the version that resonates well with your audience. For instance, if there is an 80% probability that version B is better, if you choose to publish version B, then you would see an improvement in your sales and bottom line.
For each version of your content, the result will show you different KPIs, including sample size and units, conversion rate, and sales sold per unique visitor. Let me explain what each of these KPIs stands for:
- Sample size: This is the total number of visitors who saw either of the experimental content.
- Conversion rate: This is the ratio between those who made a purchase and those who saw either of the experimental content.
For every successful experiment, sellers can easily project the impact of their experimental content for up to a year. The impact is calculated by Amazon following the publication of the winning content as an incremental unit throughout the year.
What’s more? The results can also be inconclusive. However, inconclusive results are also valuable if you take a closer look at what made the results inconclusive. The reasons can be any of the following:
- The change to your content did not influence customer behavior and decisions.
- Both versions are similar in driving sales.
- Low product traffic to determine the winning version.
- The changes you made to your content were too small for customers to observe.
If any of the above is the reason why you had inconclusive results, then your best bet is to re-evaluate your hypothesis. This will help you determine what went wrong.
Canceling an experiment
Canceling an experiment means abruptly ending the process before the termination date. There’s an option to cancel an experiment. To do that, head over to the experimental details and click on the “Cancel Experiment” button, and enter your reasons for canceling the experiment.
The reasons for canceling an experiment may be because of content error or your desired results have been achieved before the termination date. Once you cancel your experiment, shoppers will begin to see only the original content.
Also, if the experiment runs until its termination date, your audience will begin to see your original content irrespective of the winning version until you decide the version to publish.
Tips & tricks
One of the good things about using the Manage Your Experiments feature is that your experiment does not affect your Amazon search ranking; thus, giving you full confidence to tweak things around during the experiment. To stay on top of your game, you can leverage these 4 tips and tricks:
#1. Make it Different
The essence of Manage Your Experiments is to find out whether there’s better content that will better resonate with your audience. However, if the two versions are too similar, a clear winner may not emerge. So, how do you make each version unique? Here are some things you can do:
- Test lifestyle imagery against product imagery.
- Refresh existing imagery, add creative product imagery or color.
- Shorten product title length.
- Use comparison charts in A+ content
- Include your brand name in the title.
If you can tweak your content using the above tips, you are sure of getting a winning version at the end of the experiment.
#2. Let the Experiment Run for Longer
I have seen cases where sellers abruptly terminate an experiment simply because they feel the result is swaying towards a version. This is not a good practice. If you want to achieve better results, allow the experiment to run until the termination date. One way to achieve this is not to regularly check the results during the experiment. The comprehensive results will automatically be available at the end of the experiment.
#3. Use the Exact ASINs for Each Test
Another best practice is to use the exact ASINs for each test. Do not contemplate using the experiment to test different ASINs. Before activating the experiment, always check to ensure that each version of the test is correctly applied to the same ASINs.
#4. Run A/B Testing Often
The last but not the least trick is to run the experiment often and in different seasons. Understanding your audience and their pain points is a secret that you can leverage to increase your sales and bottom. And there’s no better way to understand your audience than conducting A/B testing often.
Are there any drawbacks?
Of course, there are drawbacks to using the Amazon Manage Your Experiments feature. The most significant drawback is that since the test is done live, featuring your normal content would not be possible. Half of your audience will receive inferior content, while the other half receive genuine content. However, the good news is that you can use relevant software to navigate around this phase.
But the benefits far outweigh any drawbacks. The Amazon Manage Your Experiments feature is a great way to carry out split testing. The test will help you take out the guesswork in the process of optimizing your Amazon listings and polish your product listing so that it can stand out. You can use relevant Amazon seller tools on the market to gain actionable insights, key performance indicators, and spot areas in your business that need improvements.
Frequently Asked Questions
What is a split test on Amazon?
The Amazon split testing or A/B Test is an experiment that seller or seller central owners can use to compare different variations to see which variation their audience will better resonate with. In other words, Manage Your Experiments is a split testing tool. The test usually last for at least two weeks. The elements to test are the product name, product description, main image or secondary images, product copy, or bullet points.
What are the benefits of Manage Your Experiments?
This is an internal Amazon AB testing feature. While other A/B tests are like a simulation, the narrative is different with Manage Your Experiments. One of the biggest benefits of Manage Your Experiments is that the whole testing process is done within the Amazon marketplace to give a better conversion rate. Additionally, Manage Your Experiments works with real shoppers in a real environment. The results from the test are more reliable than other A/B tests.
How do you split test Amazon’s main image?
The process of splitting Amazon’s main image is simple and straightforward. All you need to do is to update your product listing with option A and allow the test to run for at least 10 weeks. Then, rotate option B for the same time period.
How does Amazon Manage Your Experiments work?
In simple terms, Manage Your Experiments is a great Amazon feature that seller central account can use to run A/B tests on their product images, product titles, and A+ content.
What’s the best Amazon AB testing tool for you?
The best Amazon A/B testing tools are Splitly, Viral Launch Dojo Listing Application, Trendle, and PickFu. While some of these tools cost money, others are free tool.
)
