Why you shouldn’t be afraid to A/B test your ecommerce site during the holiday season

Aug 11, 2020 • 5 min read

The holiday season brings about a huge increase in ecommerce traffic. Because of the potential for sales to skyrocket, many ecommerce managers implement a sitewide freeze during this time to prevent any changes from becoming roadblocks that throw visitors off track.

While there are reasonable concerns about making site changes during an increased traffic period, there are also many rewards that can outweigh the risks. Sites that implement and execute well-planned conversion optimization experiments can see even more additional revenue on top of the already increased sales they typically see at this time of year.

Let’s take a look at why the holiday season is an especially productive time to A/B test your ecommerce site, what kinds of experiments are worth running this time of year, and how to make sure your holiday testing goes smoothly.

Why you should A/B test during the holiday season

Online spending during the holiday season has grown tremendously year after year. In 2019, a record $142.5 billion was spent online during the holidays, a 13% increase from the previous year. Sales are expected to continue to increase this year, but due to logistical changes brought on by COVID-19, some things will be different in the 2020 holiday season.

For starters, holiday sales are expected to begin even earlier this year. Many major retailers are planning to close physical stores on Thanksgiving to prevent more coronavirus outbreaks among the large crowds that begin lining up at their doors hours, or even days, in advance. Instead, they will be starting to offer their biggest deals online as early as October. With more major retailers offering sales sooner, more consumers are expected to start their holiday shopping earlier as well. Because of this, ecommerce sites can expect to see the seasonal traffic boost to start earlier as well.

More traffic = faster results

The traffic boost that sites typically see in the holiday season is one of the big reasons why testing during this time period can be so beneficial, and even more so this year with the boost beginning earlier. When a high volume of traffic runs through a test, you’re able to determine the winner in a shorter amount of time. And the sooner you get conclusive results, the sooner you can implement a conversion-increasing experience on the site.

For example, if you were to begin a test in early November, which many are predicting to be the start of the holiday rush this year, you could have clear results long before Thanksgiving. This would enable you to implement your findings before Cyber Week, helping you to achieve increased performance and a higher yield for the remainder of the year. And since the holiday season is typically the time of year when many businesses bring in the most revenue, the impact of a conversion rate increase will be amplified even more than it would be in the non-peak season.

As an added bonus, being able to complete tests quickly frees up more time to execute additional tests, helping you to find even more ways to improve conversions and increase your revenue in the most lucrative time of year.

Gain insight into new visitors

During the holiday season, shoppers have a different mindset. They have a shopping list to complete, and a limited amount of time to do it. Those who normally sift through reviews and thoroughly research a company before every purchase are more likely to skip that step simply because of time constraints.

Testing your site during the time when more of your new visitors are highly motivated to make a purchase gives you more opportunity to discover what works and what doesn’t work to convert them. Though new visitor behavior might not be the same year-round, it can still provide you with some insight into what changes might be worth testing in the regular shopping season as well.

A/B tests that have a big impact in the holiday season

As discussed in a Twitter chat hosted by Optimizely, the most impactful A/B tests during the holiday season are those that experiment with different ways to alleviate visitor doubts, fears, and uncertainties. Here are some straightforward test ideas that do just that, without requiring any major structural changes that could negatively impact your site’s performance or user experience.

Urgency messaging

Testing different urgency messaging options helps you find out if time constraints or the fear of missing out motivates your visitors to purchase. This can be a useful strategy to apply even in the non-holiday season. Try including messages like “only 2 left” next to products that are low in stock or a countdown timer showing when your holiday promotional discounts expire.

Shipping and return policies

Customers want to know if they’ll be able to exchange or return items before they purchase, especially when those items are gifts. Experiment with the best place to display your shipping and return policies to find where customers are most likely to notice them and gain the confidence to make a purchase.

Trustmarks

Shoppers have more motivation to buy during the holidays, but that doesn’t mean that they’ll trust your site right off the bat. According to our recent survey of 600 US-based consumers, 90% of customers check one or more external sources before making a purchase on an unfamiliar site. Trustmarks are one way to show new visitors your site is one they can trust, and testing them during the holiday season is a great way to find out which trustmark placements have the most impact with your customers.

Holabird Sports earned 7 TrustedSite certifications and tested the trustmarks in December 2019. They saw some amazing results–a 16% overall conversion increase, 22% conversion increase with new visitors, and 21.3% revenue increase. See how they implemented the trustmarks throughout their site in our Holabird Sports case study.

Testing TrustedSite during the holidays can be even easier with our free testing program. See if you qualify and get your test started!

Tips for testing during the holiday season

Running A/B tests during the holiday season can be very rewarding, but it’s important to thoroughly plan and monitor your experiment to avoid a negative outcome. Follow these tips to ensure things run smoothly.

Make sure your site is prepared to handle a big traffic increase. If your site goes down (as has happened to many major retailers during Cyber Week) you lose potential testing time and it will take longer to reach a conclusive result.
Communicate with key stakeholders in your company so that everyone, whether they are involved directly or indirectly with the test, is on the same page about what’s being tested. Set expectations for the actions that will need to be taken once you have your results.
Start planning your tests as early as possible. As we’ve already discussed, the earlier you can execute your tests the better. Having your experiment plan ready earlier enables you to launch earlier as well.
Check in on your tests daily. Though you might only check on tests once a week in the non-holiday season, in peak season you need to monitor for conversion decreases more often because they can be amplified just as much as conversion increases can be. Variants within tests that are losers should be eliminated rapidly to avoid harmful impacts on revenue.
Do extra QA. This can be extra manual checks on your computer, cross-browser testing, and load-speed tests. These should all be done more often than usual to make sure your site is continuing to work properly during peak season.

Testing your site during the holiday season can be very beneficial because you’re able to take advantage of a high traffic period and see results sooner. Try creating straightforward experiments to prevent distrust and ensure a better user experience for your visitors. It’s never too early to begin holiday testing plans–get started now.