How to Perform an A/B Test for SEO

Most marketers go live with a campaign understanding the initial campaign and content will not be static.

Then they gather information and review the strategy’s effectiveness after a few weeks/months of running.

They learn from the data what is needed to make necessary fixes to the campaign and updates to the content for better future results.

Unfortunately, this approach is somewhat flawed.

Applying fixes to a campaign after it goes live is better than not making any fixes at all, but theoretically it would be much better if the campaign were fixed before it went live.

Fortunately for marketers, there’s an easy way to do this.

Enter A/B testing.

What Is A/B Split Testing?

AB testing is a form of campaign analysis where a central campaign is split into two analogous yet distinct segments: an “A” campaign and a “B” campaign.

Source

Both campaigns run under identical scenarios for a given period of time, and results are collected about each.

Those test results should indicate a clear winner between the two, and the differentiating factors between them should be the root cause for its superiority.

Essentially, AB testing is a scientific experiment to apply to your marketing campaign to determine what, if anything, needs to be corrected in your strategy.

You’ll have at least two versions of the same web page (usually more) that you’ll test to see which elements perform better. Unlike some other tests, you’ll get a statistically significant result from A/B testing.

If you apply AB testing over the course of several rounds, producing new variants with each round, you’ll create a survival-of-the-fittest type of scenario that will ultimately produce a campaign better than any of the variants you have created along the way.

For example, you’ll start with an AB test, then introduce variant C if variant A proves to be better.

Then you’ll hold an AC test. If A is still the winner, you’ll introduce variant D in an AD test, and so on.

While most marketers only use AB testing in a live environment, it’s more advantageous to apply it to a testing environment first, on a much smaller scale.

That way, you’ll save the time and money of going to a live market—and you’ll only put your best foot forward for your potential new customers.

Implementing Your A/B Test

Finding the perfect scenario for your preliminary AB test can be difficult, since no matter what you’ll have to invest some time and money, but if you perform it properly and apply the necessary changes before going live, you’ll wind up with a much more cost-efficient, better performing campaign.

It’s not likely that your test results will produce false positives, since you’re measuring conversion rates based on changes you make to page elements. There aren’t really any external variables that can impact your tests.

How to Differentiate Your A and B Tests

The first step is to create a “B” variation of your already existing “A” campaign. For example, let’s say you’re working within the confines of a simple PPC Google AdWords campaign.

You’ve set your target keywords after performing extensive research, you’ve outlined your copy and headlines, and you’ve set up a landing page where your visitors will (hopefully) be persuaded to convert.

You have two main options for differentiation here: ad copy and the design of your landing page.

For most AB tests, you’ll want to start differentiating on the largest scale possible.

Landing page design changes tend to have more impact than copy changes, and will influence your conversion rate more than your click-through rate, so in this scenario, a landing page differentiator is preferable—at least to start with.

How you choose to differentiate within that segment of your campaign is up to you—there are plenty of options.

You could change the background image, the placement of your form, the number and type of fields within your form, the colors of your page, the copy in your headlines, and the addition or removal of extra features like testimonials or external links.

You can make one or more of these changes right away, but remember—it’s better to make bigger changes first.

Finding the Right Environment

Since running your campaign in a live or pseudo-live online environment will cost you money, it’s important to find the right platform for your AB test.

Let’s take the PPC example above. One of the best options is to run the test like you would a normal campaign, just with a much smaller budget. If you’re planning to run the campaign with a $1,500 monthly budget, try running it for one month with a $500 budget to determine where you stand.

Alternatively, you can test your landing pages on a different platform, such as on your social media channels. Facebook has a paid advertising feature that can be run for as little as $5 a day, and can give you great insights for a simple AB test without costing a lot of money.

You’ll also want to make sure to capture as much information as possible—use objective data like click-throughs and conversions, but also consider qualitative metrics like heat map results or user surveys to round out your analysis.

Analyzing the Metrics

Once you’ve run the AB test for a week or longer, you’ll have enough information to make a judgment about your campaign.

Obviously, in most cases, numbers will be your bottom line—the landing page that produces the greatest number of conversions is your best candidate—but you also need to pay attention to qualitative metrics to understand why the better page is better.

Source

This type of data can tell you whether something is critically wrong with your alternative landing page (so you never make the mistake again) or if there is a key feature of your main landing page that is so important it’s worth enhancing (such as a testimonials section that deserves a greater callout).

Chances are, you’ll be enlightened (or at least surprised) by some of the metrics your initial test produces.

Any positive changes you make as a result of the test will help your campaign significantly—and applying those changes before you start a full-fledged rollout can save you both time and money.

Continuing Your Test

Just because you’ve ironed out a handful of kinks before taking the campaign live doesn’t mean you’re completely out of the AB testing woods.

You should treat your campaign as an ongoing experiment, ever evolving, because conditions always change and there is always room for improvement.

After your preliminary round of AB testing, you’ll have a much better initial product, but it’s going to take continued AB testing in order to perfect your approach.

Roll out your winning landing page with a secondary differentiator—one that’s been altered in a way different from your initial test.

For example, if your first AB test focused on aesthetic design changes, try making some major copy changes. Gather information regularly, and continue to make adjustments in order to ratchet up your conversion rates.

The benefits of A/B testing

A/B testing is a critical component in Conversion Rate Optimization (CRO). It’s a key aspect in advanced digital marketing strategies and should be a standard part of your marketing budget.

The results of a split test will show you what you can change on your web pages to increase your conversion rate. When running tests with two or more variants, you can expect to generate results that will show you which areas of your site can be changed to increase revenue. For example, say you edit your call to action on a sales page. You might find that your change decreases the bounce rate for that particular page. From there, you might alter all your calls to action to see how it affects the bounce rate of your other pages.

These are some of the benefits to employing a split test on your website:

• Insight into user behavior
• Insight into how your website visitors respond to elements like colors, shapes, and placement of buttons
• Reliable data regarding multiple metrics that can tell you about customer satisfaction
• Insight into what web page elements improve user experiences
• What call to action will get your site visitors to read a blog post
• Which new features your visitors like

A/B testing is a good way to collect data

The data you collect from your split tests can be used to understand user behavior, engagement rate, your market’s pain points, and how well they respond to certain features on your website. For instance, you might get the idea to add a major section to your website, but your visitors may not seem that interested. It could be that you haven’t provided an easy way for people to find the section, but it could also be a lack of interest.

Split test for all users, mobile and desktop

When split testing, keep in mind differences between mobile versus desktop. Don’t forget to run your split tests in a way that reaches both mobile and desktop users. Website traffic that comes from a mobile device will expect your site to be mobile friendly, but also, your entire page should contain elements crated in such a way that mobile users enjoy the experience. For example, your font should be larger than usual and should fit the screen without any unnecessary sidebars. If your sidebar navigation isn’t collapsible, you may want to test variations of your sidebars, including eliminating it completely. Mobile users need content to be presented in a simple, straight forward, and compact manner or they might bounce.

Your conversion rate depends on testing elements geared toward creating a better mobile user experience. Make sure you test mobile-critical elements until you get enough data to know what your web page visitors prefer.

Don’t forget to split test your pop-ups

Pop-ups play a critical role in your ability to generate leads and achieve other conversion goals. If you use them, test every pop up with different versions of the headline, call to action, and submit button. Don’t be afraid to make significant changes to a pop up to see what works. Normally, copy won’t impact conversions as much as page elements, but when you’re working with a pop up, your copy has a significant influence on conversions.

To test your popups, run more than two variations of each one so you can split test all the elements. For instance, create different versions of While you test your popups, try not to create different versions of each web page so you can know what people see in the background isn’t affecting how they interact with your pop up.

Running multivariate testing on pop ups designed to capture leads is a great way to increase your conversion rate.

Create a system to track your tests

When you’re running multivariate tests, it helps to have a system to track your test results and future tests you plan to run.

Also, take note when you run campaigns that actively generate more traffic to your site because that will impact your conversion rate when you compare it to what you’re used to getting.

Here’s a good way to create a system for tracking the split testing aspect of your optimization program:

Create a spreadsheet, either in Excel or Google Sheets, and list all of the URLs you have on your website with the exception of blog posts. A blog post won’t necessarily need to be optimized as much as your service pages, sales pages, landing pages, and online store (if you have one).

Once you have a list of your URLs, create columns to track the date you start testing each page as well as how many versions you create and which elements you alter.

Document how much monthly conversions you already get to each of your listed URLs and then create a blank column to document your monthly conversions with each change. You might find that your conversion rate skyrockets on some pages, but not on others. When you find elements that seem to increase conversions, create variations of the page with those elements intact and start altering additional elements one each page.

A multivariate test doesn’t need an end date; it can be infinitely ongoing and that’s actually ideal. It takes time to test multiple versions of any web page because your test run needs to happen over time. Although, you can set time periods for checking and documenting website data to capture what your test concludes at specific intervals.

Split test your email marketing campaigns, too

While you’re working on split testing your web pages, don’t forget to apply A/B testing to your email marketing campaigns. Your emails have the potential to increase your traffic acquisition to get you more visitors that are ready to buy from your company.

Run an A/B test for the following:

• Subject lines. One of the biggest conversion problems with email marketing is the subject line. If you don’t use the right subject line, your subscribers won’t open your email and the whole process will be for naught. Successful email marketing campaigns begin with enticing and effective subject lines.

• Greetings. How you address your email subscribers matters. However, if you don’t collect the right information, you won’t be able to greet people effectively.

For instance, it’s best practice to user a subscriber’s first name in the greeting part of your emails. For example, you can start your emails with, “Hi Bob.” You can use phrases like, “Hi, Friend,” but this won’t feel personal and won’t generate much rapport with your subscribers. To get a subscriber’s first name, you’ll need to ask for it specifically in your sign-up web forms. From there, you can program your email marketing system to insert their first name in each email.

• The length of your emails. Email length is a good element to test. This is one variation not everyone tests, but people have short attention spans and A/B testing the length of your emails can help you determine what’s right for your particular audience.

• Your calls to action. Just like your CTAs on your website, you’ll want to split test your CTAs in your emails. Getting your email CTAs right will help you boost sales and convert new visitors faster.

What results should you look for?

When split testing, you want to find out what elements perform best, but in order to do that, you have to define your goals. For example, if you want more conversions, you have to define conversions. Is it email signups? Sales? Video plays?

A split test succeeds when it provides you with the data you need to make changes that will improve the performance of that particular web page.

The consequences of skipping A/B testing

If you’re not constantly running split tests on your website, you’re losing out on potential revenue. The more website elements you optimize, the more you’ll increase your conversions, which includes sales. It also includes email list signups, which are your future sales from your target audience.

By skipping this important element of CRO, you’ll cut yourself off from understanding visitor behavior and identifying all the ways you can improve your site to generate more conversions.

Ask for user feedback

If you want more conversions, split testing is going to help you figure out what your visitors want and which elements they respond to the best. Although, with split testing, you’ll generally be gathering data on elements your visitors respond to on an unconscious level. Another way to figure out what your visitors want is to ask them for feedback directly. This isn’t directly part of split testing, but it will complement your multivariate testing efforts.

AB tests are an inbound marketer’s best friend.

There’s simply no better way to objectively measure the effectiveness of your campaign, whether you do it before or during the official launch of your program.

Starting off with an AB test before rolling the campaign out in a live environment will help you identify any pain points preventing you from better conversion rates, and save you money in the long run.

Just remember that AB tests are something to play with throughout the entire duration of your campaign—the more you learn, the better your campaign will become.

• Author
• Recent Posts

Sam Edwards

Chief Marketing Officer at SEO.co

In his 9+ years as a digital marketer, Sam has worked with countless small businesses and enterprise Fortune 500 companies and organizations including NASDAQ OMX, eBay, Duncan Hines, Drew Barrymore, Washington, DC based law firm Price Benowitz LLP and human rights organization Amnesty International.

He is a recurring speaker at the Search Marketing Expo conference series and a TEDx Talker. Today he works directly with high-end clients across all verticals to maximize on and off-site SEO ROI through content marketing and link building. Connect with Sam on Linkedin.

Latest posts by Sam Edwards (see all)

• How to Rank Without Building Backlinks - February 8, 2023
• 14 Local SEO Strategies & Tips - February 1, 2023
• Top 200 Google Ranking Factors: 2023 Complete List - January 27, 2023