Bestemmingspagina A / B-testen

Op deze pagina leert u hoe u de conversieratio van al uw pagina's rigoureus kunt verbeteren. 

A / B 's zijn pagina-experimenten die de verandering in conversieratio (bijv. Aanmeldingen, aankopen) tussen paginavarianten testen. U zou bijvoorbeeld kunnen proberen de voordelen die u pitchen, te veranderen. Of u kunt al uw afbeeldingen vervangen. Of je kunt je pagina in twee delen knippen. 

A / B-testen is van cruciaal belang

Het kan me niet schelen hoeveel werk je op je pagina hebt gestopt, ik  garandeer je dat  de eerste iteratie niet zo performant is als een die meerdere goed ontworpen A / B-tests heeft ondergaan.

A / B's zijn geen prettige dingen; ze zijn  de enige manier  om een ​​pagina in de loop van de tijd methodisch te verbeteren. 

Eerlijk gezegd zijn ze magisch. Als u een goed A / B-testregime heeft, werkt het gedeeltelijk op de automatische piloot om uw conversie aanzienlijk te verbeteren terwijl u slaapt. Dit is de laagste-wrijving, laagste-kostenmanier om uw resultaat te verhogen.

Op deze pagina ziet u hoe u A / B-tests kunt ontwerpen, beoordelen en herhalen.

Hoe A / B-testen werkt

Dit is de A / B-testcyclus:

• Beslissen wat te veranderen.
• Gebruik een geautomatiseerde A / B-testtool om de wijziging visueel te implementeren.
• Voer de wijziging lang genoeg uit om een ​​statistisch significant aantal bezoekers te krijgen.
• Met de A / B-testtool wordt de blootstelling aan de twee paginavarianten verdeeld tussen bezoekers die in dezelfde periode aankomen. (Het test de variatie niet achter elkaar.)
• Nadat voldoende gegevens zijn verzameld, rapporteert het A / B-testprogramma of de conversiepagina aanzienlijk is gewijzigd ten opzichte van de besturingspagina. Als dat het geval is, is het aan jou om de wijzigingen in de winnende variant te coderen of te ontwerpen naar de originele pagina.
• Herhaal dit tot je zonder business bent of bent aangekomen.

Er zijn drie populaire A / B-testtools:  Optimizely ,  VWO en  Google Optimize . De laatste is gratis, volledig uitgerust en geïntegreerd in Google Analytics. Ik raad het aan.

Sourcing A / B-ideeën

U kunt op verschillende plaatsen A / B-testideeën verzamelen:

• Ondersteuning en verkooppersoneel : de mensen die het meest contact hebben met uw klanten, hebben een goed idee van wat bezoekers willen zien. Op welke vragen reageren ze regelmatig?
• User surveys: Or ask users directly. Keep in mind they may not know what they want, and they may be bad at articulating it. Read between the lines when needed.
• Best ads: Your best performing ads probably have copy and imagery that can be implemented on your page. In fact, you should also run ads for the explicit purpose of finding your most enticing page copy. (I go into depth on ads on an upcoming page.)
• Past A/B successes and failures: I'll soon discuss how to report your A/B successes and failures. You should periodically revisit them for lessons that can inform new tests.
• Competitors' sites: Identify successful competitors in your space and mine their pages for inspiration. 

🎯 If you're running surveys to better understand users before running A/B tests, survey them at the point they're most invested in your company. This is when they are most likely to respond.

For example, after they've purchased from you, present three concise questions on the post-checkout page that can be quickly answered via dropdown menus.

If you don’t yet have this data to pull from, start by asking, What do I think our ideal customers would most want to see on our page? Then test every major variation.

A/B testing and the growth funnel

Before I get into what to test, we have to first understand what we're testing for.

Consider this: If you discover an A/B variation motivates people to click a button 10x more, but this behavior doesn’t lead to greater signups or more of any other meaningful conversion event, then your variation isn’t actually better than the original.

All it's done is distract users.

So remember to consider the totality of the growth funnel when assessing the results of an A/B test. The more an A/B variation affects Revenue or Referrals versus, say, Engagement, the better it ultimately is. 

That said, while the goal of A/B testing is to increase end-of-funnel conversion, what you’re most often testing will actually be early steps in the funnel.

There are two reasons for this:

• Sample size: Earlier steps in the funnel receive greater volumes of traffic. (More people are viewing your landing page than are using your app every day.)
  
  This is important because we need sufficient traffic to run A/B tests. Otherwise, our tests take weeks to complete and we'll never get through our backlog of test ideas.
• Implementation: Changing the copy or images on your landing page requires much less work than changing the code that powers your product. 
  
  Changes to your product, while further down in the funnel and more likely to directly impact Revenue, must be more calculated in their implementation. These changes can introduce bugs, confuse existing users, and impede feature development.

That's why this page focuses on A/B testing landing pages. Plus, A/B testing your product would entail a deep discussion on product development, UI, and UX. That's outside the scope of this handbook.

What to A/B test on your landing page

For any page, you’re testing what I call either a micro or a macro variation.

Micro variations are adjustments to copy, creative, and page structure. Macro variations are significant restructurings or rewrites of your page. 

Micro variations

Here are micro variation ideas to get you started:

• Copy: Header, subheader, feature headers, feature paragraphs.
• Imagery: Header image, content images, background images.
• CTA: CTA button’s design, placement, copy.
• Social proof: Try different company logos or different forms of proof.
• Forms: Number of fields, field layout, and field copy.
• Order: The order of your page sections.
• Design: Spacing, color, and font styling.
• Discounts: Introduce time-sensitive discounts.

Micro variations sometimes significantly affect conversion. But typically they don’t. 

Changing a button’s color, or making it twice as big, usually only gets you so far. 

However, there are two notable exceptions — when micros can have a big impact:

• When you completely rewrite your header or subheader copy. (I cover these page elements on the previous page.) 
  
  Header texts are the first hook into your messaging. If you've been hitting visitors with unenticing value props, fixing this will have a huge impact.
• When you reorder near-the-fold content. What visitors see above-the-fold (before they scroll) significantly determines whether they continue scrolling or bounce.
  
  It's easy to misidentify which page sections are most enticing to visitors. So don't go on instinct. Test, say, moving your Social Proof section further down the page and swapping Feature 1 with Feature 2. Now you have Hero and Feature 2 above the fold.

Macro variations

Macro variations, meanwhile, more significantly affect conversion. However, they require considerable thought and effort: You’re forcing yourself to return to the drawing board to create an all-new page — new design, new value props, new copy.

It’s hard to summon the focus and team collaboration needed to do this in earnest. 

Which is why they're rarely done.

But macro variations are a necessity. You must see the forest through the trees. 

🎯 Since the biggest obstacle to designing macro changes is simply committing to them, I implore you to create an A/B testing schedule and rigorously adhere to it: Create a recurring calendar event for — at most — every 3 months. On that day, spend a couple hours brainstorming a macro for a pivotal page or product flow. 

Here are the two most significant sources of macro ideas:

• Inspiration from competitors' homepages: Find competitors who appear to be growing and have thoughtful, well-copywritten pages. Then mimic their tactics.
  
  Tip: Don't steal other people's creative assets when you do this 😂
• Switch up the persona you’re messaging to: Tailor your value props and overall copy to, for example, mothers instead of teenage boys. Perhaps you’re misidentifying your highest-conversion or most valuable audience.

How many A/B tests should you run?

Each A/B test — or A/B experiment — tests a primary objective: increasing landing page to signup page views, increasing signup form completion, etc.

To avoid confounding test results, I recommend running one experiment at a time. 

However, within that experiment you can have several variations. Each variation will receive the same amount of proportioned traffic and will test a different approach to successfully concluding your experiment.

For example, one variation may test switching the order of a page's elements. Another may test making the page half as long.

How to prioritize A/B tests

Each A/B test has opportunity cost; you only have so many visitors you can test against in a given time period. So prioritize tests sensibly — don't run-and-gun them.

To methodically prioritize tests, consider five factors:

• Confidence
  
  How confident are you the test will succeed? You can increase confidence by better understanding users: survey them, monitor on-site behavior, and study past A/B's.
• Impact
  
  If the tests succeeds, is it likely to significantly increase conversion? The less optimized your landing page is to begin with, or the more macro your test is, the greater the potential impact.
• Implementation
  
  How easy it is to implement? Would its implementation exhaust resources or introduce technical complexity? If so, first run your other higher-confidence, higher-impact tests that are easier to implement. Then use the learnings from them to better execute tests that have high implementation costs.
  
  Or, if you want to prioritize a high-implementation A/B, determine a minimum viable implementation that'll take roughly 20% of the time to prove 50% or more of the conversion potential. If this succeeds, then you can take the time to fully implement it.
• Uniqueness
  
  Is your new test merely a variation on a past one that failed to improve conversion? (For example, are you changing the color of a button further down the page after a previous color tweak on a button higher up on the page failed?)
  
  Be considerate of what past tests have taught you.
• Brand consistency
  
  Consider this. If, say, adding aggressively salesy copy increases conversion... but you're a funeral home that normally plays it reserved and tasteful... maybe going off-brand is not a worthwhile tradeoff for higher conversion.
  
  As we discussed on the previous page, not only does brand consistency help mold user perceptions (which you can then use to your advantage for future upselling!), but sometimes you should care more about being proud of the brand you're building than merely increasing your bottom line.

Setting up A/B tests

When creating A/B tests in your tool of choice (again, I recommend Google Optimize), you want to keep the following two implementation details in mind.

Setup: Parallel vs. sequential

Always run A/B tests in parallel — meaning, your original page and its variant are running at the same time. (A/B tools will randomly assign visitors to one or the other.)

If you run variants sequentially, visitors' traffic sources and time or day of the week won’t be controlled for. This renders your results meaningless. 

Consider how traffic sources vary wildly in the quality of visitors they send. And how people sign up for B2B services at a lesser volume on weekends.

Setup: Referrer restrictions

If a visitor began reading your blog before visiting your homepage then signing up, they may know a more about your market or product than someone who came to the homepage straight from Google. 

As a result, they may respond very differently to the copy on your homepage.

Therefore, if you have common navigational paths on your site that you suspect significantly influence conversion, setup A/B's to only run if the user either came from a specific referrer (e.g. a product page) or none at all (i.e. came directly to your site). 

When you know where they're coming from, you can tailor your copy.

Similarly, if you have recurring third-party traffic sources, such as an industry blog that frequently covers you, consider setting up A/B tests that trigger exclusively for these sources — if you know how they differentiate from typical visitors.

Your A/B testing tool will allow you to easily set up these targeting restrictions.

Assessing A/B test results

You now know what and how to test, but how do you assess your test results?

You need to be on the lookout for three things:

• Achieving sufficient sample size.
• End-of-funnel performance.
• Reasons why your test succeeded or failed.

Sample size

The principles of statistics dictate that we need a sufficiently large sample to confidently measure a boost in conversion:

• To statistically validate a 20% increase in conversion, we need at least 100 visitors.
• To validate a 6.3% increase, we need at least 1,000 visitors.
• To validate a 2% increase, we need at least 10,000 visitors.

Therefore, if you don’t have a lot of traffic, you can only afford to run macro variations — because they have the potential to make 10-20%+ improvements. 

Otherwise, you’ll be waiting forever for micro tests to complete! 

Conversely, if you have a ton of traffic, congrats, you marketing wizard. You can afford to run a bunch of copy and creative micro-optimizations to fully optimize your pages.

In the example below, we ran an experiment for a client (using Google Optimize):

As you can see, our page had 1,724 views throughout the testing period. There was a 30% (29/22) improvement in our test variation over our baseline (regular page).

Had the experiment revealed merely a 2% increase in conversions, we would have concluded  the sample size was too small to consider it viable. And, that it’d take far too long to reach 10,000 visitors to justify such a small increase in conversions. We would have pulled the plug.

Fortunately, that wasn’t the case with this experiment. Looking at the chart above, we only needed 100 visitors to validate a 20% increase in conversions.

A sign of success

One more note about the above Google Optimize screenshot: When assessing your experiment, pay attention to the column labeled Probability to be Best. 

If your variant’s probability exceeds 70% and has a sufficient number of sessions as outlined above, your A/B test should be considered for implementation into your site.

End-of-funnel performance

Increasing signup conversion is nice, but increasing the total number of paying users is what matters most.

In other words, your landing page may be really good at enticing people to sign up — but bad at enticing actual customers to sign up.

Each of your test variations can incept a different set of expectations into the user that affects their behavior later in the funnel. 

So assess which landing page variation actually resulted in end-of-funnel conversion. Your analytics tool, such as Google Analytics, automatically tracks users' initial landing pages. (Seamlessly now that Google Optimize is embedded into it.) So it's easy to associate full-funnel conversion events with their respective A/B tests. 

Reasons for success or failure

Once you find your revenue-boosting page variation, try to identify why it worked. "Try" is the key word: There's no foolproof rubric to understanding human psychology.

Consider how increasing revenue is actually just one benefit of a successful A/B test. The other is increasing your efficiency at coming up with better future tests. 

This is critical because there are only so many tests you can run in given period. You’re at the mercy of how much traffic you have.

So, at the end of every A/B test, brainstorm with your teammates to make a best guess as to what went right and wrong. 

Then consider running future tests specifically designed to verify these guesstimates.

The next page explains how to onboard users so they become addicted to your app.

Updates are coming to this handbook

🎯 Interesting in building muscle? Check out my already-released Muscle Guide.