The two essential elements in ecommerce are an online store
and a human being(with some money).
First, let’s consider the human.
The survival of our species depended on our ability to be efficient
hunters and gatherers. Natural selection
caused us to evolve an eye-brain connection tuned to rapidly absorb information
and act quickly. The content we see and
the order in which we absorb the information affects our behavior. This is why one web page performs better than
another. Any change to a web page,
however small, will impact customer behavior.. one way or another. If we want to encourage a certain behavior we
need to find out how changes impact the desired behavior. We need to make changes to our website and
measure the results.
So let’s consider the web page and focus in on a single
element, say a security trust mark, for example. Most of us would think that a simple AB test would
determine if our customers are security conscious and therefore it will show a positive
impact.. In reality, there are many questions
to answer. How much impact will it have
and which style and in which position will have the most impact. If you test the same style mark in two
different positions you will get two different results. Likewise, if you test different style or
brand of the trust mark you will get different results even in the same
position. I learned this running 100s of
trust mark tests for McAfee Secure. In
fact, in a typical test of 3 positions and 3 styles on the same page the resulting
micro conversion rate(adding item to cart for example) varied from -10% to +
15% vs baseline without the mark. So the
lesson here is to make sure you introduce sufficient variety when testing even
a single element. Variety creates more
opportunities for survival (clicks). There
is one optimal position and style of an element that will have the maximum
impact on purchase behavior. You could
set up lots of A/B tests to find the best style and position but the most efficient
method of testing with variety is known as multivariate testing.
Now let’s introduce some complexity. One page element like one gene may react
differently in the presence of another one.
If we take 5 elements and introduce 3 varieties of each it will create
125 different page combinations(versions of the page). Each combination will impact purchase
behavior differently because the content will be absorbed in a different order.
One version of an element may be drawing
attention away from another or the presence of one element without the other has
less impact. Given the amount of variety
it is highly probable that at least one of those 125 combinations will perform
well above baseline. It is also highly
probably that a large percentage of them will perform below baseline. The task of finding the top performer with
this much variety is beyond the capabilities of A/B testing. You will need to use a multivariate testing
tool or service.
There are two important considerations when selecting a
multivariate provider:
1. The most import consideration is the potential to lose money
while testing. Remember, many of these
combinations are going to perform worse than your current site. If the test splits traffic evenly between combinations
you are sure to lose money while testing.
There are several techniques to get around this problem including manual
culling elements or combination and the Tagutchi sampling method. The best solution is a real-time adaptive system
that can predict the winners and automatically adjust what is shown while the
test is running.
2. Ease of use. The best solution offer virtual implementation,
meaning no changes on your production server.
There are full and self service options and pricing varies greatly so shop
around. There are two multivariate
providers I am familiar with and can recommend:
HiConversion and Maxymiser.
One more note regarding variation and survival. All people behave differently, the same
person behaves differently at different times, and new people are constantly
coming to your web store. You cannot expect
a top performing combination that you find while testing to continue to perform
the same over time. There are some techniques
and new technologies that take advantage this time varying nature of people
and websites. (topic for a future post).