Last week I flew west to San Francisco to attend the very first OptiCon. OptiCon is Optimizely’s brand new conversion rate optimization conference. Experts from many different companies took the stage to talk about their work and shed a little light on improving CRO services. Despite the varied industries and methodologies, at the heart of each presentation were a few key principles underlying each talk.
Test Small
Don’t get caught up in questioning if a test is too small. There is no mandate in CRO that each test should be revolutionary. Rather you should seek evolutionary growth through small iterative tests. Often times it will be the multitude of small wins that make the difference over time.
You can also think of this in terms of pay offs. By making many small bets, you can quickly test new ideas each with a potentially modest pay out and with little investment. Alternatively, you could aim for massive tests. But if they failed, you would quickly lose motivation and support by heavily investing resources in tests that didn’t have any real impact.
Small tests also afford you the ability to run more tests . Thus increasing your learning and allowing you to respond and fine tune more quickly. Over time the effects of iterative tests will begin to build momentum. Similar to a PPC campaign, there usually aren’t any grand sweeping changes. Success is built on continued optimization rather than a single sleight of hand.
Structure Tests to Enhance Learning
It’s important to have a reason for testing, rather than just going off of a whim. This not only keeps the test from being arbitrary but provides a foundation for the solving the problem of why. Testing with a theory or hypothesis allows you to tailor testing to your advantage. No matter if the test succeeds of fails, you’ll at least begin to answer questions about site visitors.
Let’s consider a scenario where we want to test the navigation elements on a page. Rather than say, “I want to change this menu”, you can consider “do all these menu options make it harder to the user to convert? Users usually prefer simpler options that get to the point”.
If the minimal navigation works, you can then ask, “did conversion rate improve because the initial items were unclear or was it because those items were never needed?” Now you have a follow up test, perhaps adding some of the high priority navigation elements back into the page to see how they impact behavior.
Framing each test as a question not only directs your thinking but also requires you to consider why you are performing the test in the first place. Use this to your advantage. If you are ever challenged on why you should continue testing when many of the tests fail, you’ll be able to offer the knowledge you’ve gathered. Knowledge you wouldn’t have had if you were not testing.
Consider All Aspects of The Test
When you are testing new elements, always make sure to examine the finished product yourself. This sounds obvious but it is easy to forget.
It reminds me of one of the lessons a physics professor stressed, “no matter how well the math works, always look at the numbers you arrived at. Do they actually make sense?” It’s too easy to invest yourself in the process, only to overlook the obvious fact that you actually missed something in the process.
While the page layout may look great in your editor, it might not look that way in the wild. The live page may leave certain elements out of view for users on certain sized screens. In the worst case scenario the page cuts off before the user can see the “buy” button. The button is there but below the fold. Depending on the spacing, it may not be apparent to the user that the page continues, rendering it no more than an informational page for the product.
Other aspects to account for are the user’s expectations. Redesigns and elements under testing may differ far too significantly from common standards. This leaves users unsure of what they need to do to complete the task. An idea may look great on the drawing board but if the new page puts more of a burden on the user, they will opt to go somewhere else instead.
Always Test Something
We’ve hit on this a few times in the preceding paragraphs but you should constantly test different elements of your page. It sounds simple but it is often small things like this that end up sliding in the general bustle of a day. You might tell yourself, “Well I ran a test week so the next one can wait a bit.” Your short-term focus will hurt you over time.
While many tests will fail, the learning you gain will always pay off in the long run. The actual gains for successful tests, even if incremental, will build as well. If you still aren’t convinced, maybe you need to scare yourself into realizing that without consistent testing, you could leave a good portion of potential leads and revenue on the table. Gains that would not require anything but a few small tests.
