The music industry can teach web marketers a valuable lesson about how to maximize conversions. If you’re looking at buying multivariate testing software, this article will help you put things into perspective.
As you probably already know, multivariate testing software (such as Google Website Optimizer) is software that tells you which version of your page generates the most conversions. There’s a two-step process:
Step 1—the Content Creator: Someone creates different versions of a page (or page element).
Step 2—the Scorekeeper: Multivariate testing software keeps score of which version generates the most conversions—and declares a winner once there’s enough data.
The software is an extremely valuable part of the process. Without a scorekeeper, you’d never know which version of the page to keep. However, some newcomers appear to be under the illusion that the software is almost magical—that it can create winning pages out of thin air. It’s not true.
To understand this point, think of how the music industry works:
Step 1—the Content Creator: The artist (such as Radiohead, Coldplay or Madonna) records and promotes a song.
Step 2—the Scorekeeper: A company (such as Billboard, Gallup or iTunes) keeps score of which songs have sold the most copies—then declares the winners (by publishing the pop charts).
Now, if you were a record label looking to create a hit song, you wouldn’t just ask the guy next door to record thousands of tunes, then throw them all at the charts, hoping for a winner. You’d be much better off relying on a fantastic recording artist. That’s because the first step—creating the content—is by far the most important step. That’s presumably why Polydor Records signed a £130m (US$204m) deal with U2.
The same principle applies to your web business: you’ll only get fantastic increases in profits if you make the right changes to your webpages.
Amazon.com agrees…
In other words, what matters is what you test.
Please don’t misunderstand us: we think multivariate testing software is essential for a web business. And there are some great solutions on the market—that’s why we created a free resource, Which Multivariate?, which makes it easy for you to choose which multivariate testing software is best for your needs.
Just don’t expect the software to be more than a highly sophisticated scorekeeper.
If you want help creating amazing content, you could do a lot worse than read our free reports.
(rated 4.31 by 13 people) 
Loading …
Hey guys – great article! I appreciate the illustration. It’s one that I can use with my clients to promote the importance of website testing.
Takeaway: You’ll only get what you test. Man – I could hear that every single day and probably still benefit from it.
Really interesting, as usual, thanks a lot guys!
This is a great analogy! It can also be taken one step further in that the companies takes what works (song 1 thru 10 for example) and duplicate what they think works in them. That is why we have so many cookie cutter groups and singers. Thanks great way of explaining it.
It always helps to provide real day examples people can relate to when describing subjects such as conversion rate optimisation. Most website owners don’t understand SEO, let alone CRO! Great example.
I appreciate your material is always clear and useful… So ‘garbage in – garbage out’ isn’t as catchy a title for this piece?
Hi Joshua. By coincidence, the article actually contained a section titled “Garbage In, Garbage Out”, but we cut it out to use somewhere else!
Point well taken guys.
Regarding creating great content, I’ve been reading a lot of old school direct marketing books, everything from Claude Hopkins to more present day stuff by Robert Bly. I’ve learned a ton and have been wondering if you know of any resources that talk about video marketing.
Persuasive copy writing is one way of getting your message across but can the methods be translated into video? Any feedback or ideas on how video is different? Some of us just aren’t natural writers but may do better in front of a camera.
What might be a great follow up series would be “success stories in multivariate testing” with samples so that we can learn what to try based on what has worked in the past. Alternatively, you could create a “Top 20″ article about the best ideas to try when implementing multivariate testing. Hope that helps…
Hi Todd. Good suggestions. On the off-chance you haven’t seen it already, this article contains 108 strategies for increasing your conversion rate.
Need to get around to doing this sometime – I’m sure we can all learn a lot from split testing web pages, its just finding the time to do it. Although it’s probably one of the most important factors once you have traffic going to your site – no point in having traffic if the site can’t convert it!
Great article as always guys… the software is only as powerful as the tests being funneled in at the top. However, since we’re all striving for those great results, if you do happen to find that magic pill, please let us know!
I think its good that you guys have made it so easy to compare tools – maybe it’ll force some of the companies to drop their prices!
Lately I have been reading about Taguchi testing and honestly am a little baffled by it. What is the difference between multivariate testing and Taguchi? They basically Taguchi gets to the results much quicker but is it really as reliable as a full blown multi variate test?
Art
Hi Art. Our other site, Which Multivariate?, has plain-English explanations to your questions.