Conversion Rate Optimisation (CRO) is quickly becoming a staple in digital marketing programs as businesses look for avenues to increase the output of their website - whether it be Revenue, Leads or another metric.
The good news is that getting a testing program kicked off is easier than ever with a plethora of testing tools - whether you’re a small business owner or listed on the stock exchange.
At Data Runs Deep, we decided to explore three popular platforms to understand each of them and how their capability stacks up against each other.
The platforms we explored are:
Visual Website Optimiser (VWO)(www.vwo.com) - this has been been around since the early days of A/B testing and its owners have continuously refined their product to what it is today
Google Optimize (optimize.google.com) - part of Google Analytics 360 suite and very recently released to the marketplace (still in beta as of the date of this article)
Optimizely X (www.optimizely.com) - an enterprise-aimed platform offering deep options for experimentation such as developed aimed testing through technology stacks.
A few disclaimers before we go on...
1. Adobe Test & Target, an enterprise solution, was not included in this list as we were not able to use the platform as part of writing this blog post
2. As mentioned, Google Optimize was in beta mode when we conducted the comparison so it may not truly reflect the platform features post beta
Assessment factors
In this comparison, we’ll be looking at the following four factors to reflect our opinions on
i. Interface - how clean and intuitive is the platform’s interface?
ii. Platform features - what features does the platform offer and how powerful & useful are those features?
iii. Price - how affordable is the platform and what terms do they offer?
iv. Ease of integration with other platforms - how easy is it to integrate the platform with other analytics tools?
How we decided to conduct comparisons
For each of the platforms, we wanted to have hands on experience and create a simple test using the What-You-See-Is-What-You-Get (WYSIWYG) editor. As a Google Analytics Certified Partner, we were able to obtain a login to use the Optimize platform (currently in beta mode) and took out free 30 day trials for both Optimizely and VWO.
Here's what we found by the factors we've outlined:
1. Interface
Visual Website Optimiser (VWO) (above) - the most minimal of the three platforms. It's no surprise that their platform slogan is "The Easiest A/B Testing Platform" as the interface reflects this philosophy, but at times felt like they were dumbing things down.
However, this was possibly done on purpose and will be well-suited to those that are novices in creating CRO campaigns. The flipside to this was that we found some functions that required more mouse-clicks to access and perform - but again, overall the interface was easy to navigate.
Optimizely X (above) - the revised and released version of Optimizely has a slick interface and is a marked upgrade from Optimizely Classic (in our opinion). It can seem a bit busier than VWO but is structured logically and we didn't find it overbearing or cumbersome in any way.
One of the best features we enjoyed about the new interface is that doing critical functions are only a few (or sometimes, one) mouse click(s) away. Whilst this may seem to be quite a minor point for some, we seemed to appreciate this feature as frequent CRO test designers.
Google Optimize (above) - in a similar manner to VWO and Optimizely, Optimize was very clean and slick. It became very apparent to us that all three products shared very similar traits in terms of the user interface and appreciated how easy they all were to use.
For Optimize specifically, what we liked (and what was no surprise, come to think of it) was how the interface reflected the other Google Analytics suite of products. This brought to us a level of familiarity to the "look-and-feel" which we appreciate, and if you're a power user of Google Analytics products, you're likely to feel the same.
Overall, what we found was that all of the interfaces were easy to navigate and user-friendly.
2. Platform features
- All three platforms offer A/B testing and what-you-see-is-what-you-get (WYSIWYG) editors for creating variations of pages and adequate for "most" testing programs.
- However, we found that this is where Optimizely X stands out in terms of flexibility of how to build tests and where to perform tests
- With Optimizely X, you can run experiments using languages such as JavaScript, Ruby & Python and across devices including Over-The-Top, allowing testing on androidTV & iOS TV. This clearly sets it as the most powerful & flexible of the three platforms we tested. It may also means it's the most costly - but it would be justified given these features, always assuming you're going to use them.
- Of course, this level of testing may not be required for some (or even most) testers. For the "core" testing program of A/B, redirect or MTV testing across webpages (including mobile / responsive sites), all three platforms offer this so one may be better off choosing either Optimize or Visual Website Optimizer.
3. Price
- VWO is only solution with pricing listed on website - and we think it's very affordable.
- On low end, $US49 per month for 10k visitor quota makes the tool accessible to SMEs and/or those wanting to dabble in the “testing waters”, making it the most appropriate solution where price is the predominant factor
- Google Optimize's pricing structure has not been released yet (as of the date of this article) - however, it would be aimed at enterprises given it’s a GA360 solution so expect pricing to be aimed at larger organisations
- Similar to Optimize, Optimizely X has a customised pricing model and will be tailored based on features required by the user. With enterprise features like guided on-boarding, phone support across all paid plans and advanced security features, it's likely to be pitched at the high end.
4. Integration with other platforms
Optimizely X & VWO offer integration options with third party software such as Google Analytics for in-depth reporting. However, this integration can require multiple set-up steps and development changes to enable.
If you’re using the Google Analytics ecosystem, this is the major advantage of Optimize. Integration is native with almost no further integration steps required. For example, if you wanted to integrate Optimizely with Google Analytics which is injected into your website via Google Tag Manager, the integration could take hours when considering configuration (in all three platforms) and testing required to enable. This native integration would eliminate this process.
- The main advantage of native integration for Google products is the leveraging of data across platforms. For example, reporting using Google Analytics metrics, using Google Analytics Audiences and dataLayer variables via Google Tag Manager for powerful targeting options plus security given data is all within first party platforms. If if sounds appealing, it's because it should. Also, reporting of Google Optimise metrics would be possible in Google Analytics within existing reports (see screenshot below):
The verdict: Which one is the best?
We have a very analyst response to this: it depends.
They are all solid products and all offer unique point of differentiation to each other that should suit different scenarios in the testing market:
- If you're a large enterprise that is after a powerful, developer-flexible testing solution across multiple channels such as mobile apps and/or iOS TV, Optimizely X would be best suited
- However, if you're an SME looking to conduct more core testing such as A/B or MTV testing, price will be a large consideration so perhaps Visual Website Optimiser is worth a look
- An organisation utilising a breadth of Google Analytics 360 products will benefit greatly from the native integration of the suite's products. Here is where Google Optimize should garner significant consideration.
In the end, if you're looking for a CRO platform solution, you're in a great situation considering the quality of options that are available. We at Data Runs Deep are excited as this field of digital analytics continues to evolve and we can't wait to see what the next evolution of software platforms will offer. What a time to be alive.
Happy testing!
