After launching a number of testing campaigns with Optimizely & VWO, I wanted to share my feedback on these two tools. Here are the particular reasons why I decided to compare these two solutions:
Optimizely & VWO both let you set up AB and MVT test campaigns. They both manage the entire production chain : from editing version / campaign settings, to reporting, including traffic division. But if we go a little further into detail, what sets one apart from the other?
Let it be noted : our conversion team has no agreement or any special interest in either editor. This is our honest opinion.
The functionalities available depend on the plan you’ve chosen. On this subject, Optimizely and VWO are, in my opinion, comparable : possibility to create AB, MVT and Split URL tests, visitor targeting, turnover tracking, application tests…you’ve got everything you need. However, Visual Website Optimizer stands out here thanks to two functionalities that are not yet present with Optimizely:
It’s difficult to give a distinct opinion on this subject. VWO’s mechanism is, in my opinion, a little bit more intuitive and logical in the process of creating a campaign. You feel more assisted, step by step.
As for Optimizely, campaign creation first starts with the creation of different versions, and then you play around with the options to set up your test, which is less logical.
Concerning the interface and the look & feel, it all depends on the degree of maturity of the user when it comes to testing. Whereas VWO has more pedagogy, if you launch a large number of versions the interface quickly becomes too busy, especially when it comes to the reporting view. Personally, I prefer Optimizely’s interface which is simpler and more professional.
Functionally, these two tools are completely identical. This isn’t the aspect that changes anything for either tool.
Here again, the two tools are similar. Trackable objectives seem more numerous in VWO, but it’s not actually the case. VWO says they track form submissions, but it’s actually a goal of page views. The same for Optimizely, as tracking turnover isn’t listed but the data is collected automatically from the order confirmation page as soon as the tracking script has been installed.
Here the point goes to Visual Website Optimizer for their completely integrated contextual help (no need to play with different tabs like you have to with Optimizely).
Optimizely works in one click with:
[Update : Optimizely also works with the tools clickmap CrazyEgg & ClickTale]
In GA for example, the interface has the effect of automatically generating a custom variable for segmenting all the data displayed according to the version the user saw, remaining fully independent of your technical team.
In addition to these tools (except AT Internet), VWO interfaces with:
What interest is it to interface VWO with your e-commerce platform or your CMS ? On most of the solutions listed, you won’t need to go through production to set up the call script to the testing tool.
Here the point goes to Optimizely with a much cleaner interface and with results that are clearly readable. Ways for reading data are also much more pertinent. Two elements are clearly distinct compared to VWO:
After having created your versions and setting up your test, you can now control the rendering on different browsers. These two tools each propose two methods:
Optimizely is clearly more advanced in its preview tool. It’s possible to visualise the page “as is”, which enables the validation of the right activation of a targeted user for example (if your test is only supposed to be active on browsers in French, for example, you can simulate this type of attribute even if you are on an English browser). The tool also lets you control event launches (if a click on a form has been defined as an objective, for example).
Visual Website Optimizer’s strength is in the simplicity of its testing tool. A little module appears on the bottom of the test page, and you simply select the version you’d like. Careful though, because this module isn’t compatible with Chrome or Firefox.
Simplicity = efficiency= one more point for VWO.
Beyond simple documentation, Optimizely proposes:
VWO’s value lies in their numerous case studies.
Concerning support, there are two possibilities:
Note : Wingify, the parent company of VWO is based in India, a country where the inhabitants are extraordinarily nice of course, but with a strong accent making any discussion all the more difficult J Optimizely however started a European development strategy in 2013 with the opening of an office in Amsterdam (the Country Development Manager is Francophone: an important detail). So it’s possible that support in French may soon be available.
Both solutions depend on the same economic model: a monthly plan (yearly with a special rate) including a certain number of testable visitors.
Globally, the rates are comparable. The loss leaders are more interesting with VWO because with only $129, you can test 30000 visitors per month (the absolute maximum in my opinion) and benefit from targeting, clickmaps, MVT (beyond simple AB).
For plans with more visitors, Optimizely is more interesting. For $399/month with Optimizely, you can get 200000 visitors (without a commitment), which costs around $600 with VWO.
Here I lean towards Optimizely, where the VWO’s loss leader doesn’t offer enough visitor volume.
Here i’m certainly going to disappoint some of you, but the question “which solution is the best?” won’t have a clear response. I know that some of you may find that deeply disappointing, but don’t worry – I’m going to add a few more details.
In my opinion, the choice of a solution should be made depending on your needs and your maturity in testing and conversion issues. There are three key points: