Optimizely X and VWO’s new CRO platforms vs. Google Optimize 360

First, Google released it’s suite with the A/B-testing tool Optimize 360 (and the free version Optimize). At the same time, both Optimizely and VWO launch new platforms and improved products… In this blog post, we’ll sum up what’s new and what it’s important to keep an eye on.

Optimizely X from Optimizely was launched in October and is a experiment platform – Simultaneously VWO announced they will launch a conversion optimization platform called Evolution. And, earlier this year, Google also launched it’s new platform – Google Suite 360.

It’s clear that VWO and Optimizely are going different directions, but also that the difference compared to Google’s 360 Suite still is huge. Partly incomparable.

We want to clarify the differences from an A/B-testing perspective – and explain how the differences affect you.

Optimizely X
– An experimental platform

Optimizely have used all their experience (they are the leading A/B-testing tool in the world) and rebuilt their product.

Their new, improved product is a platform with four different segments – together called Optimizely X.

The original testing tool is still there as a “basic” version.

Optimizely X OTT – Over The Top

On the new X platform, it’s possible to run experiments on ALL devices you can connect to the internet. Like consoles, smart watches, smart speakers (eg Amazon Echo), Apple TV, drones, the web, apps… Everything.

So it’s a feature that takes experimentation to a whole new level, that follow the user digitalization and enables optimization on several devices. In other words, it’s the opposite of running test projects and experiment on element level – now its possible to analyze and optimize user behaviour in omnichannel.

Optimizely X Full stack

This news makes us happy – the Full stack allows more advanced experiments in all applications, front end as well as backend. The experimets can be developed in languages such as Python, Java, Ruby & Node (and more to come). Be a power user for real and experiment both server side and client side.

What it means? You are able to run experiments deeper in your technology – in search results, filters, trough algorithms, etc (not just in front end). And all this on different devices that has an internet connection!

Full stack simplifies your ability to create experiments that give insights into user behaviour for real.

Today, Optimizely X is unique with this feature.

Optimizely X web

Optimizely has rebulit their existing A/B-testing product for the web. With Optimizely X it’s possible to run experiments on responsive and single page applications. The interface is updated and simplified.

Full stack is applied here – you can develop experiments in your built technology.

The statistic model is still the same: frequentist statistic (compare with VWO’s and Optimize’s Bayesians statistic – how that matters is described in a separate blog post).

Optimizely X mobile

Allows experiment in iOs and Android apps – without having to wait for reviews on your app updates from App Store or Google Play. It also allows Omnichannel experiments.
This X-product too is newly developed and updated from their original product.

VWO Evolution
– A CRO-platform

An other leading A/B-testing tool is VWO. Their new platform still is a few months from launch – and they are on the same track as Optimizely. But instead of develop A/B-testing, they add new CRO segments.

Their Optimization Platform, today called “Evolution”, makes it possible to cover several CRO tools from the same platform – from tracking, analysis and insights to heat maps, recordings, documentation and A/B-testing.

That makes VWO the first tool with that many CRO components in the same platform (Optimizely does personalization as an additional feature and Google has Audience center for personaization plus Analytics 360 for analysis (personalization will become a part of Optimize 360)).

Comparison: Google Optimize 360

TLDR; A/B-testing is the core of Optimizely and VWO – They have the longest experience of A/B-testing and have A/B-tested a lot. Google has a long way to go to catch up with them… It’s almost unfair to compare the tools.

Optimize 360 is still more or less newly launched, where the basic idea is A/B-testing with seamless analysis in Analytics 360 – and it works fine.

A/B-testing with Optimize or Optimize 360 is today only possible on the web. You can’t test in backend or A/B-test native apps, etc.

There is a HTML/CSS support in the tool, but no power user mode. This means that you are limited in what and how you A/B-test. Also note that the tool isn’t suitable for single-page applications (SPA), but they’re working on a solution which today (when this blog post was published) has reached the alpha stage.

Optimize 360 has a personalization feature and like VWO they use a Bayesian statistical model (more about the differences in statistics here).

The big differences in A/B-testing functionality does not mean that Optimizely and VWO are the right choice for you. It depends on your needs.

Read more about Optimize 360 and our entire statement here.

Conclusion – What tool?

The boring answer: It depends.

Moreover, differences in price – but now, we’re focusing on the A/B-testing functionality.

Optimizely allows hardcore experiments, A/B/n-testing deep down into your technology – omnichannel – and multi-language code on all devices with internet connection. It’s impossible to come even close to that kind of experiment in any other tool today. But… it also requires that you have the skills to practically handle the functionality and the insights you get (and don’t forget the amount of traffic you need).

Optimizely is, according to us, by far the best A/B-test tool – If the need and conditions is there for this kind of powerful experiments and insights.

Google Optimize 360 may be a good option if you value easy analysis, using existing segments, attributes, and reports within Google Analytics (or Analytics 360). But also remember that you are limited in your experiments, it’s not suitable for single page applications and there’s no power user mode.

VWO is still a good A/B-testing tool – also with an other statistical engine compared to Optimizely and we’re looking forward to the launch of the new platform.

If you are looking for a free tool (not premium) – Then Optimize probably will do fine. Or A/B-test for free through GTM .

Please contact us with your reflections or questions – We use both Optimizely, VWO and Optimize. We are partners and retailers of all three of them – our main interest is the effect of your CRO work. The tool is rarely the decisive factor.

Call us now: +46 33 32 26 or contact us below.

Read more about the new platforms here: Optimizely X och VWO Conversion Optimization platform.