To ensure proper tracking of AB or MVT tests, there’s nothing quite like a little booster shot to remember the different indicators to track during testing. Several articles will be published in the altima° blog under the title “Testing Glossary” where 1 article = 1 definition. Stay tuned!
The Holy Grail… Performance KPIs
To get this series of articles going, let’s start straight off with the Holy Grail of indicators, the session value. The session value corresponds to the revenue generated for each session on a site for a given period.
It comes then as no surprise that the session value is calculated as follows:
Session value = Sales revenues / Number of sessions
For example, for a given day, 1000 sessions are recorded for your site and sales revenues of 2000€ are generated –> the Session value = 2€
In e-commerce, the average purchase and the conversion rate are both two primary indicators that are tracked to evaluate business performance. But if the average purchase increases while the proportion of visitors who convert decreases, in the final analysis, do I win? The session value allows us to have a unified view of the average purchase and the transaction rate by reconciling these two indicators in order to give an overall view of the results.
While less intuitive, there is an alternate formula for calculating the session value:
Session value = average purchase x conversion rate.
If this formula is broken down:
– Average purchase = sales revenues / number of transactions
– Conversion rate = number of transactions / number of sessions
In the end, we land on our feet:
Session value = sales revenues / number of sessions.
The average purchase and the conversation rate will thus have an underlying effect on the session value. The session value then makes it possible to reconcile these two indicators into a single Holy Grail (#humbleKPI).
… and in real terms for testing?
In the case of an A/B test: control version A and test version B, the session value will be calculated for each of the two versions.
Depending on the nature of the test, only one KPI may be targeted. As an example, when testing different highlighting of a product page’s cross-sell blocks, we will seek to increase the average purchase: that will be the reference KPI. On the contrary, it is possible that the best performing test version displays a beautiful increase in the average purchase but a slight decrease in the conversion rate.
The session value will make it possible to have a compiled view of these two indicators and will make it possible to determine whether the test version does in fact generate superior results.
Beyond the gain in sales revenues represented by the session value, this indicator has its true carryover in traffic acquisition: “each session costs me so much; up to how much more am I willing to spend to attract one more session to my site?
The increase in the session value will also allow you to improve ROI for paid levers and consequently, to spend less and earn more. The session value creates an analytical bridge between testing and traffic acquisition.
Depending on the business structure, this indicator can also apply to the visitors rather than the sessions. In that event, we refer to revenue per visitor.
Moving forward, we’ll discuss other indicators to heed in addition to the session value for optimal test tracking. Next episode, the campaign balance!