Multivariate testing is a technique for testing two or more different variables in the checkout process to determine which variable creates more revenue or a higher conversion rate. Small changes to the cart, such as a button or a change in color, can have an effect on the conversion rate. Performing multivariate testing on these changes is crucial for optimizing your checkout process.
If you are unsure about the best way to present your checkout process, then set up a multivariate test. For more information about the benefits of multivariate testing, see Best Practice: Test with MVT Campaigns.
How Multivariate Testing Works
You set up multivariate tests in the MVT Campaign portal. Different versions of the checkout process, called candidates, are set up. When a customer
An individual or business purchasing your product or service by placing an order through Cleverbridge. The customer is the end user of this product, as they are not allowed to resell the purchased products or services.
A customer is unique per client. If a customer purchases products or services from two different clients, there are 2 separate records of said customer. begins the checkout process, one of the candidates is randomly assigned. Because the candidate is randomly assigned, additional influences such as the day of the week, or the time of day won't influence the test results. However, it is advised to avoid bank holidays.
The candidate shown to the customer has a cookie so that the customer will always see the same cart if they click the link again.
After a significant number of completed paid orders, the total revenue generated by every candidate in the test is compared by looking at the results in reports. The number of paid orders you need in order An agreement between a seller and a buyer to exchange goods and/or services for money. An order can: - contain multiple products and quantities; - have multiple financial transactions. A preorder authorization is considered an order. to determine a clear result depends on the confidence level, which tells you how certain you can be. It is expressed as a percentage and represents how often the true percentage of the selected candidate lies within the confidence interval. A 95% confidence level means you can be 95% certain; the 99% confidence level means you can be 99% certain. Most researchers use the 95% confidence level. The confidence level is calculated based on a formula of the uplift (a standard deviation) and the normal distribution. Only when you reach a confidence level of 95 to 99%, the uplift is not a coincidence and can be considered trustworthy.
You then select the winning candidate to use for the checkout process. You can continue to use The MVT campaign to route traffic to the specified candidate.
Variables
A variable is the item you want to test. A variable can be anything you want to change or test in your checkout process, such as:
- Different checkout process configurations for the same market
- Price points
- Opt-in vs. opt-out
- Elements per page
- Icons, buttons (for example button size and color), colors
Multivariate tests can be set up with several different variables tested at once. You are not limited to two variables as you would be with traditional A/B testing.
Every candidate should contain only one variable for the clearest test results. For example, if you want to test an opt-in vs. an opt-out, then one candidate has the opt-in variable and another candidate has the opt-out variable.
Weight
A weight is assigned to every candidate and determines how often that candidate is randomly assigned to customers. For example, 40% of the traffic can be routed to candidate A and 60% to candidate B. The weight you decide to use depends on your business needs and goals for the test. It is advised that you leave the default value of 100, because then the equal distribution is done automatically for the active candidates.
Example: You have a 50/50 split of the weight, which means candidate A gets a weight of 100 and candidate B gets a weight of 100. If you split 66,6/33,3, it means candidate A gets a weight of 100 and candidate B gets a weight of 50.