You can use the MVT Multivariate testing (MVT) is a technique for testing two or more different variables in the checkout process to determine which variable creates more revenue or a higher conversion rate. Campaign
Different versions of the checkout process, called candidates, are set up. When a customer An individual or business purchasing your product or service by placing an order through Cleverbridge. The customer is the end user of this product, as they are not allowed to resell the purchased products or services. A customer is unique per client. If a customer purchases products or services from two different clients, there are 2 separate records of said customer. begins the checkout process, one of the candidates is randomly assigned if the candidates have the same link audience.
A candidate ranking indicator with
For information on how to implement an MVT campaign, see Basic Steps for Implementing an MVT Campaign.
To successfully manage an MVT Campaign, complete the following tasks:
Enter a unique name for the campaign.
- In the Display URL field, enter a unique identifier to be added to the end of the display URL. The first part of the URL is pre-filled. This URL is used as the campaign URL and is visible to customers.
- Click the Copy button to copy the URL and paste it in your website. The Active checkbox defines whether or not this campaign is actively used for testing.
- Add candidates to the campaign.
For more information, see Step 2: Add an MVT Candidate.
One candidate is automatically added for you. You can add as many candidates as necessary for the campaign. These candidates are randomly selected for customers during testing when the display URL is clicked. To add an MVT candidate, complete the following:
In the Candidate Name field, enter an identifier for the candidate that uniquely identifies what this candidate represents. This identifier is also used for reporting in the BI. Do not change the candidate name after you created the candidate. Each candidate gets a unique ID. It might happen that a name change results in two candidates with one ID.
The Active checkbox to the left of the Candidate Name field defines whether or not this candidate is actively used for testing.
Create the URL in one of two ways:
Create the URL in the Link Generator
- Click the View in Link Generator button to the right of the Destination URL field. The Link Generator contextual panel opens.
- Create the link in Link Generator.
- Copy the link.
- Return to the MVT campaign.
- In the Destination URL field, right-click and select Paste. The URL is pasted.
Manually Create the URL
You can determine how often your candidates are used in the campaign using the Candidate Weight field. If you give one of your candidates a higher
In MVT Campaign 1, there are two candidates and they both have a weight of 100. Both candidates are evenly distributed for customers in a 1:1 ratio.
In MVT Campaign 2, there are three candidates and they all have a weight of 100. All candidates are evenly distributed for customers in a 1:1:1 ratio.
In MVT Campaign 3, there are two candidates. Candidate A is distributed twice as much as Candidate B. Candidate A has a weight of 100 and Candidate B has a weight of 50. They are distributed to customers in a 2:1 ratio.
In MVT Campaign 4, there are three candidates. Candidate A has a weight of 100, Candidate B has a weight of 80, and Candidate C has a weight of 20. They are distributed to customers in a 5:4:1 ratio.
A fictional company called Shieldware creates an MVT campaign with three candidates. Candidate A is to be assigned 50% of the time, Candidate B is to be assigned 35% of the time, and Candidate C is to be assigned 15% of the time. To achieve this, they give the candidates the following weights:
|Candidate Name||Candidate Weight|
If Shieldware ever adds/deactivates a candidate, then they must adjust the weights of the remaining Active candidates and ensure that their combined
To Select a Weight:
There are three ways to select the weight:
- Click the circle and drag it to the appropriate weight.
- Click the up and down arrows to the right of the field.
- Click inside the field and enter a number.
Define your audience for the candidate . Select from the following options:
By default, customers are always routed to the very first candidate assigned to them, even if they return to the checkout process using the same computer in a different situation (for example, they access the checkout process from different countries). Therefore, if you want to give customers a fresh candidate everytime they access the checkout process, you must use the Reset MVT Cookie Parameter (
&resetMvtCandidate). For more information, see Cookie Settings.
When a URL is submitted that lacks a particular option (for example, Browser), and you have created four MVT candidates (three candidates with a particular option and one candidate without it), then only the candidate without the option will be considered.
Define a specific region for the candidate. This can be used to test different designs or configurations in different regions. To add a new region,
Candidate A is displayed in Germany. Candidate B is displayed in Switzerland.
To ensure the rest of the world has access to the standard checkout process, add another candidate and specify a region that covers all other regions not included in your region-specific candidates.
The Region option has a higher priority than the Language option. Therefore, if you defined two candidates as follows:
then a customer who accesses the checkout process with Spanish language settings and an IP address from the US will be given Candidate A.
|Language||Define a specific language for the candidate. The language is selected based on the customer's default browser language.|
|Browser||Define a specific browser for the candidate.|
|Operating system||Define a specific operating system for the candidate.|
Define an additional parameter for the candidate (for example,
The fictional company, Shieldware, wants to conduct a single MVT campaign, but they want to test different elements on different types of customers:
To accomplish this, Shieldware creates an MVT campaign named “Traffic” that contains five candidates. They name and define the candidates as follows:
They insert the following values in the Parameter name field for each candidate:
When customers access Shieldware’s checkout pages for the first time, these parameters wil route them to the appropriate candidates. For example, if a customer initially accesses the checkout pages through the Shieldware website, they will be randomly assigned to Audience 1A or Audience 1B.
On the other hand, if a customer initially accesses the checkout pages through a marketing email, they will be randomly assigned to Audience 2A or Audience 2B.
If you want to use the Parameter Name field to route customers to particular candidates, you must use the same values as for your traffic sources. Contact Client Experience for additional information.
To ensure that MVT filtering is functioning properly, you should also add an additional or fallback candidate with no value in the Parameter Name field.
Select whether or not to restrict the candidate for affiliates.
|HTTP referer pattern||
Define an HTTP referer pattern for the candidate.
The pattern must be written as a regular expression.
The fictional company, Shieldware, has links to the checkout process on its landing page, its home page, and one of its content pages. They want to test different elements on different customers, depending on through which web page they accessed the checkout process.
To accomplish this, Shieldware creates an MVT campaign named “Referer” that contains three candidates. They name the candidates as follows and define one HTTP referrer pattern for each one:
When customers are forwarded to Shieldware’s checkout pages through one of these web pages, they will be routed to the appropriate candidate.
Enter detailed notes about the candidate's purpose.
As the test progresses, use the notes field to keep track of the testing results, such as why a candidate was successful or not successful. That way you can keep a history of changes.
- To add another candidate, click Add Candidate. Another row displays with the new candidate. Repeat steps 1-5.
There are two ways to monitor your campaigns:
The ranking is not intended to provide sufficient information about which candidate performs best. It is only a relative result that needs more evaluation to generate insight for your business.
Rules for Candidate Ranking
The following describes how candidates are ranked:
- Ranking is eligible for active candidates testing against one another, which means two or more candidates with the same Link Audience settings.
If you have three candidates and only two have the same Link Audience settings, then only those two candidates are ranked.
If you have four candidates and two have the same Link Audience settings, and the other two have a different set of Link Audience settings, both groups of two will be ranked, but in two separate tests.
You have specified a region for candidate A (for example Europe), then you also need to specify a region for candidate B (for example USA). If you also want to test an operating system, you need to set up one more candidate with Phone as operating system and World as region.
The stars initially display as gray and inactive until ranked.
- Ranking is based on revenue per session.
The more stars that a candidate has, the more likely it is that this campaign worked better.However, the confidence level is significant. Contact Client Experience to help you evaluate the results in an Excel sheet. Client Experience can assist you with creating forms to calculate the confidence level.
Always use new candidates when you run tests. If you use an existing candidate to compare to a newly specified candidate, the users per session of the already existing candidate is taken into account.
The metrics under the Candidate name can help you determine which candidates to activate permanently . The following explains how to interpret them:
Orders are the number of transactions completed and paid for using the checkout page(s) pertaining to this candidate.
Sessions are the number of visits to the checkout page(s) pertaining to this candidate.
Conversion rate (CR) is the number of orders divided by the number of sessions.
(CR = Orders / Sessions)
Uplift is the new conversion rate divided by the control conversion rate minus 1.
(Uplift = (New CR / Control CR) - 1)
|Significance||Significance is the probability that your A/B test yielded trustworthy results. If you have a statistical significance of 90-100%, you can have a high level of confidence in the validity of your results.|
In order for these metrics to appear in the MVT Campaign
Step 4: Open an MVT Campaign
- In the main menu, select Tools > MVT Campaigns. The MVT Campaigns search opens.
- Search for the campaign you want to view.
- Select the campaign line item in the search results.
- Click Open. The campaign opens.
When you are optimizing the checkout process and performing multivariate testing, take note of several best practices for achieving the best test results. See Best Practice: Test with MVT Campaigns.
An important best practice is to use a display URL on your website with an underlying destination URL. See Best Practice: Use Display URLs.