CATEGORY: Help ArticlesCampaigns

Campaigns - Split Test campaigns

Campaigns

The Campaigns page lists all of your previously created campaigns, ordered chronologically. There are a number of different campaign types, which are described below. A brief summary of each campaign is shown on this screen, along with dropdown options to Send, Edit, Test or Delete each. There’s a Search and Filter option to allow you to locate a particular campaign quickly.    

Split Test Campaign 

The Split Test campaign type will allow you to test multiple message designs, delivery times or sender details to determine which performs best. The campaign is divided into a number of splits, each sent to a fraction of the recipients. The results and interactions are collated, allowing you to choose which of the splits is most successful and therefore is the winner. The winning split is sent to the remainder of the contacts. 

Split Settings 

The Split Settings step functions similarly to Message Selection, although it allows you to choose a different message for each split. Additional splits can be added from this step, up to a maximum of ten. The percentage to be sent to each split and the remaining split are set on this page.

Why can't I have a 50/50 split?

The maximum split you can do is 49/49 as you have to have something left for the balance. The idea is you test two or more variables and send the best performing version to the rest. Normally, a split test of 20% / 1,000 contacts will give you a statistically valid result.

Define Delivery

A group of contacts is chosen on this screen to be the recipients of your campaign. Contacts and groups of contacts can be amended within the Design Studio section. In addition, you can optionally suppress certain groups from being sent to. For example, if you have a large group of contacts with a small subsection that shouldn’t be sent to, the smaller group can be suppressed without having to make lots of manual amendments in Design Studio to accomplish the same result. Once your contacts have been chosen the Calculate Contacts feature will analyse the information and inform you of the total number of contacts that will be sent to. This information can be exported to CSV if you desire.   

Filtering

This optional step allows you to apply filtering to your campaign based on the statistics of a previous send by filtering contacts on what they’ve been sent and how they’ve interacted. A variety of filters are available to help achieve this. The Calculate Contacts feature is available on this page as well, for contact analysis. 

Summary 

The Summary page shows a brief recap of the information from the previous pages. In addition, Google Analytics and CANDDi information are shown here for each split, if either service has been turned on in Settings.   

Expanding any of the splits shows a breakdown as follows:

Test 

The Test page allows you to send a test campaign to one or more test contacts of your choice as a final stage to ensure that your campaign appears as required. 

Reports

Once the initial splits have been dispatched, Split Test campaigns have an additional step. Access the Reports section and click View full report for your dispatched Split Test campaign.   

A summary of the results will be shown, click Select winner for the chosen split and the winning split will be sent to the remaining contacts for the campaign.

 Once the winning split has been chosen the report will be amended to include it. 

N.B: For a split to be statistically viable it should include 1000 to be representative. However, if you have smaller lists there is still value in split testing, but we would recommend setting a hypothesis and testing it over a number of campaigns before drawing meaningful conclusions from your results.