A/B Split Testing Triggered Messages

Continuously improve the design and creative of your triggered emails by A/B testing different versions. What subject line is most inviting? What's the best place to place a link to get an invited user to join your site?

Our best practice for triggered message A/B Tests is to use Lifecycle Optimizer.  Using Lifecycle Optimizer for A/B testing will keep the marketer at the steering wheel for welcome series and similar send types. You can read on to learn about our legacy method of API-based A/B Split testing with triggered messaging.

The basic idea of A/B testing is simple. Create two slightly different versions of a triggered template and set up to send using the API. Reporting will determine which one gets a better response from your users, usually measured by click and open rates.

For more information on mass mail campaigns, see A/B Split Testing Campaigns.

 

Setting Up the A/B Test

1. Create a new template or edit an existing one in the Templates tab.From inside the Template Editor, click the A/B Test button and hit OK at the confirmation prompt.

2.You will confirm via a dialog box to create two versions of your template.

The names of each template will be appended with A and B. You'll be inside the template editor of version A. To view both, save and click out to the Templates tab in the main menu.

Using the Rename button to change the name of a template to end with [A] or [B] does not make it an a/b cell; the A/B Test button must be used.

3. Edit one version to be slightly different than the other.

You can give it a unique subject line as well as change the message content.

 

A/B Testing in Action

For a given triggered email sent via the API, the system will pick randomly between the different A/B versions. After a period of time, you will be able to compare the results and determine which is the most effective template.

Note: A user's placement in the A or B grouping will persist for all future triggered test sends.

 

Reporting on Testing

In the Reports tab, click on Transactional A/B Testing Report to see analytics on the different versions.

 

End Testing

Once you're satisfied that you know which is the better performing template and you want to stop testing, mark the template you no longer want to send as inactive from the Template Settings Editor. Whichever version is left will be sent from this point forward.

Note: If you mark both versions as inactive, neither will be sent. Do not delete the template as you will lose its historical performance.