A/B Split Campaigns
AB Split campaigns allow you to set two to five test segments for a campaign. You choose the list, and Engage by Sailthru will split the recipients at random, using a proportion you choose. For example, you can try out a new design or subject line on 20% of your audience and see how it performs.
Campaign reporting will provide the results of the test and help inform your planning of future campaigns.
AB Splits are typically used for longitudinal testing of an element such as personalization in email vs. a static design. Changes are tested over a longer period of time, from a few days up to weeks or months. (For a comparison of AB test campaign types, see Campaigns Overview.) Note that it is a best practice to minimize the number of elements you vary in a test, so that you can determine which one is responsible for your success.
Getting Started
To create a new campaign:
- Go to Messaging > Campaigns.
- Click Create New from the Campaigns page.
- Enter a Campaign Name. You can also rename the campaign later by clicking its name at the top of the campaign builder.
- Click AB Test, then select the AB Split campaign type
NOTE: For more information on creating a new campaign, see the Campaign Overview page.
For AB Test campaigns, you'll have the option to select the type of test for your campaign.
Once you have chosen the AB Split, the menu on the left in My Sailthru will guide you through next steps.
Each step is described on this page:
Using the menu in My Sailthru, you can move from section to section and add information in any order.
NOTE: When creating an AB Split campaign, you will see the names of each segment displayed in the left side menu. When you add a segment, it will appear with the segment letter and name of campaign.
Click the campaign name next to the paper airplane icon to change the name of each campaign segment.
Audience
You will choose the audience for your email by setting Your Target Audience and, optionally, Your Suppressed Audience. Engage by Sailthru allows you to have multiple suppression lists if needed to exclude subsets of Your Target Audience. Please reach out to your Customer Success Manager to enable this feature.
To read more about how to create, manage, grow, and track lists, see the Lists Creation section of the Audience Overview documentation pages.
Selecting Your Lists
Note: If you wish to use a control group in your testing, see Create a Control Group for A/B Testing in the Using Audience Builder documentation.On the Audience page, set who will receive your email via Your Target Audience drop down.
Once you choose Your Target Audience, you will have the option to exclude recipients via the Your Suppressed Audience drop down.
The number of valid emails refers to the number of valid emails on a list, minus hard bounces or optouts. See the User Levels Engagement page for more information. If you're using a Natural List, Created will show the date the Natural List was created.
Note: You must use a list that includes more than 10 recipients for an A/B test.
Configure
In this section of My Sailthru, you'll enter the foundational information for your email, including the Template; basic information such as the From Name, Email Subject and Reply-To Email; and choose the Data Feed and/or Ad Plan, if you're using one. This is also where you will set up your AB segments and add tests for any of the basic information.
Build From a Template
On the Configure page, choose your Template if you have not already done so.
If you have selected a template with unsaved changes, you will be prompted to review and save them. Unsaved changes will not be sent.
Any changes made to your template after you schedule your send will not be included. You must unschedule the send, make changes, and then reschedule the send to include updates or changes in your template.
Add Segments
Under Segment, you can optionally add additional test segments or change the proportional size of your segments.
- To add a segment, click the gray plus sign.
- You can have up to five segments with an AB Split Test.
- To modify the audience percentage for segments, click and drag the splitter bar between them, or click a percentage value and type a new number in its place.
- The minimum percentage is 1%.
Add the Basics>
Tests may be added to any of the Basics, including:
- The From Name and From Email that will display for recipients.
- Email Subject for the email.
- Reply-To Email to which recipients can respond.
Once you add a test, you'll see additional fields for that item which you can edit. For example, in the image below, two different Email Subjects have been entered.
NOTE: Clicking the envelope and plus symbol icon allows you to add a verified email that you have already added to My Sailthru for verification.
Add a Data Feed, Promotion, or Ad Plan (optional)
You also have the choice here to include a Data Feed in your email, with or without a test. Data Feeds are used to supply content to not only emails, but website and other online content. Engage by Sailthru utilizes the following feed types, which should contain your product or content URLs:
- Content Feeds are supplied by Engage by Sailthru's Personalization Engine product, which gathers users' interests based on website activity.
- You provide URL Data Feeds, which may be generated by your content management system. It is recommended that you add these to your feeds page in My Sailthru so that Engage by Sailthru can cache them and make them easy to access within your Template or campaign. JSON format is recommended.
- Merged Feeds contain multiple Content and/or URL Data Feeds, which Engage by Sailthru merges for you into a single feed.
See the Data Feeds Overview documentation page for more information, and note that your template must be coded to support the display of data from a feed (see Code a Template Using a Data Feed).
To insert a unique, single-use promotion code in each message, configure a Promotion, then select the name of this promotion from the drop-down. Promotion codes will be included in your A/B test sends, however, differentiating Promotions across A/B test variants is not supported.
See the Ad Targeter Setup documentation page for more information on the Ad Plan feature.
Design
The Design page is where you will modify or update content that appears from the Template you have chosen for your campaign. You can create a campaign from scratch, but it is a best practice to start with a Template. When using a Template, you can still manually edit your code for each campaign send, as needed.
Design Your Email Content
On the Design page, you will see Engage by Sailthru's side-by-side code editor. You can modify the HTML from your chosen Template as needed and use the preview window to view and test your content. Click Test Send to quickly send a test email. You will then be prompted to enter the test email address.
NOTE: When using the Test Send button, the optout link will not render in the test email.
Code from your Template will be shown in the HTML viewer and as you make changes to the code on the left-hand side, you'll see updates on the right. The Design page allows you to preview your email before making it live.
For AB campaigns, you also have the option to add a test for content creative and copy.
If you wish to change your template while working on your AB test campaign, click the Refresh from Template button.
The Confirm dialog box opens.
Select a new template and click Ok. Your selected template loads in the active window.
Note: Refresh from Template will pull in changes made to the existing template, as well. Make changes in the Template Editor or Email Composer, then select the template from the list when you wish to refresh. Changes that affect the underlying code or Zephyr used in the template will show. Changes to link tracking using the toggles in Email Composer or the Template Editor will not pull through.
On the HTML viewer, click Text and then to convert the preview to the text version of your email.
Also, above the HTML viewer are your Advanced options:
- The Zephyr Setup section will run once for every recipient of the email send, enabling dynamic headlines and content.
- Link Setup allows you to indicate whether you'll be tracking links and/or using Google Analytics to measure what happens on your site after a user opens and clicks within your email.
Using Zephyr
For AB Tests, you can use HTML to create static data, and Zephyr for dynamic information. Zephyr is Engage by Sailthru's coding language used in email subject lines and campaign Templates to increase customization. For example, Zephyr is used to incorporate data feeds to populate the Template with your latest content and to call variables associated with user's profiles, such as their geolocation to ensure relevancy for each recipient of the email.
See the For Developers documentations pages for more information on using Zephyr for Engage by Sailthru email campaigns. You can also click Zephyr Guide from the Design page in My Sailthru to access the Zephyr Syntax Overview documentation.
Viewer Options and Image Library
You can adjust your viewing options by clicking the cog icon at the bottom of the HTML viewer. This is also where the image library is located. Images you upload are available to you for all emails you design. When deleted from an email, they remain in the library for use in other emails.
You can easily bulk upload images by dragging and dropping from desktop or folder into the Image Library.
Add a Different Visual Template
To test two different visual templates, follow these steps:- Create a new A/B Split test or open an existing draft test. Follow the steps to create your test, add an audience, and configure the test.
- In the Design tab, select Add Test next to the segment tabs.
- This creates a separate tab for segment B. Select segment B.
- Select Refresh from Template. When the Confirm box opens, select a new template and then OK.
- Your new template will load.
- Continue the steps to schedule and confirm your A/B test.
Schedule
When and how your email will send is critical in getting a response from your audience. There are multiple options that let you control the send date and time to reach your audience when your content is most relevant and valuable, including the Personalized Send Time.
Scheduling Your Campaign
On the Schedule page, set up all options around how and when your email will be sent. If you choose to send On Confirmation your email will send once you click Schedule on the confirmation page. You can also set a Specific Time, which allows you to schedule an exact time and date in the future.
Choose Personalized Send Time to optimize the open rate by sending emails to individuals when they are most likely to open based on their email open history. For example, if you choose to send your email starting at 9:00AM with a 16-hour send window, the email will be sent to each user at the time between 9:00 AM and 11:00 PM when they are most likely to open it within that window. This is based on user open history in their Engage by Sailthru profile.
Note: Tests that use PST show in the Campaign List as scheduled for a specific time.
For AB Splits, you can add as many tests as you have segments to optimize performance of email based on time sent, schedule time or Personalized Send Time windows.
Confirmation and Seed Emails
Enter the Confirmation Email for the individual(s) who should receive an email when the campaign completes sending, including a confirmation of completion and total emails sent. To enter multiple confirmation email addresses, separate using a comma.
Enter the Seed Emails to which the email should be sent. Note that this will send to the email address regardless of opt-in status.
Confirmation
The Confirmation section provides you with an opportunity to review a summary of the settings you've chosen for your campaign.
Confirmation Details>
For AB Splits, the confirmation page will provide information for each segment. The tests are color-coded for each segment and the percentage for each segment is noted in the title.
TIP: Be sure to review all of your campaign details presented on this page prior to clicking Schedule. Try reading this page from the bottom up, or printing to review on paper. Changing the order or method in which you've previously reviewed information can help you to find errors or items that need updating.
If you have scheduled your campaign to send On Confirmation, a pop up will appear to confirm and give you the option to schedule for a later date.
You can always return to edit any section of your campaign prior to the send time.
NOTE: When creating an AB Test, note that each segment -- including final segments for AB Winner or AB Auto Winner campaigns -- will be viewable as separate campaigns on your Campaign page.
