Create Subject Line A/B Journeys

Note: Subject Line A/B Journeys can be created for email messages only.

You can select one of four criteria for choosing which is the best or winning version of a subject line, based on the email:

  • Number of views
  • Number of clicks
  • Revenue generated from these emails. Shop tracking results are used for this.
  • Number of orders generated from these emails. Shop tracking results are used for this.

The best version is evaluated at the specified time interval after the launch, expressed in minutes, hours, days or months. For example, the results may be evaluated one hour after the messages with the different subject line versions are sent.

Note: The winning version is selected based on unique contact actions (= criteria mentioned above).

Example:
When a contact views a specific email twice it will only count as one view for the AB winner selection using the condition number of views. This is because it only counts the views for a single unique contact.

Subject Line A/B Journeys can be launched immediately or scheduled at a given date and time.

 

Create the Subject Line A/B Journey

Setting the journey properties

Launching the journey

Pausing the journey

Canceling the journey

Taking the journey offline

Reactivating the journey

Best practices

 

Create the Subject Line A/B Journey

To create a Subject Line A/B Journey:

1. Go to the Journey entry. From the toolbar at the top select New and then A/B Test from the wizard.

2. Next, select the Subject Line A/B type.

3. Set the following properties. The journey properties are grouped by type of information. By default only the Basic fields are displayed. Switch to the All fields tab to display all other fields. On the left hand side of the properties, on-screen help is provided. This helps the user to fill out the form with accurate and comprehensive information.

Basic Fields

General

  • Name — Give a clear name to the journey.

  • Folder path — Set the Folder Path. This is the folder in which the journey is created. By default it takes the folder that is currently selected on the Start page when the journey is created. You can modify the path if required. Click the folder field to access the dialog and select a different folder.

Subject Line Test

  • Select Email — Click this field to display the Asset selector from which you can select an email message.

You can filter the emails and folders by entering some characters of the email name in the Search field, and by enabling the Starred items toggle to only show the assets marked with a star.

Note: When choosing an email message that uses assets in a folder with no access, a warning explains that this content can't be selected.

Technical Note:
The selected message can't contain a journey-defined link. This type of link allows you to define the next step within a journey. However, this is not possible after journey creation.

  • After selecting the email message, define the subject line options (at least two — option A and option B).
    By default, each language field in Option A is pre-filled with the default subject line as set in the message.
    You can expand Option A and update the subject line in each language field.

Note: The order of the languages is based on the order of the languages as they exist in the message properties.
For example French as the first language, Norwegian as the second language, and English as the third one.


The same order is used in the options when setting up the Subject Line A/B Journey.
Also, next to each option, the subject line of the first language is shown (to make it easier to identify them when options are collapsed).

While editing a value in a subject line input field, a tooltip is shown with three icons.

Clicking these icons, offers the following options:

  1. Emoji selector — At the top of the emoji selector, you can navigate through categories to see the available emojis in the selected category.
    The selectable emojis are visible in the middle section. Clicking an emoji inserts it in the input field at the location of the mouse cursor (or where it was last active).
    At the bottom, the most recently used emojis are shown, as a quick way to re-use them.

    You can search for specific emojis.


    You can select the skin tone of an emoji.


    Note: The skin tone only applies to the People and body category.

  2. Personalization fields — You can select audience, system, and dictionary fields from the left. At the right, certain formatting can be applied to the selected field. Click OK to insert the formatted personalization field into the subject line.

  3. Marigold AI offers ways to optimize your subject lines, with the help of AI.
    You can ask the AI to make your subject line shorter, re-write it, or generate 5 variations based on the current subject line.

    Example:

Additionally, a message analytics tag can be defined for each option. This allows individual tracking on this subject line in the journey reporting.

Note: You can still change the subject lines and tags in the journey properties, after creation.

  • Expand Option B and repeat the above process.

  • If you want to test more than two subject lines, click the + Add another variant link to add an additional one.

Note: You can add up to 8 subject line options.
To remove a subject line option, hover over the option header and click the bin icon in the upper right corner.

All Fields

Besides all fields from the Basic Fields tab, the All Fields tab additionally contains:

General

  • API name — The API-Name for the Subject Line A/B Journey. This name is used when the journey is executed via the API. This is a required field that gets automatically populated while entering a name above. You can modify it here if desired.
  • Description — A description is optional. It can help you and others in understanding what the purpose of the Subject Line A/B Journey is.
  • Labels — Asset labels are optional. Select one or more labels from the drop-down.
    These labels are configured in the Admin configuration section.
    Users with the proper Access permissions can also create new labels here by typing the new label value in the field and hitting the enter key.

Analytics tag

Last, set the Campaign analytics tag. The value entered for this analytics tag is passed on to a tracker (when a tracker is activated).

4. When done adding the subject lines options, press Next.

5. On the A/B Split screen, enter the settings that define the test conditions:

Who will receive a test version? — Select the percentage of the audience that will receive a test message by moving the slider left or right (e.g. 30%). The proportional breakdown of test contacts is displayed (e.g. 5604/18678 contacts). A pie chart is provided indicating the percentage of contacts getting a specific version. The winning version will be sent to the remaining number of contacts.

When will the test be evaluated? — Select the time interval after which the test should be evaluated. This can be expressed in minutes, hours, days or months.

Based on — Select how the best result will be defined:

  • Views
  • Clicks
  • Revenue. To use the revenue, the Shop Tracker should be activated.
  • Number of orders. To use the orders, the Shop Tracker should be activated.

6. When done, click Create Journey. The Subject Line A/B Journey is created and the journey properties panel is displayed:

In the navigator on the left, Basic and Advanced entries are available.

 

Setting the journey properties

Basic

The Basic entry is used to set the general properties for the Subject Line A/B Journey.

1. On this panel, the journey can be Scheduled for launch now or at a given date and time in the future. Use the buttons to indicate your choice.

Note: When a user defines a scheduled launch time on a journey, the time is expressed in their time zone. The journey will be launched in the time zone of this user at that time.
When a user with a different time zone views the journey, the scheduled time will be translated to their time zone, to reflect the local time.
Internally, the system uses a universal time zone, to make it possible to translate times to localized versions for every user. This means that updates by one user will be correctly represented in the local time zones of all other users.

Example:
User A has a time zone that is set to UTC -5 (Eastern Standard Time).
They schedule a journey to be launched at 6:00 local time (UTC -5).

User B has a user time zone that is set to UTC +1 (Romance Standard Time).
They see a scheduled launch time of 12:00 for that same journey, as their local time has a 6 hour difference compared to user A.

2. The journey audience is displayed based on the selected content. When the content uses an Engage audience, you can immediately select a segment of that audience as well.

Note: If the selected list is configured to require segment selection, you MUST select a segment for this list. The journey won't be executed as long as no segment has been selected.

The number of contacts in the audience is returned.

The icon provides additional details on what the number is based on:

  • Unsubscribes excluded
  • Segment selected or not selected
  • Deduplication enabled or not enabled

3. The audience can also be filtered.

This filter can use the following data:

  • All parent list fields
  • All data in linked lists, including 1:1 and 1:n linked lists
  • Email metrics
  • Historical data captured from the contact's channel interaction in a Custom, Recurring Batch, Single Batch and AB Journey. The user is presented a list of all Custom Journeys and a list of channels for each journey. Any interaction can be used, supporting fixed dates as well as relative dates.
  • Segments of the selected audience by including or excluding records in the selected segment

4. Campaign Analytics Tags can be set when a tracker is activated. This is optional. (For more information on this, check this topic.)

 

Advanced properties

In the Advanced entry, additional properties can be set:

Audience pre-calculation

Audience pre-calculation — For some customers, it is beneficial to pre-calculate the audience for an action in a journey well before the actual go-live, or when there is a lower load on the environment. To save time, the actual journey execution will use the pre-calculated audience when it’s available. The option is by default disabled.

When? — Only displayed when the audience pre-calculation is activated. You can choose from the following options:

  • Time before execution — The audience is calculated at a specific time interval (expressed in minutes) before the journey is executed. The journey must be scheduled in this case.
  • At a fixed time — The audience is calculated at a specific day and time.
  • When a task finishes — The audience is calculated when the selected task is finished. You can select a task from a drop-down list of tasks.
  • When a task succeeds — The audience is calculated when the selected task succeeds. You can select a task from a drop-down list of tasks.

Deduplication

Deduplication — If there are duplicate contacts in the journey's audience, you can use Deduplication to select a unique contact based on a field or a combination of fields. This avoids sending the email more than once to the same contact because the contact is in the Audience List more than once.

For each journey you can define if deduplication should be activated.

  • Unique key — You can define the field in the audience that should be used to perform deduplication. This can be a combination of fields as well, for example the name and email for a contact can be used to detect duplicates. Mail is by default selected.

Note: A maximum of 5 fields can be selected. Fields of type longtext, boolean, float and datetime are not allowed.

  • Sorting field — Here you define what field in the audience should be used to define the record that has priority over the others in case duplicate records are found. For example, you can use the last update date as a prioritization rule.
  • Sorting direction — Choose from ascending or descending to define the record that should be kept.

Journey tracking

The following option is available:

External analytics tracking

By default, tracking is enabled. Toggle the option off to disable the tracking.
External analytics tracking covers all the external trackers, like for example Google UTM, Site tracker, Shop tracker.

When this type of tracking is disabled, no parameters from external trackers will be added to domain related links and hence the information can not be used by the external tracker. This will have an impact on:

  • External analytics — E.g. Google analytics
  • Site CRM identification — The m_bt parameter will not be added for domain links and hence CRM identification will not be done.
  • Shop tracker — The parameters that are added for the shop tracker will not be added and hence this will have an impact on revenue reporting.

Note:
In some cases you don't need the journey to be tracked.
For example, a 'Password reset' journey does not require tracking.

Journey execution priority

When multiple journeys are scheduled for execution at the same time, it is possible to ensure that one journey will take priority over another. Some journeys might be more time sensitive or important than others, and this option allows you to prioritize journeys. By default the priority of the journey is set to medium, but priorities can range from low, over medium and high, to very high.

Example:
Journey A with high prio is scheduled at 09h00.
Journey B with medium priority is also scheduled at 09h00.
Journey C with high priority is scheduled at 09h01.

Journey A and B are scheduled to launch at the same time.
Journey A is executed first, as it has a higher priority than Journey B.
When the execution of Journey A is finished, Journey B and C are in the queue.
Journey C has a higher priority than Journey B and will therefore be executed before Journey B.

When done, you can Save the journey and Activate (launch) it.

 

Launching the journey

When launching the journey, you are informed about the content that will be sent, the audience and filter applied and the number of contacts that will be targeted. A dialog is displayed with this information. The counts displayed in the dialog reflect the deduplication settings defined for the journey.

Note: This dialog is only displayed to users who have the audience validation permission set. This permission is active for every user by default.

The name shown on each line represents the subject line set in the first language per option.
The first language is based on the order of the languages as they exist in the message properties.

For example French as the first language, Norwegian as the second language, and English as the third one.

This results in the French subject lines to be shown in the Audience Count dialog.

You need to confirm that the audience count is as expected and then launch the journey. If the count is not yet available, you can only cancel the journey launch.

Journeys can be scheduled as follows:

  • Launch now — The journey is launched immediately.
  • Launch later — The journey will be activated to be launched on a future date.

While journeys are not yet being executed, or for which the execution is ongoing, Cancel, Take offline, and Pause buttons are available.

  • Take offline and Pause can be used to halt the execution and make the necessary modifications to the journey, before resuming them.
  • Cancel can be used to completely stop and abandon the journey execution.

Note: When the journey is in one of the following states, all fields are grayed out/disabled.
- Executing (=currently being executed)
- Executed (=execution is finished)
- Scheduled (= planned to be executed on a later date)

 

Pausing the journey

When pausing the journey, links in emails will remain available. Only the sending of messages is paused. While a journey is paused, changes can be made, saved, validated and published. The journey can then be resumed, and newly processed messages will contain the changed content.

The toolbar contains an additional Pause button that becomes available after the journey is launched.

A paused journey can be activated again or taken offline completely.

 

Canceling the journey

It is possible to cancel a journey after it has been launched. As long the journey processing is not finished, you can cancel the journey.

The toolbar contains an additional Cancel button that becomes available after the journey is launched.

Canceling the journey will stop sending out new emails.
If currently emails are being processed, these are canceled as well to ensure these will not be delivered to the inbox of the recipients.
All links in already sent out emails will continue to work.
The status of the journey will be available from the status bar at the top:

Note: When a Journey is canceled the user is presented different alerts, depending on the execution status of the journey.
- Some messages have been processed already: A progress bar gives a very clear and visual representation of what has been processed. You can choose to cancel or keep the journey running.

- No processing data is available yet: You can choose to cancel the journey anyway or keep the journey running.

- All messages have been processed: In this case, canceling is no longer possible, and the user is informed.

Note: Canceled journeys cannot be relaunched.

 

Taking the journey offline

When taking a journey offline, no (more) messages are sent and links in emails will no longer be available.

The toolbar contains an additional Take offline button that becomes available when the journey has been activated.

An offline journey can be canceled or activated again.

 

Reactivating the journey

While a journey is paused or taken offline, it's possible to edit the content.
If you don't publish the content after you made changes to it, returning to the journey and trying to reactivate it, shows a pop-up explaining that there are unpublished content changes.

- Click OK to launch the journey with the last published version of the content (without the most recent non-published changes to the content).
- Click CANCEL to go back to the journey wizard and publish, if needed, your last content changes. That way, you can choose to reactivate the journey once more and launch it with the updated published version of the content.

 

Best practices

When setting up the Subject Line A/B Journey it is important to know what to test exactly for more accurate test results.

  • For your Subject Line A/B test to be representative, you need to test the outcome on a large number of contacts. This will provide a better representation of the behavior of the average contact in your Audience List and extreme responses will be filtered out.

  • When creating the test groups for your Subject Line A/B test, it's recommended to create a random group of contacts. By doing so, your test groups will be more similar in composition and the test results are more likely to be due to differences in the subject lines rather than to differences in the test group.

  • Before starting make sure you know what to test. When the differences between the versions in your Subject Line A/B test are rather small, it allows you to isolate the elements that have the biggest impact on the results. Or, when you go for more dramatic differences in the different versions, it allows you to discover the outliers and compare more subject lines.

  • Let the test run long enough to cover a complete business cycle. An average of 7 days is a good compromise. Do not stop a test mid-cycle. Often marketers are tempted to stop the test early when the results are in line with their expectations.