Overview

A holdout group is a group of users who is “held back” from activity, such as an email message, for example, while the remainder of the group, or segment, will receive the message. The behavior of the two groups is then compared to evaluate how their performance varies based on how the marketing campaign did or did not impact them. 

Marketers can specify a portion of Loyalty segments, and the system will randomly assign members to be held out. Communication and promotion access are suppressed for these holdout groups. The benefit of having a holdout group is to enable Loyalty clients to measure lift in purchase behavior by comparing performance in the target audience versus performance in the holdout group during the offer period. 

Please note the following features on holdout groups:

• Holdout groups can be configured with Loyalty or Static Golden Record segments.
• Segments with holdout groups can be used in Engage+ campaigns.
• Holdout group functionality is available for refreshing and non-refreshing segments. 
• The segment file format changes when holdout groups are configured (is_holdout denotes a holdout group, and if not showing, is not included in the segment)
• The holdout file name will be the same as the segment file name but with  _holdout added

Configure Holdout Groups with No Scheduled Export

1. Navigate to Members > Segments in the marketing console.   
2. Click on an existing Segment entry, or create a new one.
3. Click on the Edit Segment pop-up at the top of the page.
4. Check the box for Holdout Group.
5. Click Save.

6. In the Rules tab, click on Conditions and select/verify all options desired for your holdout group:

7. In the Results tab:
   
   a.  Refresh the segment at least once by clicking on Refresh Members.

     

         b.  Choose whether to hold out the group by a percentage or by count. 

     

         c.  Click the Save and Deploy button, then the Generate Holdout button.

         d.  Once the holdout group is generated, the results will display:

• • The total population size
  • The total number of members in the segment
  • The number of member in the holdout group

Configure Holdout Groups with Scheduled Export to Engage+

To generate and configure the Holdout Group for Scheduled Export, there is an additional step that you must perform to schedule the export. First follow steps 1 - 6 in Configure Holdout Groups with No Scheduled Export. Then, do the following: 

In the Results tab:

1. Refresh the segment at least once by clicking on Refresh Members.

2. Choose whether to hold out the group by a percentage or by count. 

3. Under the Schedule tab:

• Set the toggle to Enabled.
• Choose Once for recurrence.
• Set the future date and time that you want the export to be executed on.
• Select the Time Zone for the selected date/time of export.
• Click the Save button.

4. Back in the Results tab, Click the Save and Deploy button, then the Generate Holdout button.

          Note: You will Generate Holdout group via the button while the scheduled export is enabled            in the future. 

Result: The Holdout Group will be generated on export.