Configure a Holdout Group

Overview

A Holdout Group refers to a sub-set of Members who are "held back" from a marketing activity (such as receiving an email campaign), while the remainder of Members participate in the activity. The behavior of the two groups can then be compared to evaluate how their performance varies based on how the marketing activity did, or did not, impact them. The benefit of having a Holdout Group is to enable marketers to measure lift in purchase behavior, by comparing performance in the target audience versus the performance in the Holdout Group during the offer period. 

Marketers can specify that a Segment utilize a Holdout Group, and configure the size of the Holdout Group (either as a percentage, or as a static count). The system will randomly assign Members in the Segment to the Holdout Group.

Note the following guidelines regarding the use of Holdout Groups:

  • Holdout Groups can be enabled within the following Segment types:

    • EDP Execution Type

    • Golden Record Execution Type, if both the Dynamic and Triggered features are disabled.

  • Segments with Holdout Groups can be used in Cheetah Digital marketing campaigns. Communication and promotion access can be suppressed for the Members in the Holdout Group.

  • When Holdout Groups are enabled for a Segment, the Segment export file includes an additional column named is_holdout. The value in this field is TRUE for Members assigned to the Holdout Group.

  • When Holdout Groups are enabled for a Segment, you can optionally use the manual export feature to export only the Members in the Holdout Group (see Manually Export the Members in a Segment for more information).

Enable Holdout Groups

The option to enable a Holdout Group for a Segment is available within the Edit Segment pop-up window. See Edit Segment Properties for more information.

Configure Holdout Group Size

The platform randomly assigns Members in a Segment to the Holdout Group based on the configured size. You can configure the size as a percentage of the overall Segment, or as a static record count.

Before configuring the size of a Holdout Group, the following prerequisites must be completed:

  • Enable Holdout Groups for the Segment (see above).

  • Define the Segment Rules.

  • Publish the Segment (see Publish a Segment for more information).

  • Deploy Segment Metadata (see Deploy Segment Metadata for more information).

To configure the size of a Holdout Group:

  1. Select Members from the top navigation bar, then select Segments from the side navigation menu. The Segments screen is displayed.

  2. Search for and select the desired Segment (see Search for a Segment for more information on the available search options). The Segment Details screen is displayed. By default, the Rules tab is selected for EDP and Golden Record Segments, and the Results tab is selected for External Segments.

  3. Select the Results tab.

  4. Click Refresh Members. The platform executes the Segment logic to select Members. When the Segment execution is finished, the platform displays the list of Members selected by the Segment Rules.

  5. Click Configure Holdout and determine the size of the Holdout Group:

    • As a percentage: Select Percentage, then enter the percentage value.

    • As a fixed count: Select Count, the enter the count value.

  1. Click Save and Deploy. The platform saves the Segment and deploys Segment metadata. When finished, a success message is displayed.

  2. Click Refresh Members again. The platform again executes the Segment logic to select Members, including randomly assigning Members to the Holdout Group.

Export a Holdout Group

When the second Segment refresh process is complete, if you want to see the Members in the Holdout Group, you can manually export the Segment (see Manually Export the Members in a Segment for more information). The platform offers the following export options:

  • Segment: Download all Members in the Segment (including the Holdout Group) to your local PC. Members in the Holdout Group will have a value of TRUE in the is_holdout column.

  • Holdout Group: Download only the Members in the Holdout Group to your local PC.