Using Control Groups
- Ben Clark
Control Groups in Alterian CX are designed to work at a Rule level. Someone can be in the Control Group for one Rule but not others. This should mean that people interacting with Alterian CX are very likely to experience some optimised content. We should not have a scenario when an individual is in the control group for all Rules which would provide a dull and unhelpful relationship with your brand.
Control Groups provide a straightforward and accurate way to assess success for your Business Goals, Initiatives, and Rules. The logic is simple. Are people who are exposed to the optimised Rules more likely to do the thing you wanted than those who were in the Control Group and had a standard experience.
If those outside the Control Groups are statistically more likely to convert, we can establish that the Rule is working and importantly, how well it is working.
This allows us to examine the success of the Rule before and after changes have been made. Have the changes improved the Rule or made it worse, which CX decision generated the most success? These allow manual optimisation of the Rules and automated optimisations to occur.
Assigning Control Groups
Control Groups are enabled for an Initiative and a Control Group percentage decided upon within the Initiatives screen within the Opportunity Matrix. These values are written as parameters into the Get CXID Template for EVERY Influencer Rule that has been linked to this Initiative.
When a Rule is assigned to a Initiative with an enabled Control Group it will be updated with the Control Group information. If a Rule is removed from a Initiaitve that had a Control Group these settings will be removed.
The assigning of Rules to an Initiative in the Opportunity Matrix is what drives the application of Control Groups.
Test Iterations
Where significant changes are made to a Rule, the previous results gathered for that Rule are no longer relevant when measuring current success. The Get CXID Public Template in each Rule allows the setting of a “Test Iteration”. Test Iterations are optional, they restart the Control Group for the rule. Anyone who was in the Control Group for the previous Test Iteration is removed from the Control Group. Everyone who is exposed to the Rule moving forwards is retested to see if they should now fall into the Control Group for the new Test Iteration.
Because of this some of the people in the Control Group may have seen previously optimised content for the Rule. This should be considered when considering future conversions for these people.
Statistical significance is important when making accurate decisions and where volumes are low changing the Test Iteration frequently may mean you do not achieve statistical significance. Under these scenarios it may be better to leave the Test Iteration as the default and assess success using time as the differentiator.
Control Group Lifespan
By default, individuals remain within a Control Group for 180 days. After this they are automatically removed from the Control Group. This Control Group lifespan can be changed on a Rule by Rule basis by changing the Control Group Lifespan Parameter within the Get CXID Tile present within each Rule. This parameter is based on days to remain in the Control Group and will apply to those added to the Control Group after the change is made.
Default Outputs
The key to a Control Group is that the members do not see or experience optimised content. They should only ever see default content and therefore have a standard experience that can be compared to the Optimised experience other individuals had.
In most case Rules start with a “Realtime Input Step” (RIS). The Realtime Input Step is built to require certain fields. These are the fields that the Step designer knows are going to be needed for the Rule. They either identify the individual exposed to the Rule or provide information to allow the Rule to make decisions on the offers or content to provide back.
It’s in the RIS that we configure the default content that becomes the response back for those in the Control Groups.
The Realtime Input Step has a Default Output. This can be a HTTP code and a static value. The Static value can be set in the Template or it can be set via a Parameter which enables it to be changed in Rule Designer.
Example
Static Value
Parameter
The Carousel Selection Anti-Churn Rule forms part of a Rule Group that will show various offers on the home page.
We do not want those in the “Anti-Churn” control group to see the optimised images this Rule will create.
The creator of the RIS adds a ControlGroupDefault parameter that allows the Default value to be changed in Rule Designer. They may also create a Global Parameter if the default value is standard across all Rules.
In this case the default value for the parameter is “Default”. This is what will be passed through by the Journey Orchestation Engine through the Realtime Output Step when the person interacting is in the Anti-churn control group.
The calling solution will be configured to always have a default offer or image to show when this value is provided.
In this manner those individuals in the control group will get content but not the optimised content the other individuals will recieve.
Other Steps
Alterian Email Manager
The Email Manager plugin has been adapted to honour the control group flag. If the person is in a control group and the flow of the Rule takes them to an Alterian Email Manager step they will in skip the step and they will not recieve the email.
If required you can prepare a vanilla default email and “switch” using the Is Control Switch Public Template.