Control Groups
- Ben Clark
For each Rule, Initiative and Business Goal it is important to measure the impact they have on your customers decisions. In some cases asking for feedback will provide answers, but this feedback can prove difficult to quantify. You need a measure that can be simply and consistently gathered and is readily comparable over time.
To provide this in Alterian CX the Opportunity Matrix allows the creation of Control Groups.
Control Groups are set at a Rule level but are turned on at an Initiative level in the Opportunity Matrix.
All the Individuals exposed to a rule (and test iteration) are tested to see if they should fall into the Control Group for that Rule.
These individuals are restricted from exposure to the bespoke content or decisions that are normally provided by that specific Rules.
They should get a relevant default decision in the majority of instances. This allows us to assess how effective the optimised decisions are at encouraging conversion.
In some cases, you may want to provide a better decision than a simple default or send a simple email. As such we allow you to turn off the forced default for Control Groups. This can be turned off in Tiles containing a Realtime Output or Email Manager steps.
This functionality would normally be used with the Is Control Public template allowing you to create a flow specifically for those in the control group.
The Conversion events and the Control Groups are both linked to an Initiative and allow us to understand whether Rules have a measurable impact on the decisions made by your customers.
Were those NOT in the Control Group more likely to convert? Can we tweak the Rules to provide better results?
As a standard measure, Conversions and Control Groups help us to improve the decisions we make in our rules and in the future will allow automated optimisations to occur.
We understand that Rules can change significantly over time and therefore results gathered before the changes were made are no longer relevant. The Get CXID Public Template allows the setting of a Test Iteration that effectively stops the previous control group for this Rule and starts allocating Control groups again. We also set the default Control Group Lifetime here. More information can be found here.
Turning on Control Groups
Open the Initiative settings screen by clicking on the Cog (Settings) Icon at the top right of the Initiative screen.
Control Group - Change to Enabled to turn on Control Groups, Disabled to remove control groups. We would suggest caution in this area. Turning off the control group will remove all individuals from the current control group losing any value you had in your current tests.
Size (Percent) - What percent of the total number of Individuals exposed to the Rule (and Test Iteration) should form the control group. Please provide a number between 0 and 100. Remember that this value sets a random round robin selection for the control group whenever a new Individual is exposed to a Rule in this Initiative.
For example, a value of 20 will randomly test any new Individual by selecting a random number out of 100. If the number comes up between 0-20 they are placed in the control group. 21 and above they do not fall into the Control Group and will never be tested again for the control group for this Rule and Test Iteration.
Both these values are written into the Rules so they understand whether to honour Control Groups and what percentage to use.
Success Direction
Your Conversion Rules will create events and we will use these, and the events created by Influencer Rules to measure "Success". What we also need to understand is what success means for this Initiative.
For "PDF Download", "Website sales", "Order value" an increase in Conversion rules and any linked values is a good thing. Growth is good for these Initiatives.
For "Complaints registered" or "Abandoned baskets" a decrease in Conversion rules is a good thing.
Growth is bad for these Initiatives., More complaints or abandoned baskets is not a good result.
Open the Initiative settings screen by clicking on the Cog (Settings) Icon at the top right of the Initiative screen and toggle the Combi box to your required value, either Growth is Good, or Growth is bad.
An example of the use of “Growth is Bad” as a success direction is where the the number of complaints is being measured with a focus on reducing the volume.