Alterian Multi-Armed Bandit Processor
- Ben Clark
Overview
This plugin uses machine learning algorithms to dynamically allocate traffic to variants that are performing well, while allocating less traffic to variants that are under performing, whislt still allowing them opportunities to improve.
When started, the Multi-Armed Bandit will process rows going through the Rule. It will assign Variants, the items to be tested and count Variant Attempts and Successes the measures of success.
Until a statistically significant "winner" is calculated the Multi-Armed Bandit will continue with an even split to all active variants.
Once a Statistically Significant winner is calculated the Multi-Armed Bandit will start to send to the winner or winners.
As the Multi-Armed Bandit continues to work it will (at a frequency you select) pick a non-winning variant to test. This allows you to continually test the incumbent winners against previous losing options, and any new variants, ensuring the winner you have at any given time is tested to ensure it's still the best one.
This allows the Multi-Armed bandit to automatically adjust to changes in popularity ensuring you are not left with a previously winning but now sub-optimal Variant
If a Rule containing a Multi-Armed Bandit is restarted it will recalculate the results based on any previously collected Variants and select a winner if it can.
For example a website has a homepage banner and has to make a decision about which banner to display to a visitor to encourage a booking:
10% of the time, you split your traffic equally between the four versions of a website banner
For the other 90% of the time, you send traffic to the best performing banner.
Plugin Configuration
Within the plugin:
The following settings can be configured:
Setting | Description |
|---|---|
MAB Mode | Select Variant Allows the user to enter the Variants they would like to Test in the Multi-Armed Bandits Log an Attempt Allows the user to create a Rule to count the attempts where specific Variants were actually used. Log Success Allows the user to create a Rule to count the success for each Variant. The measurement of success may take place in a different rule to the Select Variant and Log and Attempt modes.
|
Keyspace | Tables exist in Keyspace in DDE (similar to a database). If the rule is running in DDE then this will automatically be populated with the client name and Target within DDE and the value added here (Localhost for testing) will be ignored. This can be set via a Parameter if required. |
Table | The name of the table which will be created in Cassandra. If left blank this will be the same as the keyspace field name. This can be set via a Parameter if required. |
Key (Static) | This Key is how the Multi-Armed Bandit will identify the Variants as belonging to a specific Multi-Armed Bandit and how it links the variants to other rules where Attempts and Successes for this specific MAB are performed. The Key can be a Static value set here in the Plugin. |
Key (Field/Param) | If the Key (Static) value is not populated you are also able to select either a Field from the Rule flow or a Parameter to populate the key. Using a Parameter will allow the field value to be seen and altered in the Jounrey Orchestration user interface. When creating or adapting Rules to measures Attempts or Successes for a Multi-Armed Bandit Rule ensure the Table and the Key selected is the same as the Variant Rule to ensure you are recording the correct values in the correct location. |
Experimentation Frequency | The experimentation frequencies have the following definition:
|
Field |  Field that will contain the Variants generated by the Multi-Armed Bandit. It will also be the field that should be used to count the Attempts and Successes in those modes. |
Statistical Significance | The user can choose a Statistical Significance level for the Multi-Armed Bandit. The Multi-Armed bandit will continue to send an even split to ALL variants until Statistical Significance has been reached. At this point it will decide on the “winning” variants. Statistical Significance is based on a Chi-Squared Test and its P-Value. The P-Value is is between 0 and 0.1 For ease of customer use we will accept 0-100 From a users perspective:
This can be set via a Parameter if required. |
Reporting Fields | The Alterian Multi-Armed Bandit has been designed to work automatically, calculating the “Winning” option automatically and passing in new variants to test the winning option against. Users may want more information on the decisions the Multi-Armed Bandit is making. When the Reporting Fields checkbox has been selected the fields below are automatically created by the Multi-Armed Bandit and can be output for review if required. {"MAB_StatisticalSignificance":"1.0","MAB_IsSignificant":"false","MAB_DidExperiment":"false",,"MAB_OriginalWinner":"","MAB_Options":"{"options":[{"successes":1,"name":"Variant 3","attempts":1},{"successes":1,"name":"Variant 2","attempts":1},{"successes":1,"name":"Variant 1","attempts":1}]}"}
|
Variants |  A Variant can be a static value, added to column one, or you can add a Variant from a field or from a Parameter. Here we have a static value for Variant 1 and we are using the “MAB String” field to provide the Variant for Variant 3. We are also using the “VariantfromParameter” parameter to supply the Varaint for Variant 2. Remember Parameters are available within the Journey Orchestration user interface and in this way users can easily adjust the variants used in a Multi-Armed Bandit. The Variant (Static) column is where the Variants are set when Fields or Parameters are NOT being used. In this scenario where Parameters and Fields are being used this column is only used when the Field of Parameter is not provided. Here Variant 1. In essence it becomes a default if the value is not provided in the Rule. |
Start and End Dates |  You can add Variant Start and End dates or both. The Variant will only be included in the Multi-Armed Bandit calculations if the time when the Rule is run is within this “Active” time period. If the Rule is invoked outside of this time period the Variant will not be included as a winner or a loser. Variants that have an End date but no Start date will start immediately when the Rule containing it starts and end at the End Date. Variants that have an Start date but no End date are will start at the Start Date and simply continue until changed or removed in the future. It is possible that a “Winning” Variant that has reached Statistical Significance within a restricted time period may fall out of selection due to leaving the active time period. The remaining Variants in the Multi-Armed bandit may not met the required level of Statistical Significance and therefore an general even split to all Variants recommences. |
Example Scenario
|
|
|---|---|
 |  |
| 
|