/
Alterian Multi-Armed Bandit Processor

Alterian Multi-Armed Bandit Processor

Overview

 

This plugin uses machine learning algorithms to dynamically allocate traffic to variants that are performing well, while allocating less traffic to variants that are under performing, whislt still allowing them opportunities to improve.

When started, the Multi-Armed Bandit will process rows going through the Rule. It will assign Variants, the items to be tested and count Variant Attempts and Successes the measures of success.

Until a statistically significant "winner" is calculated the Multi-Armed Bandit will continue with an even split to all active variants.

Once a Statistically Significant winner is calculated the Multi-Armed Bandit will start to send to the winner or winners.

As the Multi-Armed Bandit continues to work it will (at a frequency you select) pick a non-winning variant to test. This allows you to continually test the incumbent winners against previous losing options, and any new variants, ensuring the winner you have at any given time is tested to ensure it's still the best one.

This allows the Multi-Armed bandit to automatically adjust to changes in popularity ensuring you are not left with a previously winning but now sub-optimal Variant

If a Rule containing a Multi-Armed Bandit is restarted it will recalculate the results based on any previously collected Variants and select a winner if it can.

For example a website has a homepage banner and has to make a decision about which banner to display to a visitor to encourage a booking:

  • 10% of the time, you split your traffic equally between the four versions of a website banner

  • For the other 90% of the time, you send traffic to the best performing banner.

Plugin Configuration

Within the plugin:

 

The following settings can be configured:

Setting

Description

Setting

Description

MAB Mode

Select Variant

Allows the user to enter the Variants they would like to Test in the Multi-Armed Bandits

Log an Attempt

Allows the user to create a Rule to count the attempts where specific Variants were actually used.

Log Success

Allows the user to create a Rule to count the success for each Variant. The measurement of success may take place in a different rule to the Select Variant and Log and Attempt modes.

 

Keyspace

Tables exist in Keyspace in DDE (similar to a database). If the rule is running in DDE then this will automatically be populated with the client name and Target within DDE and the value added here (Localhost for testing) will be ignored. This can be set via a Parameter if required.

Table

The name of the table which will be created in Cassandra. If left blank this will be the same as the keyspace field name. This can be set via a Parameter if required.

Key (Static)

This Key is how the Multi-Armed Bandit will identify the Variants as belonging to a specific Multi-Armed Bandit and how it links the variants to other rules where Attempts and Successes for this specific MAB are performed. The Key can be a Static value set here in the Plugin.

Key (Field/Param)

If the Key (Static) value is not populated you are also able to select either a Field from the Rule flow or a Parameter to populate the key. Using a Parameter will allow the field value to be seen and altered in the Jounrey Orchestration user interface.

When creating or adapting Rules to measures Attempts or Successes for a Multi-Armed Bandit Rule ensure the Table and the Key selected is the same as the Variant Rule to ensure you are recording the correct values in the correct location.

Experimentation Frequency

The experimentation frequencies have the following definition:

  • Always - Always split the experimentation variant between all non-winning variants once statistical significance has been reached.

  • Very Often - Test non-winning variants around 25% of the time. The remaining 75% of the time the winning variant will be used. This is once statistical significance has been reached.

  • Normal - Test non-winning variants around 15% of the time. The remaining 85% of the time the winning variant will be used. This is once statistical significance has been reached.

  • Infrequent - Test non-winning variants around 5% of the time. The remaining 95% of the time the winning variant will be used. This is once statistical significance has been reached.

  • Never - Will always pick the winning variant once statistical significance has been reached. This removes one of the major advantages of using Multi-Armed Bandit functionality, it’s self testing mechanism and therefore is likely to only be used in testing or where the the onside factors that might affect responses will never change.

 

Field

Field that will contain the Variants generated by the Multi-Armed Bandit. It will also be the field that should be used to count the Attempts and Successes in those modes.

Statistical Significance

The user can choose a Statistical Significance level for the Multi-Armed Bandit. The Multi-Armed bandit will continue to send an even split to ALL variants until Statistical Significance has been reached. At this point it will decide on the “winning” variants. Statistical Significance is based on a Chi-Squared Test and its P-Value.

The P-Value is is between 0 and 0.1

For ease of customer use we will accept 0-100

From a users perspective:

  • 100 = 100% - this is the only likely scenario

  • 0 = % - this is the not the likely scenario

This can be set via a Parameter if required.

Reporting Fields

The Alterian Multi-Armed Bandit has been designed to work automatically, calculating the “Winning” option automatically and passing in new variants to test the winning option against.

Users may want more information on the decisions the Multi-Armed Bandit is making. When the Reporting Fields checkbox has been selected the fields below are automatically created by the Multi-Armed Bandit and can be output for review if required.

{"MAB_StatisticalSignificance":"1.0","MAB_IsSignificant":"false","MAB_DidExperiment":"false",,"MAB_OriginalWinner":"","MAB_Options":"{"options":[{"successes":1,"name":"Variant 3","attempts":1},{"successes":1,"name":"Variant 2","attempts":1},{"successes":1,"name":"Variant 1","attempts":1}]}"}


  • MAB_StatisticalSignificance = The Statistical Significance it calculated

  • MAB_IsSignificant = Was the Statistical Significance lower than the value selected in the rule?

  • MAB_DidExperiment = If the decision was Statistically Significance, did it experiment?

  • MAB_OriginalWinner = Once you hit Statistical Significance this is always set to the current winner, if did experiment, this will be different than the field output

  • MAB_Options = The options it considered when selecting a winner and the success and attempts of each

Variants

A Variant can be a static value, added to column one, or you can add a Variant from a field or from a Parameter. Here we have a static value for Variant 1 and we are using the “MAB String” field to provide the Variant for Variant 3. We are also using the “VariantfromParameter” parameter to supply the Varaint for Variant 2. Remember Parameters are available within the Journey Orchestration user interface and in this way users can easily adjust the variants used in a Multi-Armed Bandit.

The Variant (Static) column is where the Variants are set when Fields or Parameters are NOT being used. In this scenario where Parameters and Fields are being used this column is only used when the Field of Parameter is not provided. Here Variant 1. In essence it becomes a default if the value is not provided in the Rule.

Start and End Dates

You can add Variant Start and End dates or both. The Variant will only be included in the Multi-Armed Bandit calculations if the time when the Rule is run is within this “Active” time period. If the Rule is invoked outside of this time period the Variant will not be included as a winner or a loser.

Variants that have an End date but no Start date will start immediately when the Rule containing it starts and end at the End Date.

Variants that have an Start date but no End date are will start at the Start Date and simply continue until changed or removed in the future.

It is possible that a “Winning” Variant that has reached Statistical Significance within a restricted time period may fall out of selection due to leaving the active time period. The remaining Variants in the Multi-Armed bandit may not met the required level of Statistical Significance and therefore an general even split to all Variants recommences.

Example Scenario

 

 

 

 

  • Homepage - test out different banners to promote product X

  • The website calls the DDE Multi-Arm bandit rule to test out the different banner images A,B,C,D. The rule:

    • Receives the website session ID

    • The Multi-Armed Bandit (MAB) step determines which content code to show

      • Need to note what mode (Always / Very Often / Normal / Infrequently / Never) will run.

    • The real-time output steps sends what content to show in the banner

    • The MAB step logs what was shown in the specified e.g.

  • A second rule is called when a customer clicks though to the page that records that the banner has been successful. The rule:

    • Receives the website session ID and Content that was show

    • The MAB step logs that the content code / banner has been successful

  • Based on the amount of successes, the rule that determines the content / banner to show will gradually move towards winning variation, whilst still testing

 

 

Related content