Skip to main content
Grégoire_Miche2
Level 10
November 27, 2015
New

Automated A/B tests in engagement program (and not in a nested sub program)

  • November 27, 2015
  • 2 replies
  • 1700 views

It would be great to have a simple way to create A/B tests in a engagement program with some functionality that are close to what's available in email programs for the automated designation of the winner.

We could, for a given content, create 2 or more variants, with some potential changes (subject, full content), and set the winning rules.

The difference with email programs is that the winner would be determined not after a few days, but after the first n casts (n being defined by the user) and/or after a minimum of X leads. Then the program would automatically keep the winner for following casts. The 2 threshold (Number of casts and number of leads) could be with and AND (both threshold need to be reached) or with an OR (at least one of them has to be reached).

Example #1:

I set the sample to be at least 2000 leads AND 3 casts :

  • First cast : 490 leads, sending A & B. Next cast will still send A & B, as neither the 3 casts nor the 2000 leads threshold have been reached.
  • Second cast : 480 leads, sending A & B. 970 leads so far, so next cast will still send A & B, as neither the 3 casts nor the 2000 leads threshold have been reached.
  • Third Cast : 520 leads, sending A & B. 1490 leads so far, so next cast will still send A & B, as the 2000 leads threshold has not been reached, although the 3 casts has been reached.
  • Fourth cast : 530 leads, sending A & B. 2020 leads so far, both threshold reached so next cast will only send the winner
  • Fifth cast : sending the winner.

Example #2:

I set the sample to be at least 2000 leads OR 3 casts :

  • First cast : 490 leads, sending A & B. Next cast will still send A & B, as neither the 3 casts nor the 2000 leads threshold have been reached.
  • Second cast : 480 leads, sending A & B. 970 leads so far, so next cast will still send A & B, as neither the 3 casts nor the 2000 leads threshold have been reached.
  • Third Cast : 520 leads, sending A & B. 1490 leads so far, yet 3 casts have been sent, so next cast will only send the winner
  • Fourth cast : sending the winner.

-Greg

2 replies

Dan_Stevens_
Level 10
November 27, 2015

Here's a similar, yet much more general, idea:   (the key word being "AUTOMATED" - which the existing champion/challenger doesn't provide).

Grégoire_Miche2
Level 10
November 27, 2015

Hi @Dan Stevens​,

Voted

-Greg

February 6, 2020
No text available