A/B/C QA Process

Avatar

Avatar
Boost 1
Level 1
hattieh39692179
Level 1

Like

1 like

Total Posts

1 post

Correct reply

0 solutions
Top badges earned
Boost 1
View profile

Avatar
Boost 1
Level 1
hattieh39692179
Level 1

Like

1 like

Total Posts

1 post

Correct reply

0 solutions
Top badges earned
Boost 1
View profile
hattieh39692179
Level 1

10-08-2017

Hello!

We are trying to set up an ABC test in Adobe Target but are having difficulties figuring out to QA each test treatment efficiently when we are testing more than one experience against the control. This is our current process for setting up an AB test and QAing:

  1. Build out the test experience
  2. Create and apply a new audience that includes a query parameter
  3. Set traffic to the test experience to 100%
  4. Push the test live and QA using the query parameters we set up within the new audience.
  5. Once the test is QA'd and we know it's working properly, we go back into the activity, update the audience to remove the query parameter, update the traffic to an even split, and push it live.

So far this process has worked for us (but if there is a more efficient way to do this, please share!), but when introducing 2+ experiences in one activity, we can't figure out how to assign a QA parameter to each individual experience. Because of this, each time we need to QA a different experience we go back into the activity and reallocate 100% of the traffic to the next experience we are wanting to QA and repeat.

Although time-consuming, this process does work for us, but our clients like to QA the tests themselves and aren't in the platform everyday to know how to reallocate traffic and not disrupt the current test set up. Is there a way to assign a separate QA parameter to each individual experience within one activity so we can share those QA parameters with our client without having to adjust any traffic within the activity?

Any thoughts and insights will be helpful! Thank you!

Replies

Avatar

Avatar
Boost 5
Level 3
LJ_Jones
Level 3

Likes

28 likes

Total Posts

37 posts

Correct reply

8 solutions
Top badges earned
Boost 5
Boost 3
Boost 25
Boost 10
Boost 1
View profile

Avatar
Boost 5
Level 3
LJ_Jones
Level 3

Likes

28 likes

Total Posts

37 posts

Correct reply

8 solutions
Top badges earned
Boost 5
Boost 3
Boost 25
Boost 10
Boost 1
View profile
LJ_Jones
Level 3

19-08-2017

Have you used the Preview Experience links before?  From the main list of activities, click on your activity.  On the overview tab, a little down the page on the left side will be a blue link that says "Preview Experiences".  Click on this and it will open a popup.  From the popup, add a url to the page that you want to QA.  It will then generate a link for each experience.  You will then be able to click a preview link and do your QA.  Please note that these are temporary links.  If you make changes to the activity, you will have to go and generate new links.  The old links will no longer work.

The other downside is that these preview links generally ignore audience targeting.  So it won't do a great job of QA on targeting.  Also, if your activity spans more than one page, when you click to the next page, you will be out of the activity.  Unless you add back in the url parameter which enables the preview.  This url parameter can be copied and pasted to any url where you want to preview the experience.

Avatar

Avatar
Give Back 5
Employee
Jon_Tehero
Employee

Likes

61 likes

Total Posts

40 posts

Correct reply

10 solutions
Top badges earned
Give Back 5
Give Back 3
Give Back 10
Give Back
Boost 50
View profile

Avatar
Give Back 5
Employee
Jon_Tehero
Employee

Likes

61 likes

Total Posts

40 posts

Correct reply

10 solutions
Top badges earned
Give Back 5
Give Back 3
Give Back 10
Give Back
Boost 50
View profile
Jon_Tehero
Employee

17-10-2017

We updated the QA funcationlity in Target on August 22, 2017 to address some of the challenges LJ mentioned.

Check the documentation for our new Activity QA

1326631_pastedImage_3.png