Creating a Holdback set within Target, is it possible?

[csanderson99]

One of the best ways to compare long term effects of tests over time is to create a holdback set, a percentage of users that always sees the original default content no matter what has been hardcoded on the website. Is there a way to implement a long-term "control" variant like this, that can be excluded by all future tests moving forward?

[Jason_Hickey · Accepted solution]

It's possible, though I'm not sure how plausible, to use a profile script to generate a random number from 1-any integer and assign that to a users profile.  Imagine a script profile called "randNum" that generates a number between 1 and 1,000. Two audiences can then be built on this profile.   When user.randNum is between 1 and 50, assign a user to group A.  When user.randNum is greater than 50, assign a user to group B.   The last step would be to target all future activities to only Group B.   You could then have a "forever" activity on Group A with no changes, and using A4T that would become a pretty much persistent dimension about that audience to be used in Analytics segmentation.

There are, of course, gaps such as somebody clearing their cookies.    It also means you would never be able to code any test result and have to run everything, forever, through Adobe Target - which I doubt anybody would recommend.  The second caveat pretty much nullifies this tactic unless you were only wanting to measure long term effects of always on activity types like Automated Personalization or Recommendations.

If one was to be really serious about such a hold out, a hardware or server side split that feed in to separate Report Suites would be much more reasonable.