☕[AT Community Q&A Coffee Break] 2/24/21, 9am PT: Rob Hornick, Senior Product Manager for Adobe Target☕

[Amelia_Waliany]

Join us for the next monthly Adobe Target Community Q&A Coffee Break, 

taking place Wednesday, February 24th, 2021 @ 9am PT

Be sure to MARK YOUR CALENDARS with a link to this thread

We'll be joined by Rob Hornick, aka @Rob_Hornick, Senior Adobe Target Product Manager, who will be signed in here to the Adobe Target Community to chat directly with you on this thread about your Adobe Target questions pertaining to his areas of expertise:

• Recommendations
• Auto-Allocate
• Auto-Target
• Automated Personalization
• Personalization
• Machine Learning & Artificial Intelligence 

REQUIREMENTS TO PARTICIPATE  

• Must be signed in to the Community during the 1-hour period 
• Must post a Question about Adobe Target
• THAT'S IT!  *(think of this as the Adobe Target Community equivalent of an AMA, (“Ask Me Anything”), and bring your best speed-typing game) 

INSTRUCTIONS 

• Click the blue “Reply” button at the bottom right corner of this post
• Begin your Question with @Rob_Hornick
• When exchanging messages with Robert about your specific question, be sure to use the editor’s "QUOTE" button, which will indicate which post you're replying to, and will help contain your conversation with Robert

**For those who have joined an Adobe Target Community Q&A Coffee Break before: please note that we are no longer using formal registration pages for these events - just be sure to Mark Your Calendars to sign in here to the Community on 2/24 @ 9am PT, and hop onto this thread for direct answers to your Q’s from Rob.

Rob Hornick is the Senior Product Manager for Machine Learning and Personalization with Adobe Target, based in San Francisco. Rob is energized by building tools to personalize digital experiences and by putting advances in machine learning into marketers’ hands. Prior to joining Adobe, Rob was a Manager with Accenture Digital where he helped marketers optimize their processes and technology.

Curious about what an Adobe Target Community Q&A Coffee Break looks like? Be sure to check out our past 2020 Adobe Target Coffee Breaks, covering a WIDE variety of Adobe Target topics, and our latest thread from our 1/13/21 Adobe Target Coffee Break with @Robert Calangiu 

[helvoirt]

@Rob_Hornick How do you keep everything manageable and maintainable when scaling up the number of A/B tests and setting up for personalisation at scale across multiple customer journeys? Any advice or best practices you can share?

[kadcock]

@Rob_Hornick We are easing into using Profile Scripts.... are there best practices around how many is too many?  At what point do they impact page load performance?  Etc.

[Rami_Hammad]

@kadcock Here's a helpful link, it mentions all the limits in Adobe Target: https://experienceleague.adobe.com/docs/target/using/troubleshoot/target-limits.html?lang=en#orderid...

Specifically the section:

Profile scripts

• Recommended limit of active profile scripts: 300

• Recommended limit of total profile scripts per account: 2,000

Hope that helps!

[RobHornick]

---

@kadcock wrote:

@Rob_Hornick We are easing into using Profile Scripts.... are there best practices around how many is too many?  At what point do they impact page load performance?  Etc.

---

Hi @kadcock , Rami shared a good response earlier around limitations in number of profile scripts. To that I'll add some additional recommendations:

• The most "expensive" (time-consuming) operations in profile scripts tend to be "while" looping and RegEx matching. We recommend minimizing the use of these operations as much as possible.
• In contrast, scripts such as simple counters (e.g. "if page type is article, article_view_count++") are quite inexpensive to run.
• Keep in mind that all scripts are executed on each request, so you want to consider the total impact of all scripts holistically. A single loop may not be super expensive in isolation but if there are 100 such loops, it adds up quickly.
• Performance test complex profile-scripts as you add them.

More guidance on the total number of instructions and loops is available here: https://experienceleague.adobe.com/docs/target/using/audiences/visitor-profiles/profile-parameters.h... 

[FDmAmG]

Hey, @Rob_Hornick any preference or best practices on using auto-allocate vs. sample size calculator for a/b testing?

[RobHornick]

---

@FDmAmG wrote:

Hey, @Rob_Hornick any preference or best practices on using auto-allocate vs. sample size calculator for a/b testing?

---

Hi @FDmAmG ! Auto-Allocate can be used almost any time you would run an A/B test, but it can be especially good to use in the following circumstances:

• You have a limited amount of time for your test. This can be the case when a test is only relevant for a period of time (e.g. a promotion tied to President's Day Weekend or "The Big Game", or a landing page associated with an external email blast or a conference). Auto-Allocate begins shifting traffic to winners before the result is final and allows you to realize more lift in a shorter amount of time than traditional A/B tests.
• You want to test more than two experiences. Traditional A/B/N test methodologies require much more traffic when you add more experiences. Because Auto-Allocate reduces traffic to losing experiences, it's able to do a better job of finding differences between winning experiences.
• You don't want to worry about the statistical calculations. This one is self-explanatory!

You might want to run a classical A/B test under the following circumstances:

• You need a high degree of confidence about the magnitude of the difference between experiences. Because Auto-Allocate shifts traffic to winners, you may have a wider degree of uncertainty around the "true" conversion rate (or revenue, etc.) of a losing experience.
• When it's important to you to say that "there was no winner" within a fixed timeframe. Traditional fixed-horizon A/B testing has a defined end date for the test and at the end of that timeframe, you can say "We did not reject the null hypothesis; version B was not proven better than version A." Auto-Allocate doesn't have a fixed end date in mind, so you might be left with a less definite conclusion.

[FDmAmG]

The last bulleted item was something I hadn't considered! Thank you @Rob_Hornick 

[RobHornick]

Good morning and welcome! Looking forward to chatting with you all today. I'll begin answering questions shortly!

[ToddWeston]

@Rob_Hornick 

I'm struggling with understanding how to get Auto-Target or Auto Personalization to use our custom data attriubutes.  We have a number of items we send to Analytics that we would like it to consider.  1) How do we teach the ML algorithm about these.  2) Is it configurable in any way?   Thx!

[rmaisner]

@ToddWeston  @Rob_Hornick Re: using custom attributes in Auto Personalization-- it seems like Target is pulling in custom analytics segments shared to the Experience Cloud, but no other Analytics data other than standard metrics. We are specifically interested in key custom events that have been set up to define success actions on the site.

[RobHornick]

---

@rmaisner wrote:

@ToddWeston  @Rob_Hornick Re: using custom attributes in Auto Personalization-- it seems like Target is pulling in custom analytics segments shared to the Experience Cloud, but no other Analytics data other than standard metrics. We are specifically interested in key custom events that have been set up to define success actions on the site.

---

Hi @rmaisner , you are correct that Target doesn't "know" about attributes that haven't been published to Target as segments via the Experience Cloud. That said, Analytics custom events can be used as the optimization metric for both Auto-Allocate and Auto-Target activities with the new A4T support recently added to those activity types. For more detail on the specifics of A4T support for Auto-Allocate and Auto-Target, check out: https://experienceleague.adobe.com/docs/target/using/integrate/a4t/a4t-at-aa.html

(Analytics custom events cannot be used as the optimization metric for Automated Personalization as A4T support is not available for this activity type.)

[RobHornick]

---

@ToddWeston wrote:

@Rob_Hornick 

I'm struggling with understanding how to get Auto-Target or Auto Personalization to use our custom data attriubutes.  We have a number of items we send to Analytics that we would like it to consider.  1) How do we teach the ML algorithm about these.  2) Is it configurable in any way?   Thx!

---

Hi @ToddWeston! Auto-Target (AT) and Auto-Personalization (AP) will consider all out-of-box attributes and all custom attributes in the Target profile. If you send profile attributes to Analytics, to get Target to consider these attributes, you need to send them to Target as well. This can be done in a few ways: you could share segments from Analytics to Target; you could integrate with Target through the Customer Attributes service; or you could send data to Target on the page or in your API calls. More info on publishing segments from Analytics is available here: https://experienceleague.adobe.com/docs/analytics/components/segmentation/segmentation-workflow/seg-...

More info on the differences and "how-to" of other methods is available here: https://experienceleague.adobe.com/docs/target/using/implement-target/before-implement/methods/metho...

Whichever method you choose, AT and AP will automatically consider all available data attributes in modeling the best content option for a given visitor.

You also mentioned whether it's configurable in any way which attributes AT/AP consider for the algorithm. The short answer is "yes," you can choose to block certain attributes from being considered by the AT/AP algorithm. This is useful to customers in highly regulated industries (e.g. financial services) who may need to prevent certain attributes (e.g. geography, demographic affinity segment) from being used in content decisions. To block specific attributes from AT/AP, please contact Customer Care or your Customer Success Manager.

[helvoirt]

@Rob_Hornick How do you keep everything manageable and maintainable when scaling up the number of A/B tests and setting up for personalisation at scale across multiple customer journeys? Any advice or best practices you can share?

[RobHornick]

---

@helvoirt wrote:

@Rob_Hornick How do you keep everything manageable and maintainable when scaling up the number of A/B tests and setting up for personalisation at scale across multiple customer journeys? Any advice or best practices you can share?

---

Hi @helvoirt ! This is a challenge to answer concisely within the scope of a forum post, but the best answer I can give is that you need to set up a testing/personalization organization and have a repeatable process that is followed for all tests/personalization activities. The Target Welcome Kit provides some general guidance on how you might do this. For specific and concrete examples, I would point you at the excellent customer sessions our customers have shared at Adobe Summits in the past. You can see a listing of all Target sessions from Adobe Summit 2020 here:

https://www.adobe.com/summit/2020/sessions.html?Products+%26+Services=Target

The upcoming Adobe Summit 2021 features 3 sessions that I think will address your question:

https://portal.adobe.com/widget/adobe/as21/catalogsum2021?search=910

https://portal.adobe.com/widget/adobe/as21/catalogsum2021?search=908

https://portal.adobe.com/widget/adobe/as21/catalogsum2021?search=907

[Amelia_Waliany]

Hi @Rob_Hornick , thanks for taking the time today! This questions was posted by yza085

geoffreyl854413: "In as few words as possible, what is the primary advantage of using Automated Personalization vs. Auto Target" 

[RobHornick]

---

@Amelia_Waliany wrote:

Hi @Rob_Hornick , thanks for taking the time today! This questions was posted by yza085

geoffreyl854413: "In as few words as possible, what is the primary advantage of using Automated Personalization vs. Auto Target" 

---

Hi @geoffreyl854413 ! Auto-Target (AT) uses personalization to select the best experience for a visitor and builds a predictive model per experience, while Automated Personalization (AP) uses personalization to select the best combination of offers for a visitor and builds a predictive model per offer. That means:

Auto-Target is better when:

• You want to select the best experience per visitor.
• You have between 2 and 10 different experiences to choose between.
• You want to make changes across multiple pages (Auto-Target allows an experience to span multiple pages.)
• You want to consider interaction effects between offers (Automated Personalization builds a model per offer and does not consider interaction effects between them.)
• You want to use Adobe Analytics to analyze your activity or use an Adobe Analytics metric as your success metric for the activity. (Auto-Target supports Target's A4T integration.)

Automated Personalization is better when:

• You want to select the best combination of offers per visitor.
• You have dozens of possible offers resulting in hundreds or thousands of unique combinations. (Because AP builds models on a per-offer level, you can still get enough traffic to each offer to develop a prediction.)
• You want to use flexible rules to restrict combinations of offers or restrict offers to certain audiences (AP allows you to build exclusion groups and offer targeting rules) but don't mind ignoring interaction effects beyond this (AP does not consider interaction effects.)
• Avoid AP when you want to make structural or navigational changes, or changes across multiple pages -- Auto-Target is better for making non-offer-related changes.

[Amelia_Waliany]

Hi @Rob_Hornick, this question was posted by christiner93676: "Please provide best practice / recommendations on leveraging traffic allocation methods (auto-allocate to best experience vs. auto-target for personalized experiences)"

[RobHornick]

---

@Amelia_Waliany wrote:

Hi @Rob_Hornick, this question was posted by christiner93676: "Please provide best practice / recommendations on leveraging traffic allocation methods (auto-allocate to best experience vs. auto-target for personalized experiences)"

---

Hi @christiner93676 ! The choice here really depends on your goal.

Auto-Allocate will result in a non-personalized overall winning experience being chosen for most visitors.

Auto-Target will result in a personalized experience chosen for each visitor, with multiple experiences receiving traffic.

Apart from this distinction, there's one additional consideration to keep in mind. Because Auto-Allocate is non-personalized, it can get lift more quickly than a personalized model can. So Auto-Allocate is preferred in time-constrained circumstances (e.g. promotion around a holiday, landing page for an email campaign, registration page for a conference) when Auto-Target might not get to build personalized models. Auto-Allocate can also be preferred when you have many experiences to test, but little traffic (for example, if with your site's traffic, running a standard A/B test takes 30 days to reach significance, and your content changes monthly, Auto-Target will likely not be able to produce personalized models in a timely fashion to deliver lift.)