Expand my Community achievements bar.

☕[AT Community Q&A Coffee Break] WEDNESDAY 12/6/23, 8am PT: Machine Learning & AI Reporting & Analysis w/ Brent Kostak, Cristinel Anastasoaie, & Drew Burns of the Adobe Target Product Team ☕

Avatar

Administrator

11/29/23

Join us for the next Adobe Target Community Q&A Coffee Break! 

  Wednesday, December 6th from 8am - 9am PT 

We'll be joined by Adobe Target experts: Brent Kostak (@bkostak), Cristinel Anastasoaie (@cristinel), and Drew Burns (@Drburns27). In this AMA (Ask Me Anything)- like event, the experts will be logged in to chat with you over this Discussion Thread, ready to answer your any of your questions. Marisol St. John () of the Target Technical Support Engineering team will also be joining for extra support. This discussion is geared toward customers who find themselves aligning with the following Target topics:

 

Participating is easy:

  • Register by RSVPing on the right hand side button with 'Yes', 'No', or 'Maybe', as applicable and click "Add to Calendar".
  • Add your Questions below leading up to and on the day of the Coffee Break event and be sure to begin your Question with our hosts' Community Usernames: @bkostak@cristinel), and @Drburns27
  • Stand-by to reply and engage on the page as the experts weigh in during the hour, so that you can:
    • Clarify your questions
    • Ask follow up Qs and provide additional details
    • See Questions posted by other Community members to up-vote those that you find interesting
    • Discuss further with your Target Community member peers 

Meet the experts:

Amelia_Waliany_4-1701316395636.jpeg

 

 

As Product Marketing Manager at Adobe, Brent brings data-driven personalization solutions to market that connect global brands to their customers. Adobe enables enterprise marketing and engineering teams to streamline data and create a single, personalized view of their customers to deliver exceptional digital experiences. Brent has over 10 years of experience helping organizations leverage software technologies and services to better compete and serve their mission.

Amelia_Waliany_6-1701316504919.png

 

 

Cristinel is a Senior Product Manager for Adobe Target, focused on machine learning personalization in Adobe Target, Analytics for Target integration, and the product user interface. Before his current Adobe role, Cristinel held product management positions on various content management and digital marketing products focused on consumers.

 

Amelia_Waliany_5-1701316395655.pngDrew Burns is Group Product Marketing Manager for Adobe Target, the industry-leading experience optimization platform, and he has been an evangelist and a practitioner in the personalization and testing space for 15 years.  He works with progressive enterprise companies across industries on building the foundation and maturing their optimization and personalization efforts, strategies and programs leveraging manual and A.I./ML capabilities. At Adobe, he works alongside the leading enterprise companies in retail/e-commerce, travel & hospitality, financial, media & publishing, B2B/ High Tech and other verticals to improve and grow their optimization and personalization efforts, strategies and programs. 

 

 

 Something already coming to mind?

Add your related questions below any time before or during the Coffee Break on Wednesday, December 6th, when you can watch the page and be ready to add follow-up questions and discuss further with other community members!

Views

4.0K

Replies

39
Sign in to like this content

Total Likes

39 Comments

Avatar

Employee

12/6/23

Hi everyone!  We're thrilled to be bringing you another Coffee Break in the series on AI-powered testing and personalization.  Please input your questions and we'll try to answer as many as we can during the session, and follow up after with any we may not have been able to answer within the timeframe!

Auto-Target Reporting Issue - Where are my Important Attributes and Automated Segment Reports

Hello,

Looking for some support/insight from anyone that has utilized the Auto-Target feature within an A/B test. My test has been running for quite some time and I've reached the necessarily visitors and conversions levels needed for my algorithms to begin learning, 1,000 visits and 50 conversions (7,000 overall visits and 350 overall conversions) as I understand them. However, my reporting is still stating that the models have not been built yet - as indicated by the clock icon still next to my experiences. I'm also not seeing the Important Attributes report nor the Automated segment reports that are supposed to come along with this tool....presumably because the models are still being indicated as not finalize yet? I am using a Conversion goal, 'clicked an element' for our goal metric. 

This test has been running for over 100 days and it seems to be collecting the right information, but no models and no reports yet, any ideas? Are the experiences just too similar for the AI to pick up anything and create a model from? Appreciate any thoughts or comments!

 

ben_mrm_0-1701879167907.png

 

Avatar

@ben_mrm This is a strange issue that I have not come across before and hard to diagnose without a review of the activity itself.  Have you referred to Analytics for your reporting (Analysis Workspace)?  Please open a ticket with Support so they can review the activity and provide guidance on where there may be issues.

Thanks! We have looked at Analysis workspace, there is data flowing but we're losing out on the big selling point of auto-target, the automated segments and important attributes reports....as far as I know that can't be viewed in AA, right?

 

We'll have to proceed to a ticket with Support then....I'm at a lose as well. 

.....There's not additionally provisioning needed at all for these reports, right?

I've experienced this too Ben. Sharing our fix in case it's similar to your issue:

Turns out our Target default environment was set to ‘Development’ instead of 'Production'. After we set Production as the default env. Personalization Insights (automated segments) started appearing in the activity Results tab after a little while.

So basically:

  • The activity is served on www.yourcompany.com.au (where www.yourcompany.com.au is set as a Production environment host in Admin > Hosts)
  • The default environment must be set to Production for the Personalization Insights data to be available in activity results (Administration > Environment)

Hope this helps!

Avatar

Level 1

12/6/23

@bkostak@cristinel), and @Drburns27 For Automated Personalization, because I cannot use AA as the reporting source, I am limited in metrics that I want to track. Is there a way to track Bounce Rate in Target Reporting for Automated Personalization?

@MadisonDicken You may want to try setting up the activity, depending on the use case, leveraging Auto-Target which is built into the A/B testing workflow, and uses the same Random Forest Ensemble set of algorithms as Automated Personalization for 1:1 Personalization and does support A4T, which can report on bounce rate as well as any other metric you wish to track.  Keeping the variants low is optimal for these AI-powered personalization activities but multiple activities can be run concurrently for greater efficiency.

Avatar

Employee Advisor

12/6/23

Welcome community members and Target experts! Looking forward to the session to answer all AI/ML questions from those joining today. Great topics we've been hearing from the community from recent Adobe Insider Tour events. Excited for this one! 

Avatar

Level 2

12/6/23

Hello, curious to hear what people's experience is while running A/A Tests. We periodically run A/A tests (~bi-annual) on our site to confirm that our tracking is as expected. However, we've noticed that our past 3 A/A tests over a month have had inconsistent results with lifts in overall page engagement reported anywhere from 8-20% for the first 1-2 weeks before results stabilize. Would love to hear what other practitioners experience

@Officialyi Adobe Target uses a Students t-test method which is optimal for multiple variants, especially if those variants perform similarly within a test, but does require waiting until 95% confidence is reached to provide rigorous, accurate results while reducing the incidence of false positives.  You may want to run an A/A test utilizing Auto-Allocate which allows for "peeking" during the course of the activity for relative lift/confidence with accuracy at any point during the activity when testing 2 variants. 

This is configured as a A4T test with Adobe Analytics set as the reporting source. Thanks @Drburns27 our team setup the test according to best practices listed in the documentation from Adobe  

Avatar

Level 1

12/6/23

Hello everyone. We are beginners in Adobe Target and have a question regarding an A/B testing setup on the same URL. We have an A/B test where we want 50% of the users to be shown different content on the page within a single-page application. How can we distinguish the control and variant from one another within the same URL so that we can make sure we display the different content only to the variant? What are the best practices for setting up a test such as this? Thanks!

Avatar

Employee

12/6/23

@jlisic Great question!  Target, as a default, will split the test population by 50% based on the experiences you set-up in the first step of the workflow.  One of the benefits of our SPA implementation are extensions that route to a specific location within the SPA to deliver one of the experiences to the population within the test.  One other method of distinguishing or segmenting a population in distinct groups is leveraging profile scripts: How to Create Profile Scripts | Adobe Target

Thank you for that information! Does Adobe have any specific SPA online training? Many of our sites are SPAs and it would be extremely valuable if so!

Thanks very much! One additional question, does Adobe have any information about setup for sending Target data to GA4?

Avatar

Employee Advisor

12/6/23

Hi @jlisic Experience League documentation below for Target on SPAs - includes implementation best practices and a few training videos near the end! 

https://experienceleague.adobe.com/docs/target-dev/developer/client-side/at-js-implementation/deploy...

Avatar

Level 1

12/6/23

Is it possible to track and set up scroll depth for an activity within target reporting , since we don't use Adobe analytics 

Avatar

Employee

12/6/23

@JJ223 Yes, this is possible to track in Target but would need to be captured as a persistent value within the Target profile, when not using A4T/ AA.  This would require a profile script: How to Create Profile Scripts | Adobe Target

Avatar

Employee

12/6/23

I'd like to hear Drew's thoughts on leveraging the Models API to "subtractively" influence the AI inputs to Automated Personalization and Auto-Target? I could see wanting to put more influence on carefully planned profile values by removing some of the other inputs. 

Avatar

Employee Advisor

12/6/23

Hi @wozinga Can you clarify your point on 'subtractively'? Referring to Target Model API for blocklisting certain attributes, this allows customers to fine-tune the AI models to optimize success metrics specifically on remaining attributes. Are you looking for additional thoughts on how to do this? or perhaps when this approach would work for certain use cases?

Avatar

Employee

12/6/23

I'm curious about expected usage, and perhaps some examples of successful examples. I have seen cases where we felt after reviewing the results it was best to exclude the thousands of Audience Manager audiences, and test that against leaving them in. 

Avatar

@wozinga Great question!  While Auto-Target and Automated Personalization leveraging our random forest ensemble set of algorithms is super-powerful in exploring and exploiting the entire profile data for each individual, scoring over time what is most predictive, we added the ability earlier this year to block-list attributes that are not relevant per each AI-powered Personalization activity to reduce the data/reporting "noise" and focus the reports on the most relevant profile attributes and segments based on the intended goal.  This is huge in customizing each activity to the data that matter most.  Words to the wise is to ensure you are not limiting the data set too much, as the random forest ensemble does help in surfacing valuable attributes and segments that you may not be actively analyzing in its reporting, which can be valuable insights for further personalization opportunities, especially in largely anonymous or unauthenticated traffic.

Avatar

Level 1

12/6/23

Hi everyone ! I understand there are differences between audience creation options for the delay/availability of audiences (ex. Target – First Page, Adobe Analytics – 4-8h delay, etc.). Any expected effects on the relevance of Auto-Target or Automated Personalization features ?

@EdouardBT22 , both Auto-Target and Automated Personalization work with the audiences and profiles available on Target edges at the time of the delivery. However, in both activity cases models are constantly learning and any new audiences will be taken into consideration as soon as they are available in Target.

 

@EdouardBT22 Adobe Target AI-powered personalization activities will evaluate all audience segments and attributes available in the application (from any source, including Analytics) as well as the profile attributes designated in the activity (all or a subset based on block-list)  The audience segment will need to be available to Target for our activity to evaluate so requisite timeframes on ingestion of these audiences are relevant.

Avatar

Level 1

12/6/23

Any documentation on sending target data to GA4 , the one I came across only has information on sending data to universal analytics but not GA4

Avatar

Employee

12/6/23

@JJ223 Adobe Target does integrate with GA4 (many customers sending Target activity data into current Google Analytics)  "Check out "Sending Data to Google Analytics via at.js" under the link Response tokens. Hope this helps."

Avatar

Level 1

12/6/23

Any future product enhancements that are related to A4T of Auto-Target test to visualize results?

I follow this video to analyze the Auto-Target activity, but it is a lot of manual processes required. I hope that Adobe will provide some functionalities to eliminate those steps.

https://youtu.be/WLtsoNzCQTk?si=9XKUTLWnSNlf-9AT

@CocoNYC13615 We would love to hear your suggestions on ways we can reduce the manual process for you!  If you could share areas that we can streamline the process as our product and engineering teams are working on the speed of Target activities e.g. multiple improvements have been recently released for libraries and overall UI for activity set-up and execution, we would love to hear them!

Agree this is way too many steps you could set up incorrectly.

Perhaps more Analytics for Target dropdowns that you set initially in the panel, or a separate panel something like Analytics for Target (Auto-Target) that generates as close to the final workspace at https://youtu.be/WLtsoNzCQTk?t=322 5:22 as possible @Drburns27

Avatar

Level 2

12/6/23

Hi @bkostak@cristinel@Drburns27, this is probably too generic a question, but do you have any examples, or best practice of integrating 3rd party (non-Adobe) CDP audiences with Adobe Target? I came across this article, but have concerns this approach could be very manual and may impact page load speeds - 

https://experienceleague.adobe.com/docs/target-learn/tutorials/integrations/implement-data-providers...

@chris-luckett Great question!  Obviously there are many methods of bringing in this data on CDP audiences into Target, including batch profile upload api and passing the segments through the page.  Here is a good article from a partner MiaProva who has experience with these integrations: Adobe Target and Customer Data Platforms (CDPs) - MiaProva

Avatar

Level 2

12/6/23

Do you have any case studies from other Adobe customers about how they have used Automated personalisation?

Avatar

Level 3

12/6/23

@Amelia_Waliany are there upcoming Target coffee breaks at convenient APAC (Sydney) times? thanks

Event has ended
You can no longer attend this event.

Event starts:

Dec 06, 2023 - 08:00 AM (PST)

Event ends:

Dec 06, 2023 - 09:00 AM (PST)