Adobe Target

Auto-Target Reporting Issue - Where are my Important Attributes and Automated Segment Reports

Hello,

Looking for some support/insight from anyone that has utilized the Auto-Target feature within an A/B test. My test has been running for quite some time and I've reached the necessarily visitors and conversions levels needed for my algorithms to begin learning, 1,000 visits and 50 conversions (7,000 overall visits and 350 overall conversions) as I understand them. However, my reporting is still stating that the models have not been built yet - as indicated by the clock icon still next to my experiences. I'm also not seeing the Important Attributes report nor the Automated segment reports that are supposed to come along with this tool....presumably because the models are still being indicated as not finalize yet? I am using a Conversion goal, 'clicked an element' for our goal metric. 

This test has been running for over 100 days and it seems to be collecting the right information, but no models and no reports yet, any ideas? Are the experiences just too similar for the AI to pick up anything and create a model from? Appreciate any thoughts or comments!

@ben_mrm This is a strange issue that I have not come across before and hard to diagnose without a review of the activity itself.  Have you referred to Analytics for your reporting (Analysis Workspace)?  Please open a ticket with Support so they can review the activity and provide guidance on where there may be issues.

Thanks! We have looked at Analysis workspace, there is data flowing but we're losing out on the big selling point of auto-target, the automated segments and important attributes reports....as far as I know that can't be viewed in AA, right?

We'll have to proceed to a ticket with Support then....I'm at a lose as well. 

.....There's not additionally provisioning needed at all for these reports, right?

I've experienced this too Ben. Sharing our fix in case it's similar to your issue:

Turns out our Target default environment was set to ‘Development’ instead of 'Production'. After we set Production as the default env. Personalization Insights (automated segments) started appearing in the activity Results tab after a little while.

So basically:

• The activity is served on www.yourcompany.com.au (where www.yourcompany.com.au is set as a Production environment host in Admin > Hosts)
• The default environment must be set to Production for the Personalization Insights data to be available in activity results (Administration > Environment)

Hope this helps!

@MadisonDicken You may want to try setting up the activity, depending on the use case, leveraging Auto-Target which is built into the A/B testing workflow, and uses the same Random Forest Ensemble set of algorithms as Automated Personalization for 1:1 Personalization and does support A4T, which can report on bounce rate as well as any other metric you wish to track.  Keeping the variants low is optimal for these AI-powered personalization activities but multiple activities can be run concurrently for greater efficiency.

@Officialyi Adobe Target uses a Students t-test method which is optimal for multiple variants, especially if those variants perform similarly within a test, but does require waiting until 95% confidence is reached to provide rigorous, accurate results while reducing the incidence of false positives.  You may want to run an A/A test utilizing Auto-Allocate which allows for "peeking" during the course of the activity for relative lift/confidence with accuracy at any point during the activity when testing 2 variants. 

@Officialyi Is this an A4t test or Adobe Target for Analytics?

This is configured as a A4T test with Adobe Analytics set as the reporting source. Thanks @Drburns27 our team setup the test according to best practices listed in the documentation from Adobe  

@Officialyi Here is an article on best practices around A/A testing in Target: What is A/A Testing? | Adobe Target

@jlisic Great question!  Target, as a default, will split the test population by 50% based on the experiences you set-up in the first step of the workflow.  One of the benefits of our SPA implementation are extensions that route to a specific location within the SPA to deliver one of the experiences to the population within the test.  One other method of distinguishing or segmenting a population in distinct groups is leveraging profile scripts: How to Create Profile Scripts | Adobe Target

Thank you for that information! Does Adobe have any specific SPA online training? Many of our sites are SPAs and it would be extremely valuable if so!

@jlisic Yes!  Our Training team can run in-person training, and there is also documentation and videos on Experience League: https://experienceleague.adobe.com/docs/target-dev/developer/client-side/at-js-implementation/deploy... 

Thanks very much! One additional question, does Adobe have any information about setup for sending Target data to GA4?

Hi @jlisic Experience League documentation below for Target on SPAs - includes implementation best practices and a few training videos near the end! 

https://experienceleague.adobe.com/docs/target-dev/developer/client-side/at-js-implementation/deploy...

@JJ223 Yes, this is possible to track in Target but would need to be captured as a persistent value within the Target profile, when not using A4T/ AA.  This would require a profile script: How to Create Profile Scripts | Adobe Target

Hi @wozinga Can you clarify your point on 'subtractively'? Referring to Target Model API for blocklisting certain attributes, this allows customers to fine-tune the AI models to optimize success metrics specifically on remaining attributes. Are you looking for additional thoughts on how to do this? or perhaps when this approach would work for certain use cases?

I'm curious about expected usage, and perhaps some examples of successful examples. I have seen cases where we felt after reviewing the results it was best to exclude the thousands of Audience Manager audiences, and test that against leaving them in. 

@wozinga Great question!  While Auto-Target and Automated Personalization leveraging our random forest ensemble set of algorithms is super-powerful in exploring and exploiting the entire profile data for each individual, scoring over time what is most predictive, we added the ability earlier this year to block-list attributes that are not relevant per each AI-powered Personalization activity to reduce the data/reporting "noise" and focus the reports on the most relevant profile attributes and segments based on the intended goal.  This is huge in customizing each activity to the data that matter most.  Words to the wise is to ensure you are not limiting the data set too much, as the random forest ensemble does help in surfacing valuable attributes and segments that you may not be actively analyzing in its reporting, which can be valuable insights for further personalization opportunities, especially in largely anonymous or unauthenticated traffic.

@EdouardBT22 , both Auto-Target and Automated Personalization work with the audiences and profiles available on Target edges at the time of the delivery. However, in both activity cases models are constantly learning and any new audiences will be taken into consideration as soon as they are available in Target.

@EdouardBT22 Adobe Target AI-powered personalization activities will evaluate all audience segments and attributes available in the application (from any source, including Analytics) as well as the profile attributes designated in the activity (all or a subset based on block-list)  The audience segment will need to be available to Target for our activity to evaluate so requisite timeframes on ingestion of these audiences are relevant.

@JJ223 Here's another very detailed 3rd party description of Target + GA4: Sending Adobe Target A/B Testing Data to Google Analytics 4 - scandiweb

@CocoNYC13615 We would love to hear your suggestions on ways we can reduce the manual process for you!  If you could share areas that we can streamline the process as our product and engineering teams are working on the speed of Target activities e.g. multiple improvements have been recently released for libraries and overall UI for activity set-up and execution, we would love to hear them!

Agree this is way too many steps you could set up incorrectly.

Perhaps more Analytics for Target dropdowns that you set initially in the panel, or a separate panel something like Analytics for Target (Auto-Target) that generates as close to the final workspace at https://youtu.be/WLtsoNzCQTk?t=322 5:22 as possible @Drburns27

@chris-luckett Great question!  Obviously there are many methods of bringing in this data on CDP audiences into Target, including batch profile upload api and passing the segments through the page.  Here is a good article from a partner MiaProva who has experience with these integrations: Adobe Target and Customer Data Platforms (CDPs) - MiaProva