Creating Analytics for Target Panels

[#1]

How to Build the A4T Table

1. Create a New Project:
• Click the Create Project button in the top-right corner of the screen.
• Select Blank Workspace Project and click Create.
2. Set Up the Panel:
• Navigate to the Panels section on the left-hand side.
• Select Analytics for Target and drag it onto the main workspace screen.
3. Configure the Drop-Down Selections:
• Target Activity: Use the drop-down to type or select the name of the test activity you want to analyze.
• Control Experience: This will auto-populate based on the selected Target Activity. Verify it matches the control setup for your test. You should adjust this, if necessary, as this may vary by the setup of each test.
• Success Metrics: Add metrics like Orders or Revenue to evaluate the test’s success.
• Normalizing Metric: By default, this is set to Visitors. However, depending on your test, you may want to adjust.  You should use Visits as a normalizing metric when analyzing a test that measures short-term behavior, frequency per session, session-based goals, and session-specific engagement. You should use Visitors as a normalizing metric when analyzing a test that measures long-term behavior, cross-visit behavior, unique reach, and visitor-centric goals.
4. Build the Table:
• Click Build to generate the A4T table and analyze your test results.

Reading Results of the A4T Table

Using the Adobe Target Sample Size Calculator

To determine when your test will reach statistical significance:

1. Go to the Sample Size Calculator.
2. Configure the following inputs:
• Confidence Level: Set to 95%.
• Statistical Power: Set to 80%.
• Number of Offers: Include all test variations and control. You will often input ‘2’ for this section.
3. Calculate the Unique Daily Visitors:
• Drag the relevant page (e.g., Checkout) into a Freeform table in Adobe Analytics.
• Add the Unique Visitors metric and set the date range to the last 60 days. You can increase the time window if desired.
• Divide the total number of visitors by 60 and input this value into the calculator as Total Number of Daily Visitors.
4. Determine the Baseline Conversion Rate:
• Divide the total number of Orders by Unique Visitors (Orders/Unique Visitors).
5. Review the Results:
• Note the number of weeks required to complete the test and the sample size per offer needed to achieve statistical significance. Ensure the test has reached the required sample size before making decisions.

Understanding Lift and Confidence

• Lift:
• A flat lift (0% or below) indicates no statistically significant difference between the control and variation groups.
• A positive lift suggests a meaningful performance difference. Continuously monitor to ensure the lift remains above zero.
• Confidence Intervals:
• The Lift Lower and Lift Upper bounds define the 95% confidence interval for the lift.
• A narrow interval indicates greater certainty, while a wide interval suggests insufficient sample size or high variability.
• Key Criteria:
• The test must reach 95% confidence and meet the required sample size to confirm results.