A baseline for comparison for click through rate

Hi ​@KumarMu1 

When establishing a baseline for Click-Through Rate (CTR) specifically within Adobe Target, there are two crucial aspects to understand: how Adobe Target calculates baseline metrics natively, and how to configure your setup so your CTR baseline is accurate and actionable.

1. What serves as the baseline in Adobe Target?

In Adobe Target, there is no universal industry "default" CTR benchmark built into the system. Your baseline is always Experience A (the Control).

When you configure an A/B Test or Auto-Target activity in Target:

• Control (Experience A): Represents your current default experience or baseline performance.

• Lift Calculation: Adobe Target calculates Lift for variations (Experience B, C, etc.) strictly against Experience A using the formula:

  $$\text{Lift} = \frac{\text{CTR}_{\text{Experience B}} - \text{CTR}_{\text{Control}}}{\text{CTR}_{\text{Control}}} \times 100$$

2. Common Pitfalls That Distort Baseline CTR in Target

A frequent issue in Adobe Target is an artificially deflated baseline CTR. Here is why it happens and how to avoid it:

A. Mbox Impressions vs. Element Viewability

• By default, Adobe Target counts a Metric Impression as soon as the mbox request fires (typically when the page or container loads).

• If your button or link is located at the bottom of the page, Target counts an impression every time a user hits the page—even if they never scroll down to see the button.

• Result: Your calculated baseline CTR will appear extremely low (e.g., $0.2\%$), making it hard to achieve statistical confidence.

B. Fixing Viewability Baseline in Target:

• Option 1 (VEC Element Click): In the Visual Experience Composer (VEC), when setting up your Goal Metric, choose Clicked an element, and ensure you select Track clicks on elements on the same page.

• Option 2 (A4T - Adobe Analytics Integration): If using Analytics as the reporting source for Target (A4T), build a Calculated Metric for CTR:

  $$\text{CTR} = \frac{\text{Target Clicks}}{\text{Viewable Impressions (or Scroll Depth Filtered Hits)}}$$

3. General Target CTR Baselines by Element Location

If you need a rough guideline for what baseline CTRs look like across typical Target activities before running a test:

4. How Target Evaluates Baseline CTR for Statistical Significance

To determine if a variation's CTR is truly better than your baseline Control, Target uses Welch's t-test (or a binomial confidence calculation) for standard A/B tests:

1. Minimum Sample Size: Never stop a test just because Experience B shows a higher CTR early on. Use Target’s Sample Size Calculator before launching to know how many impressions your baseline CTR requires.

2. Confidence Level: Aim for 95% Confidence in Adobe Target reports before declaring a winner against your baseline.

3. Auto-Target / AP Baselines: If using Auto-Target or Automated Personalization, Target uses a Randomized Control Group (typically set to 10% or 20% of traffic) as the baseline to continuously evaluate how much lift the machine-learning model is generating over standard random serving.