Visitor count seen in Analytics is too low for a A/B test activity

[bharath_sampath]

Hello everyone,

We are not seeing a lot of visitors/views being counted for an A/B test when we pull up the reporting in Analytics. I've tried my best to explain below how the A/B test is setup, and have also explained my understanding of the issue. Any help with shedding more light into this issue, or confirming my theory, is greatly appreciated!!

This A/B test is being run within our Single Page booking engine. We wanted to deliver the test only to visitors who are clicking on the CTA (Call To Action) buttons to add a product to the cart.

We have implemented this by setting click event listeners on those buttons through an XT Activity that is delivered to all visitors to the page. So, when the visitor clicks any of those buttons, an mbox request is fired (using adobe.target.getOffer() and adobe.target.applyOffer()) with an mbox parameter. This param is configured as the qualifying audience for the A/B test, and so the mbox call returns with the A/B test.

Now, based on the experience that is delivered, the corresponding actions are performed.

1. For the control/default experience, the cart page is loaded with the selected product(different HTML page and not a different view of the same booking page).
2. For the alternate experience, a better product is suggested, which the visitor can choose or deny. Once they make their selection, the cart page is loaded with the selected product.

The A/B experiences are delivering perfectly, but the the visitor count for the A/B test in Analytics is too low. The visitor count for the XT is as expected though. We are also running another A/B test on the same page in parallel for all visitors, for which the visitor count is as expected.

We are using the target-global-mbox strategy with A4T implementation. We have set the right reporting suite, which is being used to generate reports in Analytics.

Also, we have a metric that tracks the products that are added to the cart (on the cart page). This metric shows that the products are being added (which means the same number of visitors should have been qualifying for the test), but the numbers in the reporting are too low. This lead us to believe that Analytics isn't able to register that the visitor had qualified for the campaign.

Now, I've done some investigation into this problem and found these articles on the A4T documentation:

• Minimizing inflated visit and visitor counts in A4T
• Use an Analytics tracking server

Based on the content on these pages, my understanding of A4T in a nutshell is that:

Whenever a visitor visits a page, Target loads on the top of the page (head), and Analytics loads on the bottom of the page body. When Analytics completes loading (on the bottom), Target gathers all the Activities that the visitor has qualified so far on that page, and passes it to Analytics as a server call.

And based on this understanding, I have concluded that the visitor is qualifying for the A/B test after the above said Analytics call had already completed. So, the visitor is not being counted under the test in Analytics.

The solution here, assuming that my conclusion is correct, is to fire a server call to Analytics when the visitor qualifies for the test, in order to "register" the visitor under the A/B test.

[colinr96046230 · Accepted solution]

Hey! We have run across similar circumstances in our build, and have configured a Direct Call Rule in DTM to pass an event to Analytics.

In our case, we do the following:

Run the Target test on the page (where you are running the XT currently)

Within both the test and control, build the event listener, and fire the DCR off the event listener. Off test, also make the changes when the listener fires.

As long as you don't have multiple tests needing this firing simultaneously, then you can reuse this event for different tests.

I would also strongly suggest building an mbox refresh in DTM that fires on new views such as this.

[bharath_sampath]

Thank you for sharing this Colin. I think this approach might work out. I will definitely pass this on to my team and update our findings here.

Cheers!

UPDATE: The above approach worked out and we are able to track visitors correctly. Thanks a lot!