A/B Testing Tool Integration - Data looks incorrect

[brendanj5177098]

Hi folks,

I'm trying to integrate an A/B testing tool, VWO (https://www.vwo.com) with my Adobe Analytics set up but I'm having problems.

I'm doing an A/A test where 50% of users see the control version and 50% of users see variation 1. This test is being carried out for all devices on my homepage.

I have an Adobe Analytics Workspace report to show the number of Unique Visitors who see Control and Variation 1. I'm using eVar101 for this. I have used a segment on this report to show only Unique Visitors who visited the homepage (Page Name = ES:SPA:Homepage). The problem I have is that there is a large number of Unique Visitors where eVar101 is set to 'Unspecified' and I don't understand why. Surely, if anyone visits the page, they should see the test and have eVar101 set to either Control or Variation 1?

I'm firing a second server call (Custom Link) on every page load of the page where the test is running. This server call doesn't fire on any other page. There's a screenshot below taken from the Adobe Analytics Debugger Chrome plugin.

Does anyone have any ideas what I can try to get this working?

Thanks in advance.

[nikitaf2673776]

Are you saying that when someone loads the home page they get one analytics server call, and then your second custom link one? So every time a user goes to the home page they get two analytics server calls?

[brendanj5177098]

Hi,

Yes, that’s correct. The second server call is the one shown in the screenshot.

[nikitaf2673776 · Accepted solution]

So when I visit your home page, I get one server call (hit) where Page Name = ES:SPA:Homepage and eVar 101 is not set. Then I get another one (your custom link one) where Page Name = ES:SPA:Homepage and eVar 101 is set to (let's say) Control.

Therefore I, as a unique visitor, have seen a hit with Page Name = ES:SPA:Homepage and eVar 101 is Control, and a hit where Page Name = ES:SPA:Homepage and eVar 101 is unspecified.

So in your table above, I would be in both the Unspecified and Control rows. Do you see how the unique visitors for Unspecified, Control and Same as Control add up to more than the total number of Unique Visitors?

You could either try to have your A/B tool fire before analytics, feed analytics the experience variable, and only fire one tracking call. Or you could filter your report for eVar 101 is not unspecified.

[brendanj5177098]

Thanks for that. Really interesting.

So following your logic, if I use another eVar and set it only with the custom link server call and not the first server call for say, ’VWO Page Name’ and use that in my report for the segment instead of Page Name, that should work, right?

Sounds like it could be worth trying.

Thanks for your help!

[nikitaf2673776]

That's one way of getting what you need sure. You could just as easily make some segments in workspace to clean up your reporting. I suggest:

• a hit-based segments for eVar 101 is not unspecified. For the table you screenshotted
• Either visit or unique visitor based segments for each value in eVar 101 to follow these users around and see how the different experiences impact your conversions.