Running AppMeasurement and WebSDK in parallel - data quality validation

[Franc_G]

Hi all, I have reached the stage where I want to do the data quality validation for the alloy.js (Web SDK) set up I have in place. My plan is quite simple:

1. Run both AppMeasurement and WebSDK in parallel in staging env. 
2. Review the top line metrics - pageViews, visits, unique visitors, searches and maybe a few metrics.
3. Do like for like comparison

I would really appreciate if people could share their experiences in terms of data quality validation.

Thanks,

Franc

[Jennifer_Dungan]

I assume when you say you want to run in parallel, you will be collecting to different suites in Adobe Analytics?

Or are you going to be running AA from AppMeasurement and CJA from WebSDK?

While I haven't switched to WebSDK yet, I was thinking about doing something like this from a Dev / QA environment perspective...  I am not sure about Prod though, because I would prefer not to lose the historical data of our main suite (and have to rebuild all the reports)...  Also, you should be aware that running parallel tracking could cause you to go over your server call usage....

I did a similar concession of "double tracking" a few years ago when I rolled one site from its tracking profile into our global tracking... I intended to run it for only a month or two (sending data to 2 suites)... but our data lake team didn't prioritize updating their reports for the new feed, so we ran like that for about a year.... and definitely went over our server budget.

[Franc_G]

@Jennifer_Dungan thanks for the input.

I am planning to send data into separate reporting suites. It is pretty straightforward in case of AEP - have set a new reporting suite for the web sdk data specifically. And yes, I plan to run the tracking codes in parallel only in staging env. Not brave enough to push it into PROD just yet :))) Sever usage is another consideration as well, but my plan is to capture only a few days worth of data - just to do the top line comparison.

[Jennifer_Dungan · Accepted solution]

Ah, ok,.. then yes, a staging environment should contain the cost of the server calls

If I were doing this... though I know the usage of Staging is different from Prod.. I would make two copies of some of the Key Reports (or maybe just duplicate the panels in one copy of the report)...  Set one up to the AppMeasurement staging suite, and the other to the WebSDK suite. 

This way, you can compare some core reporting in "all its glory" and make sure that the segments and calculations are all functioning as expected.

You might even want to pull some small samples of Raw Data and run a row by row comparison?

[Franc_G]

@Jennifer_Dungan thanks for the suggestion. At the moment I just want to do the smoke testing - to make sure that I can run both sets ups in parallel AND do the top line comparison. In all fairness, if it all looks okayish in Staging then, in theory, we could run the same set up in prod (with both appMeasurument and alloy in parallel). 

BUT, it would be a good practice to do as thorough validation as possible before it all goes into production. I am planning to include the analytics & insights team into this exercise once I have a data set of sufficient size. 3-4 pairs of eyes is always better than one pair    

[cserrurier]

Hello,

Before switched from AppMeasurement to Web SDK, i did a similar test : run Adobe Analytics to one reportSuite with AppMeasurement and to an another with Web SDK.
I made a workspace to compare all my metrics and did comparaison between the 2 reportSuites.
It's a big amount of work but it was very helpful to correct some issues and to be fully reassured before the big jump to the migration.
Don't forget that during the test, all your calls are doubled so the test would have a cost but in my experience, it was totally worth it.
Feel free to ask if you have any questions.

Best regards

[Franc_G]

@cserrurier yes, if I run it in staging for a few days or even a week, it shouldn't be a problem. If you push the set up into PROD, the server calls will snowball very quickly. Fully aware of that. 

I completely agree regarding discrepancies. Having a test data set at hand will highlight any deficiencies and will allow me to resolve those before going into prod. Thanks for the input.

[Jagpreet_Singh_]

Hi @Franc_G Did you find the suggestions helpful? Let us know if you need any further information! If the solution works for you, please mark the answer as correct to help others. And if you’ve found your own solution, we’d love for you to share it with the community. Thank you!