Struggling To Understand Why Android Unique Visitors & Visits Are So Much Higher Than Google Analytics' user & session metrics

[kosullivan_]

We have implemented Adobe Analytics in my organization and we are currently in the process of rolling it out across the business and using it as our source of record for all things Analytics. One issue that has been raised though is that our overall traffic in Adobe Analytics is much higher than what we see in Google Analytics. We have found that the discrepancy is being driven mainly from our mobile application traffic, and solely within the Android operating system. While I understand that the collection methods differ slightly and there will always be a difference amongst platforms, I am trying to understand why that difference is SO large for the Android platform

I took a look at a day's worth of data for Monday 7/31/23. For iOS mobile app traffic only -- I see users (unique visitors in adobe) and sessions (visits in adobe) are well within our tolerable delta with Adobe +3% and +5% on unique visitors & visits respectively:

Date	Platform	OS	Unique Visitors (Users)	Visits (Sessions)
7/31/2023	Adobe	iOS	274,459	380,860
7/31/2023	Google	iOS	266,965	361,438
∆	3%	5%

However, for this same day's worth of data -- on Android -- we see that Adobe is exponentially larger across these two metrics:

Date	Platform	OS	Unique Visitors (Users)	Visits (Sessions)
7/31/2023	Adobe	Android	247,949	381,109
7/31/2023	Google	Android	104,292	144,043
∆	138%	165%

I have consulted with my mobile application dev partner who led the implementation of the Adobe Mobile SDK on our app. They have not come back with anything concrete yet, so I was hoping I could share this with the community and see if anyone had any reasons as to why we are seeing such higher traffic in Adobe for only Android devices?

Any help and/or insights here would be greatly appreciative. I have been banging my head against the wall on this one for a while and genuinely interested to find out why this is the case. Is it real traffic that Google had not been collecting properly? Anything will help us out tremendously at this point. 

Thank you for your time.  

[Jennifer_Dungan]

Without getting into the whole GA vs AA comparison... I do have point out that for Mobile Apps, your GA data is reliant on Firebase.. and as much as we sometimes throw stones at GA.... I sometimes want to throw a boulder at Firebase.

Firebase is pretty terrible.. and if you are relying on the "default" Firebase events, I'm going to say good luck getting anything usable...

To put this in perspective.. I've worked on a few app code bases with the developers... One one of the code bases (covering 7 or 8 apps) the default Firebase tracking significantly under counted the page views, I wouldn't be surprised if it missed visits or visitors entirely.. especially for short sessions.  The other code base (covering 3 apps) significantly over counted the page views....

And how do I know this? Well I used the Firebase debugger and I could see many pages failing to track at all (though this is also a known issue of the Firebase debugger... if you don't see the event, it might have still sent... the debugger sometimes misses showing entire actions)...but when I say events were missing.. they were missing all over the place... and even when I checked the Dev suite the next day (giving time for the data to process, the hits were much lower than what I tested... and there were other people testing I hadn't even accounted for yet.....  and on the other app, I wasn't even doing anything and it was showing page views (because the automatic event detection decided that some of the content loading into the current screen "constituted a whole new page view" o_0

Suffice it to say, I don't trust Firebase's automatic tracking at all....

If you want to improve your mobile apps GA tracking, I suggest turning off Firebase's automatic tracking, and have your developers trigger page view, etc actions manually... this will also allow you to pass Page Names and Parameters into your tracking to get more meaningful information then "XXXViewController" and other garbage values.... 

However, even with manual triggers, there is still a chance that the values won't match up... GA and Adobe calculate their visits/sessions differently... it's more important that they follow similar trends and are generally within the same reach of one another... 

[kosullivan_]

Hi Jennifer, 

Thanks so much for the thoughtful response! I was pumped to actually see someone reply since this is an issue I have been grappling with for the last few weeks as we get ready to transition our business entirely over to Adobe for all of our analytics reporting. 

I was just puzzled because when I look at app data for just iOS devices -- I can see that things are matching up near perfectly and well within my tolerable delta. It is just on the Android side where I see this huge discrepancy between Adobe and Google analytics platforms. It very well could be that we are now just tracking things more accurately in Adobe and have been missing a good chunk of our Android data in Google Analytics before we made this switch -- I just don't know this as a certainty and need to be able to explain why we now see much higher traffic for mobile app all of the sudden. 

[kosullivan_]

Marked my own answer here as 'resolved' by mistake!

Thanks again for your input here. I am just trying to figure out how I can prove that my unique visitor and visits counts are correct specifically for Android devices on the mobile app in our Adobe Analytics workspace. Is there any approaches you might suggest for that?

Thanks again!

[Jennifer_Dungan · Accepted solution]

Well I'm sticking around to keep helping the issue... 

For Adobe, if you have AEP Assurance Extension installed, that is a great tool for testing your implementations. If it's not installed, I highly recommend it... this is a game changer for testing.

If you have never seen that tool, we did a session on it in our User Group, you can watch the recording here:
https://www.youtube.com/watch?v=9jgNm_cggLM

Unfortunately, as I mentioned above, GA is a pain to test!

Now, for Adobe, with the AEP Assurance extension, you will be able to see the unique user ECID value (MID / mcvisid - on all hits), as well as the Launch number (on lifecycle hits).

Also, keep in mind that how Adobe and GA record sessions may be different, and it's possible you have differences in your settings between your 2 OS for your Lifecycle settings... or maybe they are the same, but could be very different from how GA records the session... this may simply be a matter of a timeout setting?

Adobe allows for a lot of customization in the mobile app, from configuring your Lifecycle metrics, to session timeout, etc.

So while I can't really help much on debugging GA (you are at the mercy of the Firebase debugger there), Adobe should be very easy to test and make sure you are seeing the behaviours you want/expect to see.

[kosullivan_]

Thank you so much! I think it lies somewhere with the lifecycle events on the Android devices. I will need to step through an Assurance session on an Android device to see what is going on. 

Thanks for your continued support!

[JohnMa9]

I know this is a few months old, but were you able to solve this?  We are seeing a huge number of visits coming from the custom event ADBInternal: Lifecycle that mostly are collected at about the same time each day (about 1/2 of them between 5-7 AM) that do not appear to be actual customer visits (No evidence the customers opened the app in other analytics platform and would be very odd behavior for customers).  Additionally, these visits appear to have no other tag fired outside of the ADBInternal: Lifecycle - but it is doubling our visit count compared to other packages.

[Jennifer_Dungan]

Are you using In-App Messaging on your Apps? We had an issue with our vendor that caused the In App Messaging to track a Visit (even if the users weren't actively in the app)... so in the morning, approximately in the time range you have (when our Editorial Team scheduled an In-App Message for the day to send), we would get a huge spike in traffic...

We never actually got to investigate this, since we are changing our vendor I will be testing if we still have the issue.

[JohnMa9]

Thank you.  I was thinking it might have been our push notifications, but the timing didn't line up from what the team said they were sending.  Because there are no other tags associated, it is not clear.  It records a visit as well as a crash in every instance though (and we know our app is not crashing - and that the customers are not visiting).  We have been trying to solve this since we implemented the ANDROID app in July

[Jennifer_Dungan]

It certainly looks like something is triggering the tracking... pushed and in-app messaging are the most likely scenarios... but not necessarily the only ones...

Did you developers build in any sort of log collection in your app? Maybe they can see something happening at that time that could help your investigation?

[kosullivan_]

Hi Folks, 

Love that these threads remain active! Not sure if this is exactly related to your specific issue, but what we found was happening was indeed related to our Push vendor, Leanplum.

Essentially what was happening is that they had a daily job running to poll for all known devices with the app downloaded (to measure app un-installs, total audience size, etc). They would send a 'silent push' to all known users and see the % of responses returned. This was triggering a background lifecycle event for us in Adobe. We had our developers add some logic to prevent this lifecycle event from being triggered and we saw the number level out. 

Hope this helps!

[Jennifer_Dungan]

That could be a similar issue to this.. so not a "scheduled push" by the team, but a vendor specific "silent push"...  I will keep that in mind with our new vendor to make sure that if we have that, we can identify and ignore as well.

[JohnMa9]

Thank you both!  We are currently implementing a pause to fire when the app is backgrounded with Adobe feedback being that if the app is left with tracking unpaused, a push notification COULD launch a session and behave exactly as we are seeing it.  We have used Leanplum in the past as well, so I am feeding this to our developers.  Thank you so much!  I'll let you know if we solve it.

[Jennifer_Dungan]

We used Kumulos, and we did have the Pause Lifecycle when the app is in the background, but our in-app messages were still causing issues for us... it might be different with your vendor.

Just letting you know that the Pause may not fully solve the issue.

Good Luck!

[JohnMa9]

OK.  It looks like it is likely Localytics as they send a push around 6am each day to tell if the user is uninstalled or not.

@kosullivan_ do you know (or are you able to give a hint) as to how your developers were able to stop this from triggering?

Thanks for all the help!!

[Jennifer_Dungan]

Good catch on the Localytics push!

I too am interested, and I suspect others who find this thread in the future might find that useful. If your developers will provide information on their solution that would be wonderful @kosullivan_ 

[kosullivan_]

Hey @Jennifer_Dungan & @JohnMa9 so sorry for the delay here -- been a crazy week. 

Looking back through release notes and checking with my team. Will be sure to follow up here with what was done when I have the chance. 

Thanks!

[Jennifer_Dungan]

No worries @kosullivan_ I know that feeling all too well. I will be here when you are ready

[JohnMa9]

Appreciate you looking into it.  We are trying a few things, but unsure of the success level as of yet.  Just hoping to skip some steps if you have solved it - and like @Jennifer_Dungan said, future users will appreciate it as well.  I was VERY happy to have tripped over this discussion as it showed it wasn't something we may have done wrong in implementing.