Heartbeat video and Pageviews

[andrewz_webmd]

I seem to be having the opposite problem from most that i read on here.

We implemented heartbeat and now we are "missing" the page view associated to the video in the datafeed.

Scenario:

User lands on a URL which contains both a text article and a video.

page view call fires (can be seen in the debugger and our internal logs)

heartbeat ms_5 call fires (also seen in debugger and logs)

Looking at the front end we can see there are page views being recorded. When i compare the results from the front end to what can be reported via the data feed the number are different.

I have the same number of TOTAL records but i have NO "page_event = 0" records.

After several calls with Adobe consultants they are suggesting a processing rule is the issue but that does not explain why we are not seeing any records that we know to be a page view in the data feed. They also explained that "page views" in the front end are ONLY looking at "post_pagename", if it is populated it counts a page view. That is different from the past 15 years of working with this data. page views were ALWAYS event = 0. Now i have 50, 51, 52 and no zeros.

Hopefully that was coherent, I can clarify any details.

Chasing this issue for nearly 2 years, can anyone help me out here? 

[Jennifer_Dungan]

Wow.. that is an interesting question.. I think I understand what you are explaining.. but I am a bit at a loss as to how you are seeing the PV call in your debugger, but not seeing it in the report / raw data feed.... 

I'm not actually using the video heartbeat tracking, but I would love to help you solve this issue if I can (and hopefully someone using that will also join in).

Not sure if you feel comfortable sharing a URL so we can see it in action (if you don't want it public, but you trust me, you can send it as a Private Message).

But this will really just confirm what you are telling us; but maybe I can see something else in the correlation data that maybe you can check on your side. To see all sides of this, you might need support who can deep dive into the data; but let's see if we can figure something out here first.

[andrewz_webmd]

Sure! 

Dont know if you have access to internal ticketing systems but here are some details:

CaseID: E-000647601 as a followup to Incident: 220119-000296. Both have been closed at this point.

You should be able to access the page via: https://www.staging.medscape.com/recap/962029?uac=176742HK

Some data comparisons between the GUI and data warehouse which actually show 2 different issues.

First: if there are 191 visitors and 191 visits how is it possible there is only 96 page views? (This is the missing page view records in question)

Second: In the data feed, there are no "page event 0's" - if there are no "page views" in the raw data, where are the page views in the front end coming from? 

I also have some downloaded cloud debugger sessions with pre/post processing columns if needed.

[Jennifer_Dungan]

Nope, definitely don't have access to the cases,... I'm just another user like you. I don't work for Adobe, I just have been recognized as a power user that can help people out.

However, you are right, this is strange... Do you by any chance have Processing Rules in places that could be changing the data on your page view event?

From what I see here, you are correct... the Page View is firing...  going to dig a bit more....

[andrewz_webmd]

We have been using Adobe since 2006, we have MANY processing rules however i can not identify one that would change the type of call being recorded.

Even if there was a rule changing it, the fact that there are a matching number of records being reported via the front end compared to the data warehouse makes me believe that page event = 0 is NOT the only type of record that is counted as a page view.

[Jennifer_Dungan]

I hear that! I was wondering if maybe the Page Name (which seems to be your current filter) was potentially be overwritten to something else..

I completely agree though, these is definitely still something really odd going on with the PVs, UVs, Vs that needs investigation.

Do you have someone on your Data Team who is working with the ETL process that can help investigate on their site (not to say it's all there), there seems to be a few issues... and they are interconnecting making this difficult to diagnose (we've all been there, its not a fun place to be)

[yuhuisg]

I haven't used Page Events very much myself, nor have I used Heartbeat metrics.

Having said that, could you be confusing Page Views with Page Events? https://experienceleague.adobe.com/docs/analytics/components/metrics/page-events.html?lang=en#compar...

[Jennifer_Dungan]

"Page Events" in this case is the the raw data column (nothing like using very similar names):

https://experienceleague.adobe.com/docs/analytics/export/analytics-data-feed/data-feed-contents/data...

PageViews should be there as 0 in the raw data, but they aren't... however the debugger clearly shows that the page view has been sent... which is why I am wondering if there's a processing rule maybe changing the page name (which is currently being used as the filter).

However, since this data has obviously gone through an ETL process...  (those aren't the standard column names), the other possibility is that the ETL has stripped it out....

I also don't use the Video Heartbeat events, but it really wouldn't make sense for that function to prevent the PV from appearing in the report.  Though looking at the Workspace again, we can see the PVs... so I am more convinced the ETL is causing some of the issue... however, for the Media Stream Start / Media Stream Close; Those haven't been pulled in the Workspace, and @yuhuisg is correct, I don't think you should be using Page Events like this in the Workspace.....

[andrewz_webmd]

Agreed BUT in the workspace only 96 page views are reported from 191 visitors and visits. Which indicates this issue is intermittent. I can identify the 96 records which are being counted as a page view and they are page event 50 which i know to be "Heartbeat Video: Media stream start" And the only reason i can identify them is because we do have custom events for tracking PVC on our sponsored pages (post_event_list) which is the only thing that made them different than the others.

[Jennifer_Dungan]

Yeah, Definitely not giving up on this.. but just out of curiosity... do you need the default heartbeat tracking?

I track videos, but I just fire custom tracking calls on start, pause, complete, 10%, 25%, 50%, 75% and 90% mark... which looks very close to what you are doing...  However, I just don't use Adobe's media plugin... 

I just created some custom events and eVars (start, pause, progress, and complete), and some eVars to hold the video title, id, progress, stuff like that.. and triggered custom tracking for all the times I wanted to track.

It seems like you are doing something similar to those breakdowns.. and if the default isn't working, maybe try your own thing? I thought the real power of the heartbeat (and I could be wrong) was to send a heartbeat at regular intervals (every 5 or 10 seconds of the video) and since you aren't doing this; maybe you would be better off with doing your own thing?

[andrewz_webmd]

We are indeed sending the default 5-10 sec heartbeat ping and we are using the time spent values that are being returned. There are a bunch of additional ad hoc calls (i.e. start/play etc) because we have not been able to rely on the heartbeat data. Ideally we would love to get rid of all the ad hoc video progress calls and only use HB is we can get it working correctly.

[Jennifer_Dungan]

Oh... I didn't see any of those calls when I was looking the other day.. but it was a dev environment so maybe there were changes being made.... any luck from support yet? I will continue to investigate from here.

[VaniBhemarasetty]

@andrewz_webmd I see that you have raised client care, which seems to have been resolved by our team. please reopen from the admin console if you need more assistance so that we can assist you on this further

[Jennifer_Dungan]

Still, @andrewz_webmd if there are any details that could potentially help other users, please share here! I am certainly interested. 

[andrewz_webmd]

Still digging into the data on our end. We have not been able to identify the issue. Again same number of records BE vs FE but no "page event 0's" in the BE data.

[andrewz_webmd]

This ticket has been opened, closed and reopened several times. Absolutely not resolved.

Its basically ME looking at it and running tests so if i take more than 48hrs to respond to a ticket it gets closed.

[Jennifer_Dungan]

Ah... okay, I will try and find some time to help take another look!

[andrewz_webmd]

Considering we collectively think there must be a processing rule at fault here, i added another processing rule which sets page name. 

Now we have double the number of page views as expected via the front end and still only 1 "page event = 0" records in the data warehouse.... 

All the suggestions from the Adobe consultants and ideas found here have provided zero results. Both myself and dev team as at a loss on what to look at next.

[andrewz_webmd · Accepted solution]

@Jennifer_Dungan and anyone else that might be following along here....

After exhaustive investigations between myself, my Data Management team and several calls with Adobe folks we have figured it out! 

In short we were wrong  More specifically the logic we have been using for 14+ years against the datafeed was incorrect.

We always used the following to count page views in our warehouse: 

sum(case when page_event_Cd = 0 then 1 else 0 end) as PAGEVIEWS

Turns out while all page event 0's are indeed page views they are not the only records which are counted as page views. According to Adobe documentation the correct way to count page views is "Count the number of rows where a value is in post_pagename or post_page_url." With no mention of page event column.

Did some comparisons and sure enough there are plenty of rows in the datafeed that are page event 50 (Media stream start (non-Primetime)) but the post_pagename field is populated. Including these rows our reports from the warehouse are matching to the front end!! 

TLDR - If you are using the data feed in a warehouse, you might want to check your calculations to not use page event.

[Jennifer_Dungan]

WOW! Thanks for letting us know... The data feed issue (with the page_event) is likely more impacted when using something like the Media Plugin... but this is really good to know. 

I will let our data team know that this could be an issue in their processing (but we don't use the Media plugin, we just did our own custom media events)