How to reconcile Data feed with workspace report

[GabrielaNa1]

Hi,

I am coming back with another question regarding calculations from Data Feeds. 

I exported - for test - all columns for a certain report suite. The event.TSV has these rows, besides the "Instance of eVarxxx" and "Custom Event yyy":

• 1 Purchase
• 2 Product View
• 10 Cart Open
• 11 Checkout
• 12 Cart Add
• 13 Cart Remove
• 14 Cart View

In order to get Revenue and Orders I follow the instructions in Calculate metrics | Adobe Analytics

In the data feed I have extracted hits for July 16 and July 17. All rows in my extract have duplicate_purchase = 0.
The next step is to "include only rows where event_list contains the purchase event". According to the above list I am picking only the rows where post_event_list contains 1 (post_event_list like '1,%').

Now, if I count the rows, as indicated in the documentation, I get Orders=743 (total for both days).

Then I go in the AA workspace and create a free form table. I select Orders and Revenue and I get completely different numbers: 1237 orders for both days. The Revenue, calculated by parsing post_product_list for these orders, is obviously different too from the 162456 CAD in the image below.

Am I missing something? Is it possible that the data feed doesn't include everything?

Any suggestions will be appreciated.

Thanks.

[Jennifer_Dungan · Accepted solution]

How often are you sending data feeds? Hourly? Daily?

Do you have a delay on your feeds to ensure that all data is processed before the feed files are compiled and sent?

Do you have mobile app in your data? If so, do you allow for "offline data" from your apps? It's possible that data might be held in a queue and sent later, which could stitch data into your suite after the fact and result in the data potentially missing from your data feeds.... (it seems unlikely that this would be the case, given you need internet to make a purchase... I would think... )

I've never seen discrepancies like this in our data... but this is the second such issue I have seen on the forum in a few days... so I am starting to wonder if there is a bug....

[GabrielaNa1]

Thank you for your answer. Indeed, I reran the data feed jobs for the two days and now the data seems complete, and the metrics verify between my scripts and AA free form table. 

Indeed, I didn't have any delay set up in my jobs, so this why some data was missing.

However, this now raises another question: how much of a delay to provide? I can understand why the lack of a delay might have brought the 17th data incomplete, as it ran very quickly after the day ended. But the data for the first day, July 16, was downloaded on the 17th, sometimes around 1 PM, which means many hours after the day of 16th ended. I see that the max available delay is 120 minutes, which means that it would have solved nothing in my case, right?

Well, I believe it is a discussion I must have internally, to understand if there are processes that might bring data in AA late, and coordinate my data feed accordingly.

In any case, for now I am good, I can build my formula confident that I can use the front end to reconcile.

[Jennifer_Dungan]

120 mins seems to be the max, and it should account for the processing.. however, having offline mobile data could very much change that balance, since the user may not open the app to send the data for some time..

You are right that the data you downloaded should have been there... 

I am concerned though that I am seeing this issue from multiple sources.. I wonder if something is out of balance and needs to be investigated by Adobe.... 

Glad I could at least sort you out a bit... but as someone whose team uses the raw data feeds, I'm concerned for our data as well now.....