Classification data to Adobe Analytics data Feed

[#1]

Description -

1. AWS Web Analytics product team uploads AWS blog meta data via classification importer feature in Adobe Analytics (AA). Also we use Classification rule builder feature to automatically classify AWS product pages meta data in AA reporting. Unfortunately these classification data can't be sent to downstream AWS s3 location via Adobe Analytics data feed feature. We have a downstream bigger data lake system where we send hourly data feed from Adobe Analytics. Every time my team works on the classification addition or modification needs to be manually worked by AWS downstream data engineering team to add the same details in their database to bring parity to AA and other system reporting. This is time consuming and repetitive task.

2. There are limited eVars (250) available for Adobe Analytics. Classification is a great feature to separate out concatenated data being passed to an eVar and store individual values in different coulmns associated with that eVar. This feature saves number of eVars to be used for this kind of implementation. Unfortunately AWS Web Analytics team can't utilize this feature because we send hourly data feed to downstream data lake where classification data can't be sent.

How would you like the feature to work - Classification columns (eVars columns) needs to be included in data feed eVar list.

Why is this feature important to you - This feature is important because this will save significant time for AWS data engineering team and also reporting will be in sync automatically.

[#2]

This would be an awesome feature, but I don't see how it would / could work...

Classification data first off takes time to process, it's not immediately (newly files can take up to 24 hours to process, Classification Rules process data every 4-6 hours, I would expect the imported rules to be a similar latency), so I doubt that processing would be done in time for the Data Feed to run successfully with that information included. Secondly, classification data isn't permanent... it can be re-processed for historical content (needs change, a mistake was made, etc)...

Data Feeds are essentially a one-time export, normally destined for in house Data Lakes, pulling and mapping data from the source is a more standard practice. 

While I understand that it would be nice to have synced data, logistically I don't think it can work like this.

[#3]

Hello,

thanks for providing our valuable input. There is an option to set data feed interval/ frequency (hourly/ daily). currently we have hourly data feed set up. I agree with you that classification data doesn't show in the report instantly due to processing time. But one good thing about classification is that there is no data loss and it is applied retroactively. So technically if we send the classification data to data feed at some of of time data lake reporting should match with AA. Stakeholders needs to be educated with this expectation and solution will bring lots of value to the customers because of removing duplicate effort.

Best Regards,

[#4]

While we are uploading the classification file to AA via importer, we can also put that classification file into the mentioned bigger data lake and augment the data there directly to create the same result.

[#5]

Yes, AWS team following this process, but this is duplication of effort and needs coordination with two teams on timely manner.Ideally one system should be source of data and other should consume it automatically. Also if we can send classification data from AA to downstream system this will save lots of eVars for future custom implementation purpose.

[#6]

Right, but the issue still remains that Classification Rules process every 4-6 hours... the max delay you can add to a Data Feed is 2 hours.... I just don't see how you could possibly get all the data unless you are extremely lucky, and your last "daily" processing happens just after midnight, allowing your classifications to process, and your data feed to send after everything is processed...

However, there is no inherit differences between an hourly or daily feed (in what is available), and adding in fields that won't exist in many cases is going to lead to confusion for a lot of customers. 

While I get that it would improve your process (assuming it even works out based on when things run, which isn't even guaranteed), I just don't see this being something that Adobe is going to put a lot of focus on... particularly since classification data can be updated at any time and affect data going back 6 months.... 

Raw Data is the data as it was collected...data that doesn't change (or at least not without significant effort like the Data Repair API). This is fundamentally different than classifications which can change on a whim... 

However, neither @leocwlau nor I work for Adobe... the Adobe product team might decide to look into this.. I am just warning you that it seems unlikely and to not get your hopes up.... A lot of ideas get posted here, many don't make it to the roadmap.