Best Solution to store Assets Audit trail and User Activity Stream in multiple publisher environment

[rajeev_yadav]

We are using AEM 6.1 & need capture Asset audit trail(View & downloads) & user activity stream for each asset in  publisher DAM. There are multiple publish instances with TAR MK. Various reports need to be published to logged in user in website. 

There are couple of ways that I can think of:

1. Store asset Audit trail & User Activity Streams in OAK repository on each publish instance.

2. Setup a separate shared database(MongoDB or MYSQL) to store this data and use the API code to generate reports from database data.

Which one is recommended approach by Adobe &  better  solution? In Option 1, what are the different ways to synchronize the data between multiple publish instances? As I understand, reverse replication is not supported anymore in 6.1 except for pages in UGC. Are there any performance implications in approach 1 or 2 due data size?

Suggestions are welcome and much appreciated. :)

[Jitendra_S_Toma · Accepted solution]

Rajeev,

I totally in support of your approaches which you have thought through. Let me highlight my perspective on each point.

1. Store asset Audit trail & User Activity Streams in OAK repository on each publish instance.

Ideally, This is a good approach and clustering solves the bigger problem with regards to syncing the publish instances but as you know Apache Oak does not support clustering and that's create whole problem. Anything updated in one publish instance can't be updated to another and user may not find correct data if user goes to another publish instance.

As you know, Above problem does happen in case of load balancing. In AEM, Reverse replication is an another option however reverse replication also has not been enabled in AEM 6.1 for all type of content.

2. Setup a separate shared database(MongoDB or MYSQL) to store this data and use the API code to generate reports from database data.

This is a good option to keeping the auditing & UGC content which can be store outside of AEM and every user activity can be performed in central repository.

From performance point of view, This would be right option to choose. However, This does have one cons which is that every user activity would be a webservice call and central repository must be available.

[rajeevy89244319]

Thank you Jitender. For approach 2, is it necessary to use SRP API and buy Community License? Again, if the data is stored in OAK then we have ActivityStream API to retrieve and populate data. I think this internally uses Lucene indexes which improves the performance. But if we are storing in Mongo, can ActivityStream API retrieve the data? Also,I believe we might have to setup another search engine(Solr) to index the data stored in Mongo?

[Jitendra_S_Toma]

rajeevy89244319 wrote...

Thank you Jitender. For approach 2, is it necessary to use SRP API and buy Community License? Again, if the data is stored in OAK then we have ActivityStream API to retrieve and populate data. I think this internally uses Lucene indexes which improves the performance. But if we are storing in Mongo, can ActivityStream API retrieve the data? Also,I believe we might have to setup another search engine(Solr) to index the data stored in Mongo?

With the given use cases, it is worth having distributed approach because reporting, indexing & searching all within AEM isn't going to be easy thing to manage. However, it is up to you. I prefer this way.

FYI

Lucene does not do indexing by default in AEM. You have to do indexing of your content by oak configuration.

[rajeevy89244319]

I understand that in OAK it doesnot index by default.. we need to create indexes.. but the good thing is that Lucene is already integrated and just requires creation of indexes. I agree distributed approach would be a better solution in this.