Content Sync between 2 or more Publishers

[rajkumark10]

Hi,

I am trying to synchronize the content with the help of Sling Distribution. I have followed below URL for the same.

User Synchronization

I have followed the same steps. After configuration i am able to sync the users and groups created in Publisher, but not able to sync the content (page, assets etc) between publishers. I have searched for examples and no luck. Kindly let me know is there way to do it in same way or we need to go for replication agents. If there are other ways also, Please share.

[smacdonald2008]

Using Rep agents is the best way to move content as discussed here -- Replication

[Gaurav-Behl]

If your use case is to sync publish servers without author involvement, then there are couple of options available based on your aem version and other requirements:

1) Use packagemanager and do it manually for each content tree.

2) Use grabbit, rysnc, crx2oak, rcp etc.

links-

http://jackrabbit.apache.org/filevault/rcp.html

https://helpx.adobe.com/experience-manager/6-3/sites/deploying/using/using-crx2oak.html

https://helpx.adobe.com/experience-manager/kb/how-to-clone-an-AEM-TarMK-instance-AEM.html

http://adamcin.net/net.adamcin.recap/

https://github.com/TWCable/grabbit

I've read that there are ways to use sling distribution for content as well but I don't think that's the principal use case of this feature.

[rajkumark10]

Hi Gaurav,

Thanks for your reply, We are using 6.4 version. I will check the URL you have shared. 

Di Donald,

Thanks, Till now we were trying to achieve the same using Replication, But we are looking for other options as Gaurav mentioned.

[rajkumark10]

Hi,

I have checked the URLs you have shared, I can see these are used for Migration of content.

But our requirement is below

Getting asset as input from user end and will place the  asset in One publisher, That data need to be sync with all other publisher. It will not be an single time process. Whenever user upload, The synchronization need to be done between publishers.

[Gaurav-Behl]

The mentioned utilities would work for assets as well, check - Assets Migration Guide

Couple of things to keep in mind are

- size of content/assets to sync

- if sync is a periodic/trigger based activity vs a one-time activity

- performance overheads of the solution that you pick

- maintenance/cleanup of assets

Is there any specific reason to abandon usual replication from author to publish and enable publish-sync feature? Would the user still upload assets on author or directly on publish server?

Have you checked binaryless replication where you can use a shared datastore to reduce replication/transport overheads?

Assets Offloading Best Practices

[rajkumark10]

Hi,

Yes, Users will upload the files in front end and that will be placed in Any one publisher instance (It will not move to author).

Can you please let me know which way is the best one to proceed further.

1. Do reverse replication and then move the data from author to all publisher (But in this case, the modified data will change in all publisher, when replication take place. We dont want that to happen)

2. Do replications between publishers (Data will affect when the replication from author instance triggered for same path, since the asset will not be available in author)

These are the ways we planned and the blockers we have. That is the reason we are looking for new approach to proceed further with this sync.

[Gaurav-Behl · Accepted solution]

This is a use case of user generated content (assets uploaded by end-user in your case) and I would recommend SRPs to handle this.

check -

Recommended Deployments

SRP - Community Content Storage

How to design your architecture for UGC in AEM 6.1​ -- old link but good enough

http://aempodcast.com/2017/aem-resources/aem-ugc-different-approaches/#.XEb9klxKh3g  -- old link but good enough

The best way to move forward would depend on existing architecture and other constraints, if any. There are several things to keep in mind-

- Is existing author/publish setup a clustered based or a farm or mongo based?

- operational/maintenance cost/strategy etc

- end-to-end architecture and process flows

- traffic volume of uploaded assets/UGC

- performance aspects including sync related activities

- real-time vs non-real time sync, error scenarios, stale content issues, ongoing maintenance/deployments etc.

- BC/DR strategy

- application specific dependencies/use cases on UGC?

- It seems that you already have something up & running. You may tweak it to keep going otherwise build entire solution from scratch

[rajkumark10]

Thanks Gaurav for the update