Expand my Community achievements bar.

July 31st AEM Gems Webinar: Elevate your AEM development to master the integration of private GitHub repositories within AEM Cloud Manager.
SOLVED

Continuously synchronise between contents in dev and prod environments

Avatar

Level 2

I need to move all contents from AES development to production environments. To do this, I used package manager. The problem was that some folders have large sizes, so I had to build each subfolder and upload them to the development branch. Additionally, I need to copy all changes made in the future from development to production. It is not possible to accomplish this with package manager. Please suggest some appropriate mechanism for achieving this.

1 Accepted Solution

Avatar

Correct answer by
Employee Advisor

Hi @Anas_R!

There are multiple tools for content transfer between AEM environments available. Have you already seen the following community article?

Automated Content Sync

 

Depending on your exact requirements, you might also want to look at the following list of tools for content transfers:

 

Please note that not all tools will work with AEM Cloud Services and some might be a bit outdated.

 

Hope this helps!

View solution in original post

14 Replies

Avatar

Correct answer by
Employee Advisor

Hi @Anas_R!

There are multiple tools for content transfer between AEM environments available. Have you already seen the following community article?

Automated Content Sync

 

Depending on your exact requirements, you might also want to look at the following list of tools for content transfers:

 

Please note that not all tools will work with AEM Cloud Services and some might be a bit outdated.

 

Hope this helps!

Avatar

Community Advisor

@Anas_R . In addition to @markus_bulla_adobe suggestions, please refer to https://adapt.to/2023/schedule/lightning-talks/acs-content-sync 

https://adobe-consulting-services.github.io/acs-aem-commons/features/contentsync/index.html

 

It incrementally synchronizes content between AEM Author instances

 

 

The comparison between various tools is available on: https://adapt.to/2020/schedule/content-sync-automation

 


Aanchal Sikka

Avatar

Level 2

@aanchal-sikka Tried ACS Content Sync tool as mentioned in the article. Configured the source environment with the url, user name and password, gave the path to synchronise and then on clicking 'Submit' button, I got an error

building catalog from https://author-p42404-e166196.adobeaemcloud.com/apps/acs-commons/content/contentsync/jcr:content/sync.catalog.json?root=%2Fcontent%2Fportfolio&strategy=com.adobe.acs.commons.contentsync.impl.LastModifiedStrategy
java.io.IOException: Failed to fetch data from https://author-p42404-e166196.adobeaemcloud.com/apps/acs-commons/content/contentsync/jcr:content/sync.catalog.json?root=%2Fcontent%2Fportfolio&strategy=com.adobe.acs.commons.contentsync.impl.LastModifiedStrategy, HTTP [401]

Could you please tell me what is the issue?

Avatar

Employee Advisor

Hi @Anas_R!

The error message clearly mentions an HTTP 401 Unauthorized status code when the tool is trying to access your Cloud Service environment.

 

There are different things to try to get to the root cause:

  • Please double check on the credentials provided as well as permissions granted for the account you are using for the content sync. Have you tried a manual login with these credentials? Have you tried accessing the path mentioned in the error message via browser?
  • Have you checked the log files on the instance where the sync tool is running for additional information? Are there other errors, stack traces of relevant log messages to be found that can help to guide you to the root cause?
  • Have you also checked the log files (including access.log and request.log) on the target Cloud instance?

 

Hope this helps!

Avatar

Level 2

@markus_bulla_adobe @aanchal-sikka 

  • Tried a manual login with these credentials? Yes, that was fine
  • Tried accessing the path mentioned in the error message via browser? Yes, I was getting the required json response from this.
  • Checked the log files on the instance where the sync tool is running for additional information? I couldn't find any log file from this tool. Only log I got is from the activity log area. That was a stack trace which I mentioned in the question.
 
  • What is access.log and request.log ? Where can I find it? Do we need access to system/console to get these log files ?

Avatar

Employee Advisor

@Anas_R  schrieb:
  • Tried a manual login with these credentials? Yes, that was fine
  • Tried accessing the path mentioned in the error message via browser? Yes, I was getting the required json response from this.

One more check would be to try a call from the source AEM host, e. g. logging in to the command line and using curl, wget or similar. That way you could verify if there are any components on network level that might be blocking the call (e. g. because the target is only accepting requests from a dedicated list/range of IP addresses or similar restrictions). There could also be a WAF in place that might block the request based on certain criteria (including IP ranges, user agents and others).

@Anas_R  schrieb:
  • Checked the log files on the instance where the sync tool is running for additional information? I couldn't find any log file from this tool. Only log I got is from the activity log area. That was a stack trace which I mentioned in the question.

You could try and setup a dedicated logger (via OSGI web console as explained here), ideally with a DEBUG or even TRACE log level for the relevant java classes of the tool. Apart from that, I recommend to check the error.log of the instance to see if there is anything interesting.

 


@Anas_R  schrieb:
  • What is access.log and request.log ? Where can I find it? Do we need access to system/console to get these log files ?

Your target environment does seem to be a Cloud Service setup. In that case you won't have direct access to neither system console nor the file system where log files are stored. You need to go through Cloud Manager and Developer Console or use the Adobe I/O CLI Cloud Manager plugin as explained in this article. This documentation provides detailed information on available log files and log levels for AEM Cloud Services.

 

Just to double check: What AEM versions/flavors are you working with?

The instance running the sync tool is AEM 6.5? Is that self hosted or AMS based?

The instance that you are trying to reach is AEM Cloud Service?

Avatar

Level 2

@markus_bulla_adobe I was not able to get enough details from the logs. I am using AEM cloud service and my target environment also. Version is 2023.11.14227.

Avatar

Employee Advisor

Have you been able to add a logger for the sync tool and increase the log level to DEBUG?

Avatar

Level 2

No @markus_bulla_adobe . Will check that and get back to you. Thanks for your responses.

Avatar

Level 3

@markus_bulla_adobe @aanchal-sikka  Could you please elaborate on what to do to add a Custom log file for this end?

Avatar

Employee

In AEM CS it is not possible to add custom log files but you can add a specific logger for the relevant classed/packages that will the log into the standard log files.

 

The following documentation outlines the process:

 

https://experienceleague.adobe.com/docs/experience-manager-cloud-service/content/implementing/develo...

 

You will need to add a logger OSGI configuration and deploy it to your instance.

Avatar

Level 3

If I want to add the dependency acs-aem-commons-6.3.2 how would the log file look like?

 

 

Avatar

Level 6

Hi @Anas_R ,

We are also facing similar issue. Was you able to resolve the same and what was the issue ?

 

Thanks,

Somen