Content Migration from higher to lower environments (AEMaaCS)

[ahegde-1]

Hi all,
I have a java service to create a package everyday on the basis of given query, this runs in PROD environment and the package is getting created (there is a scheduler to trigger this service everyday at 1 AM). 
I want to download the same package from PROD, in DEV environment. As all the environments are AEMaaCS tried to download,upload and install the packages using java is seeming impossible. I have tried to run curl command using java itself to download, upload and install the package. It runs fine in local and package is getting installed in local server. But the same code to download the package is getting EXIT CODE : 23 in DEV.

Here is the code snippet of curl command runner using java to download the package

// Download page package from remote env
            ProcessBuilder pageDownloadProcessBuilder = new ProcessBuilder("bash", "-c",
                    String.format("curl -u %s:%s %s%s --output %s", queryUser, QUERY_PASSWORD, authorUrl, pagePackagePath, PAGE_PACKAGE_NAME));
            Process pageDownloadProcess = pageDownloadProcessBuilder.start();
            pageDownloadProcess.waitFor();
            int pageDownloadExitCode = pageDownloadProcess.exitValue();
            logger.info("pageDownload Exit Code : {}", pageDownloadExitCode);
            pageDownloadProcess.destroy();

the actual curl command runs is : 

curl -u user_name:user_pwd https://<prod_url>/etc/packages/my_packages/myPackage.zip --output myPackage.zip

error :

2023-05-22 12:00:00.164 INFO [com.myproject.core.services.impl.CurlJavaServiceImpl] pageDownload Exit Code : 23

Any input would be helpful. Thanks in advance.

[nitesh_kumar]

Hi @ahegde-1  ,

Check if this local user exists on the instance. Also, Have you tried with the access token?

curl -H "Authorization: Bearer <access-token-here>" -X

https://experienceleague.adobe.com/docs/experience-manager-cloud-service/content/implementing/develo...  

https://experienceleague.adobe.com/docs/experience-manager-learn/getting-started-with-aem-headless/a... 

FYI, the content copy feature is now available on AEM Cloud, do check it out if not already.

https://experienceleague.adobe.com/docs/experience-manager-cloud-service/content/implementing/develo... 

Regards,

Nitesh

[arunpatidar]

Download (the package)

curl -u admin:admin http://localhost:4505/etc/packages/export/name_of_package.zip > name of local package file

Upload a new package

curl -u admin:admin -F package=@"name_of_package.zip" http://localhost:4505/crx/packmgr/service/.json/?cmd=upload

Upload a package AND install

curl -u admin:admin -F file=@"name of zip file" -F name="name of package" -F force=true -F install=true http://localhost:4505/crx/packmgr/service.jsp

Upload a package DO NOT install

curl -u admin:admin -F file=@"name of zip file" -F name="name of package" -F force=true -F install=false http://localhost:4505/crx/packmgr/service.jsp

https://gist.github.com/anupammaiti/8c8a9c4f54018c27331f365313f3fc01 

[Shiv_Prakash_Patel]

Hi @ahegde-1 ,

Please check this article for content migration between AEM instance - https://unlocklearning.in/package-migration-via-approval-workflow-process/

Regards,

Shiv

[aanchal-sikka]

Hello @ahegde-1 

The following command works for us on AEM as a Cloud Service:

Download:

curl  -H "Authorization: Bearer <access-token-here>" http://localhost:4502/etc/packages/Content_Sync/test14_28042023130839.zip --output ../downloads/test14_28042023130839.zip

Upload & Install

curl -H "Authorization: Bearer <access-token-here>" -F file=@./test14_28042023130839.zip -F name=test14_28042023130839 -F force=true -F install=true http://localhost:4502/crx/packmgr/service.jsp

We have created Service Credentials via Developer Console in Cloud Manager and using the access tokens.

The service user should have read access to /etc/packages to be able to download the packages from source instance

The service user should have write access to /etc/packages to be able to install the packages on target instance.

You can either generate new Token from Developer console or using library https://github.com/adobe/aemcs-api-client-lib 

[ahegde-1]

hi @aanchal-sikka thanks for the input.

but I am having confusion now, the curl command I am using is 

curl -u user_name:user_pwd https://<prod_url>/etc/packages/my_packages/myPackage.zip --output myPackage.zip

here user_name:user_pwd is the user in PROD instance having write/read access to /etc/packages. If I use the command you given, there you are not using the user created in PROD. Instead you are using "Authorization: Bearer <access-token-here>" , and using localhost URL. 

If I download a package from PROD in DEV, then where in the file system it will get downloaded? Because the package with same name downloads everyday with different filter paths and each time package get overrides by the new one in my local. I want to make sure today's package is replaced by the new package downloaded tomorrow. So that it won't cause unnecessary file storage in cloud. 

[aanchal-sikka]

Hello @ahegde-1 

Ideally one shouldn't overwrite packages in CRX, even if its a backup.

You should either increment the version number, or add a timestamp to the name of the package.

Could you possibly update the script thats creating the package?

[ahegde-1]

hello @aanchal-sikka , Im not referring over writing the package in CRX, I'm concerned about the package file downloaded in cloud file system.

Yes I can update the script which is creating the package, because its a java service i am trying to run curl command.

[DPrakashRaj]

You might not have permission to download the file at the location provided on cloud