AWS libraries for S3 connector configs not resolved in AEM as a cloud service instance

[kiranparab]

Hi Team,

AEM TO S3 Blocker UseCase ->

1. AWS libraries for S3 connector configs not resolved in AEM as a cloud service instance

Environment – AEM As a cloud service

Issue –

Dependency not resolved –

We are using the AEM as a cloud service environment and we are finding issues in using the AWS SDK libraries using as a maven dependencies. We found the way to update the couple of configurations in AEM quickstart folder which is not possible in AEM as cloud service straight forward as we don't have access to unix box to update the same. Developer console also give access for Felix configs.

       Approach 1

       References -

       Configuring node stores and data stores in AEM 6 | Adobe Experience Manager

Failed as we are looking for system configs to get added and as per suggestion from Adobe Added the same configs in Runmode but this doesn’t support the technical usecase.

Approach 2

Reference –

AEM Project Structure | Adobe Experience Manager

Failed as AEM as a cloud service won’t allow to install any package under /libs as it is only having read access and needs to be build this package through bundle.

Approach 3

ZIP building for project codebase and deploy as a package in direct instance as Approach 2 failed.

Reference –

AEM Project Structure | Adobe Experience Manager

Failed as AEM as a cloud service won’t allow to install any package under /apps as it is only having read access and needs to be build this package through bundle.

Approach 4

ZIP building from project codebase as Approach 2 and 3 failed.

Reference –

1. AEM Cloud Connector Installation and Configuration – Smartling Help Center

2. Maven Repository: com.amazonaws » aws-java-sdk-s3 (mvnrepository.com)

This will allow to add S3 connector zip from project bundle as per given reference. But that way of implementation also didn’t work as it is giving same error as,

14:00:46,076 [main] [ERROR] Error resolving dependencies for project com.pacificlifeinsurancecompanydam:aem-pacificlifeinsurancecompanydam-project.all

14:00:46,077 [main] [ERROR] Failed to collect dependencies at com.adobe.granite:com.adobe.granite.oak.s3connector:zip:1.10.10

Failed as a given reference of updating embedded packages for Vendor packagesof s3connector, adding s3 depdencies and <allowIndexDefinitions>true</allowIndexDefinitions> still not worked.

Approach 5

Creating the non-osgi aws-java-sdk-osgi bundle to osgi compatible manner. Added the dependencies but still does not work.

Failed as loads of depdencies not resolved with the osgi package in felix.

Thanks and Regards,
Kiran Parab

[Shashi_Mulugu]

@kiranparab I think you can refer to ACS commons code on how they have done the dependency setup to be compatible with OSGI, though their S3 connectors not compatible with AEM Cloud, baseline code deploys and resolves in AEM cloud as well.

[Vijayalakshmi_S]

Hi @kiranparab,

Could you please let know your use case with respect to AWS S3 in AEMasCS instance. Are you making use of the same/its functionality as part of your code base (core module) ?

[kiranparab]

Yes we are using AWS SDK to use in our codebase to move AEM to S3 assets and JSON metadata. We are finding issue when adding the respective depdencies like,

Maven Repository: com.adobe.granite » com.adobe.granite.oak.s3connector » 1.10.10 (mvnrepository.com... and https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-osgi/1.12.13

[Vijayalakshmi_S]

Hi @kiranparab,

https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-osgi/1.12.13 is AWS Java SDK with support for OSGi. It is an OSGi bundle that can be installed directly in Felix but when tried in my local AEM 6.5.0 instance, some of its run time dependencies are not resolved. 

So in order to resolve this, 

• If you are using com.amazonaws.* family (AWS Java SDK V1), you can directly use https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-s3/1.12.14
• If you are using maven-bundle-plugin for building your core module, we can solve it using Embed-Dependency entry within.
• But if you are using bnd-maven-plugin (which is the case with latest AEM Maven Archetype projects), we need to build and install as separate OSGi bundle. But not sure if it is allowed in the context of AEM as Cloud Service instance.
  • (At organization level, you can have/maintain separate maven project for building third party dependency as single bundle. However, I suggest to have a word with your project architect and Adobe CSE for streamlining the process)

If you would like to try in your local AEMasCS SDK instance, 

• Steps to build and install a maven dependency as OSGi bundle:
  • https://myaemlearnings.blogspot.com/2021/02/createbuild-and-install-osgi-bundle-of.html
• Recently, I have used AWS S3 Java SDK V2(software.amazon.awssdk.* family) in my code base and used below pom.xml for creating OSGi bundle. (Download and amend the pom.xml. It has few other third party dependencies too, you can remove everything and add your desired dependencies and respective Export-Package entry)
  • https://github.com/Viji1304/demoproject-thirdparty/blob/master/pom.xml (Use command - mvn package install)

[kiranparab]

Because of mutable concept we can't create non OSGI to osgi bundle and deploy. We need to keep this as a part of codebase through embed dependency only.

[shelly-goel]

@kiranparabYou would have to add this bundle in the embed-dependency section as described here:

<Embed-Dependency>*;scope=compile|runtime;inline=true</Embed-Dependency>

https://stackoverflow.com/questions/56697999/how-to-embed-external-jars-dependency-in-osgi-bundles