Expand my Community achievements bar.

AEM6.5.17 Integrate with Amazon S3 Bucket Path

Avatar

Level 5
Level 5
9/19/23 4:17:42 PM

Hi Team,

 

Currently we are using AEM 6.5.17 (amazon EC2 Servers) . We need to connect with S3 Bucket URL  from AEM.

 

Our requirement is some of assets (pdfs) need to store and retrieve from S3 instead of Dynamic Media or AEM Instance.

 

Do you we need install the Amazon S3 Connector  or Do we need to handle with  OSGI Services. Moreover I am not able to find Amazon S3 connector zip file  to install and configure . Please help here to share the instruction best practices to integrate with AEM

 

Regards

Vara

 

 

Views

1.0K

Replies

5
Sign in to like this content

Total Likes

Translate
Translate
5 Replies

Avatar

Community Advisor
Community Advisor
9/19/23 8:50:50 PM

@varaande To enable the S3 data store functionality, a feature pack containing the S3 Datastore Connector must be downloaded and installed (for example, com.adobe.granite.oak.s3connector-1.10.0.zip). Go to the Adobe Repository and download the latest version from the 1.10.x versions of the feature pack.

Please refer to Steps mentioned in below documentation URL:

https://experienceleague.adobe.com/docs/experience-manager-65/deploying/deploying/data-store-config....

 

Views

987

Replies

3
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Level 5
Level 5
9/20/23 7:11:02 PM
In response to Bhuwan_B

Hi Bhuwan_B

 

I had installed com.adobe.granite.oak.s3connector-1.10.0.zip , Kindly let know how to validate the whether able to add/retrieve content /binary to Amazon S3 bucket 

what are parameter we need configure (ex:acesskey,s3bucket,secret key  etc.) , Is there any UI make sure that connector is installed successfully 

 

varaande_0-1695262020760.png

 

Here is details for S3 I had received ,i am confused about two security key fields which one i need use

 

1) service account Secret key: username
2) assume role Secret key: assumeRoleArn

 

 

 

 

Views

963

Replies

1
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Level 5
Level 5
10/5/23 11:44:15 PM
In response to varaande

@varaande You should be able to get the below details from your AWS S3 implementation details. So that we can establish a connection from AEM to that AWS S3 Instance.

1) service account Secret key: username
2) assume role Secret key: assumeRoleArn

Views

915

Replies

0
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Level 5
Level 5
10/10/23 1:53:25 PM
In response to Bhuwan_B

Hi Bhuwan_B

I had installed the S3 connector and configured access key and security key at /configMgr

 

varaande_0-1696971063474.png

 

varaande_1-1696971156372.png

 

 

But getting below error & when upload the asset (& publish ) not able to see the same in S3 Bucket URL

"04.10.2023 10:18:09.909 *ERROR* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak.blob.cloud.s3.S3Backend Error"

 

 

Log Snippet:

04.10.2023 10:18:09.903 *INFO* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak-blob-cloud Service [org.apache.jackrabbit.oak.blob.cloud.s3.S3DataStore,9165, [org.apache.jackrabbit.oak.plugins.blob.AbstractSharedCachingDataStore, org.apache.jackrabbit.oak.plugins.blob.AbstractSharedCachingDataStore]] ServiceEvent REGISTERED
04.10.2023 10:18:09.904 *INFO* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak.blob.cloud.s3.S3DataStoreService Initializing the DataStore with homeDir [/opt/aem/crx-quickstart/repository]
04.10.2023 10:18:09.904 *INFO* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak.blob.cloud.s3.Utils Configuring Amazon Client from property file.
04.10.2023 10:18:09.908 *INFO* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak.blob.cloud.s3.Utils S3 service endpoint [s3.us-gov-east-1.amazonaws.com]
04.10.2023 10:18:09.909 *ERROR* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak.blob.cloud.s3.S3Backend Error
java.lang.IllegalArgumentException: Cannot create enum from us-gov-east-1 value!
at com.amazonaws.services.s3.model.Region.fromValue(Region.java:311) [com.amazonaws.aws-java-sdk-osgi:1.11.330]
at org.apache.jackrabbit.oak.blob.cloud.s3.S3Backend.init(S3Backend.java:175) [org.apache.jackrabbit.oak-blob-cloud:1.10.1]
at org.apache.jackrabbit.oak.plugins.blob.AbstractSharedCachingDataStore.init(AbstractSharedCachingDataStore.java:161) [org.apache.jackrabbit.oak-blob-plugins:1.22.15]
at org.apache.jackrabbit.oak.plugins.blob.datastore.AbstractDataStoreService.activate(AbstractDataStoreService.java:90) [org.apache.jackrabbit.oak-blob-plugins:1.22.15]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invokeMethod(BaseMethod.java:242) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod.access$500(BaseMethod.java:41) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod$Resolved.invoke(BaseMethod.java:678) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invoke(BaseMethod.java:524) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:318) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:308) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:342) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:115) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:984) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:957) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:766) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:736) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:675) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.configurationUpdated(ConfigurableComponentHolder.java:436) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.RegionConfigurationSupport.configurationEvent(RegionConfigurationSupport.java:340) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.RegionConfigurationSupport$2.configurationEvent(RegionConfigurationSupport.java:115) [org.apache.felix.scr:2.1.20]
at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.sendEvent(ConfigurationManager.java:1704) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.run(ConfigurationManager.java:1646) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.UpdateThread.run0(UpdateThread.java:138) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:105) [org.apache.felix.configadmin:1.9.12]
at java.base/java.lang.Thread.run(Thread.java:829)
04.10.2023 10:18:09.909 *ERROR* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak-blob-cloud bundle org.apache.jackrabbit.oak-blob-cloud:1.10.1 (4695)[org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore(382)] : The activate method has thrown an exception (org.apache.jackrabbit.core.data.DataStoreException: Could not initialize S3 from {component.id=382, maxErrorRetry=10, component.name=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore, s3Region=, writeThreads=30, socketTimeout=120000, connectionTimeout=120000, s3Bucket=, maxConnections=40, service.pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore})
org.apache.jackrabbit.core.data.DataStoreException: Could not initialize S3 from {component.id=382, maxErrorRetry=10, component.name=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore, s3Region=, writeThreads=30, socketTimeout=120000, connectionTimeout=120000, s3Bucket=, maxConnections=40, service.pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore}
at org.apache.jackrabbit.oak.blob.cloud.s3.S3Backend.init(S3Backend.java:254) [org.apache.jackrabbit.oak-blob-cloud:1.10.1]
at org.apache.jackrabbit.oak.plugins.blob.AbstractSharedCachingDataStore.init(AbstractSharedCachingDataStore.java:161) [org.apache.jackrabbit.oak-blob-plugins:1.22.15]
at org.apache.jackrabbit.oak.plugins.blob.datastore.AbstractDataStoreService.activate(AbstractDataStoreService.java:90) [org.apache.jackrabbit.oak-blob-plugins:1.22.15]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invokeMethod(BaseMethod.java:242) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod.access$500(BaseMethod.java:41) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod$Resolved.invoke(BaseMethod.java:678) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invoke(BaseMethod.java:524) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:318) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:308) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:342) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:115) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:984) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:957) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:766) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:736) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:675) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.configurationUpdated(ConfigurableComponentHolder.java:436) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.RegionConfigurationSupport.configurationEvent(RegionConfigurationSupport.java:340) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.RegionConfigurationSupport$2.configurationEvent(RegionConfigurationSupport.java:115) [org.apache.felix.scr:2.1.20]
at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.sendEvent(ConfigurationManager.java:1704) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.run(ConfigurationManager.java:1646) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.UpdateThread.run0(UpdateThread.java:138) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:105) [org.apache.felix.configadmin:1.9.12]
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalArgumentException: Cannot create enum from us-gov-east-1 value!
at com.amazonaws.services.s3.model.Region.fromValue(Region.java:311) [com.amazonaws.aws-java-sdk-osgi:1.11.330]
at org.apache.jackrabbit.oak.blob.cloud.s3.S3Backend.init(S3Backend.java:175) [org.apache.jackrabbit.oak-blob-cloud:1.10.1]
... 27 common frames omitted

 

 

Views

883

Replies

0
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Administrator
Administrator
10/9/23 7:01:20 AM

@varaande 

Did you find the suggestions from users helpful? Please let us know if more information is required. Otherwise, please mark the answer as correct for posterity. If you have found out solution yourself, please share it with the community.



Kautuk Sahni

Views

897

Replies

0
Sign in to like this content

Total Likes

Translate
Translate
Related Conversations
Need Help on AEM integration with Microfrontend along with steps

Views

438

Likes

0

Replies

11
Issue with OSGi Bundle in Installed State After Adding Apache PDFBox to bnd-baseline-maven-plugin

Views

428

Likes

0

Replies

11
Integrate splunk with aemaacs

Views

286

Like

1

Replies

6
AEM AS Cloud Service integration with Splunk

Views

230

Likes

0

Replies

2
Massive update of nodes under a specific path

Views

243

Likes

0

Replies

4