AEM6.5.17 Integrate with Amazon S3 Bucket Path

[varaande]

Hi Team,

Currently we are using AEM 6.5.17 (amazon EC2 Servers) . We need to connect with S3 Bucket URL  from AEM.

Our requirement is some of assets (pdfs) need to store and retrieve from S3 instead of Dynamic Media or AEM Instance.

Do you we need install the Amazon S3 Connector  or Do we need to handle with  OSGI Services. Moreover I am not able to find Amazon S3 connector zip file  to install and configure . Please help here to share the instruction best practices to integrate with AEM

Regards

Vara

[Bhuwan_B]

@varaande To enable the S3 data store functionality, a feature pack containing the S3 Datastore Connector must be downloaded and installed (for example, com.adobe.granite.oak.s3connector-1.10.0.zip). Go to the Adobe Repository and download the latest version from the 1.10.x versions of the feature pack.

Please refer to Steps mentioned in below documentation URL:

https://experienceleague.adobe.com/docs/experience-manager-65/deploying/deploying/data-store-config....

[varaande]

Hi Bhuwan_B

I had installed com.adobe.granite.oak.s3connector-1.10.0.zip , Kindly let know how to validate the whether able to add/retrieve content /binary to Amazon S3 bucket 

what are parameter we need configure (ex:acesskey,s3bucket,secret key  etc.) , Is there any UI make sure that connector is installed successfully 

Here is details for S3 I had received ,i am confused about two security key fields which one i need use

1) service account Secret key: username
2) assume role Secret key: assumeRoleArn

[Hemalatha]

@varaande You should be able to get the below details from your AWS S3 implementation details. So that we can establish a connection from AEM to that AWS S3 Instance.

1) service account Secret key: username
2) assume role Secret key: assumeRoleArn

[varaande]

Hi Bhuwan_B

I had installed the S3 connector and configured access key and security key at /configMgr

But getting below error & when upload the asset (& publish ) not able to see the same in S3 Bucket URL

"04.10.2023 10:18:09.909 *ERROR* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak.blob.cloud.s3.S3Backend Error"

Log Snippet:

04.10.2023 10:18:09.903 *INFO* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak-blob-cloud Service [org.apache.jackrabbit.oak.blob.cloud.s3.S3DataStore,9165, [org.apache.jackrabbit.oak.plugins.blob.AbstractSharedCachingDataStore, org.apache.jackrabbit.oak.plugins.blob.AbstractSharedCachingDataStore]] ServiceEvent REGISTERED
04.10.2023 10:18:09.904 *INFO* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak.blob.cloud.s3.S3DataStoreService Initializing the DataStore with homeDir [/opt/aem/crx-quickstart/repository]
04.10.2023 10:18:09.904 *INFO* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak.blob.cloud.s3.Utils Configuring Amazon Client from property file.
04.10.2023 10:18:09.908 *INFO* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak.blob.cloud.s3.Utils S3 service endpoint [s3.us-gov-east-1.amazonaws.com]
04.10.2023 10:18:09.909 *ERROR* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak.blob.cloud.s3.S3Backend Error
java.lang.IllegalArgumentException: Cannot create enum from us-gov-east-1 value!
at com.amazonaws.services.s3.model.Region.fromValue(Region.java:311) [com.amazonaws.aws-java-sdk-osgi:1.11.330]
at org.apache.jackrabbit.oak.blob.cloud.s3.S3Backend.init(S3Backend.java:175) [org.apache.jackrabbit.oak-blob-cloud:1.10.1]
at org.apache.jackrabbit.oak.plugins.blob.AbstractSharedCachingDataStore.init(AbstractSharedCachingDataStore.java:161) [org.apache.jackrabbit.oak-blob-plugins:1.22.15]
at org.apache.jackrabbit.oak.plugins.blob.datastore.AbstractDataStoreService.activate(AbstractDataStoreService.java:90) [org.apache.jackrabbit.oak-blob-plugins:1.22.15]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invokeMethod(BaseMethod.java:242) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod.access$500(BaseMethod.java:41) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod$Resolved.invoke(BaseMethod.java:678) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invoke(BaseMethod.java:524) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:318) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:308) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:342) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:115) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:984) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:957) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:766) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:736) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:675) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.configurationUpdated(ConfigurableComponentHolder.java:436) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.RegionConfigurationSupport.configurationEvent(RegionConfigurationSupport.java:340) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.RegionConfigurationSupport$2.configurationEvent(RegionConfigurationSupport.java:115) [org.apache.felix.scr:2.1.20]
at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.sendEvent(ConfigurationManager.java:1704) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.run(ConfigurationManager.java:1646) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.UpdateThread.run0(UpdateThread.java:138) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:105) [org.apache.felix.configadmin:1.9.12]
at java.base/java.lang.Thread.run(Thread.java:829)
04.10.2023 10:18:09.909 *ERROR* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak-blob-cloud bundle org.apache.jackrabbit.oak-blob-cloud:1.10.1 (4695)[org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore(382)] : The activate method has thrown an exception (org.apache.jackrabbit.core.data.DataStoreException: Could not initialize S3 from {component.id=382, maxErrorRetry=10, component.name=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore, s3Region=, writeThreads=30, socketTimeout=120000, connectionTimeout=120000, s3Bucket=, maxConnections=40, service.pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore})
org.apache.jackrabbit.core.data.DataStoreException: Could not initialize S3 from {component.id=382, maxErrorRetry=10, component.name=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore, s3Region=, writeThreads=30, socketTimeout=120000, connectionTimeout=120000, s3Bucket=, maxConnections=40, service.pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore}
at org.apache.jackrabbit.oak.blob.cloud.s3.S3Backend.init(S3Backend.java:254) [org.apache.jackrabbit.oak-blob-cloud:1.10.1]
at org.apache.jackrabbit.oak.plugins.blob.AbstractSharedCachingDataStore.init(AbstractSharedCachingDataStore.java:161) [org.apache.jackrabbit.oak-blob-plugins:1.22.15]
at org.apache.jackrabbit.oak.plugins.blob.datastore.AbstractDataStoreService.activate(AbstractDataStoreService.java:90) [org.apache.jackrabbit.oak-blob-plugins:1.22.15]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invokeMethod(BaseMethod.java:242) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod.access$500(BaseMethod.java:41) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod$Resolved.invoke(BaseMethod.java:678) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invoke(BaseMethod.java:524) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:318) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:308) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:342) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:115) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:984) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:957) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:766) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:736) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:675) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.configurationUpdated(ConfigurableComponentHolder.java:436) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.RegionConfigurationSupport.configurationEvent(RegionConfigurationSupport.java:340) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.RegionConfigurationSupport$2.configurationEvent(RegionConfigurationSupport.java:115) [org.apache.felix.scr:2.1.20]
at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.sendEvent(ConfigurationManager.java:1704) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.run(ConfigurationManager.java:1646) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.UpdateThread.run0(UpdateThread.java:138) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:105) [org.apache.felix.configadmin:1.9.12]
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalArgumentException: Cannot create enum from us-gov-east-1 value!
at com.amazonaws.services.s3.model.Region.fromValue(Region.java:311) [com.amazonaws.aws-java-sdk-osgi:1.11.330]
at org.apache.jackrabbit.oak.blob.cloud.s3.S3Backend.init(S3Backend.java:175) [org.apache.jackrabbit.oak-blob-cloud:1.10.1]
... 27 common frames omitted

[kautuk_sahni]

@varaande 

Did you find the suggestions from users helpful? Please let us know if more information is required. Otherwise, please mark the answer as correct for posterity. If you have found out solution yourself, please share it with the community.