How to Validate Amazon S4 bucket Intergration with AEM

[varaande]

Hi Team,

Please let me know how to test or validate once integrated with Amazon S3 bucket URL 

Here is steps followed :

1)Installed the Amazon S3 Connector 

2) Configure the Configuration for S3 Connector 

Regards

Vara

[Ravi_Pampana]

You can perform a simple test within AEM by uploading a file to the S3 bucket through AEM's interface or retrieving a file from the S3 bucket. Verify that the operations are successful.

[varaande]

HI Ravi/Team

we are not able to see the asset (binary) after upload to AEM author(& publish) instance after integrate with connector 

Any suggestion, Do we need to get the latest connector(if yes where to download the latest connector)  or other options to integrated AEM with Amazon S3 Bucket as OOTB

Here Error snippet :

ERROR* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak.blob.cloud.s3.S3Backend Error
java.lang.IllegalArgumentException: Cannot create enum from us-gov-east-1 value!

Log snippet :
04.10.2023 10:18:09.903 *INFO* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak-blob-cloud Service [org.apache.jackrabbit.oak.blob.cloud.s3.S3DataStore,9165, [org.apache.jackrabbit.oak.plugins.blob.AbstractSharedCachingDataStore, org.apache.jackrabbit.oak.plugins.blob.AbstractSharedCachingDataStore]] ServiceEvent REGISTERED
04.10.2023 10:18:09.904 *INFO* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak.blob.cloud.s3.S3DataStoreService Initializing the DataStore with homeDir [/opt/aem/crx-quickstart/repository]
04.10.2023 10:18:09.904 *INFO* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak.blob.cloud.s3.Utils Configuring Amazon Client from property file.
04.10.2023 10:18:09.908 *INFO* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak.blob.cloud.s3.Utils S3 service endpoint [s3.us-gov-east-1.amazonaws.com]
04.10.2023 10:18:09.909 *ERROR* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak.blob.cloud.s3.S3Backend Error
java.lang.IllegalArgumentException: Cannot create enum from us-gov-east-1 value!
at com.amazonaws.services.s3.model.Region.fromValue(Region.java:311) [com.amazonaws.aws-java-sdk-osgi:1.11.330]
at org.apache.jackrabbit.oak.blob.cloud.s3.S3Backend.init(S3Backend.java:175) [org.apache.jackrabbit.oak-blob-cloud:1.10.1]
at org.apache.jackrabbit.oak.plugins.blob.AbstractSharedCachingDataStore.init(AbstractSharedCachingDataStore.java:161) [org.apache.jackrabbit.oak-blob-plugins:1.22.15]
at org.apache.jackrabbit.oak.plugins.blob.datastore.AbstractDataStoreService.activate(AbstractDataStoreService.java:90) [org.apache.jackrabbit.oak-blob-plugins:1.22.15]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invokeMethod(BaseMethod.java:242) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod.access$500(BaseMethod.java:41) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod$Resolved.invoke(BaseMethod.java:678) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invoke(BaseMethod.java:524) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:318) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:308) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:342) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:115) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:984) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:957) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:766) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:736) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:675) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.configurationUpdated(ConfigurableComponentHolder.java:436) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.RegionConfigurationSupport.configurationEvent(RegionConfigurationSupport.java:340) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.RegionConfigurationSupport$2.configurationEvent(RegionConfigurationSupport.java:115) [org.apache.felix.scr:2.1.20]
at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.sendEvent(ConfigurationManager.java:1704) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.run(ConfigurationManager.java:1646) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.UpdateThread.run0(UpdateThread.java:138) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:105) [org.apache.felix.configadmin:1.9.12]
at java.base/java.lang.Thread.run(Thread.java:829)
04.10.2023 10:18:09.909 *ERROR* [CM Event Dispatcher (Fire ConfigurationEvent: pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore)] org.apache.jackrabbit.oak-blob-cloud bundle org.apache.jackrabbit.oak-blob-cloud:1.10.1 (4695)[org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore(382)] : The activate method has thrown an exception (org.apache.jackrabbit.core.data.DataStoreException: Could not initialize S3 from {component.id=382, maxErrorRetry=10, component.name=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore, s3Region=, writeThreads=30, socketTimeout=120000, connectionTimeout=120000, s3Bucket=, maxConnections=40, service.pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore})
org.apache.jackrabbit.core.data.DataStoreException: Could not initialize S3 from {component.id=382, maxErrorRetry=10, component.name=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore, s3Region=, writeThreads=30, socketTimeout=120000, connectionTimeout=120000, s3Bucket=, maxConnections=40, service.pid=org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore}
at org.apache.jackrabbit.oak.blob.cloud.s3.S3Backend.init(S3Backend.java:254) [org.apache.jackrabbit.oak-blob-cloud:1.10.1]
at org.apache.jackrabbit.oak.plugins.blob.AbstractSharedCachingDataStore.init(AbstractSharedCachingDataStore.java:161) [org.apache.jackrabbit.oak-blob-plugins:1.22.15]
at org.apache.jackrabbit.oak.plugins.blob.datastore.AbstractDataStoreService.activate(AbstractDataStoreService.java:90) [org.apache.jackrabbit.oak-blob-plugins:1.22.15]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invokeMethod(BaseMethod.java:242) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod.access$500(BaseMethod.java:41) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod$Resolved.invoke(BaseMethod.java:678) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invoke(BaseMethod.java:524) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:318) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:308) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:342) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:115) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:984) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:957) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:766) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:736) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:675) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.configurationUpdated(ConfigurableComponentHolder.java:436) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.RegionConfigurationSupport.configurationEvent(RegionConfigurationSupport.java:340) [org.apache.felix.scr:2.1.20]
at org.apache.felix.scr.impl.manager.RegionConfigurationSupport$2.configurationEvent(RegionConfigurationSupport.java:115) [org.apache.felix.scr:2.1.20]
at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.sendEvent(ConfigurationManager.java:1704) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.run(ConfigurationManager.java:1646) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.UpdateThread.run0(UpdateThread.java:138) [org.apache.felix.configadmin:1.9.12]
at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:105) [org.apache.felix.configadmin:1.9.12]
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalArgumentException: Cannot create enum from us-gov-east-1 value!
at com.amazonaws.services.s3.model.Region.fromValue(Region.java:311) [com.amazonaws.aws-java-sdk-osgi:1.11.330]
at org.apache.jackrabbit.oak.blob.cloud.s3.S3Backend.init(S3Backend.java:175) [org.apache.jackrabbit.oak-blob-cloud:1.10.1]
... 27 common frames omitted

Regards

Vara

[sherinregi]

Hi @varaande 

You can refer to the below video link which talks about the integration and also validation steps

https://www.youtube.com/watch?v=t4oV_4EeDe0

1) look in system console for the blob files status related to s3

2)Go to the error log and verify the s3 related connection information

3) Upload a new file in AEM for testing, AEM will push the content to s3 bucket and will see a file there.

[varaande]

HI Team,

Connection successful as per log . But when we upload or upload followed by publish asset/s not storing Amazon S3 bucket URL . I am missing some thing here . I am continuing troubleshooting the issue 

[sherinregi]

@varaande  Can you share the error logs you are getting and if the upload is not successful or time out or any permission issues it should show up in the error logs.

[kautuk_sahni]

@varaande Did you find the suggestions from users helpful? Please let us know if more information is required. Otherwise, please mark the answer as correct for posterity. If you have found out solution yourself, please share it with the community.