Installing AEM 6.5 with S3 connector and install Service Pack 4 too

[#1]

Hi,

I'm trying to install AEM with AWS S3 as DataStorage and I cannot connect my instance with S3 bucket.

I'm following the installation guide at:

https://docs.adobe.com/content/help/en/experience-manager-65/deploying/deploying/data-store-config.h...

I tried to install the latest version of s3connector but there are activation problems on Oak Cloud Blob Store bundle so now I'm trying with s3connector v1.10.8:

https://repo.adobe.com/nexus/content/groups/public/com/adobe/granite/com.adobe.granite.oak.s3connect...

I'm following this steps:

1. Extract the contents of s3connector zip file into a temporary folder
2. Go to AEM install dir and unpack the quickstart jar file
3. Go to crx-quickstart folder and create an install folder inside
4. Copy the s3connector files located at jcr_root/libs/system/install into the crx-quckstart/install folder
5. Copy the s3connector config files located at jcr_root/libs/system/config into crx-quickstart/install folder.

At this point I only need these files:

org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore.config

org.apache.jackrabbit.oak.segment.SegmentNodeStoreService.config

At SegmentNodeStoreService.config we have this:

customBlobStore=B"true"

And at S3DataStore.config we have this:

accessKey="[bucket-access-key]"

secretKey="[bucket-secret-key]"

s3Bucket="[bucket-name]"

s3Region="eu-west-1"

s3EndPoint="s3-eu-west-1.amazonaws.com"

connectionTimeout="120000"

socketTimeout="120000"

maxConnections="40"

maxErrorRetry="10"

s3RenameKeys=B"false"

path="/opt/aem65/author/crx-quickstart/repository/datastore"

s3Encryption="SSE_S3"

uploadThreads=I"10"

stagingPurgeInterval=I"300"

stagingSplitPercentage=I"10"

stagingRetryInterval=I"600"

writeThreads="10"

cacheSize="21630578304"

secret="shareddatastore"

I start the installation using a script where runmodes are:

export CQ_RUNMODE='author,dev,nosamplecontent,crx3tar-nofds'

And at the end I use crx-quickstart/bin/start script.

The result is that I cannot connect with the s3 bucket. The bucket remains empty.

It’s suppose also that at datastore folder I should see only 3 folders:

• download
• tmp
• upload

But what I can see it’s a lot of folders. It’s like it’s not configuring the S3 connection and it’s installing AEM as usual (like we usually do when starting AEM locally by double clicking on the quickstart jar file).

My questions are these:

Am I doing something wrong? Are the parameters in the configuration files correct?

Am I following the steps in the correct order? What about installing Service Pack 4? Should I install it before installing S3 connector or after?

Am I using the correct s3connector version?

Please, if someone answer these questions it could be useful a brief step by step explanation.

Thank you.

[#2]

Check similar issue:

https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/installing-aws-s3-connecto...

Do you get any type of error ?

[#3]

This link is another thread I've openend where we couldn't reach the underlying problem and was marked as resolver I don't know why....

First of all I would like to now if the procedure I follow is correct and if the properties are right.

The error I can see at logs it's what follows:

26.05.2020 13:12:55.499 *ERROR* [FelixStartLevel] org.apache.aries.jmx.whiteboard.MBeanHolder register: Failure registering MBean org.apache.jackrabbit.oak.plugins.blob.BlobStoreStats@18b847a3
javax.management.InstanceAlreadyExistsException: org.apache.jackrabbit.oak:name=DataStoreBlobStore,type=BlobStoreStats
at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
at org.apache.aries.jmx.whiteboard.MBeanHolder.register(MBeanHolder.java:114) [org.apache.aries.jmx.whiteboard:1.2.0]
at org.apache.aries.jmx.whiteboard.JmxWhiteboardSupport.registerMBean(JmxWhiteboardSupport.java:88) [org.apache.aries.jmx.whiteboard:1.2.0]
at org.apache.aries.jmx.whiteboard.Activator$MBeanTracker.addingService(Activator.java:102) [org.apache.aries.jmx.whiteboard:1.2.0]
at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:943)
at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:871)
at org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked.java:256)
at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:229)
at org.osgi.util.tracker.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:903)
at org.apache.felix.framework.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:990)
at org.apache.felix.framework.EventDispatcher.fireEventImmediately(EventDispatcher.java:838)
at org.apache.felix.framework.EventDispatcher.fireServiceEvent(EventDispatcher.java:545)
at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4833)
at org.apache.felix.framework.Felix.registerService(Felix.java:3804)
at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:328)
at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:302)
at org.apache.jackrabbit.oak.osgi.OsgiWhiteboard.register(OsgiWhiteboard.java:79) [org.apache.jackrabbit.oak-core-spi:1.10.8]
at org.apache.jackrabbit.oak.spi.whiteboard.WhiteboardUtils.registerMBean(WhiteboardUtils.java:115) [org.apache.jackrabbit.oak-core-spi:1.10.8]
at org.apache.jackrabbit.oak.spi.whiteboard.WhiteboardUtils.registerMBean(WhiteboardUtils.java:99) [org.apache.jackrabbit.oak-core-spi:1.10.8]
at org.apache.jackrabbit.oak.plugins.blob.datastore.AbstractDataStoreService.registerMBeans(AbstractDataStoreService.java:154) [org.apache.jackrabbit.oak-blob-plugins:1.10.8]
at org.apache.jackrabbit.oak.plugins.blob.datastore.AbstractDataStoreService.activate(AbstractDataStoreService.java:110) [org.apache.jackrabbit.oak-blob-plugins:1.10.8]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invokeMethod(BaseMethod.java:228) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.inject.methods.BaseMethod.access$500(BaseMethod.java:41) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.inject.methods.BaseMethod$Resolved.invoke(BaseMethod.java:664) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invoke(BaseMethod.java:510) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:317) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:307) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:340) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:114) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:982) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:955) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:765) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.manager.AbstractComponentManager.enableInternal(AbstractComponentManager.java:666) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.manager.AbstractComponentManager.enable(AbstractComponentManager.java:432) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.enableComponents(ConfigurableComponentHolder.java:665) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.BundleComponentActivator.initialEnable(BundleComponentActivator.java:338) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.Activator.loadComponents(Activator.java:382) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.Activator.access$200(Activator.java:49) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.Activator$ScrExtension.start(Activator.java:264) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.AbstractExtender.createExtension(AbstractExtender.java:196) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.AbstractExtender.modifiedBundle(AbstractExtender.java:169) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.AbstractExtender.modifiedBundle(AbstractExtender.java:49) [org.apache.felix.scr:2.1.16]
at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:488)
at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:420)
at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:232)
at org.osgi.util.tracker.BundleTracker$Tracked.bundleChanged(BundleTracker.java:450)
at org.apache.felix.framework.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:915)
at org.apache.felix.framework.EventDispatcher.fireEventImmediately(EventDispatcher.java:834)
at org.apache.felix.framework.EventDispatcher.fireBundleEvent(EventDispatcher.java:516)
at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4817)
at org.apache.felix.framework.Felix.startBundle(Felix.java:2336)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1539)
at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
at java.lang.Thread.run(Thread.java:748)
26.05.2020 13:12:55.500 *INFO* [FelixStartLevel] org.apache.jackrabbit.oak-blob-cloud Service [842, [org.apache.jackrabbit.oak.api.jmx.CacheStatsMBean]] ServiceEvent REGISTERED
26.05.2020 13:12:55.501 *ERROR* [FelixStartLevel] org.apache.aries.jmx.whiteboard.MBeanHolder register: Failure registering MBean org.apache.jackrabbit.oak.cache.CacheStats@71d88d87
javax.management.InstanceAlreadyExistsException: org.apache.jackrabbit.oak:name=BlobStore-MemCache,type=CacheStats
at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
at org.apache.aries.jmx.whiteboard.MBeanHolder.register(MBeanHolder.java:114) [org.apache.aries.jmx.whiteboard:1.2.0]
at org.apache.aries.jmx.whiteboard.JmxWhiteboardSupport.registerMBean(JmxWhiteboardSupport.java:88) [org.apache.aries.jmx.whiteboard:1.2.0]
at org.apache.aries.jmx.whiteboard.Activator$MBeanTracker.addingService(Activator.java:102) [org.apache.aries.jmx.whiteboard:1.2.0]
at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:943)
at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:871)
at org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked.java:256)
at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:229)
at org.osgi.util.tracker.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:903)
at org.apache.felix.framework.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:990)
at org.apache.felix.framework.EventDispatcher.fireEventImmediately(EventDispatcher.java:838)
at org.apache.felix.framework.EventDispatcher.fireServiceEvent(EventDispatcher.java:545)
at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4833)
at org.apache.felix.framework.Felix.registerService(Felix.java:3804)
at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:328)
at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:302)
at org.apache.jackrabbit.oak.osgi.OsgiWhiteboard.register(OsgiWhiteboard.java:79) [org.apache.jackrabbit.oak-core-spi:1.10.8]
at org.apache.jackrabbit.oak.spi.whiteboard.WhiteboardUtils.registerMBean(WhiteboardUtils.java:115) [org.apache.jackrabbit.oak-core-spi:1.10.8]
at org.apache.jackrabbit.oak.spi.whiteboard.WhiteboardUtils.registerMBean(WhiteboardUtils.java:99) [org.apache.jackrabbit.oak-core-spi:1.10.8]
at org.apache.jackrabbit.oak.plugins.blob.datastore.AbstractDataStoreService.registerMBeans(AbstractDataStoreService.java:159) [org.apache.jackrabbit.oak-blob-plugins:1.10.8]
at org.apache.jackrabbit.oak.plugins.blob.datastore.AbstractDataStoreService.activate(AbstractDataStoreService.java:110) [org.apache.jackrabbit.oak-blob-plugins:1.10.8]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invokeMethod(BaseMethod.java:228) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.inject.methods.BaseMethod.access$500(BaseMethod.java:41) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.inject.methods.BaseMethod$Resolved.invoke(BaseMethod.java:664) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invoke(BaseMethod.java:510) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:317) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:307) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:340) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:114) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:982) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:955) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:765) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.manager.AbstractComponentManager.enableInternal(AbstractComponentManager.java:666) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.manager.AbstractComponentManager.enable(AbstractComponentManager.java:432) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.enableComponents(ConfigurableComponentHolder.java:665) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.BundleComponentActivator.initialEnable(BundleComponentActivator.java:338) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.Activator.loadComponents(Activator.java:382) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.Activator.access$200(Activator.java:49) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.Activator$ScrExtension.start(Activator.java:264) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.AbstractExtender.createExtension(AbstractExtender.java:196) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.AbstractExtender.modifiedBundle(AbstractExtender.java:169) [org.apache.felix.scr:2.1.16]
at org.apache.felix.scr.impl.AbstractExtender.modifiedBundle(AbstractExtender.java:49) [org.apache.felix.scr:2.1.16]
at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:488)
at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:420)
at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:232)
at org.osgi.util.tracker.BundleTracker$Tracked.bundleChanged(BundleTracker.java:450)
at org.apache.felix.framework.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:915)
at org.apache.felix.framework.EventDispatcher.fireEventImmediately(EventDispatcher.java:834)
at org.apache.felix.framework.EventDispatcher.fireBundleEvent(EventDispatcher.java:516)
at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4817)
at org.apache.felix.framework.Felix.startBundle(Felix.java:2336)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1539)
at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
at java.lang.Thread.run(Thread.java:748)

Regards,

[#4]

I know this thread is a bit old, but I hit a similar issue a while back and through lots of trial and error I found that when installing AEM 6.5 you *must* use version 1.10.0 of the S3 connector initially. 

After you have AEM successfully running you can then proceed with upgrading the connector to the latest and the latest AEM SPs. 

If someone found a different way would love to hear it but I spent quite a bit of time trying to get the latest version working on a fresh AEM 6.5 install and everytime it failed. I walked the versions back using the same configs until I hit a version that worked perfectly.

[#5]

Yes, I had to do the same.

Thank you for your comments!!!