Implementing S3 store for AEM datastore

[maddy_23]

I am implementing S3 bucket for datastore, have some open question and could not found clarity anywhere so posting here. I followed the document (https://experienceleague.adobe.com/en/docs/experience-cloud-kcs/kbarticles/ka-16057)

1. Which version of s3 connector is compatible with 6.5.18, how we can check which version of s3 connector we should be using for respective AEM version. Document only say use latest s3 connector for latest SP version.

2. When we use S3 for existing AEM setup do we need to manually move the datastore from /crx-quickstart/repository/datastore/ to /crx-quickstart/repository/repository/datastore/upload, after moving need to deleted the filestore configuration from /crx-quickstart/launchpad/config/org/apache/jackrabbit/oak/plugins/blob/datastore/

3. After uploading all assets I observed same datastore folders been added in /crx-quickstart/repository/repository/datastore/download is this expected, or we can delete it.

4. We should only binary less replication with S3 datastore or normal replication also will work.

5. Where we can see or confirm if the assets is loading from S3.

[AmitVishwakarma · Accepted solution]

1. S3 Connector Version Compatibility: Using the latest compatible version of the S3 connector for AEM 6.5.18 is indeed the recommended approach. The AEM version and connector compatibility are typically outlined in Adobe documentation, and since AEM 6.5 is a stable release, using the latest stable S3 connector compatible with it is the right choice.
2. Moving the Datastore to S3: The procedure of manually moving the datastore directory to /repository/repository/datastore/upload/ and then cleaning up old configurations is the correct migration procedure when setting up an S3 datastore. This is part of the standard process for configuring external datastores in AEM.
3. download Folder Behavior: The download folder appearing in the datastore is expected behavior because AEM will temporarily store assets that are being downloaded from the S3 bucket in this folder. It’s safe to leave it as AEM automatically handles it.
4. Replication Strategy: Binary-less replication is the best practice with S3-based datastores. This avoids replicating large binary assets, which would be inefficient. While normal replication can technically work, it’s not optimal for setups using S3 as a datastore, and it's recommended to use binary-less replication in such cases.
5. Confirming Assets from S3: The approach to confirm assets being served from S3 by checking logs, asset URLs, and using the S3 console is completely accurate. This is the best method to verify that your setup is working as expected.

[maddy_23]

@AmitVishwakarma Thank you. So latest S3 connector will work with any version of AEM SP like 6.5.15 or 6.5.18 or latest SP 22. Is that correct?

When I used latest connector version with 6.5.0, the bundles were not resolved.

[AmitVishwakarma]

Yes, the latest S3 connector should work with any AEM 6.5 Service Pack version, including 6.5.15, 6.5.18, or the latest SP 22, as long as you are using the appropriate version of the connector for your AEM version.

If you're seeing issues where the bundles are not resolving in AEM 6.5.0, that suggests a compatibility issue. The bundle dependencies or versions might not align with the version of AEM you're using, and the connector may be expecting newer versions of certain bundles or APIs that were introduced in later service packs.

Possible solutions:
1.Upgrade to a Later SP (Service Pack): If possible, consider upgrading AEM to a more recent service pack (e.g., 6.5.15 or 6.5.18). This could resolve compatibility issues related to the bundles not being resolved.
2. Check Bundle Versions: Verify that the bundles are correctly installed and that there are no missing dependencies. Sometimes, the connectors require certain versions of other AEM bundles, which may only be available in later SPs.
3. Bundle Resolution Debugging: You can check the AEM log files (error.log and other logs) for any specific errors related to the connector. This can give you insight into which bundle or class might be missing or causing the issue.