AEM Upgrade 6.5 - Huge Datastore size

[ChitraMadan]

Hello,

We are doing AEM 6.4 on-prem upgrade to 6.5 on-prem and in order to upgrade all our environments, we are trying to build a base image with 6.5 and DAM packages from our existing production instance, so that we don’t spend time installing DAM on each instance separately. This does not yet contain any other code/content package.

We have observed that this new instance that we have created to be used as a base image is around 155 GB as opposed to current production which is around 80 GB. We have tried the below steps to reduce the size of the repository, but none of it has made a huge difference.

Disabled DAM update Asset workflow, while installing DAM Packages
Deleted DAM packages, after installation
Ran Online compaction from Daily and Weekly Maintenance tasks
Ran offline compaction

The DAM packages were uploaded and installed via CURL commands (Total size of all DAM packages is about 55GB). Author servers were scaled up to support upload and install of huge packages, and was later downsized. After the activity was completed
datastore size - 155GB
segment store size - 1.1GB

Has anyone else been in the same situation? Any idea if this is expected with 6.5 or something we can do to bring this down?

Thanks,

Chitra

[B_Sravan]

Seems like you already tried all the solutions that are possible as far as I know. Let me also check if there's any solution or if this is expected on 6.5 (as in with the upgrade, the sizing up may not be from our code base too right?)

[Lokesh_Vajrala]

@ChitraMadan You may want to delete the DAM packages from the package manager after installation, which will not remove the assets from the DAM. After that, run offline compaction to remove any references in the data store to the deleted DAM packages, which may reduce the size of the data store.

[B_Sravan]

@Lokesh_Vajrala I believe that they tried it already according to the description in the question, am I right @ChitraMadan?

[ChitraMadan]

Yes, we have tried that @Lokesh_Vajrala 

[vishalk63643534]

Hi @ChitraMadan Were you able to find a resolution to this issue?

[Jörg_Hoh · Accepted solution]

* Make sure that you also remove the snapshot packages, these are not deleted by default, and in your case can consume a lot of space.

* Then make sure that you do an offline compaction to clean up all references to the binaries in the datastore. Online compaction will remove nodes which deleted less than 24h ago (not sure about the exact number, but something in that range), but offline does not have that limit.

* Then run a datastore garbage collection.