Revision Cleanup and Datastore Garbage Collection questions.

[UserSD]

Hi All, 

Need help with a few clarifications.

I have an AEM 6.5.21 on prem instance running on RHEL VM. TarMK is being used as the node store and File System Data Store for datastore. The datastore size currently is ~ 260GB.

Under Operations dashboard I can see:

Under Daily Maintenance Task (Runs from 02:00 – 05:00):

I have Lucene Binaries Cleanup and Revision Clean Up. Both Tasks currently have a green checkmark displayed next to it with a message succeeded, which makes me believe they successfully ran.

Under Weekly maintenance task (Runs every Saturday from 01:00 – 02:00):

I can see Data Store Garbage collection has successfully run previous Saturday and is scheduled to run upcoming Saturday at 01:00:00 . When I open the configuration for DataStoreGarbageCollectionTask, I can see the checkbox for  granite.maintenance.mandatory is not selected.

The Workflow Purge is successful and AuditLog Maintence (Failed) tasks are also present.

Under the  SegmentNodeStoreService configuration in system/console/ConfigMgr,

We have pauseCompaction=B"true" . Compaction has never been run in this system since it was created.

I have following questions:

1. What functions does granite.maintenance.mandatory checkbox in the DataStoreGarbageCollectionTask configuration serve?
2. How can I decrease the size of my Datastore which is currently at 260 GB?. Considering the fact that DataStoreGarbageCollectionTask is running weekly on Saturdays (1:00 to 2:00) the size of my datastore is still around 260 GB. We have around 100 XDP files that we maintain inside /content/dam/formsanddocuments which we use with output service to create PDFs. Apart from this we don’t maintain much assets. Our site mainly has AEM Adaptive forms embedded inside AEM Site pages, ~20 AEM site pages and ~20 AEM Adaptive forms. Our users do upload some documents while submitting forms and the node count inside /tmp/fd/af is ~5000 . These mainly appear to be the user uploaded documents and on closer inspection these appear to be only for the current months upload, the documents from previous months upload seems to be removed.
3. Will enabling compaction help in reducing the size of the AEM datastore? Or is compaction only for segment store?
4. Considering the fact that we have never run compaction and this being a live prod system, what approach would be recommended online or offline? If we go for offline, how long would the first compaction run typically last, any rough benchmarking?
5. Considering that my current daily maintenance window configured in the operations dashboard is for 3 hrs (02:00 to 05:00) and with the default settings of doing full compaction during Sundays and tail type compaction for the rest of the days, what happens if during the initial run of compaction, it exceeds the 3 hrs maintenance window? Will compaction still run until it completes or will it roll back? If it rolls back, how can I make sure it -  fully runs during the initial compaction run without timing out?

Best Regards,

[kautuk_sahni]

@Tethich @MukeshYadav_ @PRATHYUSHA_VP @Jörg_Hoh @muskaanchandwani @ShivamKumar @AMANATH_ULLAH @abhishekanand_ @anupampat  when you get a chance, could you review this question? Your input would be greatly appreciated!

[Jörg_Hoh]

Hi,

Running the revision cleanup (aka online compaction, see https://experienceleague.adobe.com/en/docs/experience-manager-65/content/implementing/deploying/depl...)  periodically should be able to keep the growth of the repository itself (the TarMK) managable, as "overwritten" and delete content is being removed. It can also remove references to binaries in the datastore, which would then get eligible for purge.

The log should indicate if the online compaction ever completes, or if it terminates because it takes too long.

And when the compaction has completed, a run of the Datastore Garbage Collection should be able to cleanup the datastore and remove all binaries which are no longer referenced.