AEM 6.5 - Reducing size/ clean up for 'repository/segmentstore'

Hi @pavan_kumarti ,

To clean up and reduce the size of the 'segmentstore' in AEM 6.5, particularly focusing on the 'data*.tar.bak' files, you can follow these steps. This will help you safely manage and clean up the repository before your upgrade:

Manual Cleanup

1. Stop the AEM Instance: Before making any changes to the repository files, ensure the AEM instance is stopped to prevent data corruption.
   ./aem stop

   1. Backup: Before performing any cleanup, ensure you have a full backup of your repository and any other critical data.

   2. Run Offline Compaction: Run offline compaction to clean up the segment store. This process will remove old and unused segments.
      java -jar aem-quickstart.jar -v -r compaction
      Delete Old Backup Files: After running offline compaction, you can manually delete the 'data*.tar.bak' files. These are backup files created before compaction and are typically safe to remove after a successful compaction.

      Automatic Cleanup Configuration

      To configure automatic cleanup of old backup files and manage repository size, you can use the following OSGi configurations and repository maintenance tasks:

      1. Configure TarMK Compaction: In the AEM Web Console, configure the TarMK compaction to run automatically and clean up old segments:

         • Go to AEM Web Console: http://localhost:4502/system/console/configMgr
         • Search for and configure "Apache Jackrabbit Oak Segment Tar Compaction":
           • Set "Compaction Mode" to custom
           • Configure other parameters such as "Compaction Retained Generations", "Compaction Force Timeout", etc., according to your needs.

      2. Enable Revision Cleanup: Configure automatic revision cleanup which includes compaction and other maintenance tasks:

         • Go to AEM Web Console: http://localhost:4502/system/console/configMgr
         • Search for and configure "Apache Jackrabbit Oak Revision GC":
           • Set "GC Type" to tar
           • Schedule the maintenance task according to your requirements (e.g., daily, weekly).

      3. Configure Blob Garbage Collection: To clean up unused blobs in the data store:

         • Go to AEM Web Console: http://localhost:4502/system/console/configMgr
         • Search for and configure "Apache Jackrabbit Oak Blob Garbage Collection":
           • Enable and schedule the blob GC process.

      Example Configuration Steps

      Here are some example steps to configure the automatic cleanup in AEM:

      1. Access AEM Web Console: Open your browser and go to http://<AEM-HOST>:<PORT>/system/console/configMgr.

      2. Configure TarMK Compaction:

         • Search for "Apache Jackrabbit Oak Segment Tar Compaction".
         • Set "Compaction Mode" to custom.
         • Configure the schedule (e.g., 0 0 2 * * ? for daily at 2 AM).

      3. Configure Revision GC:

         • Search for "Apache Jackrabbit Oak Revision GC".
         • Set "GC Type" to tar.
         • Schedule the GC process (e.g., 0 0 3 * * ? for daily at 3 AM).

      4. Configure Blob GC:

         • Search for "Apache Jackrabbit Oak Blob Garbage Collection".
         • Enable and schedule the process according to your needs.

      Important Notes

      • Test in Non-Production: Always test these configurations and clean-up processes in a non-production environment before applying them to production.
      • Regular Monitoring: Regularly monitor the segment store size and the success of cleanup tasks. Ensure the tasks are running as expected.
      • Documentation and Logs: Keep documentation of your configurations and review AEM logs to troubleshoot any issues related to repository maintenance.

      By following these steps, you can effectively manage and reduce the size of your 'segmentstore' in AEM 6.5, ensuring a smoother upgrade and backup process.