Garbage collection taking longer than the normal to finish

[santhoshm687661]

Hi All,

When we are trying to schedule a garbage collector job for every week during the week end it's taking almost 22-23 hours to finish the job.

We have been using s3 connector to store the content.

Below are the errors from the logs that i have noted.

12.09.2021 13:00:00.063 *INFO* [sling-oak-observation-15] org.apache.jackrabbit.oak.blob.cloud.s3.S3Backend getRecord:Identifier [6c0a68bb82c7ef943da8c1232b85e17923aa222892a8064fa7c46be8e455bdc9] not found. Took [9] ms.
12.09.2021 13:00:00.063 *WARN* [sling-oak-observation-15] org.apache.jackrabbit.oak.plugins.blob.MarkSweepGarbageCollector Error occurred while deleting blob with id [6c0a68bb82c7ef943da8c1232b85e17923aa222892a8064fa7c46be8e455bdc9#345892]
org.apache.jackrabbit.core.data.DataStoreException: com.amazonaws.services.s3.model.AmazonS3Exception: Not Found (Service: Amazon S3; Status Code: 404; Error Code: 404 Not Found; Request ID: JNQTJXM2RKFGG35D; S3 Extended Request ID: GrTSGQTR2hb9r/tfzPxrC+7jyIAV/WB5SXO6fmc8MAiYGDRyVBazi8EU1mO6NoQN6Dc4p9e+Nj8=), S3 Extended Request ID: GrTSGQTR2hb9r/tfzPxrC+7jyIAV/WB5SXO6fmc8MAiYGDRyVBazi8EU1mO6NoQN6Dc4p9e+Nj8=

Also i have tried to find the above blob id's in s3 bucker and repository but am unable to find the particular blob id.

Please check and let me know the pointers.

[Vijayalakshmi_S]

Hi @santhoshm687661,

Could you please let know the AEM version and Oak version

[Raja-kp]

Hi @santhoshm687661  - Please let us know the below:

1. AEM repository size ? why its taking 22-23 hours to complete the job ?

2. How are we performing the data store garbage collection - Maintenance operation window or JMX Console ?

[santhoshm687661]

aem version 6.5.8

oak version 1.22.4

@Vijayalakshmi_S 

[santhoshm687661]

working on maintenance operation window because when we run through jmx console we are seeing the errors mentioned in forum ticket description

@Raja-kp 

[Raja-kp]

@santhoshm687661 - Does it take 20hrs+ When you are performing via maintenance window ?

[santhoshm687661]

@Raja-kp Yes.

[Raja-kp]

@santhoshm687661 Your AEM repository size ?

[santhoshm687661]

@Raja-kp It will be round 300 GB

[Raja-kp]

@santhoshm687661 

1.Perform the offline compaction - calculate the repo size

2.Perform the datastore garbage collection - calculate the repo size

Please raise a Adobe support case if you face the same issue again.

[santhoshm687661]

@Raja-kp This all steps i have did bro

Please let me know if there is any other alternate options.

[Raja-kp]

@santhoshm687661 Better raise a Adobe support case and please post the solution if they have provided.