Expand my Community achievements bar.

SOLVED

Garbage collection taking more time and creating reference files under meta folder

Avatar

Level 4

Hi All,

 

We are using a shared datastore for author and publishers in our platform.

As we are currently facing the reference files are creating in s3 under meta folder which is causing take huge memory.

 

When we run the garbage collection for boolean value as a false then i am seeing some of the bold ids are missing and when i try to search those specific id's in our s3 datastore in s3 am unable to find.

 

Could you please let me know if any once has encounter the same issues?

 

Thanks,.

 

 

1 Accepted Solution

Avatar

Correct answer by
Community Advisor
4 Replies

Avatar

Level 4

configurations for org.apache.jackrabbit.oak.segment.SegmentNodeStoreService.config

 

#repository.home=crx-quickstart/segmentstore

#Maximum size of a segment in MB. The default maximum is 256MB.
#tarmk.size=256

#Boolean value indicating that a custom data store is used. The default value is false.
customBlobStore=B"true"

#pauseCompaction=B"true"

#Determines if this Node Store should be started in standby mode.
standby=B"false"

blobTrackSnapshotIntervalInSecs=L"0"

service.ranking=I"100"

Avatar

Community Advisor

Hi @santhoshm687661 

If the datastore is shared between repositories, we need to 

  • run the GC with markOnly as true in all the instances (mark)
  • Then run with markOnly as false in one of the instances (sweep)

Detailed explanation from official docs :

Avatar

Level 4

@Vijayalakshmi_S I have tried all this steps still am seeing the same error.

25.12.2021 14:14:01.504 *ERROR* [sling-oak-observation-1] org.apache.jackrabbit.oak.plugins.blob.MarkSweepGarbageCollector Not all repositories have marked references available : [021fa42f-8c70-4475-a907-e9d65107b4ed, 09715601-7a9a-4597-a01e-512c7fdfc867, 0f716341-91c3-420e-b25b-3664047ab228, 136d9d62-c81f-47fa-9e25-f634a2069c11, 20cc00bb-9019-4258-80c7-aff9db8e0f4a, 3ced1932-be00-4411-a951-c87a30812243, 3e2e7f45-edf8-4df5-a3d5-191875a85e23, 408f85f6-7af2-4e64-9601-c778c122f08f, 528679c5-ce01-49fb-a5fd-209b6f58bba9, 56a03dde-bbb1-454f-8bb9-e14230d211d4, 76c1b1fd-904c-4d01-b8d3-c5a18da088e9, 850d9a41-188b-484e-a6d2-5e3378227625, 85945b20-2b01-473a-9e1b-fd33f912a7f9, 8667776f-54c8-4341-a4ff-c34d6017d73d, 97707dc2-b211-4b1d-97ba-ffadc54d56a4, ab7ccab7-a21a-4e86-b54d-0f01c067b9f9, b04cbe4d-bdcd-4e07-8555-a473737bffbd, cfe6bf7c-a558-4bdf-b323-42931341b6e9,

de68dc5e-19ad-4a73-b060-2801f3fbf691, e9850d87-4a1a-4169-8106-50fe76cfa34e]
Please let me know if any inputs.

Avatar

Correct answer by
Community Advisor

@santhoshm687661 

Please check the solution from this thread (similar scenario as yours) - https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/s3-datastore-growth-issue/...