Issue with AEM Author ACC Indexing

[MartynaWi]

Hello,

----

The scope is an AEM author on premise instance. 

----

we seem to have a SegmentNotFoundException on one of our AEM author instances (integration/test layer)

This error causes indexisng not to work properly on the instance, which has impacts on

1) page moves (link rewriting does not work

2) the Lucene search functionality (some of our custom code is also impacted.

I tried to follow the usual steps:

- https://experienceleague.adobe.com/en/docs/experience-cloud-kcs/kbarticles/ka-16542

- https://github.com/cqsupport/fix-instructions/tree/master/count-nodes

----

the normal check fails immediately with an exception:

[aem@xxx-xxx-aema1 crx-quickstart]$ java -jar oak-run-1.10.6.jar check --bin=1 repository/segmentstore

Apache Jackrabbit Oak 1.10.6

org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment 39523625-57bf-499d-a39e-f82c4f584cd9 not found

at org.apache.jackrabbit.oak.segment.file.AbstractFileStore.readSegmentUncached(AbstractFileStore.java:284)

at org.apache.jackrabbit.oak.segment.file.ReadOnlyFileStore$1.call(ReadOnlyFileStore.java:124)

...

----------------

The check passes without finding ano problem IF we specify the "--head" option

(last good revision reported as a half an hour old -> fine, as instance was shut down for about that long)

[aem@xxxxx-xxx-aema1 crx-quickstart]$ java -jar oak-run-1.10.6.jar check --bin=1 --head repository/segmentstore

Apache Jackrabbit Oak 1.10.6

Checking revision 21a0e286-0193-4788-a2c3-b93a4970bf8c:220

Checking head

Checking /

Checked 1,853,539 nodes and 8,884,964 properties

Path / is consistent

Searched through 1 revisions and 0 checkpoints

Head

Latest good revision for path / is 21a0e286-0193-4788-a2c3-b93a4970bf8c:220 from Apr 2, 2024, 2:08:32 PM

------------------

Is there something else we could try before we have to create a content package, then recreate that instance from scratch?

Thanks,

Martyna Wilczynska

[arunpatidar]

Hi @MartynaWi 

When encountering the same issue, we followed these steps:

1. Identified the correct state of the instance and verified the date.
2. Created a content package from the identified date to the current date.
3. Restored the AEM instance from the VM backup based on the correct state.
4. Deployed the content that was changed after the correct state date back onto the instance.

[MartynaWi]

Thank you @arunpatidar , but this solution seems not to be working for us, because:

- we dont know how to identify the last correct state;

- we only have 1 day of backup.

BR,
Martyna

[arunpatidar]

Hi @MartynaWi 
You already have a date, right?

Latest good revision for path / is 21a0e286-0193-4788-a2c3-b93a4970bf8c:220 from Apr 2, 2024, 2:08:32 PM 

You can ask run repo check with previous day or 2nd April backup and check if it is repo is not corrupted.

you can ask Adobe, Adobe takes backup everyday.

[MartynaWi]

Hello @arunpatidar 

follow up from our Devs:

No, this is the date reported by the --head option (which we don't know exactly what it does)
It reports the repository as being healthy. But the error shows up if we do the complete check, or try to index anything.

[arunpatidar]

Hi @MartynaWi 
Please check following articles for details

https://experienceleague.adobe.com/en/docs/experience-cloud-kcs/kbarticles/ka-16550 

https://www.tumblr.com/cq-ops/156640065474/useful-oak-oak-run-commands 

https://gist.github.com/andrewmkhoury/3a3b5e69900f4e873743 

[kautuk_sahni]

@MartynaWi  Did you find the suggestions helpful? Please let us know if more information is required. Otherwise, please mark the answer as correct for posterity. If you have found out solution yourself, please share it with the community.