AEM 6.1 to AEM 6.3 Upgrade Index issue
Hi,
I am working on an Inplace upgrade from AEM 6.1 to AEM 6.3. I ran all the pre upgrade checks and they went fine. Also i checked the Repository consistency check. It was fine. Also ran the offline compaction. Then ran the upgrade. The unpacking and migration both went fine. But when i started the server back up using the 6.3 jar. The server is not starting back up. i left the server over night and i only see over and over only one exception. i understand that it is reindexing, but it looks like reindexing is failing. Also does anybody know how to fix the BlobId issue. kind of stuck here, on what to do next
30.12.2017 12:50:16.246 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.IndexUpdate Removing corrupt flag from index [/oak:index/cqPageLucene] which has been marked as corrupt since [2017-12-30T03:39:42.087-06:00]
30.12.2017 12:50:16.246 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing will be performed for following indexes: [/oak:index/slingeventJob, /oak:index/versionStoreIndex, /oak:index/cqTagLucene, /oak:index/workflowDataLucene, /oak:index/cqProjectLucene, /oak:index/ntBaseLucene, /oak:index/authorizables, /oak:index/commerceLucene, /oak:index/cqPageLucene]
30.12.2017 12:50:21.225 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate Marking [/oak:index/authorizables] as corrupt. The index is failing since Sat Dec 30 03:39:46 CST 2017 ,3 indexing cycles, failed 6607 times, skipped 0 time
30.12.2017 12:50:21.226 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate Marking [/oak:index/cqPageLucene] as corrupt. The index is failing since Sat Dec 30 03:39:42 CST 2017 ,3 indexing cycles, failed 1 times, skipped 0 time
30.12.2017 12:50:21.226 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorContext Stored the cloned index definition for [/oak:index/workflowDataLucene]. Changes in index definition would now only be effective post reindexing
30.12.2017 12:50:21.228 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorContext Stored the cloned index definition for [/oak:index/slingeventJob]. Changes in index definition would now only be effective post reindexing
30.12.2017 12:50:21.229 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorContext Stored the cloned index definition for [/oak:index/versionStoreIndex]. Changes in index definition would now only be effective post reindexing
30.12.2017 12:50:21.231 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorContext Stored the cloned index definition for [/oak:index/commerceLucene]. Changes in index definition would now only be effective post reindexing
30.12.2017 12:50:21.232 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorContext Stored the cloned index definition for [/oak:index/cqProjectLucene]. Changes in index definition would now only be effective post reindexing
30.12.2017 12:50:21.234 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorContext Stored the cloned index definition for [/oak:index/damAssetLucene]. Changes in index definition would now only be effective post reindexing
30.12.2017 12:50:21.235 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorContext Stored the cloned index definition for [/oak:index/ntBaseLucene]. Changes in index definition would now only be effective post reindexing
30.12.2017 12:50:21.236 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorContext Stored the cloned index definition for [/oak:index/cqTagLucene]. Changes in index definition would now only be effective post reindexing
30.12.2017 12:50:21.237 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorContext Stored the cloned index definition for [/oak:index/lucene]. Changes in index definition would now only be effective post reindexing
30.12.2017 12:50:21.239 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorContext Stored the cloned index definition for [/oak:index/authorizables]. Changes in index definition would now only be effective post reindexing
30.12.2017 12:50:21.240 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.IndexUpdate Removing corrupt flag from index [/oak:index/authorizables] which has been marked as corrupt since [2017-12-30T03:39:46.261-06:00]
30.12.2017 12:50:21.240 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorContext Stored the cloned index definition for [/oak:index/cqPageLucene]. Changes in index definition would now only be effective post reindexing
30.12.2017 12:50:21.241 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.IndexUpdate Removing corrupt flag from index [/oak:index/cqPageLucene] which has been marked as corrupt since [2017-12-30T03:39:42.087-06:00]
30.12.2017 12:50:21.241 *INFO* [async-index-update-async] org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing will be performed for following indexes: [/oak:index/slingeventJob, /oak:index/versionStoreIndex, /oak:index/cqTagLucene, /oak:index/workflowDataLucene, /oak:index/cqProjectLucene, /oak:index/ntBaseLucene, /oak:index/authorizables, /oak:index/commerceLucene, /oak:index/cqPageLucene]
30.12.2017 12:50:22.455 *ERROR* [FelixStartLevel] org.apache.jackrabbit.oak.plugins.index.lucene.BadIndexTracker Could not access the Lucene index at [/oak:index/cqTagLucene] . since 9.755 h ,0 indexing cycles, accessed 2363 times
java.lang.IllegalStateException: Attempt to read external blob with blobId [8d15416c34093b37fbd9435128c5ff5bf2a45325#333321] without specifying BlobStore
at org.apache.jackrabbit.oak.segment.SegmentBlob.getBlob(SegmentBlob.java:247)
at org.apache.jackrabbit.oak.segment.SegmentBlob.getLength(SegmentBlob.java:256)
at org.apache.jackrabbit.oak.segment.SegmentBlob.length(SegmentBlob.java:112)
at org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory$OakIndexFile.<init>(OakDirectory.java:409)
at org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory$OakIndexInput.<init>(OakDirectory.java:589)
at org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory.fileLength(OakDirectory.java:176)
at org.apache.jackrabbit.oak.plugins.index.lucene.directory.IndexSanityChecker.isThereASizeMismatch(IndexSanityChecker.java:81)
at org.apache.jackrabbit.oak.plugins.index.lucene.directory.IndexSanityChecker.check(IndexSanityChecker.java:53)
at org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier.checkIntegrity(IndexCopier.java:250)
at org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier.wrapForRead(IndexCopier.java:121)
at org.apache.jackrabbit.oak.plugins.index.lucene.reader.DefaultIndexReaderFactory.createReader(DefaultIndexReaderFactory.java:102)
at org.apache.jackrabbit.oak.plugins.index.lucene.reader.DefaultIndexReaderFactory.createReaders(DefaultIndexReaderFactory.java:61)
at org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.open(IndexNode.java:68)
at org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker.findIndexNode(IndexTracker.java:227)
at org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker.acquireIndexNode(IndexTracker.java:179)
at org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex.getPlans(LucenePropertyIndex.java:239)
at org.apache.jackrabbit.oak.query.QueryImpl.getBestSelectorExecutionPlan(QueryImpl.java:1018)
at org.apache.jackrabbit.oak.query.QueryImpl.getBestSelectorExecutionPlan(QueryImpl.java:976)
at org.apache.jackrabbit.oak.query.ast.SelectorImpl.prepare(SelectorImpl.java:280)
at org.apache.jackrabbit.oak.query.QueryImpl.prepare(QueryImpl.java:641)
at org.apache.jackrabbit.oak.query.QueryEngineImpl.prepareAndSelect(QueryEngineImpl.java:294)