Hi All,
We are upgrading the AEM Server from 6.1 to 6.3 version, completed all the Upgrade activities like maintenance , pre-upgrade , tar compaction & oak migration except the replacing the AEM 6.3 jar and bringing up instance is not successful.
Command
java -Xmx4096m -XX:MaxPermSize=2048m -jar cq-quickstart-630-author-p4502.jar
Also we are using the osgi configuration for File Data storage & Node storage.
org.apache.jackrabbit.oak.plugins.blob.datastore.FileDataStore
org.apache.jackrabbit.oak.segment.SegmentNodeStoreService
I have analyzed the issue and following are the observations. Please suggest me how to solve this issue.
1) Generated 5 GB of error.log with so many warn and error message.
2) Project related bundles, apps, etc are missing in crxde.
3) Start page and Welcome page (/aem/start.html, /libs/cq/core/content/welcome.html) are not opening throwing exception.
stderr.log
21.03.2018 06:26:24.231 *ERROR* [FelixStartLevel] ERROR: Bundle '360' EventDispatcher: Error during dispatch. (java.lang.NullPointerException)
java.lang.NullPointerException
at com.day.cq.dam.pim.impl.sourcing.selection.ProductAssetSelector$1.addingService(ProductAssetSelector.java:219)
at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:941)
at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:870)
at org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked.java:256)
at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:229)
at org.osgi.util.tracker.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:901)
at org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:991)
at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:839)
at org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:546)
at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:4558)
at org.apache.felix.framework.Felix.registerService(Felix.java:3550)
at org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:348)
error.logs(some error message)
-> 21.03.2018 06:28:35.912 *WARN* [FelixStartLevel] com.day.cq.cq-replication [com.day.cq.replication.impl.ReplicatorImpl(463)] Could not get service from ref [com.day.cq.replication.AgentManager, org.apache.sling.event.jobs.consumer.JobExecutor]
21.03.2018 06:28:35.912 *WARN* [FelixStartLevel] com.day.cq.wcm.cq-wcm-core [com.day.cq.wcm.core.impl.PageManagerFactoryImpl(1839)] Could not get service from ref [com.day.cq.replication.Replicator]
21.03.2018 06:28:35.912 *ERROR* [FelixDispatchQueue] com.day.cq.cq-replication FrameworkEvent ERROR (org.osgi.framework.ServiceException: Service factory returned null. (Component: org.apache.sling.event.impl.jobs.JobManagerImpl (637)))
org.osgi.framework.ServiceException: Service factory returned null. (Component: org.apache.sling.event.impl.jobs.JobManagerImpl (637))
at org.apache.felix.framework.ServiceRegistrationImpl.getFactoryUnchecked(ServiceRegistrationImpl.java:380)
at org.apache.felix.framework.ServiceRegistrationImpl.getService(ServiceRegistrationImpl.java:247)
at org.apache.felix.framework.ServiceRegistry.getService(ServiceRegistry.java:344)
at org.apache.felix.framework.Felix.getService(Felix.java:3700)
at org.apache.felix.framework.BundleContextImpl.getService(BundleContextImpl.java:470)
at org.apache.felix.scr.impl.manager.SingleRefPair.getServiceObject(SingleRefPair.java:72)
at org.apache.felix.scr.impl.inject.BindMethod.getServiceObject(BindMethod.java:648)
at org.apache.felix.scr.impl.manager.DependencyManager.getServiceObject(DependencyManager.java:2201)
-> 21.03.2018 06:19:53.133 *ERROR* [FelixDispatchQueue] org.apache.felix.http.whiteboard FrameworkEvent ERROR (org.osgi.framework.ServiceException: Service factory returned null. (org.apache.felix.http.base.internal.service.HttpServiceFactory@662cc759))
org.osgi.framework.ServiceException: Service factory returned null. (org.apache.felix.http.base.internal.service.HttpServiceFactory@662cc759)
-> 21.03.2018 06:20:25.008 *WARN* [OsgiInstallerImpl] org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl Unable to revert package content adobe/granite:com.adobe.granite.activitystreams.content:0.0.14. Snapshot missing.
21.03.2018 06:20:25.024 *WARN* [OsgiInstallerImpl] org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl Unable to revert package content adobe/granite:com.adobe.granite.backup.content:0.0.32. Snapshot missing.
21.03.2018 06:20:25.039 *WARN* [OsgiInstallerImpl] org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl Unable to revert package content adobe/granite:com.adobe.granite.cloudsettings.content:1.0.28. Snapshot missing.
21.03.2018 06:20:25.055 *WARN* [OsgiInstallerImpl] org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl Unable to revert package content adobe/granite:com.adobe.granite.cluster.content:0.0.36-CQ610-B0001. Snapshot missing.
21.03.2018 06:20:25.071 *WARN* [OsgiInstallerImpl] org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl Unable to revert package content adobe/granite:com.adobe.granite.contexthub.content:0.0.276-CQ610-B0001. Snapshot missing.
-> 21.03.2018 06:21:37.328 *WARN* [OsgiInstallerImpl] org.apache.jackrabbit.oak.security.user.UserImporter New member of Group 'winter-sports-admins': No such authorizable (NodeID = f4f0918c-f09e-3217-89b9-040b43601144)
21.03.2018 06:21:37.341 *WARN* [OsgiInstallerImpl] org.apache.jackrabbit.oak.security.user.UserImporter New member of Group 'surfing-admins': No such authorizable (NodeID = 89b46f84-2f98-31de-a505-9ae063311f57)
21.03.2018 06:21:37.348 *WARN* [OsgiInstallerImpl] org.apache.jackrabbit.oak.security.user.UserImporter New member of Group 'surfing-members': No such authorizable (NodeID = 27c55f02-efd9-39da-a63d-bb455048cd27)
-> 21.03.2018 06:23:56.506 *INFO* [OsgiInstallerImpl] com.adobe.granite.installer.factory.packages.impl.PackageTransformer Unable to check content package {}
java.io.IOException: Stream is not a content package. Missing 'jcr_root'.
-> 21.03.2018 06:24:41.222 *WARN* [OsgiInstallerImpl] org.apache.jackrabbit.vault.fs.io.Importer Error while committing changes. Retrying import from checkpoint at /. Retries 3/10
21.03.2018 06:24:41.718 *ERROR* [OsgiInstallerImpl] org.apache.jackrabbit.vault.fs.impl.io.DocViewSAXImporter Error during processing of /libs/cq/core/content/nav/tools/communities: javax.jcr.nodetype.ConstraintViolationException: No matching node definition found for communities
21.03.2018 06:24:41.718 *ERROR* [OsgiInstallerImpl] org.apache.jackrabbit.vault.fs.io.Importer E /libs/cq/core/content/nav/tools/communities (javax.jcr.nodetype.ConstraintViolationException: No matching node definition found for communities)
21.03.2018 06:24:41.719 *ERROR* [OsgiInstallerImpl] org.apache.jackrabbit.vault.fs.impl.io.DocViewSAXImporter Error during processing of /libs/cq/core/content/nav/tools/communities/componentguide: javax.jcr.nodetype.ConstraintViolationException: No matching node definition found for communities
21.03.2018 06:24:41.719 *ERROR* [OsgiInstallerImpl] org.apache.jackrabbit.vault.fs.io.Importer E /libs/cq/core/content/nav/tools/communities/componentguide (javax.jcr.nodetype.ConstraintViolationException: No matching node definition found for communities)
21.03.2018 06:24:41.798 *ERROR* [OsgiInstallerImpl] org.apache.jackrabbit.vault.fs.io.AutoSave error during auto save - retrying after refresh...
-> Caused by: org.apache.jackrabbit.oak.api.CommitFailedException: OakConstraint0001: /libs/cq/core/content/nav/tools[[nt:folder]]: No matching definition found for child node communities with effective type [nt:unstructured]
-> 21.03.2018 06:26:10.261 *ERROR* [FelixDispatchQueue] com.adobe.granite.workflow.core FrameworkEvent ERROR (org.osgi.framework.ServiceException: Service factory returned null. (Component: org.apache.sling.event.impl.jobs.JobManagerImpl (637)))
org.osgi.framework.ServiceException: Service factory returned null. (Component: org.apache.sling.event.impl.jobs.JobManagerImpl (637))
Views
Replies
Total Likes
This command worked well for me migrating from 6.1 to 6.3
java -Xmx4096m -jar aem-author-6.jar -v -x crx2oak -xargs -- --load-profile segment-no-ds --skip-checkpoints
Views
Replies
Total Likes
Hi Hemanth,
We have used the below command for migration from 6.1 to 6.3, it was successfully completed but during bringing the instance up seeing so many errors.
java -Xmx4096m -XX:MaxPermSize=2048m -jar cq-quickstart-630-author-p4502.jar -v -x crx2oak -xargs -- --load-profile segment-fds
profile: In our project there is File Data store configuration because of which we used the profile as "segment-fds" instead of "segment-no-ds".
Views
Replies
Total Likes
Can you share the command that you used for migration ?
Views
Replies
Total Likes
Command used for migration is: java -Xmx4096m -XX:MaxPermSize=2048m -jar cq-quickstart-630-author-p4502.jar -v -x crx2oak -xargs -- --load-profile segment-fds
Views
Replies
Total Likes
You need to specify your datastore configuration.
Add --src-datastore=
/path/to/datastore to the command
Where /path/to/datastore represents the path to your File Datastore.
Refer to Performing an In-Place Upgrade
Views
Replies
Total Likes
Hi,
I have added --src-datastore=<datastore_path> to my command and tried it yesterday but no luck.
Views
Replies
Total Likes
Can you try making the configurations correct ? Have you referred to the doc below ? https://helpx.adobe.com/experience-manager/6-3/sites/deploying/using/data-store-config.html
Views
Replies
Total Likes
Thank you Hemanth for your reply.
I have modified the configuration according to AEM 6.3, still getting the same error messages during the AEM 6.3 startup.
Configurations
org.apache.jackrabbit.oak.plugins.blob.datastore.FileDataStore.config
path="crx-quickstart/repository/datastore"
minRecordLength=I"4096"
org.apache.jackrabbit.oak.segment.SegmentNodeStoreService.config
repository.home="crx-quickstart/repository"
tarmk.size="256"
customBlobStore=B"true"
Stderr Log
21.03.2018 12:43:09.711 *ERROR* [FelixStartLevel] ERROR: Bundle '360' EventDispatcher: Error during dispatch. (java.lang.NullPointerException)
java.lang.NullPointerException
at com.day.cq.dam.pim.impl.sourcing.selection.ProductAssetSelector$1.addingService(ProductAssetSelector.java:219)
Error Log
21.03.2018 12:41:07.768 *ERROR* [OsgiInstallerImpl] org.apache.jackrabbit.vault.packaging.impl.ZipVaultPackage Error during install.
javax.jcr.nodetype.ConstraintViolationException: OakConstraint0001: /libs/cq/core/content/nav/tools[[nt:folder]]: No matching definition found for child node communities with effective type [nt:unstructured]
21.03.2018 12:41:07.768 *ERROR* [OsgiInstallerImpl] com.adobe.granite.installer.factory.packages.impl.PackageTransformer Error while processing install task of day/cq561/social/component-guide:community-components-guide-pkg:1.1.69.
org.apache.jackrabbit.vault.packaging.PackageException: javax.jcr.nodetype.ConstraintViolationException: OakConstraint0001: /libs/cq/core/content/nav/tools[[nt:folder]]: No matching definition found for child node communities with effective type [nt:unstructured]
21.03.2018 12:41:34.234 *ERROR* [OsgiInstallerImpl] org.apache.jackrabbit.vault.packaging.impl.ZipVaultPackage Error during install.
javax.jcr.nodetype.ConstraintViolationException: OakConstraint0001: /libs/cq/core/content[[nt:folder]]: No matching definition found for child node heartbeat with effective type [nt:unstructured]
21.03.2018 12:41:28.292 *ERROR* [OsgiInstallerImpl] org.apache.jackrabbit.vault.fs.io.Importer Error while committing /libs/cq/reporting/widgets/source/charts/ChartData.js: javax.jcr.nodetype.ConstraintViolationException: OakConstraint0001: /libs/cq/core/content[[nt:folder]]: No matching definition found for child node heartbeat with effective type [nt:unstructured]
21.03.2018 12:41:28.292 *ERROR* [OsgiInstallerImpl] org.apache.jackrabbit.vault.fs.io.Importer Error while committing /libs/cq/reporting/widgets/source/charts: javax.jcr.nodetype.ConstraintViolationException: OakConstraint0001: /libs/cq/core/content[[nt:folder]]: No matching definition found for child node heartbeat with effective type [nt:unstructured]
21.03.2018 12:41:28.292 *ERROR* [OsgiInstallerImpl] org.apache.jackrabbit.vault.fs.io.Importer Error while committing /libs/cq/reporting/widgets/source: javax.jcr.nodetype.ConstraintViolationException: OakConstraint0001: /libs/cq/core/content[[nt:folder]]: No matching definition found for child node heartbeat with effective type [nt:unstructured]
Views
Replies
Total Likes
Hey Indra,
Can you confirm if the migration was complete and you got the COMPLETED message on the CLI ? Can you share the upgrade.log at the time of migration was done ?
Views
Replies
Total Likes
Hi Kunwar,
I have analyzed the upgrade log and found the following error in upgrade.log. For download the log from below link.
Upgrade file link:https://drive.google.com/file/d/1Z3V3JZiEaWqT9xCvv5edcG2qtLx-rg8T/view
22.03.2018 16:55:02.579 *ERROR* [FelixStartLevel] com.adobe.cq.upgradesexecutor.Activator OSGI installer finished and the upgrade task condition was not satisfied. Executing installed upgrade tasks but most likely one of the mandatory upgrade tasks was not executed. Please check the log file.
22.03.2018 16:55:39.014 *INFO* [FelixStartLevel] com.day.cq.compat.codeupgrade.impl.cq62.Cq62ProjectsCodeUpgrade Done updating old projects in /content/projects
22.03.2018 16:55:39.014 *INFO* [FelixStartLevel] com.day.cq.compat.codeupgrade.impl.cq62.Cq62ProjectsCodeUpgrade Begin adjusting projects resource types.
22.03.2018 16:55:39.016 *WARN* [FelixStartLevel] com.day.cq.compat.codeupgrade.impl.cq62.Cq62ProjectsCodeUpgrade Exception in Cq62ProjectsCodeUpgrade.run()
java.lang.IllegalArgumentException: Value for key jcr:mixinTypes can't be put into node: [Ljava.lang.String;@5ae86d27
at org.apache.sling.jcr.resource.internal.JcrModifiableValueMap.put(JcrModifiableValueMap.java:458)
at org.apache.sling.jcr.resource.internal.JcrModifiableValueMap.put(JcrModifiableValueMap.java:47)
at com.day.cq.compat.codeupgrade.impl.projects.ProjectsAdjuster.checkResource(ProjectsAdjuster.java:90)
at com.day.cq.compat.codeupgrade.impl.projects.ProjectsAdjuster.checkResource(ProjectsAdjuster.java:96)
at com.day.cq.compat.codeupgrade.impl.projects.ProjectsAdjuster.adjustProjectResourceTypes(ProjectsAdjuster.java:64)
at com.day.cq.compat.codeupgrade.impl.cq62.Cq62ProjectsCodeUpgrade.doUpgrade(Cq62ProjectsCodeUpgrade.java:124)
at com.day.cq.compat.codeupgrade.impl.AbstractCodeUpgradeTask.run(AbstractCodeUpgradeTask.java:64)
at com.adobe.cq.upgradesexecutor.Activator.runUpgradeTasks(Activator.java:350)
at com.adobe.cq.upgradesexecutor.Activator.finishUpgradeExecution(Activator.java:220)
at com.adobe.cq.upgradesexecutor.Activator.start(Activator.java:170)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:697)
at org.apache.felix.framework.Felix.activateBundle(Felix.java:2227)
at org.apache.felix.framework.Felix.startBundle(Felix.java:2145)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1372)
at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.jcr.nodetype.NoSuchNodeTypeException: Node type cq:Project does not exist
at org.apache.jackrabbit.oak.util.TreeUtil.addMixin(TreeUtil.java:217)
at org.apache.jackrabbit.oak.jcr.delegate.NodeDelegate.addMixin(NodeDelegate.java:392)
at org.apache.jackrabbit.oak.jcr.session.NodeImpl$26.performVoid(NodeImpl.java:967)
at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performVoid(SessionDelegate.java:274)
at org.apache.jackrabbit.oak.jcr.session.NodeImpl.addMixin(NodeImpl.java:956)
at org.apache.sling.jcr.resource.internal.NodeUtil.handleMixinTypes(NodeUtil.java:63)
at org.apache.sling.jcr.resource.internal.JcrModifiableValueMap.put(JcrModifiableValueMap.java:449)
... 15 common frames omitted
Views
Replies
Total Likes
For a test, You can try to package the /content/projects folder and later remove all the contents under and redo the upgrade to make sure it succeeds. If it works, You can open a daycare case, I can have a look with deeper analysis on what could have happened.
My hunch is nodetype cq:Project is somehow not getting registered in the nodetype index but would need deeper analysis on this.
Views
Replies
Total Likes
Hi kunwar,
I have deleted the "/content/projects" from server and reran the upgrade but still seeing some more exception in upgrade log file. Also we have raised the Day care ticket.
Have you ever faced the javax.jcr.AccessDeniedException: Root node is not accessible Exception while upgrading the AEM. Now only Cq62ProjectsCodeUpgrade activity is successful out of below 20 upgrade activities. Please let me know if there is any solution for fixing the Root node is not accessible.
[Cq561SocialContentUpgrade, Cq60SocialContentUpgrade, Cq62WorkflowContentUpgrade, CQ63ProcessingProfileConfigUpdate, Cq62TargetContentUpgrade, CQ63MetadataSchemaConfigUpdate, CQ63InboxItemsUpgrade, Cq62TargetCampaignsContentUpgrade, Cq62MediaContentUpgrade, Cq62MobileContentUpgrade, Cq62PrimaryComponentsUpgrade, CQ62Html5SmartFileUpgrade, Cq62ProjectsCodeUpgrade, CQ63MonitoringDashboardsConfigUpdate, CQ63AssetsMetadataFormsUpdate, CQ63MobileAppsContentUpgrade, CQ63ToolsMenuEntriesContentUpgrade, Cq62ConfContentUpgrade, Cq62FormsContentUpgrade, CQ63AssetsSearchFacetsUpdate, CqMergeBackupUpgrade]
Views
Replies
Total Likes
I think something is messed up, Can you create a daycare case with us and I can do a deep dive and understand what’s happening here.
make sure you upload the upgrade.log and what all steps you took for this upgrade for review
Views
Replies
Total Likes
I have also faced the same problem as yours. I went to a lot of migration iterations just to settle with the steps below.
Please check the instructions below.
Let me know if it helps you.
------ STEPS -------
Performing an In-Place Upgrade
https://helpx.adobe.com/experience-manager/6-3/sites/deploying/using/in-place-upgrade.html
Performing The Upgrade
Views
Replies
Total Likes
Views
Likes
Replies
Views
Likes
Replies