Content Replication Queue Blocked Detected-

[ashishr1501]

We have a big team of content authors and one of them was trying to publish and we got an alert saying - Blocked Replication Queue, which i already cleared and copied the logs for investigation. Can anyone please help to guide, if anything is wrong or the authors are publishing in a wrong manner?

2025/03/12 13:57:07:544 - INFO - Successfully applied package with id dstrpck-1741787823269-91c4fd10-6d60-45de-a58b-67d282f22df1, type ADD, paths [/content/kpmg/us/en/webcasts/2025/ask-the-experts, /content/dam/kpmg/event-hero/2025/ask-the-experts-2000x1125.jpg]
2025/03/12 14:07:34:120 - INFO - Request accepted with distribution package PackageMessage(pubSlingId=f1f2ea2c-c9f7-4757-b555-a51323af536b, reqType=ADD, pkgId=dstrpck-1741788454042-08512221-7f2c-4f12-ac0f-f14a5e2d0c01, pkgType=journal_filevault, pkgLength=52176, pubAgentName=preview, userId=replication-service, paths=[/content/kpmg/us/en/articles/2024/strategic-workforce-planning-in-the-age-of-genai], deepPaths=[], metadata={}) at offset=72840959, queueSize=1, queueSizeDelay=0
2025/03/12 14:07:37:879 - WARN - Message: Failed attempt (0/infinite) to import the distribution package PackageMessage(pubSlingId=f1f2ea2c-c9f7-4757-b555-a51323af536b, reqType=ADD, pkgId=dstrpck-1741788454042-08512221-7f2c-4f12-ac0f-f14a5e2d0c01, pkgType=journal_filevault, pkgLength=52176, pubAgentName=preview, userId=replication-service, paths=[/content/kpmg/us/en/articles/2024/strategic-workforce-planning-in-the-age-of-genai], deepPaths=[], metadata={}) at offset=72840959 because of 'javax.jcr.nodetype.ConstraintViolationException: OakConstraint0026: /content/kpmg/us/en/wireframe-designs/chitvan-wireframing-space/strategic-workforce-planning-in-the-age-of-genai/jcr:content/image/file[[nt:file]]: Mandatory child node jcr:content can not be removed', the importer will retry later,
Stacktrace: org.apache.sling.distribution.common.DistributionException: javax.jcr.nodetype.ConstraintViolationException: OakConstraint0026: /content/kpmg/us/en/wireframe-designs/chitvan-wireframing-space/strategic-workforce-planning-in-the-age-of-genai/jcr:content/image/file[[nt:file]]: Mandatory child node jcr:content can not be removed
at org.apache.sling.distribution.serialization.impl.vlt.FileVaultContentSerializer.importFromStream(FileVaultContentSerializer.java:159)
at org.apache.sling.distribution.packaging.impl.InMemoryDistributionPackageBuilder.installPackageInternal(InMemoryDistributionPackageBuilder.java:119)
at org.apache.sling.distribution.packaging.impl.AbstractDistributionPackageBuilder.installPackage(AbstractDistributionPackageBuilder.java:156)
at org.apache.sling.distribution.monitor.impl.MonitoringDistributionPackageBuilder.installPackage(MonitoringDistributionPackageBuilder.java:92)
at org.apache.sling.distribution.serialization.impl.vlt.VaultDistributionPackageBuilderFactory.installPackage(VaultDistributionPackageBuilderFactory.java:317)
at org.apache.sling.distribution.journal.bookkeeper.PackageHandler.installAddPackage(PackageHandler.java:78)
at org.apache.sling.distribution.journal.bookkeeper.PackageHandler.apply(PackageHandler.java:61)
at org.apache.sling.distribution.journal.bookkeeper.BookKeeper.importPackage(BookKeeper.java:164)
at org.apache.sling.distribution.journal.impl.subscriber.DistributionSubscriber.processQueueItem(DistributionSubscriber.java:397)
at org.apache.sling.distribution.journal.impl.subscriber.DistributionSubscriber.fetchAndProcessQueueItem(DistributionSubscriber.java:348)
at org.apache.sling.distribution.journal.impl.subscriber.DistributionSubscriber.processQueue(DistributionSubscriber.java:324)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: javax.jcr.nodetype.ConstraintViolationException: OakConstraint0026: /content/kpmg/us/en/wireframe-designs/chitvan-wireframing-space/strategic-workforce-planning-in-the-age-of-genai/jcr:content/image/file[[nt:file]]: Mandatory child node jcr:content can not be removed
at org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:226)
at org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:213)
at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.newRepositoryException(SessionDelegate.java:738)
at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.save(SessionDelegate.java:552)
at org.apache.jackrabbit.oak.jcr.session.SessionImpl$9.performVoid(SessionImpl.java:459)
at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performVoid(SessionDelegate.java:300)
at org.apache.jackrabbit.oak.jcr.session.SessionImpl.save(SessionImpl.java:456)
at com.adobe.granite.repository.impl.CRX3SessionImpl.save(CRX3SessionImpl.java:220)
at org.apache.jackrabbit.vault.fs.io.AutoSave.saveWithBackoff(AutoSave.java:204)
at org.apache.jackrabbit.vault.fs.io.AutoSave.save(AutoSave.java:176)
at org.apache.jackrabbit.vault.fs.io.Importer.run(Importer.java:532)
at org.apache.sling.distribution.serialization.impl.vlt.FileVaultContentSerializer.importFromStream(FileVaultContentSerializer.java:151)
... 11 more
Caused by: org.apache.jackrabbit.oak.api.CommitFailedException: OakConstraint0026: /content/kpmg/us/en/wireframe-designs/chitvan-wireframing-space/strategic-workforce-planning-in-the-age-of-genai/jcr:content/image/file[[nt:file]]: Mandatory child node jcr:content can not be removed
at org.apache.jackrabbit.oak.plugins.nodetype.TypeEditor$1.onConstraintViolation(TypeEditor.java:110)
at org.apache.jackrabbit.oak.plugins.nodetype.TypeEditor.constraintViolation(TypeEditor.java:235)
at org.apache.jackrabbit.oak.plugins.nodetype.TypeEditor.childNodeDeleted(TypeEditor.java:323)
at org.apache.jackrabbit.oak.spi.commit.VisibleEditor.childNodeDeleted(VisibleEditor.java:110)
at org.apache.jackrabbit.oak.spi.commit.CompositeEditor.childNodeDeleted(CompositeEditor.java:135)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeDeleted(EditorDiff.java:165)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:581)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:147)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:597)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:147)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:444)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:650)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:147)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:597)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:147)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:444)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:650)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:147)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:444)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:650)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:147)
at org.apache.jackrabbit.oak.segment.MapRecord$3.childNodeChanged(MapRecord.java:430)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:517)
at org.apache.jackrabbit.oak.segment.MapRecord.compareBranch(MapRecord.java:595)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:496)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:420)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:650)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:147)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:444)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:650)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:147)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:444)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:650)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:147)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:444)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:650)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:147)
at org.apache.jackrabbit.oak.composite.CompositeNodeState$WrappingDiff.childNodeChanged(CompositeNodeState.java:309)
at org.apache.jackrabbit.oak.composite.CompositeNodeState$ChildrenDiffFilter.childNodeChanged(CompositeNodeState.java:256)
at org.apache.jackrabbit.oak.segment.MapRecord$4.childNodeChanged(MapRecord.java:471)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:517)
at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:462)
at org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:650)
at org.apache.jackrabbit.oak.composite.CompositeNodeState.compareAgainstBaseState(CompositeNodeState.java:163)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:51)
at org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:54)
at org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(CompositeHook.java:60)
at org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(CompositeHook.java:60)
at org.apache.jackrabbit.oak.composite.CommitHookEnhancer.processCommit(CommitHookEnhancer.java:55)
at org.apache.jackrabbit.oak.segment.scheduler.Commit.apply(Commit.java:99)
at org.apache.jackrabbit.oak.segment.scheduler.LockBasedScheduler.execute(LockBasedScheduler.java:299)
at org.apache.jackrabbit.oak.segment.scheduler.LockBasedScheduler.schedule(LockBasedScheduler.java:270)
at org.apache.jackrabbit.oak.segment.SegmentNodeStore.merge(SegmentNodeStore.java:212)
at org.apache.jackrabbit.oak.composite.CompositeNodeStore.merge(CompositeNodeStore.java:150)
at org.apache.jackrabbit.oak.core.MutableRoot.commit(MutableRoot.java:261)
at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.commit(SessionDelegate.java:395)
at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.save(SessionDelegate.java:550)
... 19 more

[DPrakashRaj · Accepted solution]

Log suggest that this file has a problem.

content/kpmg/us/en/wireframe-designs/chitvan-wireframing-space/strategic-workforce-planning-in-the-age-of-genai/jcr:content/image/file

From log 

org.apache.jackrabbit.oak.api.CommitFailedException: OakConstraint0026: /content/kpmg/us/en/wireframe-designs/chitvan-wireframing-space/strategic-workforce-planning-in-the-age-of-genai/jcr:content/image/file[[nt:file]]: Mandatory child node jcr:content can not be removed
at org.apache.jackrabbit.oak.plugins.nodetype.TypeEditor$1.onConstraintViolation(TypeEditor.java:110)

suggestion: when posting logs on this forum try to mask the sitename directly for more security and my be to comply with your office IT policy

[giuseppebag]

The error occurs because the mandatory jcr:content node is missing or removed from the nt:file node under the specified path. Here’s a quick attempt to fix:

1. Verify the node structure in CRXDE Lite and ensure the jcr:content node exists under the file node.
2. If missing, recreate the jcr:content node with the required properties (jcr:mimeType, jcr:lastModified, etc.).
3. Investigate scripts or processes that might be removing the node to prevent future issues.

If needed, use AEM logs or audit trails to identify the root cause.

[AmitVishwakarma]

Hi @ashishr1501 ,

It looks like the replication failure happened due to a missing jcr:content node under an nt:file node. This mandatory node must exist, and any attempt to remove or modify it incorrectly causes a ConstraintViolationException.

Steps to Fix:

1. Check Node Structure in CRXDE Lite:

     - Navigate to /content/kpmg/us/en/wireframe-designs/chitvan-wireframing-space/strategic-workforce-planning-in-the-age-of-genai/jcr:content/image/file
     - Ensure that the jcr:content node exists under the file node.

2. Recreate the jcr:content Node (If Missing):

     - Add a jcr:content node under the file node.
     - Set the required properties:

          - jcr:mimeType
          - jcr:lastModified
     - Any other required metadata.

3. Check for Any Custom Code or Workflows:

     - Identify scripts or workflows that might be incorrectly deleting or modifying this node.
     - Review the replication logs and audit logs to see when this node was removed or modified.

4. Retry Replication After Fixing:

     - After restoring the correct structure, clear the replication queue again and attempt to republish the content.

Regards,
Amit Vishwakarma