Issue with Dispatcher Flush Agents

[sambashiva0808]

Hi Everyone,

We're trying to clear the Dispatcher Cache of the Vanity URLs using some custom logic, where it is not exactly working.
We've created a page and configured some Vanity URLs for that page.  Now when we activate the page all the URLs(Page URL and Vanity URLs) are passing through the Dispatcher Flush Agent of Author, but the Flush Agent configured in Publish is not getting the the Vanity URLs but the page URL is passing through the agent.

Example:
Created a page "/content/vanity-test" and configured "/test-vanity-1" and "/test-vanity-2" to that page. Now when I activate the page, all the URLs:

• /content/vanity-test
• /test-vanity-1
• /test-vanity-2

are passing through the Author Dispatcher.

But the Vanity URLs are not getting into the Publish Dispatcher, only the page URL is passing through it.

Custom Code:

for (Map.Entry<String, Agent> e : agentManager.getAgents()

.entrySet()) {

     if ((e.getValue().isCacheInvalidator()

     && (e.getValue().getConfiguration().getTransportURI().contains(IntegrationConstants.CACHE_INVALDATION_URL))||

     e.getValue().getConfiguration().getTransportURI().contains(IntegrationConstants.AKAMAI_INVALDATION_URL))) {

           Agent dispatcherFlush = e.getValue();

          try {

               ReplicationAction ra = new ReplicationAction(

               ReplicationActionType.DELETE, path);

               ReplicationContent rc = dispatcherFlush.buildContent(session, ra);

               dispatcherFlush.replicate(ra, rc,

               new ReplicationOptions());

               LOGGER.debug("$Activator cache flush requested. Check queue");

               } catch (ReplicationException ex) {

                      LOGGER.error(ex.getMessage());

               }

       }

}

Please suggest why it would happen, where in Author Agent the Vanity URLs are going through but not in Publish Agent.

Note :

• The Flush Agent configured in Author is only for testing, main Flush Agent is configured in Publish instance only.
• The above code is in separate bundle and is deployed only in author not in publish. Is that could be the reason for the above issue ?
  If yes, how can I invoke my custom code in Publish instance as I'm relying on the Preprocessor Class when I replicate the page, where as this activation process cannot happen in Publish how can I invoke the custom code for clearing the cache of Vanity URLs. Please suggest. Thank you
  
  

[kautuk_sahni]

Jörg Hoh​ Need your help here.

[Jörg_Hoh]

Have you checked the "Alias update" box in the configuration dialog of the invalidation agent on the publish instance (tab "Settings" at the very bottom)?

Jörg

[sambashiva0808]

I tried checking for that but "Alias Update" Check box is not available in the Configuration Dialog. We're using AEM 6.1 version.

[sambashiva0808]

For your reference, Please find the screenshot of the configuration dialog of the invalidation agent.

[Jörg_Hoh]

Check in the JCR repository (using CRXDE) if on your authoring instance on the /etc/replication/agents.author/publish/jcr:content node there is a property "aliasUpdate" with the value "true" (boolean type). And if it is missing on the agent on the publishing instance.

The code is already there to handle it, although the UI for AEM 6.1 does not yet expose that feature.

Jörg

[creative83]

The location you gave is default Publish agent. We have our Custom Publish agents but these are configured on Author box only as those are not needed in Publish box. In Publish box there are only flush agents configured.

Can you please let us know where this aliasUpdate property should be added? Only in other or in Author publish replication agents and Publish Flush agent both?

[Jörg_Hoh]

Check on the system, where this is working, and compare to those where it does not. Adjust the agent names accordingly.

Jörg

[creative83]

Thanks for your quick response. As of now in 6.1 none of the systems this attribute is present as it is not present in UI as well as we never used this functionality.

As you have advised I have added that "aliasUpdate" as true in my Author box "Publish Replication Agent". I referred Replication documentation as well and it was same as you mentioned and not asking to do anything additional.

Do I need to change any thing in Publish Flush Agent or Dispatcher?

[Jörg_Hoh]

The question for me is now: Why does it work on one instance and not on the other (the ones you shared the screenshots from) if you have configured these invalidation agents the very same way.

Can you please check your logs of both systems and paste the log statements like these:

Setting up replication with options: [...]

That should indicate if the aliasUpdate mechanism of the replication itself is used or if something else is causing the replication/invalidation of the vanity URLs.

Jörg

[creative83]

The screenshot shared above of Author box Dispatcher is wrong I guess. We should not have Dispatcher configured at all right? We jsut need Replication Agents. Below changes I have done:

- I have disabled Dispatcher Flush agent in Author box.

Added aliasUpdate property to my publish replication agent in Author box.

Anything else in Author or Publish?

[Jörg_Hoh]

Please do not change anything yet; let's validate first. Can you get the logs?

Jörg

[sambashiva0808]

Hi Jörg,

• We've created a sample page (/content/vanity/vanity-test) and authored a Vanity URL(/content/vanityTest) on that page.
• We've updated the Default Replication Agent of Author instance with "aliasUpdate" property to true and also updated the Dispatcher Flush Agent of Publish with "aliasUpdate" property to true.
• Now when we activate the page it is not throwing any error in siteadmin, but we see there's some exception in error logs. Meanwhile, the dispatcher cache of the page is getting cleared in this process where as the cache of the Vanity URL is not getting cleared).

Note : It is throwing the same exception even if we add the "aliasUpdate" property to Default Replication Agent of Author and not in Flush Agent of Publish.

Exception in error.log (Author Instance):

20.03.2018 19:34:46.547 *INFO* [0:0:0:0:0:0:0:1 [1521554686546] POST /bin/replicate.json HTTP/1.1] com.day.cq.replication.impl.ReplicatorImpl Setting up replication with options: ReplicationOptions{synchronous=false, revision='null', suppressStatusUpdate=false, suppressVersions=false, filter=null, aggregateHandler=null}

20.03.2018 19:34:46.606 *INFO* [0:0:0:0:0:0:0:1 [1521554686546] POST /bin/replicate.json HTTP/1.1] com.day.cq.wcm.core.impl.PageManagerImpl Created new version 1.4 for /content/vanity/vanity-test

20.03.2018 19:34:46.607 *INFO* [0:0:0:0:0:0:0:1 [1521554686546] POST /bin/replicate.json HTTP/1.1] com.day.cq.wcm.core.impl.VersionManagerImpl Create new version RevisionImpl{id='89b595cb-c810-4f36-a47f-1da4b7d47716', label='1.4', existingPath='/content/vanity/vanity-test'} for modified page /content/vanity/vanity-test.

20.03.2018 19:34:46.607 *INFO* [0:0:0:0:0:0:0:1 [1521554686546] POST /bin/replicate.json HTTP/1.1] com.day.cq.replication.Agent.test_and_target Creating content for path /content/vanity/vanity-test

20.03.2018 19:34:46.608 *INFO* [0:0:0:0:0:0:0:1 [1521554686546] POST /bin/replicate.json HTTP/1.1] com.day.cq.replication.Agent.publish Creating content for path /content/vanity/vanity-test

20.03.2018 19:34:46.623 *INFO* [0:0:0:0:0:0:0:1 [1521554686546] POST /bin/replicate.json HTTP/1.1] com.day.cq.replication.impl.ReplicationContentFactoryProviderImpl Created new repository content at /var/replication/data/8cd83ed8-9543-4178-8c55-874ba606c0bf/83/835fad67-5e7d-4352-9115-1d531d1a871c (size=1311, lastmod=2018-03-20T19:34:46.617+05:30)

20.03.2018 19:34:46.631 *INFO* [0:0:0:0:0:0:0:1 [1521554686546] POST /bin/replicate.json HTTP/1.1] com.day.cq.replication.Agent.test_and_target.queue Replication request queued for test_and_target at /content/vanity/vanity-test.

20.03.2018 19:34:46.638 *INFO* [pool-12-thread-6-com_day_cq_replication_job_test_and_target(com/day/cq/replication/job/test_and_target)] com.day.cq.replication.Agent.test_and_target.queue Processing job for agent test_and_target

20.03.2018 19:34:46.638 *INFO* [pool-12-thread-6-com_day_cq_replication_job_test_and_target(com/day/cq/replication/job/test_and_target)] com.day.cq.replication.Agent.test_and_target.queue Job for agent test_and_target processed in 0ms. Ok.

20.03.2018 19:34:46.643 *INFO* [0:0:0:0:0:0:0:1 [1521554686546] POST /bin/replicate.json HTTP/1.1] com.day.cq.replication.Agent.publish.queue Replication request queued for publish at /content/vanity/vanity-test.

20.03.2018 19:34:46.646 *INFO* [0:0:0:0:0:0:0:1 [1521554686546] POST /bin/replicate.json HTTP/1.1] com.day.cq.replication.impl.ReplicatorImpl Processed replication: setup 0ms, checked 0ms, pre 60ms, build 17ms, queued 19ms, status 3ms, total 99ms

20.03.2018 19:34:46.654 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish.queue Processing job for agent publish

20.03.2018 19:34:46.654 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish Sending POST request to http://localhost:4503/bin/receive?sling:authRequestLogin=1

20.03.2018 19:34:46.704 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish sent. Response: 200 OK

20.03.2018 19:34:46.704 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish ------------------------------------------------

20.03.2018 19:34:46.704 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish Sending message to localhost:4503

20.03.2018 19:34:46.704 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish >> POST /bin/receive HTTP/1.0

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish >> Action: Activate

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish >> Path: /content/vanity/vanity-test

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish >> Handle: /content/vanity/vanity-test

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish >> Referer: about:blank

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish >> ...spooling 1311 bytes...

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish --

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish << HTTP/1.1 200 OK

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish << Date: Tue, 20 Mar 2018 14:04:46 GMT

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish << X-Content-Type-Options: nosniff

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish << Content-Type: text/plain; charset=UTF-8

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish << Content-Length: 30

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish << Server: Jetty(9.2.9.v20150224)

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish << X-Powered-By: Jetty(9.2.9.v20150224)

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish <<

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish << ReplicationAction ACTIVATE ok.

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish Message sent.

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish ------------------------------------------------

20.03.2018 19:34:46.705 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish Replication (ACTIVATE) of /content/vanity/vanity-test successful.

20.03.2018 19:34:46.706 *INFO* [pool-12-thread-12-com_day_cq_replication_job_publish(com/day/cq/replication/job/publish)] com.day.cq.replication.Agent.publish.queue Job for agent publish processed in 52ms. Ok.

20.03.2018 19:34:46.724 *ERROR* [pool-25-thread-49] org.apache.sling.extensions.threaddump.internal.Activator Uncaught exception in Thread Thread[pool-25-thread-49,5,main]

java.lang.IllegalArgumentException: null

at com.day.cq.wcm.msm.impl.Utils.appendPath(Utils.java:98)

at com.day.cq.wcm.msm.impl.SourceTargetMap$TreeConfig.getTargetEndPoints(SourceTargetMap.java:343)

at com.day.cq.wcm.msm.impl.SourceTargetMap.index(SourceTargetMap.java:186)

at com.day.cq.wcm.msm.impl.SourceTargetMap.addSourTargetFilter(SourceTargetMap.java:61)

at com.day.cq.wcm.msm.impl.LiveCopyIndexImpl.buildIndex(LiveCopyIndexImpl.java:334)

at com.day.cq.wcm.msm.impl.LiveCopyIndexImpl.getTargetLiveCopies(LiveCopyIndexImpl.java:143)

at com.day.cq.wcm.msm.impl.LiveCopyManagerImpl.getDependentLiveCopies(LiveCopyManagerImpl.java:513)

at com.day.cq.wcm.msm.impl.LiveRelationshipManagerImpl.isSource(LiveRelationshipManagerImpl.java:322)

at com.day.cq.wcm.msm.impl.MSMEventProcessor.filterBlueprint(MSMEventProcessor.java:94)

at com.day.cq.wcm.msm.impl.MSMEventProcessor.access$000(MSMEventProcessor.java:39)

at com.day.cq.wcm.msm.impl.MSMEventProcessor$PageEventBuilder.run(MSMEventProcessor.java:120)

at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

[Jörg_Hoh]

I don't think that the exception is directly related to your problem. You should raise a daycare ticket and ask if there is a fix for that exception (or in which CFP the fix is contained).

In your initial posting of this problem, you added a screenshot which shows that for a vanity URL an additional invalidation request has been created, and that this additional request is visible in the replication queue. So somehow it is possible to achieve that behaviour you want to achieve, but we don't know how.

Sorry, I ran out of ideas at the moment. Have you already contacted Adobe Support for this?

Jörg

[creative83]

before we close this thread I have few questions to you....

1) What is the expected behaviour of aliasUpdate flag? Were you expecting that flag to invoke invalidate script for vanity URL as well?

2) What we are trying to achieve is not a custom solution. AEM OOTB provides Vanity and that can be achieved without having REDIRECT. In this case dispatcher creates cache. What is AEM's solution for clearing this dispatcher cache?

[Jörg_Hoh]

* The aliasUpdate property should trigger logic which checks for configured vanity URLs and Aliases, and creates invalidation requests for them too.

* and as stated, in AEM 6.3 there is UI to configure it, but AFAIK the logic behind it is present also in earlier versions.

You should definitly follow up with Adobe Support on this.

Jörg

[creative83]

@Jörg Hoh I am seeing below error when I activate a page which is having Vanity URL associaed with it. THis errors I see in Replication log of Author itself where I have added aliaUpdate=true. Thought to share this with you before raising Adobe ticket.

Source Page:/content/XXX/offers/xx-offers/abc-plus-holiday

Vanity assocaited with it : /content/XXX/holiday

27.03.2018 13:30:44 - INFO - 10-59-2-69-24611 : ------------------------------------------------

27.03.2018 13:30:44 - INFO - 10-59-2-69-24611 : Replication (ACTIVATE) of /content/XXX/offers/xx-offers/abc-plus-holiday successful.

27.03.2018 14:22:11 - INFO - 10-59-2-69-24611 : Creating content for path /content/XXX/holiday

27.03.2018 14:22:11 - ERROR - 10-59-2-69-24611 : Error while building replication content com.day.cq.replication.ReplicationException: RepositoryException during serialization

27.03.2018 14:22:11 - INFO - 10-59-2-69-24611 : Sending POST request to https://10.59.2.69:24611/bin/receive?sling:authRequestLogin=1

27.03.2018 14:22:12 - INFO - 10-59-2-69-24611 : sent. Response: 200 OK

27.03.2018 14:22:12 - INFO - 10-59-2-69-24611 : ------------------------------------------------

[Jörg_Hoh · Accepted solution]

I cannot provide that much help based on this shared error message. Please go on and raise the ticket.

regards,

Jörg