Getting the replication status of a content fragment, problems on publish instance

[DanielMa16]

Hello

In one of our functionalities, we use a servlet that recovers the information of some content fragments in the project, however our client is asking us to dismiss content fragments that exist but are not published.

To achieve this, we have tried a few things, this being the most promising from what we have researched.

Resource cfResource = resolver.getResource(pathCf);
ReplicationStatus replicationStatus = cfResource.adaptTo(ReplicationStatus.class);
JsonObject isCfPublished = new JsonObject();
isCfPublished.addProperty("isPublished", replicationStatus.isActivated());
Boolean statusCf = isCfPublished.get("isPublished").getAsBoolean();
LOGGER.warn(pathCf+" isPublished "+statusCf);

this works well in showing only published Content fragments, however if one content fragment is unpublished, and then published again, the publish instance of our AEM as a cloud service instance is still recognizing it as deactivated. When in author it is showing the content fragment as valid again.

Is there a delay from author to publish as far as recovering the replicationStatus? is publish somehow unable to recover the updated value?

Is there a more effective way we could do this operation, that would work on publish?

Any advice is welcomed.

Thanks in advance

[kautuk_sahni]

@partyush @RajaShankar @Vijayalakshmi_S @Manasi29 @Nupur_Jain @rk_pandian @himanshu @Aaron_Dempwolff Curious to hear your perspectives on this question. Do you all mind sharing your thoughts?

[rk_pandian]

ok, I see the issue here, the content fragment when deactivated and activated again from author has cq:lastReplicationAction_publish as Deactivate in the publish instance, but if you simply activate the same content fragment again, the status gets modified correctly in publish with the latest timestamp. I spun up a new cloud instance and literally it was empty, so could be a bug in the product.

@DanielMa16 looking at your requirement from a different perspective - you want to remove the content fragments that are not published, right?

is my understanding correct that you want to remove them from the author instance? because if they were never published or unpublished at any point of time, then they will never cease to exist in the publish instance as well, so I'm a bit curious on why you are interested in the status from the publish instance.

Please correct me if I'm wrong.

Regards,

Ramkumar

[DanielMa16]

Hello @rk_pandian our client wants to have unplublished content fragments present in the environment to publish them when they please, but they dont want us to take them into account when they are unpublished.

Best Regards.

[Manasi29]

@kautuk_sahni 
My understanding for the query is that in author instance once the content fragment is unpublished the status is deactivated and when republished it's activated but in Publish Instance even though the unpublished CF is republished the status remains to be deactivated.

Please correct me if there's any gap in my understanding.

But if I got it right, @DanielMa16 below are few steps I would like you to follow and see if the issue still persists,

1. First step you can take is checking the replication queue to check if there are any errors occurring, as in AEM as cloud it may have delays.
2. Please check if in your project you are caching the content fragments, if so try checking the logs to see if the status is updated after clearing the dispatcher cache.
3. If clearing cache resolves the issue it indicates that you have to check the dispatcher caching config, as sometimes the stale content is served due to dispatcher not invalidating the cache. So make sure that you are invalidating or deleting the dispatcher cache after publishing.
4. You can try using workflow API to check the content 1fragment publication status which will programmatically ensure a more accurate status of content replication.

From my understanding of the issue and experience working with AEM, I strongly believe this could be a cache issue so please take a close look on that.

[DanielMa16]

The curious thing is that after republishing them, they show up as activated in the repository browser of the developer console, they show up as "Activate" as their replication status, both in author and publish, but when accessing the value through our java servlet, the response for 

replicationStatus.isActivated()

is false in publish instance.

Regardless we will investigate what you mention.

Best Regards.

[Manasi29]

As you are mentioning the replication status is correct in the repository browser but replicationStatus.isActivated() returns false leads my thought process to the question that how is the replication status is being cached or retrieved within your servlet session.

You can try checking few more things -
1. Along with isActivated try accessing JCR properties as I believe they could provide more accurate status as you mentioned in repository browser the status is correct.
2. From the code snippet you provided I am unable to guess where are you acquiring the resource resolver from but make sure that you are using a service user (system user) here that has correct permissions, instead of using a request or cached session.
3. Also you can try explicitly refreshing the session or resource resolver to ensure getting the latest data as JCR session cache might return stale data.
4.Moreover there's a possibility that status is updated in repository browser but servlet's still hitting the node that's being processed so you could try adding some delay before checking the replication status.
5. And last but not least as I mentioned in my previous reply have a close look at dispatcher cache invalidation as it may happen that the AEM instance has up to date data but the status is being served from the cache.

[Jörg_Hoh]

I wonder why need this logic on publish at all; because by definition all content on publish is already published. Or do I miss something?

Also, the replication status information is added after the replication has taken place; and the replication process itself replicates the content fragment as it is (with or without replication status information).

So as a timeline

Assume you have a content fragment called CF, and it is not replicated yet. For that reason it has the state CF0 (no replication status information.

At time t1 you replicate that CF

-> on publish you have CF

After completion on author the replication status is added to CF, so it's now CF1

At time t2 you make some changes to the content fragment, and you activate it again.

-> on publish you have CF1 now.

After completion on author the replication status is update, so it's now CF2

That means on publish you can never rely on the replication status information.

[DanielMa16]

I see, our client is asking us to not add these unpublished CFs in some calculations, but still have them in the system, since the replication status is not releable enough i assume it would be better to either add a field in the CF to take this into account or use another field to exclude them.

I imagine that would be more reliable.

Best Regards.

[Jörg_Hoh]

Where do these calculations happen? On author? In that situation the replication status is accurate. It's just that the information on publish is not accurate, but at that point you don't really need it at all.

[DanielMa16]

We get the CFs in a call from a servlet, which is where the code snippet we shared comes from, where we take the information of those CFs and we compare it with some information with a middleware client, checking if they are unplublished to see if we add them to the comparison at all, thats why we need to know in publish the replication status of the Fragment.

Best Regards.

[Jörg_Hoh · Accepted solution]

But when you request that information from a publish instance, isn't the replication status always (and implicit) "replicated"? Because otherwise these CFS would not be present on a publish instance.