Highlighted

Replication agent config group by runmode

kgio

23-09-2016

There seems to be some inconsistency between how run modes are handled by the replication agent configuration and by other configurations in AEM.

For reference, i am using AEM 6.2, com.adobe.granite.replication.core:6.0.14

At issue here is the fact that when multiple run modes are appended to a replication agent group, they are evaluated with an OR operation.
Let me give an example. The obvious run modes that everyone will be familiar with is "author" and "publisher"; thus, you will typically have replication agents found under both /etc/replication/agents.author and /etc/replication/agents.publish.

Now, we know that when doing OSGI configurations, a group of configurations applies if, and only if, it matches all of the runmodes. Thus, the run modes can be used to restrict the configurations. Lets say that for your production set you applied the run mode "prod" to all of the AEM instances.
It would then make sense to have configurations that are:

  • config.author            //applies to all author instances
  • config.author.prod    //applies to only prod author instances
  • config.publish           //applies to all publish instances
  • config.publish.prod   //applies to only prod publish instances
  • config.prod               //applies to all prod instances, both author and publish
  • config                       //applies to all instances

Now that we understand the behaviour for OSGI configurations; i would expect the same sort of run mode based configuration logic to apply to replication agent configurations as well. However, if i use the following replication agent group:

  • /etc/replication/agents.author.prod

it does not mean agents for prod author instances (as it would with OSGI configurations). Instead, it means that these replication agents are applicable for any author or prod instance. So an instance with the run modes "prod, publish" would apply because it has prod, as would "author, test" because it has author. 

I fail to see any utility in this. Adding more run modes doesn't further constrain the configurations, it actually relaxes the constraints. 
Additionally, when looking at the code I see that there is a special carve-out for using * to mean "all run modes". Given how this actually works, that could have simply been accomplished by placing replication agents at /etc/replication/agents.author.publish. That special case, which doesn't need to exist, makes me feel like the run modes should have been evaluated with an AND condition, in which case there might be some use to having a wildcard.

We would like to place our replication agents under SCM, and not have to juggle having different repositories for each set of servers. We already do this with everything else (such as OSGI) configurations, such that deploying to or updating an instance can largely be automated. As it stands now, we can't do that with replication agents because I cant make configurations like the following:

  • agents.prod.author
  • agents.prod.publish
  • agents.test.author
  • agents.test.publish
  • etc.

in the above example, on an instance with the run modes "prod, author". The first 3 sets of replication agents all apply. Is i said; I fail to see any utility in this behaviour.
If you wanted some configuration to apply to two different run modes, then why wouldn't you just add a common run mode to all those machines, as they surely share something in common.

Does anyone have an example where the current behaviour is of any benefit? 
Is there any way, with the existing behaviour, to accomplish what we want?
Changing this behaviour consists of changing 2 lines in "AgentConfigGroupImpl", does anyone see any reason the behaviour shouldn't be changed to just be the same as other configurations?

 

Thanks, 

Kyle

Replies

Highlighted

smacdonald2008

23-09-2016

This level of detail goes far beyond the product docs: 

https://docs.adobe.com/docs/en/aem/6-2/deploy/configuring/replication.html

 I have not heard of this many replication configurations. 

Do you expect all of this to be in the AEM docs? 

Highlighted

Feike_Visser1

Employee

23-09-2016

To me what you can do is to have packages, with the right config, and put these into folders like install.prod.author and so on.

Then these packages will be installed when the correct runmode is configured.

Highlighted

kgio

23-09-2016

There really aren't all that many replication configurations. It's just multiple sets, like "test", "prod", "staging", "preview", etc. So each set has its own specific publishers, flush agents, etc. 
We could just keep all the configurations separate and know that we need to install the staging replication agents on staging, and the production replication agents on production, and so on, It's just much more convenient when there is only one thing to do and any instance can behave like any other instance simply by setting the run mode. 

The docs largely assume that you would only ever do "agents.author" and "agents.publish"; but it is documented that any run modes are evaluated with an OR condition. I don't expect more from the docs, I just found the way it works to be counterintuitive and problematic.

 

The suggestion to use install folders to install the proper packages may be a workable solution. It would be a more complex setup, but would probably do the trick. I will play with that and see if its workable.

Highlighted

Opkar_Gill

Employee

25-09-2016

Hi Kyle,

it seams the behaviour you have seen is the OOTB behaviour as defined at[0]

Just thinking out loud here but, what if you used a run mode that does not include the period character, such as "/etc/replication/agents.agentsprodauthor/repAgentName" and "/etc/replication/agents.agentsprodpublish/repAgentName", which would only match that exact text and not a series of tokens? It would be a case of adding another run mode just for the replication agent configuration. The drawback is that you are adding a run mode specifically for the replication agents and it won't be the same as for the other OSGI configurations.

Regards,

Opkar

[0]https://helpx.adobe.com/experience-manager/kb/RunModeDependentConfigAndInstall.html

Highlighted

mark_chester

22-05-2017

Hi Kyle,

As someone who is rather frustrated with this exact problem, I've had to do the following:

Create runmodes for each instance that combine the values I need, for example, my dev servers have:

  • publish, dev, d, publish-dev
  • author, dev, d, author-dev

And then I have to use the hyphenated value in my replication agents.

  • /etc/replication/agents.author-dev
  • /etc/replication/agents.publish-dev

It's janky, and I wish Adobe would make all runmode handling consistent across all configs.  But it works*, and allows all agents to be held in your code/config repo.

*ONE CAVEAT: AEM seems to trip up on runmodes in replication agents a bit.  In some cases, editing agents results in weird UI issues.  For example, all agents appear to be disabled, but they are enabled when you edit them.

Highlighted

kgio

23-05-2017

That is the solution we had to go with as well.

A feature request was made to have the runmode configurations for the replication agents match the runmode configurations for everything else. 

That ticket is GRANITE-14360.  I was told that the feature wouldn't even be considered until at least 6.4. If you ask about it then reference that ticket number and maybe it will get more attention.

 

kyle

Highlighted

konradw58070066

05-05-2020

Even with bundle `com.adobe.granite.replication.core` in Version 6.2.86 this has not been fixed (i.e. multiple run modes separated by "." are interpreted as OR not as AND). This is the version which ships with AEM 6.5(.0)