Hello there,
In the last couple weeks, the nlserver6 service fails to start with the following error:
systemd[1]: Starting LSB: Neolane Communication Suite...
...
...
nlserver6[...]: Starting nlserver6: [ OK ]
systemd[1]: New main PID <nlserver_watchdog_pid> does not belong to service, and PID file is not owned by root. Refusing.
systemd[1]: New main PID <nlserver_watchdog_pid> does not belong to service, and PID file is not owned by root. Refusing.
systemd[1]: Failed to start LSB: Neolane Communication Suite.
systemd[1]: Unit nlserver6.service entered failed state.
systemd[1]: nlserver6.service failed.
Upon investigation, it is found that this was caused by a recent systemd package update to version systemd-219-67.el7_7.1.x86_64/systemd-219-67.el7_7.2.x86_64 which includes the following change which in turn introduced stricter PID files and MAINPID handling (as /var/run/nlserver6.pid is owned by 'neolane' user, but systemd expects it to be owned by root).
core: be stricter when handling PID files and MAINPID sd_notify() messages (#1663143) -> https://github.com/systemd/systemd/commit/db256aab13d8a89d583ecd2bacf0aca87c66effc
Excerpt from https://github.com/systemd/systemd/commit/db256aab13d8a89d583ecd2bacf0aca87c66effc#diff-ab78220e1270... wherein the above error originates from:
if (st.st_uid != 0) {
log_unit_error(UNIT(s), "New main PID "PID_FMT" does not belong to service, and PID file is not owned by root. Refusing.", pid);
return -EPERM;
}
Looks like a bug has been requested https://github.com/systemd/systemd/issues/8085 to fallback the above change as it breaks several daemons and a PR https://github.com/systemd/systemd/pull/8098 also made to relax the restriction. But, this doesn’t seem to be included in the CentOS 7.7.1908 update yet or probably planned in a future update and so we have downgraded systemd to the last version 219-62.el7_6.9 which doesn't include the above (PID restriction) change.
Steps to downgrade systemd to 219-62.el7_6.9 (from 219-67.el7_7.1/219-67.el7_7.2):
- Downgrade systemd to 219-62.el7_6.9:
>> yum downgrade http://vault.centos.org/7.6.1810/updates/x86_64/Packages/systemd-219-62.el7_6.9.x86_64.rpm http://vault.centos.org/7.6.1810/updates/x86_64/Packages/systemd-libs-219-62.el7_6.9.x86_64.rpm http://vault.centos.org/7.6.1810/updates/x86_64/Packages/systemd-sysv-219-62.el7_6.9.x86_64.rpm
Note: Downgrade other libs (like libgudev1) which requires latest systemd udpate (219-67.el7_7.1/219-67.el7_7.2)
- Disable systemd package on /etc/yum.conf:
# Disable systemd packages for install/update temporarily to workaround `core: be stricter when handling PID files and MAINPID sd_notify() messages (#1663143)` (https://github.com/systemd/systemd/commit/db256aab13d8a89d583ecd2bacf0aca87c66effc) included in 219-67.el7_7.1/219-67.el7_7.2
exclude=systemd*
Hope this helps for others who are facing the above issue.
Adobe Campaign Support Team - Suggest you to implement a fix for this on nlserver6 service script (possibly?) (/etc/init.d/nlserver6) as the above systemd PID file handling restriction is likely to be included in a future release again. Thanks!
Cheers,
Selva
