Write AEMasaCS logs directly to ElasticSearch | Community
Skip to main content
R
RustamMussabay
Level 2
April 18, 2024
Solved

Write AEMasaCS logs directly to ElasticSearch

  • April 18, 2024
  • 2 replies
  • 2793 views

Hello,

 

Is it possible to configure AEMasaCS project to write logs directly to external ElasticSearch to monitor logs in Kibana?

I know that we can download/tail logs via aio and then use beats to submit data to ES, but I was told to investigate if there is a way to send data directly. I've checked ACS Commons and saw a SyslogAppender feature that is not compatible with AEMasaCS, and I believe the entire ch.qos.logback package is marked as Deprecated with a "This internal logback API is not supported by AEM as a Cloud Service." message. So, I'm wondering if this is even a valid approach. Also, I'm not sure how much performance overhead it will cost us to send logs via network.

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by Peter_Puzanovs

Hi Rustam,

 

We have very limited direct access to FS in CS version of AEM.

 

There's no such functionality in AEM as a CS to share files directly with Elastic.

 

There's an option for Splunk(Not Elastic) to setup forwarder which will get you a bit closer to your instance, but then you still may get blocked depending on your Adobe Azure security settings when connection to your organisation.

 

AFAIK cron job in logstack and aio API is the best option I found so far.

 

May be someone else in community found a better approach.

 

Regards,

Peter

2 replies

Peter_Puzanovs
Community Advisor
Peter_PuzanovsCommunity Advisor
Community Advisor
April 19, 2024

Hi Rustam,

 

Sending logs directly through the jvm process to Elastic is definitely not good, it may indeed impact many parts of the Adobe application.

 

Adobe IO gives you the opportunity to download logs via [1] download logs API,

In ELK stack we can utilize logstack cron based events system to import data into your index[2]. Once it's in index we can do all of the monitoring needed. This way you will utilise aio events that live outside of your jvm process.

 

[1] https://developer.adobe.com/experience-cloud/cloud-manager/reference/api/#tag/Environments/operation/downloadLogs

[2] https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http_poller.html

 

Regards,

Peter

    R
    RustamMussabayAuthor
    Level 2
    April 19, 2024

    Hi Peter,

     

    Thanks for your response!

    Yes, I'm aware of the aio utility to download/tail logs and my suggestion was to use it to gather the logs and then feed it to ES. But I also want to understand whether it's possible to send logs from AEM application or not, and if not, why. If it was a regular Java app, I would add logback-elasticsearch-appender dependency, but it depends on ch.qos.logback packages that are not compatible with AEM as a CS.

    Nevertheless, AEM as a CS uses something to store logs in a file system, so may be there is a documentation I couldn't find or something undocumented that sheds a light on this topic. 

      Peter_Puzanovs
      Community Advisor
      Peter_PuzanovsCommunity AdvisorAccepted solution
      Community Advisor
      April 19, 2024

      Hi Rustam,

       

      We have very limited direct access to FS in CS version of AEM.

       

      There's no such functionality in AEM as a CS to share files directly with Elastic.

       

      There's an option for Splunk(Not Elastic) to setup forwarder which will get you a bit closer to your instance, but then you still may get blocked depending on your Adobe Azure security settings when connection to your organisation.

       

      AFAIK cron job in logstack and aio API is the best option I found so far.

       

      May be someone else in community found a better approach.

       

      Regards,

      Peter

        joerghoh
        Adobe Employee
        joerghohAdobe Employee
        Adobe Employee
        April 26, 2024

        I am aware of some thoughts to support log-forwarding to other systems than Splunk. You might want to contact customer support which can probably get you in contact with the right Adobe folks.

          R
          RustamMussabayAuthor
          Level 2
          April 29, 2024

          Hello Jorg,

          Apologies for the delay in responding.

          I reached out to our contact at Adobe via email about the issue and quoted your previous response.

           

          Your help is really appreciated.

            Terms & ConditionsAccessibility statement

            Loading footer...