ACC: Incorrect OS and browser name and version in tracking logs

[isahore]

Hi there,

We are not seeing the exact operating system and browser names and versions in the tracking logs captured from the tracking server. When looking at the tracking log file there is the complete user agent string which contains data like this for the logs generated:

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.1 Safari/605.1.15 (when opening a tracked link in Safari, Mac OS)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36 (when opening a tracked link in Google Chrome, Mac OS)

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/18.17763 (when opening a tracked link in MS Edge, Windows)

Question 1: How is Campaign extracting the browser and OS name and version from this string? Or is there another source from where this information is captured and read? While doing some digging, I came across an option variable named "NmsTracking_UserAgentRules" which contains some rules defined in two different axes. Does it have a role to play in determining which OS or browser version is reflected in the tracking logs table? I tried making changes to it but the changes did not reflect in the resulting tracking logs.

Question 2: Is there a way to reflect the full user agent string in an existing or additional field in the tracking logs table?

Question 3: If yes, can the tracking update job be modified in some way to do that? Any underlying mechanism that can be changed?

P.S.: The instance is on-premise, both the app and the redirection server.

Update: I was able to edit the option variable "NmsTracking_UserAgentRules" and finally see the changes reflect in the logs. However, what I am not able to do is capture the complete user agent string (example given above) through it. I am not able to add any custom field to be captured. The only ones captured and populated are browser name,browser image,browser version, OS name, OS image, OS family. Any pointers here are really appreciated.

@Vipul_Raghav , @Adhiyan, @Amit_Kumar , @david--garcia 

Thanks much,

Ishan

[Adhiyan]

Hello Ishan,

The option "NmsTracking_UserAgentRules" is used to normalize the data present in the user agent , in a user friendly manner , as captured in the raw tracking logs.

The tracking data is normally used to prepare and view reports by a marketer.

The data present in tracking logs on the server would not make much sense for a marketer if presented as it is.

Also , there are Out-of-box reports that depend on the user agents data from the nms:userAgent table.

So can you please detail what other data are you trying to extract from the user agent string and what is missing in the current context.

Also , Campaign is not meant to be used as a analytical tool hence the data on user_agents will be limited to an extent in Campaign.

Regards,

Adhiyan

[isahore]

Hi Adhiyan,

I need the complete user agent string to pass it on to another analytics tool, not to use it in Campaign for any analysis. I did some further study about the use of the option variable " "NmsTracking_UserAgentRules" and realized it might get overridden over time to add more OS and browser names and versions.

I also came across a method in the "nms:remoteTracking" schema, called "GetTrackingLogs"

GetTrackingLogs (String instance, String dbId, Number version, String pointer, Number logCount)

 .

https://experienceleague.adobe.com/developer/campaign-api/api/sm-remoteTracking-GetTrackingLogs.html

But I don't know from where the first three parameters should be fetched. Can you share some details about this method implementation so that I can try it out?

Thanks,

Ishan

[Adhiyan]

Hi Ishan,

If you want to simply forward the string to another tool , I think there are better ways of achieving that without causing regression risks to your existing campaign setup.

1. Forward the logs received in the nl6/var/<instance_name>/redir/logs/ folder to your tool as it is or apply some filtering commands ( awk /sed ) to only forward the data for user_agent strings.

This can be achieved by running a shell script which will extract the lines and send only the column containing the user_agent string.

2. Use a log forwarder service ( e.g - Splunk , new relic etc. )  to forward your redirection logs directly to the analytics tool.

assuming you have a linux environment , you can access the apache logs in the 

/var/log/apache2/ directory and they will be name access.log or ssl_access.log 

Additionally , NmsTracking_UserAgentRules will be overridden each day if you have the Refresh for deliverability workflow running on the instance. 

That workflow will overwrite the rules with the one present on the global deliverability server.

The Method GetTrackingLogs is only used to fetch data from a Mid-sourcing server where there is a Hybrid setup ( MKT has the database and workflows running and email sending is done via Mid ) .

In this setup , the Mid server acts as a tracking server but the logs are pulled into the MKT instance by the tracking workflow using this method.

Regards,

Adhiyan

[isahore]

Hi Adhiyan,

The setup that I am working on is the same as the one you mentioned. Mid-sourcing server is used to send the emails and also acts as the tracking server.

Can you guide on the implementation of the method GetTrackingLogs? Apparently this method is called by the tracking workflow, but the implementation is hidden. Specifically the question about the (first three) input parameters to this method.

Thanks,

Ishan

[Sukrity_Wadhwa]

Hi @isahore,

Were you able to resolve this query with the help of the given solutions? Do let us know.
Thanks!

[isahore]

Hi @Sukrity_Wadhwa,

Unfortunately not. I adopted another approach which does not use the method GetTrackingLogs.

Thanks,

Ishan

[Sukrity_Wadhwa]

Hi @isahore, 

Could you please share your approach here to help the rest of the Community with the solution as well? 
Thanks!

[isahore · Accepted solution]

Hi @Sukrity_Wadhwa,

Below are the details of the custom approach:

The solution works by exposing a custom API method on the tracking server that reads the tracking log entries from the tracking log files on the server. This API method is called from the application server via a scheduled workflow running every 1 hour.

This method takes 4 input parameters which determine which of the logs should be read from the tracking log files and returns the logs in the form of a JSON string which are then saved in a new field in the nms:trackingLogRcp schema.

The parameters of the API method are as follows:

Input parameter(s): startPointer (the pointer value from where to start parsing the next logs), lastSyncDateTime (timestamp of the last successful synced log), maxLogs (maximum number of logs to retrieve in one API call), 

endPointer (last pointer value synced from the tracking server

by the OOTB tracking update process).

Output parameter(s): logsStr (JSON String with logs and the updated input parameters to

be used to make the next API call)

The endPointer value in the input parameter is taken from the xml stored in the OOTB option variable "nmsTracking_Pointer". Other input parameters are also configured in option variables for reading and setting them with new values with every execution.

Drawbacks (as I see it): Tracking log files have to be read twice. Once by the OOTB tracking update workflow, and the second time by this custom workflow. An overhead which I do not know how it could be avoided.

Benefits: gets the work done pretty fast, without pausing any OOTB process.

P.S.: Do not hesitate to comment if there is a better way to do this.

Thanks,

Ishan

[Sukrity_Wadhwa]

Thanks a lot @isahore!