How to get notification if feed import fails?

[vinay16]

Hi everyone,

We receive an email notification whenever a workflow gets failed in AC classic. We are importing data using feed. Sometimes, it gets failed. Is it possible to get an email whenever feed import gets failed or there has been no update since 24 hrs?

I have tried to find answer however, I couldn't find anything related to this.

Thanks,

Vinay

[Manoj_Kumar_]

Hello @vinay16 

There are multiple ways to add notifications.

• In case of workflow Failure - You can get notifications if the workflow is failed. To get this notification, ensure all the workflow has a supervisor group assigned to them.
• In case of no counts -  you will have to add a test activity in the workflow to check the counts, and if the counts are 0, then send the WF transition to an alert activity. You can choose a group/operator to receive notifications. 

[vinay16]

Hi Manoj,

Thanks for your reply. I have already implemented these things. However, I was asking how I can implement a workflow / trigger for sending out an alert in case of import fails in entire system not for only 1 workflow. Can you please share your insight for that?

[Sreevarshini-P]

Hi @vinay16 

• You can use alert activity where you can provide the assignee ( operator) to whom the alert should be intimated.
• In other way around you can also try using a separate workflow . Create a workflow specifically for monitoring the feed import status or data updates.
• Create a WF with activities like scheduler (make it run every 24 hrs) by this you can find whether there is any update or not . If not then try setting a alert activity and send the mail.
• You can use query activities or decision activities to evaluate the status or last update time of the feed import.
• If the feed import is successful, the workflow can end. If the feed import fails or there has been no update, the workflow proceeds to the email delivery activity.

Try this way and see whether its working

[vinay16]

Hi Sreevarshini,

Thanks for steps by step suggestions. I have 2 questions:

• Create a WF with activities like scheduler (make it run every 24 hrs) by this you can find whether there is any update or not . If not then try setting a alert activity and send the mail. --- Can you please let me know how to set up an alert activity and send the mail ?
• You can use query activities or decision activities to evaluate the status or last update time of the feed import. --- I have been trying to find how to use query activity for checking last updated time or count of feed import. Can you please suggest any particular option - like which table to use and what should be the query etc ?

[david--garcia]

A robust solution would be to create an "ELT Log" custom schema to store import/export logs for all feeds incoming and outgoing of your instanc along with a technical workflow that will execute a daily morning summary of these logs to your techops team.

Monitoring

• Each import workflow to be enhanced to write details of record counts/successful imports/exclusions per feed to ETL log.
• Include a trailer record in your feed (at the very last line) to inform the system how many records are in the feed being consumed .e. "T|2329 " you will need a logic to process this line, or you can include it in your file's name and read it from there.
• Your import workflow will log in the ETL schema the expected records (via trailer record) and then successful records and failed records. (the difference is then used in a summary report to highlight feeds where there is a differente for futher analysis.
• Each import/export worflow will have a supervisor distribution list in case of error whilst processing the feed.
• You can include an attribute in your etl log to set the frequency of each feed, and have your technical workflow analyse if a feed was supposed to be imported in the next hour and there is no log, then raise an email alert.

Reporting

• Execute an email summary report before the 1st campaign launch concerning the feeds that have been loaded into the system or create a custom dashboard report/webapp to get a view of real time stats for members not having access to the instance.

Notification

• Email/SMS/Push/Slack notification/API, you can virtually use any channel to notify a third party system, a user, group of users, or even go the extra mile and have another reporting system to directly read the logs (for instance, I had a client that used Microsoft SCOM to gain direct access to a hosted ACC instance's database to handle the alerting side) this is where having a custom ETL log schema reap the benefits of a more robust solution (integration).
• You can have an itemized list in your etl log, to set the type(s) of alerts to execute if the feed fails, for instance(email and sms) or just email, or others.

[vinay16]

Hi David,

Thanks a lot for detailed steps and information.

There's already record logs for this. There are separate technical workflows which import / export records and there is an entry of each import under "feed import logs".

Each technical workflow is having "workflow supervisor" in case any one of them gets failed. 

However, there isn't any workflow / mechanism that triggers an alert in case there is no feed import record. I am trying to find a way to implement this..