Unified Alert message configuration for ETL process

[gowthamr123]

Hi All,

We have  5 ETL process which runs daily basics, In  all 5 ETL process i have set up an alert activity which i get an notification separately.

Is there a way to get consolidated alert message that can be configured.?

Kindly any one help on this.

[LakshmiPravallika]

Hi @gowthamr123 ,

Please try setting the required variables in the Alert activity in the Options for all the 5 Workflows having ETL Jobs and then try using those Global Variables(Options) inside the Alert activity of your 5th ETL Job Workflow.

You can create option variables under "/Administration/Platform/Options/".

Regards,

Pravallika.

[akshaaga]

@gowthamr123 ,

Can you expand on the request, please?

A. Is the condition of all the existing processes similar?

B. With what criteria are you expecting the alert to be merged? I assume all the process activities would have an alert mechanism based on different conditions.

The configuration of this activity is similar to that of an Approval.

You can use the default delivery template which is used to alert operators is ‘alertAssignee’!

[gowthamr123]

Hi akshaaga

As i said before we have 5 ETL process with separate alert notification configured.

1.1st ETL workflow we are updating 3 schema's and for all three we have separate alert notification

2.2nd ETL workflow we are updating 2 schema's and  for all two we have separate alert notification 

3.3rd  ETL workflow we are updating 2 schema's and  for all two we have separate alert notification 

4.4th ETL workflow we are updating 4 schema's and  for all four we have separate alert notification 

5.5th ETL workflow we are updating 2 schema's and  for all two we have separate alert notification 

Now my requirement is if all the 5ETL process completed i need one Alert notification need to be triggered.

Please let me know is there a way that can be configured.

Thanks in advance.

[Milan_Vucetic]

Hi @gowthamr123 

one approach involves implementing a central workflow or orchestrator. While this workflow won't execute significant actions, its purpose is to aggregate alert messages from ETL workflows, finaly sending a consolidated notification. You can even use this workflow to manage the initiation of these 5 workflows, either sequentially or in parallel. Following their execution, they can provide distinct responses (such as "true" or "false"). Establish 5 event listeners in orchestartor workflow, and from each ETL workflow, trigger the relevant status via JavaScript node. Once all are concluded, check whether you've received 5 "true" values, sending unique self-notification. Conversely, if one status is "false" (indicating an issue), send a negative notification.

https://experienceleague.adobe.com/developer/campaign-api/api/sm-workflow-PostEvent.html

Cheers,

Milan

[ParthaSarathy · Accepted solution]

Hi @gowthamr123 ,

Step-1:

In your 1st ETL Workflow, Begin the workflow with a scheduler activity.

And in other ETL Workflow's, begin the workflow with 'External signal'. (Make sure that internalName of the external signal activity is 'signal' (open external signal activity > advanced tab > Name: signal)

Step-2:

In 1st ETL Workflow as you're updating 3 schemas in it, connect the 3 update data activity's transition to 'AND join' activity, followed by a JavaScript code activity.

(In 'AND-Join' select any of the transition as primary set)

Step-3:

In JavaScript activity, trigger the 2nd WKF using xtk.workflow.PostEvent method

Syntax:

xtk.workflow.PostEvent("internalName_of_next_Workflow", "internalName_of_next_Workflow's_Signal_Activity", "", <variables/>, false);

//Example: if the 2nd ETL WKF's internal name is 'ETL_WKF2' then,
xtk.workflow.PostEvent("ETL_WKF2", "signal", "", <variables/>, false);

 Step-4:

Repeat the same in all other ETL workflows, by connecting all update data activity with AND join and with JS to trigger next Workflow.

Step-5:

In the last ETL Workflow, instead of JavaScript activity, use 'Alert' activity, and configure the message as 'ALL ETL Workflows Ran successfully'.

So now you will be receiving a Unified alert message.

Note:

1. All the ETL workflow should be in started state.

2. In all workflow's property  > General tab > select a 'Supervisor' group to get alert if any Workflow fails.

[Sukrity_Wadhwa]

Hi @gowthamr123,

Were you able to resolve this query with the help of the given solutions or do you still need more help here? Do let us know. In case the given solutions were helpful, then kindly choose the one that helped you the most as the 'Correct Reply'.
Thanks!