I have a workflow which needs to end by sending a custom notification to a particular group or user , is there any way, where we can send custom notifications to users or groups programatically(from Process step) in AEM.
This is the closest implementation to what you are asking:- Adobe Experience Manager Help | Creating custom AEM workflow steps that send email messages
// This article will discusses about how to develop an AEM custom workflow step by using AEM Workflow Java APIs. The custom workflow step sends email messages.
To get email that belongs to a user - you need to use the User Manager API. See: User ("The Adobe AEM Quickstart and Web Application.")
So in your custom step - you should use the JackRabbit API to obtain an email addresses and then email the users.