It sounds as if you've worked with Azure AD support, so I may be mistaken in this suggestion, but if not, it might be a quick fix.
I believe that Azure AD has a configuration in either the application or the claim that allows you to specify which groups would be included the group claim. Presuming you have fewer than 150 AEM-relevant groups, if that configuration exists, you should be able to have Azure AD in effect filter the group claim to only specific groups that are relevant.
I may be thinking of Okta or ADFS and if so, my apologies. But if correct, this will be far easier than a custom SAML handler.
Hope it works!
Beau