That's something my org is doing. It works the same way as enabling SSO/SAML within your instance of Admin Console, and is already pretty good granularity of permissions that can be specified within product profiles. As a note, there are some workflows within these apps that might require some access to AEP as well. So it would be best practice to put your CJA/CJO users in a given user group and then assign both the corresponding product+AEP product profiles to that user group.
Also, once you have SSO enabled, you should check out the the User Sync tool to shift your user management to Active Directory (or another directory service)--we've had a great experience using this script.