Since there is no way to define which handler on the same path to trigger for authentication you would have to do some customization.
You could implement a custom servlet /bin/auth and configure the authentication handlers under subpaths of the servlet - for example /bin/auth/saml. When the user accesses the AEM site via method requiring SAML then set a cookie to remember their original entry point and redirect to that servlet with a querystring saying which type of authentication. For Example:
/bin/auth?type=saml
or
/bin/auth?type=oauth
Then redirect the user to the respective Auth Handler's path to trigger its normal authentication sequence (note that you should have stored a cookie remembering which page they started on before being redirected to /bin/auth?... so you could use that to send them back:
Auth handler OSGi configs:
SAML Handler "Path":
/bin/auth/saml
IDP would be configured with SAML Response to be sent to:
/bin/auth/saml/saml_login
OAuth Handler "Path":
/bin/auth/oauth