Hi,
SP integration has to be configured in Azure AD properly to work in the first place. This is the trickiest part because permissions need to be set correctly (i did share that in the past somewhere in the community, so this should be searchable). Once this is set every user approves access individually since these are delegated permissions.
We made it to work, but it was a pain and still it doesn't work 100% optimal