The popup is not part of the workflow itself, but part of the "activation" logic.
Workflows run async and are not directly connected to a browser window, thus they cannot render HTML and send it as response to a user.
If you have a custom workflow, you can either replicate the activation feature (most likely to can reuse the complete logic and UI then), start the reference search first and as a final step put all that information into your custom workflow. This allows a limited amount of interactive work, but it's more complicated than a simple workflow, which just runs in the background and has no feedback channel.
Or you just implement a workflow step (as usual) and use the ReferenceSearch class [1] to find all other objects in the repo which are referencing your page.
[1] ReferenceSearch (The Adobe AEM Quickstart and Web Application.) (note that this is not a service, but a simple object!!)