I usually create a segment "Event X Exists", just a standard hit level segment - then I can use it for other purposes if I need (most of the time it will be only available in the Workspace, I don't make it a public segment unless this is something I will need to use over and over again).
Then in the Workspace, I stack this segment with the metric I want, like Unique Visitors.
Because I used Hit on the segment, the daily breakdown will ensure that only hits on that day are included; stacking the UV metric with segment ensures that I get the UVs who performed the action.
(If you try and create the actual segment at the Visitor level it will be all time... Any visitor who triggered the event ever, which isn't what you want)