I think that your current method of using a previous page breakdown is probably the best way. The other option would be to use a flow visualization.
If you try and build this as a segment, you're going to run into several limitations. The main one being that segments are either hit, visit, or visitor level. If you use a hit level segment with page = 404, you still need to break it down by the previous page. If you use visit or visitor level, it's going to pull in all of the pages, not just the one directly before the 404.
The best thing I would suggest is having something set up to capture a page as a 404 (a specific page name, a unique event that fires, etc.). And then break that down by previous page so you can see where they were before hitting a 404 page. You can do the same type of thing with a flow visualization. Have the piece of information that identifies a page as a 404 as the main column, and then you can see what pages they were on before or after that 404.
I actually did a lot of 404 reporting in my old role. We used the clickstream data feeds to join together information to determine what page people were on and what links they clicked that took them to a 404 so we could trend them out over time and identify problem areas. But if you don't have data feeds to work with, the previous page dimension in workspace is going to be your best option.
