Finish and Stop are both options for terminating a live journey. The difference is in how they treat the individuals that are currently in the journey.
When you Finish a journey, any individuals who are currently in the journey will remain and finish out the journey, but no new individuals will be able to enter the journey.
Stop is for emergencies, when you notice something wrong with the way you set up the journey and want to pull the plug. When you Stop a journey, any individuals currently in the journey drop out, and no one new will enter.
If you want to make changes to a published journey you can Create a new version. Once you publish the new version, it will supersede previous versions (new entrants will go into the new journey version, but anyone remaining in the previous version will finish that journey). If you finish a journey, the previous version will become active again unless you had finished or stopped it.