A segment is basically saying: "Find me all of the data points for my scope where there is at least one hit containing such-and-such specifications."
Consider 2 visitors:
John: visited your website 2 days ago through your campaign tracking code and converted.
Mary: visited your website last month through your campaign tracking code but didn't convert, then came back to your site yesterday and converted.
And let's say your report's date range covers the last 7 days.
Then:
Visitor-based segment would include both John and Mary's data. John: because his visit from 2 days ago falls under the 7-day range of your report, and Mary because of her visit from yesterday AND she qualified in the segment because of that visit last month. Unique Visitors = 2 (John + Mary), Conversions = 2 (John + Mary).
Visit-based segment would include only John's data. John: because his visit from 2 days ago falls under the 7-day range of your report. Mary is excluded because her visit within that 7-day range did not include that tracking code. Unique Visitors = 1 (John), Conversions = 1 (John).
Hope that helps to clarify what you're seeing.