Average Time on Site with Segments not working right?

I am having an issue with Average time on Site. This metric is calculated by sequences depending on the dimension. For example I would look do the following:

a --> b --> a-->a

It would look into the total seconds of a divided by the sequence 2.

But one issue I am running into is when using segments for couple specific referrer and I would create a chart like that:

Dimension                         Metric avg. Time on Site

page a

breakdown  segment 1          Infinity

                    segment 2          Infinit

The segment is hit-based and only looks into specific referrers.

When creating it just like shown above it doesnt work at all. when I would put the segment breakdown above the metric I get values back. It seems like something is not working right on this case.

Plus when would anyone use time spent per visit or average time on Site?

Due to this issue I am not sure which would be the right one or shows the right values?