The idea can be ignored. Instead of creating calculated metrics and putting it left, I should have dragged the (underlying) segments over there,and the metric unique visitors at the top. In that case the Percent Change can be calculated, and I have my desired result.