I see what you mean now. I tried on mine and was able to replicate it. The only thing it lets me do is "edit label," but even that will leave the number and green box. May be worth submitting as an Idea to remove it completely - I agree, it's very annoying and can be misleading.