I have uploaded a background image to a collection - but it doesn't look like that I can remove the picture again. I can only replace it with a new one.
Can it really be true?
Currently yes. I keep a 1 pixel transparent image around just for this situation. It's on the list of user interface improvements for a future release. It's high on the list.