You can delete comments on the updates tab if the "Never allow users to delete comments" is NOT checked in the Access Level settings under Set additional restrictions.
We recently found a bug in our instance because the Delete option was not showing. I would contact support if you are experiencing the same.
Editing comments (updates, posts, etc) on projects is one of the more popular ideas in the innovation lab. Workfront has labeled it 'under consideration.' See the idea for more and give it an upvote!