Hi Richard,
I don't have custom fields readily available that simulate what you're doing, so I used the "priority" field that is available in both "issue" and "task". Using your initial grouping method, I saw similar results (you could either get task to group or issue, but not both). I stripped the textmode lines in the group statement to their bare bones, and the following groups like values together:
group.0.linkedname=Priority group.0.valueexpression=IF(ISBLANK({task}.{priority}),{opTask}.{priority},{task}.{priority}) group.0.valueformat=HTML textmode=true
In the case of "priority", the group titles are just numeric values, but I belive that is due to the fact that this field, in and of itself, really is just a numeric value. However, with your custom field, you may be just fine. It's at least an idea to play with a bit, and hopefully you can get closer to what you're looking for.
Good luck!
Kathy