I referenced the below link for any modifications to the RTE.
https://docs.adobe.com/docs/en/aem/6-2/administer/operations/page-authoring/rich-text-editor.html
Once you override the RTE component, you should be able to add:
<subsuperscript
jcr:primaryType="nt:unstructured"
features="*"/>
to your rtePlugin node.
Then update the toolbar to reference the subsuperscript:
<inline
jcr:primaryType="nt:unstructured"
toolbar="[#format,#paraformat,#justify,#lists,links#modifylink,links#unlink,tracklinks#modifylinktracking,misctools#specialchars,-,control#close,fullscreen#start,-,control#save]">
<popovers jcr:primaryType="nt:unstructured">
<format jcr:primaryType="nt:unstructured"
items="[format#bold,format#italic,format#underline,subsuperscript#subscript,subsuperscript#superscript]"
ref="format"/>
...