I'm experiencing the same issue (not seeing change thumbnail button). Running AEM 6.4.3. Should this functionality be available out of the box for AEM? I've installed FFMPEG.
Also configuring "video profiles" requires that you use the OOTB video component. The OOTB video component only works with static templates. I am using editable templates.
Any tips?
P.S. foundation components are being deprecated, which includes the video component.
The button is visible only if you run Dynamic Media or Scene 7 mode.