I'm experiencing the same issue (not seeing change thumbnail button). Running AEM 6.4.3. Should this functionality be available out of the box for AEM? I've installed FFMPEG.
Also configuring "video profiles" requires that you use the OOTB video component. The OOTB video component only works with static templates. I am using editable templates.
P.S. foundation components are being deprecated, which includes the video component.