Hi @akaria!
For videos, there is a wide range of client-side video players available as well as the standard HTML5 video player. You can leverage any of these in a custom component or base your implementation on the Embed Core Component.
A basic implementation should be quite straight forward.
While there is no need to migrate your video content (they can be left in the /content/dam structure), you will have to migrate resource types of existing page content to reflect the new components resource type. (Rewriting the sling:resourceType property from "foundation/components/video" to the future resource type, e. g. "/apps/core/wcm/components/embed/v1/embed".) While doing so, please ensure that the referencing property pointing to the actual video binary should either keep the same name (e. g. "fileReference") or must be rewritten as well.
One side note: depending on your load profile you may want to evaluate external video hosting/delivery solutions. While it's totally possible to serve videos from AEM, this is usually a quite load-hungry process and external solution may give benefits in terms of delivery speed, video optimization and similar (and often also come with a dedicated video player). You may want to check Dynamic Media and its video capabilities.
Hope that helps!