Thanks for the info @Lupin !
I ended up finding this: https://www.raspberrypi.org/forums/viewtopic.php?t=130294 and updated it with your information.
So this is my working setup:
In /etc/emulationstation/es_systems.cfg, add:
<system> <name>Media</name> <fullname>Movies</fullname> <path>/home/pi/RetroPie/roms/movies</path> <extension>.avi .avi .mp4 .MP4 .mkv .MKV</extension> <command>/opt/retropie/supplementary/runcommand/joy2key.py /dev/input/js0 kcub1 kcuf1 kcuu1 kcud1 0x70 0x71 & omxplayer -o hdmi %ROM% killall joy2key.py</command> </system>Now create a folder called movies under /home/pi/RetroPie/roms/and place your video-clips in there.
Bonus: If you want to hide the text that shows up when skipping forward/pausing and so on, change to omxplayer --no-osd -o hdmi %ROM% in the code above