Thanks for the info @Lupin !
I ended up finding this: https://www.raspberrypi.org/forums/viewtopic.php?t=130294 and updated it with your information.
So this is my working setup:
In /etc/emulationstation/es_systems.cfg, add:
<system>
<name>Media</name>
<fullname>Movies</fullname>
<path>/home/pi/RetroPie/roms/movies</path>
<extension>.avi .avi .mp4 .MP4 .mkv .MKV</extension>
<command>/opt/retropie/supplementary/runcommand/joy2key.py /dev/input/js0 kcub1 kcuf1 kcuu1 kcud1 0x70 0x71 &
omxplayer -o hdmi %ROM%
killall joy2key.py</command>
</system>
Now create a folder called movies under /home/pi/RetroPie/roms/and place your video-clips in there.
Bonus: If you want to hide the text that shows up when skipping forward/pausing and so on, change to omxplayer --no-osd -o hdmi %ROM% in the code above