@GreenHawk84 which version of Emulationstation are you running?
There has been a corner case [1], which escaped my mind in the first place, but then has been corrected by @pjft .
If your ES is below 2.10 (which I assume):
Either update to 2.10 onwards
If you don't want to update use this workaround: Edit your ~/.emulationstation/es_settings.cfg while ES is not running and check the value of SlideshowScreenSaverCustomImageSource. Make it read false (you can also do it during ES is running in the Screensaver Options Menu).
Rationale: Your behaviour can happen on ES < 2.10 if you set your slideshow screensaver to use a custom image source and then switch to video screensaver (without disabling the image source flag).
Please report back if it solves your issue. And if it does, mark this topic with [solved] in your initial posting header.
[1] https://github.com/RetroPie/EmulationStation/commit/a8ccd0a75ba81b3e4af88ed3ebcfe08e57520ab6