@NastyButtler322 not quite. To the best of my recollection from reading through that code for quite a while a few months ago, what's implemented in ES is a setting that sets a threshold of memory.
When that threshold is crossed, it will explicitly unload textures from memory in order to load new ones. I'm not sure it's as much VRAM in the normal sense of the word as it is a ES texture memory manager, though I can't say for sure as I didn't implement it.
That's why I have a suspicion that keeping it to high is as bad as having it too low, though I haven't been able to get anyone to test that. Too low means it won't have enough memory reserved to render all it needs to. Too high (I believe) will result in white screens if your Pi runs out of memory before the threshold is hit, as it won't unload textures to load new ones.
Edit: see VRAM mention here:
https://github.com/pjft/EmulationStation/blob/master/es-core/src/resources/TextureDataManager.cpp
The method to unload VRAM just deletes the texture from the GL texture manager. The method to remove from RAM deletes the actual texture data from memory.
https://github.com/pjft/EmulationStation/blob/master/es-core/src/resources/TextureData.cpp