Try gpsp's non-retroarch version. With this version, you'll probably have less lag on GBA games, since it consumes less resources than the default retroarch version.
I've tested this version on a Raspberry Pi 1 B+, and I remember playing some GBA games without lag. It was on 2015, and back then, if my memory serves, there was no lr-gpsp yet.
I don't know if it'll work on your Pi Zero, but assuming Pi Zero and Pi 1 has approximately the same hardware, there's the chance that it can work.