I use a bezel that I made. Get asteroid.png and asteroid.cfg from here if you're interested. https://github.com/busywait/retroboxes/tree/main/overlay/Asteroids
I've tried lr-mame2003-plus and lr-fbneo. They need different ROMs, get the correct one from a relevant romset.
lr-mame2003-plus has more control over vector lines, but the screen is slightly squashed in height (black unused space at the top and bottom). The screen shot below shows that problem.
asteroids game with a bezel running in RetroArch
You can scale the picture to be a little taller, but then motion doesn't look as smooth. So, I use lr-fbneo.
With either core I used with the "normal" resolution, and integer scaled 2x to fit my bezel in the retroarch video options.
I don't use a shader. I do run with video_threaded = "false" to reduce latency by a frame. (I can't remember if I tested if that actually made any difference! I assume that it's taken away 16mS of lag).