@neurocrash Just in case anyone else gets the same idea, I was able to run Dos Vectrex Emulator in Retropie Dosbox, and while it looks great, it is far too slow on my Raspberry Pi 3. I think it runs at about 50% of the original speed.
@neurocrash Nevermind, it was supposed to be opened with the Hotkey+X button, but on the diagrams for controller buttons reverse the X and Y keys, so I had them mapped backwards.
I finally used the retroarch menu (Select+X) and remapped the controls here. It works fine.
I think that the problem was the special syntax for Vectrex.