Update:
Updated the RetroPie-Setup script and all OS packages and restarted the Pi. This got the Bluetooth to connect (unreliably, it still fails to register about 1/4 of the button pushes I make). Frame rates are still very poor but I think that might just be the state of RetroPie on Pi 4 at the moment.