As @TheDataCereal said, open the RA menu (Select + X) while in a game and change your Input settings, you probably want Player 1 to be the NES gamepad instead of the arcade Joystick. I don't think this is caused by the auto-save-on-exit settings.
As an alternative, use @meleu's joystick configuration and you can configure - per system - which controller is which player.
Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.
Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.