I don't think the N64 emulator mapping are switched (each controller should have its own mapping), but rather that the most recent configured controller takes the place of P1 or they're just arranged alphabetically.
You can check this in the mupen64plus.cfg file and see the order in which they appear and their device index.