Can you give more details about your system ? See https://retropie.org.uk/forum/topic/3/read-this-first; you don't have to list all gamepads in the info, but what is important is what platform are you running on and how are the gamepads connected.

EmulationStation should save the controller info after the gamepad is configured, this is stored in $HOME/.emulationstation or (if you're using file shares to access your system) in \\retropie\configs\emulationstation. The filename is es_input.cfg and you can check its contents after each controller is configured to see if an entry for it is added.
It's better to add them one by one and - after adding one controller - restart EmulationStation and check that the controller works fine.

[..] when I run games it goes to the 4 arcade joysticks.

That's probably because they're added as Player 1 - 4 in the order of their detection and they're always detected first.

I have set retroarch to show 13 ports so I can finally see all the controllers

Not sure how useful this is unless you want to play a 13 player game - not a very common scenario.

. I also have the setup of the controllers get mixed up in emulation station and I have to go back into setup each one or restart the system.

That may happen if different controllers have the same name and Vendor/Product IDs. The name and VID/PID serve as an identifier to the configuration of the controller (not only in EmulationStation, but emulators also). That's not something that can be fixed easily or by configuration: if 2 different controllers identify themselves similarly, then how can the system tell them apart ?

Emulation missing file libICE.so.6