Thanks mitu - I didn't know about the es_input.cfg file. That's what I was looking for. My inputs don't overlap between the two boards, so I thought I'd be able to edit that file manually and create something that allowed for more than one player to control ES UI, but I couldn't get it to work. Not a big deal - player one controls the ES UI. As far as I can tell, using four players across all the boards works in the retroarch.cfg files for each platform and that's what really matters. Thanks for your help.

Also a reminder that arcade games commonly shared a single set of controls, for example pacman. So player 2 actually uses player 1 inputs when it's their turn. There were also games that had distinct player controls such as tmnt or mortal kombat. It would help to specify the game your trying to play to determine if that plays any part in your issue.

@iandaemon thats exactly what i did and didnt catch thank you