Ok, after some investigation, here are some afterthoughts. If I go in and remap the controller in Emulation Station, and switch my Start and Select buttons, it seems to be fine (other then not being mapped to the button I want them to be on the cabinet). With this set up, if I go into, say the NES emulator, the Start and Select work fine in the game and pressing both together allows me to exit the emulator. Now if I switch them back, Start works fine, Select immediately exits the emulator. If I map them to 2 other random buttons in my 6 button layout, everything works fine in the emulator, but pushing them together will not exit the emulator.
One thing that I did notice, is looking at the layout instructions that came with the Xin-Mo, it shows for player 1 that the Start button should be #9, and the Select should be #10. When I map the controller in ES, the Start button shows up as 9, but the Select shows up as 8. So, it's possible I have the Select button wired to the wrong input on the Xin-Mo board. This maybe a stupid question, but would it make that much of a difference? I mean, I'm telling it to map a control to a certain button.
I'm going to pull the control apart tonight and double check my wiring.