@Gate I wonder if my advice is just too far removed from your configuration. I don't have any controllers--only a keyboard interface (IPAC-4). All of my controls are wired directly to the keyboard interface, so none of the joypads apply for me.
I am surprised, however, that you can't get the MAME GUI to appear with the TAB key on a keyboard. It can be very handy to see inside the emulator what MAME thinks you have mapped to a given control. I wish I had more advice to troubleshoot.