OK.. I fixed the issue, but this is a little odd.
Apparently, Controller 1 (Player 1) has to be in USB1 on these 3040's, Controller 2 (Player 2) in USB2, etc, etc.
Is this a unique situation for the 3040 or is this a universally known fact in RetroPie land? If it is, then I apologize for not doing more research.

This may just me being a little more Windows oriented than Linux oriented, but does it really matter which slot the controllers are plugged into? I'd think that the programming of the controllers would follow the unique ID and not care about the USB Slot at all.

Thoughts?