Somewhat solved.
I went back into Bluetooth and removed the XBox controller from the paired listing, and then added it back in the usual way, and this time got no error (maybe due to the XPadNeo being installed ?), and then did the Input configuration again, and now the Middle Button + X brings up the menu to quit the games etc, like the Pi3B+ did.

Its super weird as I re-did the Input configuration 3 times and that did not solve it, before I tried removing the controller completely and adding it in again.

There certainly seems to be some sort of issue here, but I am unsure what it is exactly.

@mitu Yes, it works with the select button, so it is fine, and defnitelly it does not worth to take so much of your time. Thank you very much for all that help.

All what I can say is for sure it used to work with the home button but since some update it stopped working anymore. I wonder if there is a difference between the xbox one controllers. The two controllers I have are the ones with the USB B connector, not the newest one with USB C.

But yes, I guess I can call it a day and use as it is. Thanks again!

@quicksilver you're right in Mame2003+ simply open the Retroarch Menu/GUI > Quick menu > Settings (not Controls indeed) and head for the Control Mapping setting. Change the value either to Analog (left stick) or Digital (Dpad). Thanks for the tip as I completely missed that opion!

@Sldb please post all the info that was requested of you. Which model of Xbox one controller are you using? It's in the battery compartment. Are you connecting via USB or Bluetooth?