Several other users are reporting the same issue on the github for xpadneo:
https://github.com/atar-axis/xpadneo/issues/259
I paired the controller in windows and copied the link key to my raspberry pi as one of the users on github mentioned. Now the pairing works and I can use the controller. Now the Xbox-Button doesn't work as hotkey for both controllers anymore. But this might be an issue because of updating the whole system.
For getting the keys I followed these instructions:
https://bbs.archlinux.org/viewtopic.php?pid=1678405#p1678405
Problem solved! :)