Had the same issue with the Xbox One S controller.
https://www.instructables.com/id/Fixed-Setup-for-Xbox-One-Bluetooth-Controller-With/
This fixed it for me, but I suggest that you keep it connected via USB and ditch bluetooth.
If you connect it back via USB cable (because you wanna charge it), once you're battery is charged, and you unplug it, you will have to remove it from devices and register it again... it's a pain in the butt.