@yowl00 Welcome to the forum.
There was a fix to Daphne by @werpu one and a half year ago (see here). It added a command line option -joynum followed by the controllers system index number, e.g. 0 or 1 to set the controller being used.
It isn't clear to me, though, if the number is the joystick index (jsX) or the event index (eventX). So, here the commands to show both of them:
evtest # lists all devices connected to an event
jstest /dev/input/js0 # change 0 to 1, 2, 3 etc. to see the device using it. Exit with Ctrl-c.