pi5 usb port priority change with 2 dragonraise joystick controllers

[giandeejay]

I have a PI5, and retropie v4.8.8 (latest), installed (manually), after Raspberrypi OS 64 Bit Lite (bookworm), sometimes, my joysticks are inverted (expecially during a reboot), why ?
If i test usb ports with "jstest" sometimes, joystick (for player1) is 0, sometimes is 1...
Is there a method to fix the dragonraise controller, to a fixed port ?

the Pi5 has 4 usb port (the first 2 are usb 3) and normally I use the fisrt "up-left usb3" port with hdd ssd 1 terabyte (with usbromservice), the second "down-left usb 3" port, with a wireless keypad (mouse+keyboard), and the other 2 usb 2.0 for the 2 dragonraise controller joystick (and 10 buttons for each player)...
So the up-right usb2 is for player1 (0) and the down-right usb2 is for player2 (1),
with all my pi4 (used in over 10 identical arcade cabinet) i have no issue, but with pi5 sometimes retropie inverts the 2 players,
so player1 became the player2 (1), and the player2 became 1 (0).

I remeber in PI3b+ this USB priority schema:
up-left port 0
down-left port 1
up-right port 2
down-right port 3

so if you want to connect HDD USB + wifi usb keypad + 2 joystick , the best schema was:

up-left port 0 - HDD USB (usbromservice)
down-left port 1- USB JOYSTICK1 + 10 BUTTONS (dragonraise)
up-right port 2 - USB JOYSTICK2 + 10 BUTTONS (dragonraise)
down-right port 3 - WiFi usb dongle with keypad

But the same hardware with PI5 this sometimes don't works...
CAN YOU EXPLAIN HOW CAN I SOLVE THE 2 PLAYERS CONTROLLERS (SOMETIMES) ARE INVERTED ?