Dualshock 2 Shawan adapter odd connectivity issue on system power-on

[avengin]

So system info:
Pi 4b 4gb
Retropie 4.6 Rpi4 image fresh downloaded on 512GB microsd
Controllers used, xbox 360, ps3 official, ps2 official dualshock 2 with shawan usb adapter
bluetooth mini keyboard for wifi input connectivity
PSU official Pi4 off amazon 5.1v 3A
Monitor CCTV 12" 800*600 hdmi (for briefcase build)
Connected: only controllers (1 or at most 2), power and hdmi

So what I've done. Normal first intial boot everything works fine. I load into emulation station setup up initially the xbox controller to navigate for bluetooth and wifi connectivity then using retropie-setup adding the driver "ps3controller" then setting up the Shawan third party adapter. So I have no issues with the xbox 360 controller or the official ps3 controller on usb. After getting this driver installed I can setup the dualshock 2 usb adapter and I have no problems using it for emulators and all the fun stuff until I shutdown or reboot. I know the adapter shows up as shawan as I've tested this on a lakka image and have had no issue when booting.

What I've noticed is when the system powers on I get the weirdness happening. If the shawan adapter is connected it does not get seen when ES loads, if I let the system sit for something like 5mins to 10mins it will get seen eventually by the system. If I connect it after it loads it does the same thing. When it boots up if I plug in either of my 2 USB controllers or have them already connected then the shawan adapter is instantly seen and usable. So far I cannot get the adapter seen on boot without needing to plug in an extra controller and then unplugging it, or waiting really long.

This is the adapter PS2 to USB
I've seen the list of working usb adapters thread and this is whats listed, of the 2 dualshock 2 controllers I have only one actually works with it.

The reason I'm using the dualshock 2 with adapter is easy of portability as my setup has a ps2 and a pi 4 in it so 1 controller to take with is easier. Now I know there is another way I can set this up with a ps3 to ps2 controller adapter a Gam3Gear Brook so that may become my fall back, but if someone has some pointers I'd love to hear it. I've used the retropie setup to remove all drivers except the shawan one but it makes no difference either.