Raphnet NES2USB Struggles

hansolo77 · 29 Nov 2017, 21:15

This thread will be a "repository" of sorts, used to document my experience in solving a critical issue in my RetroPie Build.

For Starters - What My Goal Is

I have built a RetroPie system using an old and broken NES console. My plan was to use a wireless Xbox360 controller as my primary controller. Then, as desired, have access to the NES ports already built into the case, so I can play NES games with their original controllers. I also have installed 2 DB9 plugs above the NES ports that connect to a Mayflash adapter that converts those plugs into usable Sega and Atari ports. The idea being, like with the NES ones, that should I desire to use original controllers, I would have that capability.

Original Implementation - How it Worked Before

And yes, it DID work before. After inquiring here in the forums, I learned that a 3rd party driver could be installed that would essentially emulate an Xbox controller by directing inputs from ANY controller to a keyboard input. That input is then transferred to RetroArch by means of a "fake" controller. The experience with that is documented in THIS thread. Much thanks go out to @mediamogul for writing up the original documentation on how to make this all work. Originally I had to use this approach to be able to map a controller for use with the "Streets of Rage Remix Port". The problem I was having was due to the port only excepting input from "js0", which was not my Xbox controller. To get around that, I used the XBOXDRV software to make my controller emulate a keyboard, which the game also used. After learning how to make that work, it was as simple matter of repeating the steps to enable the use of the dedicated controllers in other emulators. The beautiful thing was that the XBOXDRV software would create an additional controller for RetroArch to use, then unload itself after gaming finished. And to top it all off, I could use my Xbox controller as well as the other controllers at the same time, so I could use it AS NEEDED. As I said, this all worked before, and I never had any problems.

FIRST PROBLEM - When I Noticed I Had a Problem

The first indication that something was wrong came when I updated RetroArch to 1.6.9. The RetroArch program would crash whenever I navigated inside RetroArch to do things like change settings, look at my Achievements, or attempt to connect to an online game. When I asked the forums here, I learned that the problem was probably related to the customizations I had in my retroarch.cfg file, and was advised to replace it with the generic one. While that worked for my Xbox controller, it inevitably meant losing all the custom inputs for the keyboard which broke all the keyboard emulated controllers I had with XBOXDRV. Once I was able to replace the input keys with what I had them mapped to, things looked like they were fixed, until I tried to trigger the RunCommand menu to change a setting for which emulator to run for a certain rom.

RUNCOMMAND - It Stopped Working!

Originally discussed and documented on the forums here. I thought the cause of my problem with RunCommand was due to the update of RetroArch. In order to update it, I had to download a new version of the RetroPie-Setup scripts. I thought that doing that is what caused my RunCommand to stop working. Then I discovered it would work on most systems, just not the ones I had created XBOXDRV scripts for. I then learned that my Atari, Sega, and NES systems wouldn't trigger RunCommand unless I used the controllers that had been remapped. But another issue came up where it didn't work 100% of the time. That prompted me to dig further into the runcommand-onstart.sh script to see what was going on.

SECOND PROBLEM - Why runcommand-onstart.sh Wasn't Always Working

As it turns out, reviewing my script was indeed an eye opener. I had created the various XBOXDRV controller scripts to work with inputs existing in the /dev/input/js# fashion. While this method works, it doesn't work 100%. Various testing of my scripts revealed that the js# would sometimes get mapped to the wrong controller. Sometimes when I boot up my Pi, it enumerates the joysticks in a different order. Most of the time it's correct, but other times it's wrong. This typically happens as a result of having the Xbox controller off when the system boots up. Since it's not on, the Pi doesn't assign it a js#. That led me to return to the forums and update the thread about the XBOXDRV to see if there is an alternative way to map a controller. @mediamogul was once again very helpful in suggesting I try the /dev/input/event# instead. Before going to far into it, I tested different scenarios and found that it too had the same problem as the /dev/input/js# in that the numbers would associate to different controllers based on the situation. So that wasn't a solution either. I then went back to the documentation, and saw a possible fix in where Bluetooth devices can be given a static event name by creating a SYMLINK in the hardware enumeration rules. I thought that would be my fix, but it was giving me complications because the Raphnet Adapter was only providing one /dev/input/by-id. Since I have 2 controllers, I need to have 2 by-id numbers. I asked @mediamogul again and he said I could just use the /dev/input/by-path option as well, so long as I don't change the ports the adapters are connected to on the Pi. I checked, but unfortunately this also only indicated 1 by-path instance for the Raphnet adapter.

THE ROOT CAUSE Finally What Caused all the Problems

As it turns out, this is a KNOWN issue with the Raphnet adapters. As indicated on their support page, the multi-port adapters will be recognized in Linux as a single device, yet provide multiple events. According to their website, that issue was patched and sent into the Kernel upstream. However, RetroPi's kernel isn't using that latest version with the patch, so a manual fix had to done. I tried it, and it didn't work. Since learning of that, I have done a complete Kernel upgrade, installing all the Raphnet patches and the "softpatch" fix to the /boot/cmdline.txt file. It still doesn't work. At this point, I'm out of options. I have reached out to Raphnet Support but have yet to receive a response.

GOING FORWARD - What Will Happen Next

Seeing as how a solution for the Raphnet NES2USB adapter is still in the works, I have a potential here to fix my other controllers. The problem with the Raphnet is that although it shows 2 event numbers, it still only shows 1 device. I can't program my runcommand-onstart.sh with only one device id. I can, however, map my other controllers that use the MayFlash adapter. THAT adapter, thankfully, indicates 2 event id's AND 2 device id's. So all I have to do is change my script to grab the /dev/input/by-id devices instead of it's /dev/input/js. This will be my next project until I get a response from Raphnet.

So there you have it. Lengthy, I know. But I try to be thorough in my documentation, should somebody else come up with a similar problem, or if I need to remember for myself should I need to do it again. I am, however, looking for input from the community as to a possible solution. If somebody has had this happen to them, and you have a working setup, please feel free to enlighten me! I will update this OP as progress is made.

Thanks all!

mitu · 29 Nov 2017, 22:10

@hansolo77 Just a quick observation - from watching the previous topic and this new one - you didn't have to patch and install a new linux kernel.
The current Raspbian Jessie image - which RetroPie uses - already has a recent enough kernel (4.9.35) which should include the patches mentioned on the Raphnet support page (applied in kernel version 4.2.0).

Rion · 29 Nov 2017, 23:02

@hansolo77 Have you heard of retrousb?

They have a few USB kits they sell to modify your original controller to USB.
Just a taught.

hansolo77 · 29 Nov 2017, 23:26

@mitu said in Raphnet NES2USB Struggles:

@hansolo77 Just a quick observation - from watching the previous topic and this new one - you didn't have to patch and install a new linux kernel.
The current Raspbian Jessie image - which RetroPie uses - already has a recent enough kernel (4.9.35) which should include the patches mentioned on the Raphnet support page (applied in kernel version 4.2.0).

See, I thought that was the case. But lastnight after performing the long kernel upgrade, the dreaded NORTHWEST DRIFT issue came back. So I rebuilt the kernel again with the included patches. The drift is gone, but the 2-device issue is still present. I don't know what I could be doing wrong. The process I'm using to update the kernel is documented here. Raphnet does have an additional patch, with a link to a forum post about it. I suspect I need to install that too, but when I tried to do that this morning (copy/paste into a .diff file) it gave me errors asking if I wanted to resume. Either way, it still doesn't work. Default (using the retropie-setup scripts) or Manually (using the other link from @obsidianspider).

@rion said in Raphnet NES2USB Struggles:

@hansolo77 Have you heard of retrousb?

They have a few USB kits they sell to modify your original controller to USB.
Just a taught.

I have, and consdidered them at one point. But my thought was, why go with an inline USB adapater if I can use the existing plugs? If I wanted to go directly USB, I would have been ahead to just buy some USB NES controllers. In hindsight, the Raphnet adapter was probably not the best course of action. There are certainly other adapters I could have used out there. I was quick to buy that one when I first started my project because I didn't know any better, and had read reviews that said they worked. And honestly, they DO work. I have no problem with the adapter. I can control Player 1 and Player 2 out of the box right now. ES and RetroArch works with them. It's only in my specific special use case that I'm having the problem. It's because I want to be able to use them only as needed. For some odd reason, if I have them mapped in ES, RetroArch stops working the Xbox controller. If I switch it over to use that, then the NES controllers stop working. The only way to have BOTH is with XBOXDRV. Now I just need to figure out how to get the system to give me 2 devices instead of 1.

Rion · 30 Nov 2017, 00:18

@hansolo77 I highly recommend people not to buy cheap Replica Nes/Snes/MS etcetera controllers. But then again I can see why you didn't want to open up your original controllers.

hansolo77 · 30 Nov 2017, 05:39

POTENTIAL UPDATE
I posted a request on a Facebook Group asking for assistance. Through a hopeful suggestion, somebody linked to a thread about a similar issue. In fact, the reply was the most hopeful. @JoargTheBard suggested that the solution is to add this to the /boot/cmdline.txt:

usbhid.quirks=0x289b:0x0002:0x040

The interesting thing here, is that it is "almost" what Raphnet's script is supposed to have, which I already added. However, their script adds this:

usbhid.quirks=0x289b:0x0002:0x40,0x289b:0x0003:0x40,0x1781:0x0a9d:0x40

There a couple of things I noticed here. First of all, the obvious thing is that the Raphnet addition is a LOT longer. The 2nd thing is the last bit of the "suggestion", where he has it as 0x040, Raphnet has it as 0x40. This got me thinking. @JoargTheBard also suggested looking up information on the "Xin-Mo Controller". I've never heard of that, but Googled it anyway. What I found was a page in the RetroPi Github that also says to use 0x040. The page also says what the various numbers are (Score!) such as Vendor ID and Product ID, as well as a command to locate them on MY device. Well I did it, and sure enough the codes match from the suggestion, Raphnet, and My device. The only difference is the 0x40 vs 0x040 bit (and the other stuff added to the rest of the Raphnet "patch"). I suspect those other codes, after the comma, are there to fix other Raphnet devices that might have the same problem.

THE FIX?
I'm going to try erasing all the stuff after the comma on my current /boot/cmdline.txt first to see if that solves the problem. If not, I will try changing to the 0x040. I have high hopes. I'm just to tired to mess with it tonight.

QUESTION TO THE MASSES
Is anybody familiar with what that 0x040 is? Is it documented somewhere? So if the first part of that "code" is to explicitly indicate the vendor id and the product id, what does that 3rd part actually do? Seems kinda strange you would need to put some 5-digit code in a boot file to split a device into 2. Also.. what do you guys think? Does my "Sherlocking" sound right, that the 0x040 is the key? I'd like to get some feedback before I try it. But if not, I'll update tomorrow the results.

hansolo77 · 30 Nov 2017, 06:11

I definitely think the problem is with the 0x040 bit being 0x40. Everywhere I look, I find solutions from people using 0x040. In fact, here is a list I found on a "Recalbox" build:
https://github.com/recalbox/recalbox-buildroot/blob/master/board/recalbox/fsoverlay/etc/modprobe.d/usbhid.conf
They don't list the Raphnet device there, but I'm sure it would be there if that was still being updated. I tried to research the usbhid.quirks bit, and actually see a LOT of information. So I don't think I'll delve too much into it. Strangely, I didn't see any mentions though as to what 0x040 does, unless it's a special part of the device's chip programming that the designer of the circuit can utilize. That's getting too technical for me. I'll just be happy if this is my solution.

hansolo77 · 30 Nov 2017, 07:00

DIDN'T WORK
I was tossing in bed, this whole thing running around in my mind. I couldn't sleep, so I decided to do it now, and see if it fixes the problems. Long story short, it didn't work. The first thing I tried to do was to just change all the 0x40 bits into 0x040. Rebooted, and then did a ls /dev/input/by-id. Same results as before; only one joystick entry for the Raphnet. The next thing I did was to replace the original 0x40 but then erase all the other "devices" from the line. Rebooted, retested, same results. The last thing I tried was to replace the 0x040 to the line, but with the other "devices" removed. Reboot, Retest, Same Results. So at this point, I'm back in the "waiting to hear from Raphnet" mode. I could have sworn this was the solution I needed. Oh well. While sleeping tonight, I've decided to try and give the kernel another rebuild attempt, this time WITHOUT the Raphnet patches. I've had a couple of people mention now that the Raphnet fixes should be all included into the kernel. So maybe I'll get lucky, and the problem will be solved in the morning. For all I know, it could be a result of BOTH fixes. I need the latest kernel (without the patches because they're already included) AND a change to the /boot/cmdline.txt to have the 0x040 instead of 0x40. Here's hoping anyway.

obsidianspider · 30 Nov 2017, 13:43

Since I was tagged in this thread I'll comment a bit about my setup. I'm not sure that it will help.

I'm using genuine Nintendo controllers. Even the NES controller that I converted to use a SNES plug is a "real" controller from Nintendo.

I'm only using the two Super Famicom controller ports. I've not tried to use Bluetooth controllers, or any additional controllers. I thought about it, but since the Raphnet controller adapter shows up as four controllers I frankly didn't want to mess with the complexity of doing other controller options and instead got some cheap SNES extension cables and things work fine for me.

Every time there is a kernel update I do have to patch manually or I get the "northwest" problem, but I'm not sure if the issue has been fixed in Stretch. Since I'm still using the RetroPie images and updating through there I've not tried updating manually from Jessie as I understand that there are still somethings not supported in Stretch.

hansolo77 · 30 Nov 2017, 17:13

@obsidianspider Thanks for your comment. It's nice to know that I wasn't imagining it, and that the drift problem is still and issue. I'll get this all figured out eventually. :)

hansolo77 · 30 Nov 2017, 20:14

@mediamogul Do you know if I need the --detach-kernel-driver flag in my settings? I had a thought that maybe that's why the RunCommand stops working, because I've turned off the driver for the controller. I will test.

UPDATE
I've re-wired the USB plugs again, and have successfully managed to get the Raphnet to now always come up with the same event# regardless if my Xbox controller is on or off. All I had to do was connected it to port 0 (top left) of the Raspberry Pi. Now it always comes up as event0 and event1. Doing that caused my Xbox controller settings to get jacked up though. Because now RetroArch is looking for the Raphnet controller as the default Player 1 regardless of the emulator. I hope I fixed it though. I cleared out the retroarch.cfg settings again and re-created the controller mapping to the Xbox. I then set it so it would always use the Xbox controller as the default.

hansolo77 · 30 Nov 2017, 20:49

UPDATE 2
I have now configured a working XBOXDRV config to load via the runcommand-onstart.sh file. I've tested it with the Xbox controller both on and Off, and it works exactly like it should now. So YAY there. However, there is still an issue with RunCommand. I tried to # comment out the lines in the XBOXDRV config to see if I needed that --detach-kernel-driver flag. Apparently, I absolutely do need it. Without it, all I get is a black screen. The option to trigger RunCommand never displays, nor does the game. So I put them back into the config.

NEW ISSUE Back to RunCommand
Now that I've got the runcommand-onstart.sh configured and running like it should be, I'm back to the issue of not being able to trigger the RunCommand menu with anything other than the controller I programmed via XBOXDRV. Apparently, the RunCommand menu only excepts input from event0, js0, or a keyboard. By using XBOXDRV to remap the Raphnet controllers to a keyboard, I'm able to trigger RunCommand, and navigate, but I can't select any options. I feel like this a symptom of RunCommand limitations. In that it won't receive input from ALL controllers, and the only option to enter a menu is via the ENTER key. I would really appreciate it if somebody more in the know (dev's, etc) could look at this and consider updating it to allow condition. For instance, why does triggering require a specific controller (make it work with ALL controllers and ALL buttons). And then while navigating, allow input from all HATS, D-PADS, STICKS, etc. And then for selecting, allow input from "A" as well as "ENTER".

mediamogul · 30 Nov 2017, 20:49

@hansolo77 said in Raphnet NES2USB Struggles:

Do you know if I need the --detach-kernel-driver flag in my settings?

~~It probably doesn't matter in your case, but it's a good thought. I'd try it with and without to see it it makes a difference.~~

I've tested it with the Xbox controller both on and Off, and it works exactly like it should now. So YAY there...

I tried to # comment out the lines in the XBOXDRV config to see if I needed that --detach-kernel-driver flag. Apparently, I absolutely do need it. Without it, all I get is a black screen.

Progress, nice! What kind of controller are you using again?

mitu · 30 Nov 2017, 20:58

@hansolo77 said in Raphnet NES2USB Struggles:

Apparently, the RunCommand menu only excepts input from event0, js0, or a keyboard

Take a look at https://github.com/RetroPie/RetroPie-Setup/blob/3928a0570727f850ec8895cc18f77cad04d0d86d/scriptmodules/supplementary/runcommand/runcommand.sh#L106, on how the joy2key.py script is started by runcommand. I think you might want to fall on the else part of the test, which - from what I can tell from the joy2key sources - adds all input devices as possible sources.

hansolo77 · 30 Nov 2017, 21:02

@mediamogul said in Raphnet NES2USB Struggles:

Progress, nice! What kind of controller are you using again?

Player 1 - Global Default - Xbox 360 Wireless Controller
Player 2 - Global Default - Xbox 360 Wireless Controller
Player 3 - Global Default - Xbox 360 Wireless Controller
Player 4 - Global Default - Xbox 360 Wireless Controller
NES Player 1 (via XBOXDRV) - Raphnet NES2USB, Event0, Original NES Controller
NES Player 2 (via XBOXDRV) - Raphnet NES2USB, Event1, Original NES Controller
SegaMD Player 1 (via XBOXDRV) MayFlash, by-id, Original Sega Controller
SegaMD Player 2 (via XBOXDRV) MayFlash, by-id, Original Sega Controller
Atari (2600,5200,7800,800,etc) Player 1 (via XBOXDRV) - MayFlash, by-id, Original Atari Joystick
Atari (2600,5200,7800,800,etc) Player 2 (via XBOXDRV) - MayFlash, by-id, Original Atari Joystick

hansolo77 · 30 Nov 2017, 21:08

@mitu said in Raphnet NES2USB Struggles:

@hansolo77 said in Raphnet NES2USB Struggles:

Apparently, the RunCommand menu only excepts input from event0, js0, or a keyboard

Take a look at https://github.com/RetroPie/RetroPie-Setup/blob/3928a0570727f850ec8895cc18f77cad04d0d86d/scriptmodules/supplementary/runcommand/runcommand.sh#L106, on how the joy2key.py script is started by runcommand. I think you might want to fall on the else part of the test, which - from what I can tell from the joy2key sources - adds all input devices as possible sources.

Ah, that's insightful. Now if I only know how to cause the else statement. By using jsX, than means "ALL" right? I wonder if I can just add an extra mapping in the XBOXDRV to have it send ENTER and TAB along with what I have mapped to A and B. That would at least get me working INSIDE RunCommand. But I still don't have a way to trigger it with my Xbox controller if I don't have the NES controllers hooked up.

mitu · 30 Nov 2017, 21:10

@hansolo77 said in Raphnet NES2USB Struggles:

Ah, that's insightful. Now if I only know how to cause the else statement

Well, for testing purposes you can copy the else branch code over the if branch. If that works, then you can see you get on that branch.

hansolo77 · 30 Nov 2017, 21:12

@mitu I don't really follow. Are you saying I should edit the script locally and change the programming to not include anything BUT what's in the else bit? IF that works, what's to keep it from going back in the future?

mitu · 30 Nov 2017, 21:34

@hansolo77

for testing purposes

hansolo77 · 30 Nov 2017, 21:34

The problem with RunCommand is directly associated having the XBOXDRV becoming active via the onstart script. I was able to remap the A and B buttons on my NES controller to be ENTER and TAB, as per the notations in the runcommand.sh script. I'm not able to map the buttons simultaneously though. If I try to do both, it only takes whatever is last entry. So it's either A=A and B=B or A=START and B=TAB. So changing the maps in XBOXDRV isn't the solution.

I wonder if a script can be made to launch with the emulator, that would allow the Xbox controller to still remain as js0? The more I think about it though, that's probably just nonsense. It's so weird how it all works. RetroArch will use the Xbox controller as the default no matter what. But also uses the XBOXDRV overtop of that config because it's using a keyboard. I remember at one point trying to map the Xbox controller as a 3rd device, but that pretty much broke RetroArch. I'm probably just going to end up stuck with what I have unless the devs are willing to look at and update RunCommand to not require js0 if it exists.