An input lag investigation
-
@dankcushions said in An input lag investigation:
this is fantastic! :) great job!
Thanks a lot! Hopefully it will get things going regarding this tired old subject that never seems to go anywhere. :P
Do we know why Dispmanx is one frame faster than OpenGL on the Raspberry Pi?
vanafel (the person who wrote the retraorch dispmanx driver) is active on the libretro forums so hopefully he can answer
Yep, he has responded to me over there. I've yet to respond to him, though... Too much to do and too little time.
Is there any way of disabling the aggressive smoothing/bilinear filtering when using Dispmanx?
this has literally just been fixed! if you recompile retroarch from source you will get no filtering (unless video_smooth is turned on, of course)
Yep, vanfanel told me and I tested it right away. Worked like a charm! Nice to actually be able to use dispmanx.
Even when using Dispmanx, the Raspberry Pi is slower than Windows 10 by 1 frame. The same seems to hold true for Linux in KMS compared to Windows 10. Maybe it's a Linux thing or maybe it's a RetroArch thing when running under Linux. I don't expect anyone to actually do this, but it would be interesting if an experienced Linux and/or RetroArch dev were to look into this.
i have no experience in the KMS stuff, but according to vanfanel there is KMS/DRM support coming to raspian - see http://libretro.com/forums/showthread.php?t=4648 (also some general dispmanx info here also). this sounds like a good thing for retropie in general.
Yep, I've been following that as well. Hoping for any improvement, but give the fact that Linux/KMS on a PC couldn't outperform Win 10, I'm not holding my breath.
It would also be nice if any emulator devs could comment regarding the input handling and if that's to blame for an extra frame of input lag, like we can now suspect. Also, would be interesting to know if the input polling could be delayed until the game actually requests it.
there's some options in retroarch that might be relevant here:
- input driver: it's udev by default, but what about linuxraw? udev is via the device manager... possibly slower than linuxraw?
I believe I did a quick and dirty test of linuxraw a while ago. Granted, it was with a 60 FPS camera, but I couldn't detect any difference between that and the default driver. More investigation work might be needed, but I think there might be other avenues to explore first.
- poll type behaviour: there's a setting in the RGUI input menu called "poll type behavior" that may help when set to "late" vs "early" or "normal" - what does this do re: input lag? i think 'late' is what you want, but who knows.
Yeah, it's set to "late" by default. The way the nestopia and snes9x-next cores are currently implemented, I don't think this setting makes a difference. They both poll the input right before kicking off the emulator's main loop anyway. Even if they were designed to poll right when they actually read the input (which seems to be once per frame), this wouldn't help much in many cases anyway. The emulator renders a frame as fast as possible and it usually takes a few milliseconds at most (although certain games can take the majority of a frame interval to render on a Pi). If the frame rendering is fast, reading the input at the beginning or at the end will make little difference.
However, I have found one thing that does make a difference: At least in snes9x-next, frame rendering is made so that the emulator's main loop begins by rendering what was calculated in the last call to the main loop. Having finished rendering the frame, it continues by reading the input and calculates the next frame. It then exits the main loop and the new frame isn't rendered until the main loop is executed once again. This causes the emulator to have two frames of lag even in the ideal case.
See this post here for the whole story (and the fix): http://libretro.com/forums/showthread.php?t=5428&p=41376&viewfull=1#post41376
-
Interesting reading and good jobs.
Some comment:
I think you'll need to measure the switch deboucing timing of your game controller and probably you'll want to exempt it from your data. That has nothing to do with the performance of RetroArch and its underlying drivers, with respect to the handling of user input. And I think this timing might contribute huge percentage of amount to the total delay!Okay let me explain why I think so - a wee bit of technical thing and some behind the scenes of what's happening when you press the button on your controller.
When you press a button on the game controller, the MCU (a micro-controller, kind of a small computer, resides in your game controller) will almost detect the press right away. Due to the nature of the mechanical switches that tend to bounce in unpredictable manner for a short period of time, however, the MCU will not proceed to acknowledge the press and send a report across the USB bus. Instead it will do many sampling and if all the results are matching, then only the MCU will acknowledge and register the press and send a report across USB. That extra sampling time that is imposed in order to counter the nature of switch bounce is what referred as 'switch debouncing time'.
I think you really want to distinguish this timing out, so that we can see more clearer picture in term of how much each component of a system contributes to the delay. As I alluded earlier, the amount contributed by switch debouncing might surprise you.
(Reference: http://www.ganssle.com/debouncing.htm)
-
if anyone is still following this thread, @Brunnis 's lag fix for lr-snes9x-next is now live! update from source to get it :)
-
@dankcushions said in An input lag investigation:
if anyone is still following this thread, @Brunnis 's lag fix for lr-snes9x-next is now live! update from source to get it :)
Thanks for letting people know! I haven't actually tried it on RetroPie myself yet, but I will probably get around to doing it tomorrow. :)
-
I hate to pop in here all ignorant, simply looking for answers.....
But what are the answers?
I mean, (I did read your whole post, and what I got from it was you stating that there is ALWAYS going to be some input lag that you CANNOT fix "Because Pi." basically. Is this correct? you stated that an actual computer plays SNES better than a Pi?
Have you found any commands that can be entered into command line, or into a config to fix things?
Whats this about an update to snes9x? Is this a fix for input lag? How do you install it? Is it as simple as going to "update retropie" in the menu system?Im a bit noobish, I could always use the help, thank you!
-
@scrappbrapp the first post links to a thread in the libretro forums which has more of the technical details.
code fixes have been done on the default snes and NES emulators that reduce the input lag. they can gotten by updating the individual lr-snes9x-next and lr-fceumm emulators by binary in the 'main' section of the packages section of the retropie-setup script.
-
Hi !
First of all : great job !!! So basically, I guess those 2 more frames of delay (ogl vs dispmanx and linux vs win) is what I perceive and what bugs me :) Hmm, I'm so addicted to pixel shaders, it'll be difficult to switch to dispmanx ! If it were possible to use scanlines with dispmanx, it would be a nice solution.
With the same display and joysticks, I compared the same NES / PC Engine games on my FPGA computer (MiST) and the Raspberry Pi, and the difference is important (meaning : important enough to notice it without making measures). Some fast PC Engine games like Star Soldier seem unplayable on the Pi (because the ship moves extremely fast) while I perceive no delay on the MiST. In some other games, it's barely noticeable. Mainly depends on the sprite speed I guess !
Personally, I made the configuration tweaks everyone does (enable hard sync, disable multithreaded video, set frame delay) but I'm unable to measure. It may make things better but input delay is still noticeable. There are also options to poll input at different times.
It's pretty cool some people are paying attention to this issue. I guess casual gamers don't mind, but it makes a big difference when playing fast arcade games seriously !
-
BTW, Brunnis, thanks a lot for your fix too ! Will it get into the main tree ? Would be great if libretro devs checked the other cores :)
-
@dankcushions could you walk me through?
Im taking a guess of how to update .. from the main menu, do I go: retropie > Retropie setup > Manage packages > Update all installed packages OR Manage main packages > lr-fceumm/lr-snes9x > and then do I choose update from binary OR update from source?
Thanks
-
they can gotten by updating the individual lr-snes9x-next and lr-fceumm emulators by binary in the 'main' section of the packages section of the retropie-setup script.
-
Well I guess I followed it correctly, and I think I do see the differencee, thank you. When I mash sword in zelda for example, I feel like its reacting different than it use to.
Whats the difference from updating by binary, and source?
-
First of all, I just want to say great work to the OP for doing such a thorough investigation and thanks also to the devs for implementing the delay-reducing change into lbr-snes9x-next and lbr-fceumm.
I recently updated my Retropie from my old 3.7 setup to the latest 3.8.1 just so I could install the updated emulators with these changes, and here are my findings so far:
- Running on the default OpenGL display driver with Floob's Video Manager overlays/shaders applied to most of my emulators (NES, Genesis, SNES... etc.), I didn't really notice a discernible reduction in input delay from before the update.
- Then I switched to the Dispmanx display driver, and I noticed a signifcant reduction in input delay. It was pretty freaking fantastic, as even tighter platformers like DKC and Super Mario World felt nearly native.
- BUT running on Dispmanx means that I can not use any of the shaders/overlays that I've enjoyed up until this point.
So now I'm faced with a tough decision. Do I want to enjoy the awesome CRT overlays/shaders that give all of my console emulators that authentic look and feel at the expense of some input delay, or do I want my gameplay to be as responsive as possible but with only a simple bilinear filter applied and no overlays?
I'm having a hard time choosing. Why is it that OpenGL inherently has more input lag than Dispmanx? Is there theoretically anything that can be done to achieve parity between the two?
-
@ScOULaris said in An input lag investigation:
I'm having a hard time choosing. Why is it that OpenGL inherently has more input lag than Dispmanx? Is there theoretically anything that can be done to achieve parity between the two?
The GL driver has to do lots of extra processing (i.e. running the shader) that the Dispmanx driver doesn't. Keep in mind the shader has to be run for every pixel on the screen - which for a 1080 screen is about 1.5 million times. You can see where the extra lag comes from.
Lag parity isn't possible because of this. With a really fast video card it could be reduced but that's not an option on the Pi.
-
Question about the NES and SNES input lag fix. Should I remove the video hard sync and frame settings after applying the fix? I know those were previously known ways to fix the input lag slightly but will that throw anything off with these new fixes in place?
-
Video hard sync does nothing on the rpi
-
@BuZz Should this be removed from the Git page then?
https://github.com/retropie/retropie-setup/wiki/Overclocking#improving-input-lag-and-delay -
@silentq yes. The wiki is edited by the community so mistakes do happen.
-
@davej said in An input lag investigation:
@ScOULaris said in An input lag investigation:
I'm having a hard time choosing. Why is it that OpenGL inherently has more input lag than Dispmanx? Is there theoretically anything that can be done to achieve parity between the two?
The GL driver has to do lots of extra processing (i.e. running the shader) that the Dispmanx driver doesn't. Keep in mind the shader has to be run for every pixel on the screen - which for a 1080 screen is about 1.5 million times. You can see where the extra lag comes from.
Lag parity isn't possible because of this. With a really fast video card it could be reduced but that's not an option on the Pi.
Hm. So I understand how a shader can add more input delay (increasingly in correlation with the complexity of the shader). Would lowering the Retroarch rendering resolution to 720p instead of the default 1080p for SNES have any impact on input delay since it's half of the pixels that need to be processed by the shader?
-
The default is not 1080p. The default is the video output resolution, so depends on your screen. afaik it will affect it, as the final scale from render res to video output res is done via dispmanx. I tend to run my tv in 720p for retropie anyway switching it in
/boot/config.txt -
@BuZz said in An input lag investigation:
The default is not 1080p. The default is the video output resolution, so depends on your screen. afaik it will affect it, as the final scale from render res to video output res is done via dispmanx. I tend to run my tv in 720p for retropie anyway switching it in
/boot/config.txtYeah for me the video output resolution is 1080p, so the default render resolution right now is 1080p for all of my emulators as well. Should lowering the render resolution to 720p theoretically lessen input lag because the shader overhead is lessened, or would it actually increase input delay because it would add an extra scaling step to bring the render resolution up to the video output res?
Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.
Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.