New emulation method to eliminate input lag

[mediamogul]

Man, I just can't stay away from this. I've learned that the method being spread online for determining how many frames of input lag a game has, is usually missing one step. I, myself have posted:

For anyone who doesn't know, you can test how many frames of lag a game has by using the RetroArch pause feature, then hold down a button that will invoke an action in game. Then, use the frame advance feature to move forward one frame at a time until you start to see the action take place in game. However many frames it took to begin the action animation is how many frames of lag exist and it can vary from game to game.

However, it should read:

For anyone who doesn't know, you can test how many frames of lag a game has by using the RetroArch pause feature, then hold down a button that will invoke an action in game. Then, use the frame advance feature to move forward one frame at a time until you start to see the action take place in game. However many frames it took to begin the action animation , minus one is how many frames of lag exist and it can vary from game to game.

This test has yielded a surprising result on a few games. Most notably is 'Battletoads'. Both it and 'Mike Tyson's Punch-Out!!' are always the main offenders when discussing input lag. After running the test, 'Mike Tyson's Punch-Out!!' had a whopping, yet unsurprising five frames of lag. However, 'Battletoads' only had one, like almost any other NES game. This means that the margin for error under ideal circumstances on stage 11 was simply programmed intentionally to offer one of the most unforgiving windows of user input in video game history, allowing no more than a few frames on each corner before your speed boost will fail and bring the spinning wheel a little closer each time.

As noted above, before testing the actual existing amount of lag, I was able to do very well on this stage by setting the run-ahead frames to '3'. However, since this is two more frames than actually exist, a setting of three would be tantamount to cheating, in the same way as using anything similar to alter the inherent game play difficulty, such as slow motion. Anyway, I've spent more time on this than I should have today, so I'll stop here, but I have to say that this is truly an amazing feature that's bound to dispel many more preconceived notions of input lag before it's all said and done, as well as dramatically improve how we control our games when used correctly.

[pjft]

@mediamogul so, now that you have managed to get me some work for my very little free time, tell me what exactly do I need to do to get this running:

• update RetroArch from source
• configure fceumm and picodrive (any others?) with 5 and 2 run ahead frames.

What's that with the 2 RetroArch instances?

[mediamogul]

@pjft said in New emulation method to eliminate input lag:

update RetroArch from source

Actually, the latest binary (1.7.3) already has this feature.

configure fceumm and picodrive (any others?) with 5 and 2 run ahead frames.

First, you should test to see how much lag the game in question has. It seems as though one frame is the most common for NES if you were to perhaps want to make that a baseline setting for the system. The test noted above is simple in it's proceedure, but my own input settings conflicted in such a way that made it way more complicated and the same will likely prove true for others.

I ended up creating a preference file that I could load separately with default keyboard input settings that also ignore automatic controller mappings. This was still somewhat inconvenient, so I believe I'll be doing further testing on my desktop computer and applying any changes to the Pi afterwards, as the amount of lag did not vary between the two. To keep it as maintenance-free as possible, I've chosen a baseline setting for a whole system and configured the outliers on a game-by-game basis as I feel it's necessary. This is a good opportunity to note this feature's biggest obvious drawback, in that there is no truly universal setting. This being the case, some games will always need their own custom configuration, like 'Mike Tyson's Punch-Out!!'.

What's that with the 2 RetroArch instances?

Setting this seems to ensure a greater stability and appears to be necessary past a few frames with the cores that can handle it. It runs a second instance of RetroArch to run-ahead. This in some way prevents audio stuttering in cores where audio is affected after a save state is restored. It also allows the rewind feature to operate without glitching the graphics in at least lr-fceumm. In my testing of lr-fceumm with a stock Pi3b, I was able to maintain a steady 60 fps with two instances, rewind enabled and five frames of run-ahead, which is really a step beyond where most of us thought performance would be with this new feature.

[tekn0]

I am really glad this is working out for you! I agree on the new level of smoothness. I felt great zipping through mega man games and contra on the NES. Also sonic felt wonderful on the Genesis.

I personally just keep it at 2 frames. That seems to be a sweet spot for all the games I tested. Hopefully one day a snes9x patch could allow this feature to run on the pi3.

[mediamogul]

@tekn0 said in New emulation method to eliminate input lag:

I personally just keep it at 2 frames. That seems to be a sweet spot for all the games I tested.

There's talk of including the amount of lag each game has in a downloadable database that would set each automatically. You should be fine with 2 frames in many, maybe even most situations. However, keep in mind that by overextending the run-ahead, you're still going to run into timing problems in some games. The only difference is that your character will overreact a frame or two early by skipping frames of action instead of the normal problem of being a frame or two late. Still, having your timing off by such a small amount in either direction isn't that big of a deal in most games.

[mediamogul]

@mediamogul said in New emulation method to eliminate input lag:

After running the test, 'Mike Tyson's Punch-Out!!' had a whopping, yet unsurprising five frames of lag.

After playing a few rounds against 'Kid Dynamite' last night, I started to notice that on occasion, he would nail me with an uppercut before my character had actually moved back from a successful dodge. I believe this is happening because the current lag test fails to consider an animation cycle with intentional delay. By me employing a run-ahead of five frames, based on viewing the basic jab animation, of which three frames were apparently intended to be inactive, my run-ahead was over-extended for those three frames, which led to 'Iron Mike' punching me from a future that wasn't supposed to exist.

If I'm correct, this illustrates the timing problems that can arise by having run-ahead over-extended. The solution is to test for lag on a clear cut example of a visual cue taking place when a button is pressed. For 'Mike Tyson's Punch-Out!!', I retested at the title card screen, where pressing 'start' is obviously intended to roll the card upward and start the match. Here, the game showed only two frames of lag that, while much less than before, is still twice that of most other NES titles.

[quicksilver]

@mediamogul said in New emulation method to eliminate input lag:

@mediamogul said in New emulation method to eliminate input lag:
By me employing a run-ahead of five frames, based on viewing the basic jab animation, of which three frames were apparently intended to be inactive, my run-ahead was over-extended for those three frames, which led to 'Iron Mike' punching me from a future that wasn't supposed to exist.

[mediamogul]

@tekn0 said in New emulation method to eliminate input lag:

Hopefully one day a snes9x patch could allow this feature to run on the pi3.

I believe lr-snes9x may have room to be optimized for this feature. After reading a bit about it directly over at the libretro forums, it seems that most SNES games have an inherent 2 frames of lag. Once run-ahead was enabled, my framerate dropped to between 32 and 37 FPS. What's interesting though, is once I enabled a second instance, the framerate picked up to a steady 53-58 FPS.

Prior to release, everyone assumed the real killer to Pi performance would be the second instance, but while admittedly taxing, it doesn't appear to be a huge problem. In fact, I've found that it irons out quite a few issues, such as speed, corrupted graphics and sound. If lr-snes9x and it's counter-parts were to perhaps be further optimized for the necessary barrage of save states it requires to run this, SNES run-ahead on the Pi could be a real possibility. What's more, it would likely have the side-affect of allowing much smoother Pi Netplay support.

In the interest of transparency, I tested lr-snes9x2010 with no shaders or overlays enabled, at my typical render size of 800x600 with a run-ahead of two frames, both with and without a second instance. Here, I only used one test game, which was 'Super Star Wars'. I imagine framerate would be further impacted by Super FX/2 chip titles, as well as games utilizing transparency blending, such as 'Jurassic Park' and 'Kirby’s Dreamland 3'.

[mediamogul]

Here's a very positive development for lr-snes2010 on the Pi3b. I had read that some improvements were recently made to optimize for run-ahead, so I updated from source and was surprised to find that almost all games will now run at 60FPS. I did see the occasional momentary dip to 59.7, but it didn't happen often. This time I did all my tests at 1920x1080 and everything still managed to hold up as long as I had a second instance enabled. Once the second instance was deactivated, the frame rate plummeted. I also tested a few of the potential troublemakers noted above and there's no surprise that they didn't do nearly as well. Below are the lowest and highest frame rate I got with each of those tests.

'Star Fox' - 52-58 FPS
'Star Fox 2' - 43-51 FPS
'Super Mario World 2: Yoshi's Island' - 43-56 FPS
'Jurassic Park' - 50-57 FPS (With transparency blending) 59.7- 60.2 (Without)
'Kirby's Dream Land 3' - 46-55 FPS (With transparency blending) 59.7- 60.2 (Without)

Every other game tried (ten or so) - 59.7- 60.2

[Darksavior]

I tried Genesis Plus GX again from source and that works. Again, like fceumm, it gets jerky in controls and tearing when the screen scrolls if you go to 3-4 (and it's 60fps all the time) so I keep it at 2.

For snes9x-2010 (built from source), I'll keep it off. It varies way too much per game and I'm lazy to thoroughly test each one. My usual go-to game to test is Megaman X 1's Octopus water level. ~45fps with run-ahead set to 1. I also have speed hacks on "compatible" which is a bigger game changer than this run-ahead thing:P

pce-fast (also from source) works with 2 but I get frame drops at 3 which means this emulator is probably not optimized for it.

I'm using a pi3b+ oc'd to 1.5ghz at 1080p with crt-pi shader.

[mediamogul]

@darksavior said in New emulation method to eliminate input lag:

like fceumm, it gets jerky in controls and tearing when the screen scrolls if you go to 3-4
For snes9x-2010 (built from source), I'll keep it off. It varies way too much per game
pce-fast (also from source) works with 2 but I get frame drops at 3

Are you activating the second core instance for these situations? It's probably a good idea just to leave it on in all situations, as the benefits are huge and according to the developer, there aren't really any drawbacks. If there's still an issue, it may be the crt-pi shader.

[Darksavior]

@mediamogul I'm not enabling the second core. Is there a difference?

[mediamogul]

@darksavior said in New emulation method to eliminate input lag:

Is there a difference?

Definitely. Without it being active, I get a drop of 20-30 FPS past a 2 frame run-ahead in both lr-fceumm and lr-snes2010 (The only two cores I've tested). I'm actually able to get a steady 60 FPS with no stutter in lr-fceumm with 5 frames of run-ahead and rewind enabled, which of course is a decent hit to the processor in and of itself.

[Darksavior]

@mediamogul Ah ok, that helps a bit. Unfortunately, even with the crt-pi shader off, I still notice and feel the jerkyness when set to 3 or 4 (really noticeable at 4) on fceumm/genesis gx so I'll leave both instances enabled at 2. I have the overclock setting on for NES which it's really needed in megaman 3 and probably tons of others.

It also doesn't help when I enable the fix slowdown hack in snes9x2010, which I'll need to turn off and actually look into what games actually need it. So far it's just super rtype.

[mediamogul]

@darksavior

Interesting. Results with run-ahead on the Pi may end up varying greatly with users, depending on a number of factors. Fortunately for you specifically, if you do plan to continue using it, 1-2 frames generally seems to be all that is needed in most cases.

[Keltron3030]

Are these input lag settings available on 1.7.1?

[dankcushions]

@keltron3030 no

[Keltron3030]

Shouldn't updating retroarch from source get me to 1.7.3? It still shows 1.7.1 on the GUI. Sorry for a likely dumb question.

[dankcushions]

@keltron3030 you have to update your retropie-setup script first.

[Keltron3030]

@dankcushions

Oh, got it thanks. Now I'm on 1.7.3 and I see latency settings; thanks. Now my hot keys don't work, dammit! 🤪