New emulation method to eliminate input lag

[tekn0]

@mediamogul I was also able to improve performance further on my setup by changing the audio driver to sdl2. Alsa seems to have some cracking issues on my setup.

All snes cores did not work or were too slow. Also mame2003 would error in all kinds of ways, but things are very stable and smooth with 2 frames on fceumm and picodrive.

I am not sure if this applies, but I was reading stretch has some performance issues in general with OpenGL (https://github.com/RPi-Distro/repo/issues/79) I did try with other video drivers and performance was about the same.

[dankcushions]

@tekn0

I am not sure if this applies, but I was reading stretch has some performance issues in general with OpenGL (https://github.com/RPi-Distro/repo/issues/79) I did try with other video drivers and performance was about the same.

you should read the full comments - that’s about opengl software rendering. retropie uses gles (not opengl) hardware (not software) rendering, and there’s not performance issues with that as far as i’m aware.

[tekn0]

@dankcushions Thank you for clarifying, I was not sure.

[markwkidd]

@tekn0 said in New emulation method to eliminate input lag:

mame2003 would error in all kinds of ways

I think this would only work for mame2003 drivers which have savestate support. Do you know if you were trying a game with savestates?

I've read reports of people using runahead with mame2003 but I don't know which games they were playing. I assume anything that works with netplay in mame2003 is a good candidate, but I'm not aware of a list of those games either

[tekn0]

@markwkidd I only tested 2 games, Aliens, and Aliens vs Predator. They both support save states.

[markwkidd]

@tekn0 said in New emulation method to eliminate input lag:

@markwkidd I only tested 2 games, Aliens, and Aliens vs Predator. They both support save states.

well dang, there goes that theory

[tekn0]

@markwkidd I also just tested the same games with the appropriate rom set versions for FBA. I was only able to get 1 fame on Aliens vs predator. Frame-rate was to slow for even 1 frame on aliens.

The most important thing to note was that for FBA, run-ahead has to be enabled after the game is running. If you make it a core option at boot the rom will error. I has second instance enabled during all tests. It seems to not impact performance.

Performance was the same on fba2012 and fba current.

Other games tested on FBA
Metal Slug = 1 Frame
Street Fighter Alpha = 1 Frame
Aliens vs predator = 1 Frame
Street Fighter 3 = Slow
Street Fighter 2 = Slow
Aliens = Slow

Other emus
Fceumm = 2 Frames (all games tested) (core option worked fine)
picodrive = 2 Frames (all games tested) (core option worked fine)
Genesis-GX = 1 Frame
Snes9x (all versions) = Slow
mame2003 = crashes and errors (as core option and enabled after)

[mediamogul]

I finally got around to testing RetroArch's run-ahead feature tonight and at first blush, I can say that it's freaking fantastic. As an initial test, I was able to reclaim three frames of input lag on 'Battletoads' for the NES. A quick Game Genie code got me right to the infamous 'Clinger Winger' stage and with very minimal effort I was able to get about two thirds through it. After that the level starts to get really insane and it would take some good old fashioned practice on my part to proceed. However, for the very first time this stage feels beatable under emulation. Time is something I don't have a lot of at the moment, but I can't wait to sit down with this and knock a few "impossible" retro games off my bucket list.

Edit: Moved this to a more appropriate thread.

After experimenting with this a little bit more in lr-fceumm, I found I was able to successfully set the run-ahead frames as high as '5' without affecting performance. Previously I saw stuttering issues past '2' on some titles, but after reading more about the feature, it was indicated that audio stuttering can occur without setting a second instance of RetroArch. It's been assumed that the Pi couldn't handle this, but once enabled, the stuttering went away completely and allowed for a much higher run-ahead frame count. Granted, most titles don't need anything more than a setting of '2' and giving those games a setting higher than they need results in some interesting character behavior, but it shows that a run-ahead of 1-5 frames is not out of the question when needed for this core at the very least.

[mediamogul]

Man, I just can't stay away from this. I've learned that the method being spread online for determining how many frames of input lag a game has, is usually missing one step. I, myself have posted:

For anyone who doesn't know, you can test how many frames of lag a game has by using the RetroArch pause feature, then hold down a button that will invoke an action in game. Then, use the frame advance feature to move forward one frame at a time until you start to see the action take place in game. However many frames it took to begin the action animation is how many frames of lag exist and it can vary from game to game.

However, it should read:

For anyone who doesn't know, you can test how many frames of lag a game has by using the RetroArch pause feature, then hold down a button that will invoke an action in game. Then, use the frame advance feature to move forward one frame at a time until you start to see the action take place in game. However many frames it took to begin the action animation , minus one is how many frames of lag exist and it can vary from game to game.

This test has yielded a surprising result on a few games. Most notably is 'Battletoads'. Both it and 'Mike Tyson's Punch-Out!!' are always the main offenders when discussing input lag. After running the test, 'Mike Tyson's Punch-Out!!' had a whopping, yet unsurprising five frames of lag. However, 'Battletoads' only had one, like almost any other NES game. This means that the margin for error under ideal circumstances on stage 11 was simply programmed intentionally to offer one of the most unforgiving windows of user input in video game history, allowing no more than a few frames on each corner before your speed boost will fail and bring the spinning wheel a little closer each time.

As noted above, before testing the actual existing amount of lag, I was able to do very well on this stage by setting the run-ahead frames to '3'. However, since this is two more frames than actually exist, a setting of three would be tantamount to cheating, in the same way as using anything similar to alter the inherent game play difficulty, such as slow motion. Anyway, I've spent more time on this than I should have today, so I'll stop here, but I have to say that this is truly an amazing feature that's bound to dispel many more preconceived notions of input lag before it's all said and done, as well as dramatically improve how we control our games when used correctly.

[pjft]

@mediamogul so, now that you have managed to get me some work for my very little free time, tell me what exactly do I need to do to get this running:

• update RetroArch from source
• configure fceumm and picodrive (any others?) with 5 and 2 run ahead frames.

What's that with the 2 RetroArch instances?

[mediamogul]

@pjft said in New emulation method to eliminate input lag:

update RetroArch from source

Actually, the latest binary (1.7.3) already has this feature.

configure fceumm and picodrive (any others?) with 5 and 2 run ahead frames.

First, you should test to see how much lag the game in question has. It seems as though one frame is the most common for NES if you were to perhaps want to make that a baseline setting for the system. The test noted above is simple in it's proceedure, but my own input settings conflicted in such a way that made it way more complicated and the same will likely prove true for others.

I ended up creating a preference file that I could load separately with default keyboard input settings that also ignore automatic controller mappings. This was still somewhat inconvenient, so I believe I'll be doing further testing on my desktop computer and applying any changes to the Pi afterwards, as the amount of lag did not vary between the two. To keep it as maintenance-free as possible, I've chosen a baseline setting for a whole system and configured the outliers on a game-by-game basis as I feel it's necessary. This is a good opportunity to note this feature's biggest obvious drawback, in that there is no truly universal setting. This being the case, some games will always need their own custom configuration, like 'Mike Tyson's Punch-Out!!'.

What's that with the 2 RetroArch instances?

Setting this seems to ensure a greater stability and appears to be necessary past a few frames with the cores that can handle it. It runs a second instance of RetroArch to run-ahead. This in some way prevents audio stuttering in cores where audio is affected after a save state is restored. It also allows the rewind feature to operate without glitching the graphics in at least lr-fceumm. In my testing of lr-fceumm with a stock Pi3b, I was able to maintain a steady 60 fps with two instances, rewind enabled and five frames of run-ahead, which is really a step beyond where most of us thought performance would be with this new feature.

[tekn0]

I am really glad this is working out for you! I agree on the new level of smoothness. I felt great zipping through mega man games and contra on the NES. Also sonic felt wonderful on the Genesis.

I personally just keep it at 2 frames. That seems to be a sweet spot for all the games I tested. Hopefully one day a snes9x patch could allow this feature to run on the pi3.

[mediamogul]

@tekn0 said in New emulation method to eliminate input lag:

I personally just keep it at 2 frames. That seems to be a sweet spot for all the games I tested.

There's talk of including the amount of lag each game has in a downloadable database that would set each automatically. You should be fine with 2 frames in many, maybe even most situations. However, keep in mind that by overextending the run-ahead, you're still going to run into timing problems in some games. The only difference is that your character will overreact a frame or two early by skipping frames of action instead of the normal problem of being a frame or two late. Still, having your timing off by such a small amount in either direction isn't that big of a deal in most games.

[mediamogul]

@mediamogul said in New emulation method to eliminate input lag:

After running the test, 'Mike Tyson's Punch-Out!!' had a whopping, yet unsurprising five frames of lag.

After playing a few rounds against 'Kid Dynamite' last night, I started to notice that on occasion, he would nail me with an uppercut before my character had actually moved back from a successful dodge. I believe this is happening because the current lag test fails to consider an animation cycle with intentional delay. By me employing a run-ahead of five frames, based on viewing the basic jab animation, of which three frames were apparently intended to be inactive, my run-ahead was over-extended for those three frames, which led to 'Iron Mike' punching me from a future that wasn't supposed to exist.

If I'm correct, this illustrates the timing problems that can arise by having run-ahead over-extended. The solution is to test for lag on a clear cut example of a visual cue taking place when a button is pressed. For 'Mike Tyson's Punch-Out!!', I retested at the title card screen, where pressing 'start' is obviously intended to roll the card upward and start the match. Here, the game showed only two frames of lag that, while much less than before, is still twice that of most other NES titles.

[quicksilver]

@mediamogul said in New emulation method to eliminate input lag:

@mediamogul said in New emulation method to eliminate input lag:
By me employing a run-ahead of five frames, based on viewing the basic jab animation, of which three frames were apparently intended to be inactive, my run-ahead was over-extended for those three frames, which led to 'Iron Mike' punching me from a future that wasn't supposed to exist.

[mediamogul]

@tekn0 said in New emulation method to eliminate input lag:

Hopefully one day a snes9x patch could allow this feature to run on the pi3.

I believe lr-snes9x may have room to be optimized for this feature. After reading a bit about it directly over at the libretro forums, it seems that most SNES games have an inherent 2 frames of lag. Once run-ahead was enabled, my framerate dropped to between 32 and 37 FPS. What's interesting though, is once I enabled a second instance, the framerate picked up to a steady 53-58 FPS.

Prior to release, everyone assumed the real killer to Pi performance would be the second instance, but while admittedly taxing, it doesn't appear to be a huge problem. In fact, I've found that it irons out quite a few issues, such as speed, corrupted graphics and sound. If lr-snes9x and it's counter-parts were to perhaps be further optimized for the necessary barrage of save states it requires to run this, SNES run-ahead on the Pi could be a real possibility. What's more, it would likely have the side-affect of allowing much smoother Pi Netplay support.

In the interest of transparency, I tested lr-snes9x2010 with no shaders or overlays enabled, at my typical render size of 800x600 with a run-ahead of two frames, both with and without a second instance. Here, I only used one test game, which was 'Super Star Wars'. I imagine framerate would be further impacted by Super FX/2 chip titles, as well as games utilizing transparency blending, such as 'Jurassic Park' and 'Kirby’s Dreamland 3'.

[mediamogul]

Here's a very positive development for lr-snes2010 on the Pi3b. I had read that some improvements were recently made to optimize for run-ahead, so I updated from source and was surprised to find that almost all games will now run at 60FPS. I did see the occasional momentary dip to 59.7, but it didn't happen often. This time I did all my tests at 1920x1080 and everything still managed to hold up as long as I had a second instance enabled. Once the second instance was deactivated, the frame rate plummeted. I also tested a few of the potential troublemakers noted above and there's no surprise that they didn't do nearly as well. Below are the lowest and highest frame rate I got with each of those tests.

'Star Fox' - 52-58 FPS
'Star Fox 2' - 43-51 FPS
'Super Mario World 2: Yoshi's Island' - 43-56 FPS
'Jurassic Park' - 50-57 FPS (With transparency blending) 59.7- 60.2 (Without)
'Kirby's Dream Land 3' - 46-55 FPS (With transparency blending) 59.7- 60.2 (Without)

Every other game tried (ten or so) - 59.7- 60.2

[Darksavior]

I tried Genesis Plus GX again from source and that works. Again, like fceumm, it gets jerky in controls and tearing when the screen scrolls if you go to 3-4 (and it's 60fps all the time) so I keep it at 2.

For snes9x-2010 (built from source), I'll keep it off. It varies way too much per game and I'm lazy to thoroughly test each one. My usual go-to game to test is Megaman X 1's Octopus water level. ~45fps with run-ahead set to 1. I also have speed hacks on "compatible" which is a bigger game changer than this run-ahead thing:P

pce-fast (also from source) works with 2 but I get frame drops at 3 which means this emulator is probably not optimized for it.

I'm using a pi3b+ oc'd to 1.5ghz at 1080p with crt-pi shader.

[mediamogul]

@darksavior said in New emulation method to eliminate input lag:

like fceumm, it gets jerky in controls and tearing when the screen scrolls if you go to 3-4
For snes9x-2010 (built from source), I'll keep it off. It varies way too much per game
pce-fast (also from source) works with 2 but I get frame drops at 3

Are you activating the second core instance for these situations? It's probably a good idea just to leave it on in all situations, as the benefits are huge and according to the developer, there aren't really any drawbacks. If there's still an issue, it may be the crt-pi shader.

[Darksavior]

@mediamogul I'm not enabling the second core. Is there a difference?