White flicker on moving through UI after update to 4.2.3

TMNTturtlguy

@pjft I am not sure that your new changes have caused a new way to get glget errors. Take a look at my more detailed testing and report on the PR comments. I have additional notes, testing and another video there. I did or want to duplicate my posts on both sites, so I kept the forum post to a synopsis of my larger post.

As you and I both found, we had been able to reproduce glget errors with several systems running on the comic book theme prior to your modification. I had found that it ran fine at VRAM 100 but in order to operate as smooth as possible and avoid Glget errors it was beneficial to run at a different VRAM, slightly lower or higher than 100. I was also never able to scroll at extremely high speeds as I can now. Lastly, as noted on GitHub I cannot create glget errors on other themes such as carbon or futura.

pjft

@TMNTturtlguy Thanks for the reply. I'll make sure to dutifully read both posts, but thanks for the extensive posting. Hopefully you had a great family weekend! :)

paffley

Hi,

Don't know if this helps but I managed to get rid of the White Flash altogether on my created theme.

Rpi3
Latest RetopiePie build
Video & Marquee support enabled
18 Systems and adding
Scraped boxart within retropie
Game shot images dl'd from web
different backgrounds for system view and gamelist view (PNG 30kb, 6kb)
10 extra image elements added to both system view and gamelist view* (PNG 85kb, 4kb, 2kb, 8kb, 6kb, 10kb, 4kb, 4kb, 31kb, 8kb)

What I did...
I changed my 'background png' from 1920x1080 to 1280x720 and batch shrank my scraped and dl'd images (boxart/game shot) down from approx. 300kb to 50kb - This seemed to stop the white flash completely on my theme and runs smooth on all parts.

My thoughts
I believe it was the weight of my theme (pretty light) and the weight of the scraped boxart/game shot images together causing the White Flash in my opinion, only happened since RP4.2.

Hope this helps...

Cheers,

pjft

Thanks @paffley for sharing. Indeed, there's plenty of things that can - and should! - be done at a theme level, no doubt about that, as well as for the artwork people use in their own libraries.

The main thing that has come up recently, as well, is that since a few ES builds ago there have been an increase of odd white flashes, not fully related to the themes themselves - as even Carbon was now suffering from them - and that's kind of the issue we had been trying to tackle here.

@josete2k - I put together a new build. Same place, same instructions. It should have reverted the changes that caused the white flashes to start showing up. Please try it. It is not the latest - the final one will have the option to Re-Use SVGs as it helps saving memory in larger themes. That is not the case in Carbon, and it should work well.
@TMNTturtlguy - You can also find a new build in my repository if it helps. It will have the option to Re-Use SVGs. Don't use the build I'm sharing with @josete2k as it will fail miserably on your theme. Heavy themes still have problems with the limited memory on the Pi, even after reverting the changes that caused the white screens to become more prevalent. The Re-Use SVG option is the one you want to use for your theme.

In general, and after testing and profiling your theme fairly extensively recently, a few recommendations:

Definitely get all systems to point to the same SVG border file.
Select the "Re-Use SVG" option to reduce memory consumption.
Use "Fade" transition rather than "Slide", especially on the Pi. Slide works best when it can load every single system in memory, so that it doesn't have to load things on demand. It's not that it won't work, but if you hold your finger on the right or left button to scroll a lot of systems, you'll see a lot of black screens as it frantically tries to load the textures from memory. This also causes white flashes on the logos after returning from a game, as it tries to load all the textures at once again. This also happened in the old build :(
Don't set your VRAM to more than 100, unless you have increased your memory split further. Indeed 120MB makes it run as smooth as butter, but that's because it thinks it can load everything at once into memory :) You'll run out of memory after a few systems (10-15, I think) and then you get glGet errors :) I still haven't fixed ES to recover gracefully from that, so it just results in blank screens. I actually run your theme with 70MB of VRAM, I find it a good compromise, as well as using fade.

You may test the last known good build here. You will need to remove the background color from the theme's systemInfo text, as it wasn't supported at the time.

I'll still tinker with this in the coming days, as I'd rather not have the re-use SVG option as an option, but rather as a default. The problem is that it causes the flashing in Carbon, as its logos are SVGs. I have some thoughts on how to adjust that, but will see.

I'm close to calling it a day for these developments. We'll see.

Sorry the outcome isn't extremely better, but at least it should not be worse than where we were a few months ago - hopefully slightly better even.

Thanks.

EDIT: @josete2k - I have just updated the build, it should now have the option as well. You'll want to set it to false (so, do NOT re-use SVGs).

TMNTturtlguy

@pjft Thanks! You are working so hard on this, we all owe you big time.

For your information regarding my Comic Book Theme, Part of the optimization i have been working on for both the 4:3 and 16:9 versions has been reducing files sizes and deleting non used information and pointing all artwork and sound clips that are shared to one location in the art folder, so yes, all systems point to the border .svg in one location now. The testing i rand yesterday and today all incorporated those changes already.

I will run some extensive testing tonight to see what i can come up with and re-create. I think you have done an amazing job of improving to this point, I don't think any of us are expecting you to be able to make the demanding themes work perfectly, there are limits to what we can do! Get some rest!

Oh. by the way, do you want me to post my testing results to the PR comments? I can do a short summary here and a more detailed report on github? Or i can just comment here if you prefer. Thanks

pjft

@TMNTturtlguy Either way works - I suppose here is probably good!

Thanks for the encouraging words - I'm actually thankful you insisted on this being a regression, so much that I started noticing it more and it annoyed me enough to get me to spend time on this.

Unfortunately, though, it seems I won't take it a lot further, but at least it's back to where it was - with warts and all :)

No need for extensive testing, really - you're investing a lot of time in this, which we all certainly appreciate, but at this stage the changes I'm proposing are so few now that, unless you notice something really breaking, it shouldn't be a cause for significant concern.

You may test if things work acceptably for you without enabling the re-use SVG option, though, as I'd love to be able to remove it. But it will be a memory hog without it, so it's kind of a mixed feeling for me. I'f love to at least sort that one out.

But no, just enjoy your RetroPie setup and have fun - otherwise you remind me of me enjoying spending so much time reading about old games that I never find the time to play them :)

TMNTturtlguy

@pjft Reporting back on further testing!

Comic Book Theme - VRAM at 100

RE-USE Theme SVGS ON - this runs super smooth with 17 systems in slide transition I have no black flashes at all. When i enter a game and then exit the game and go back to system selection screen, when i scroll right i still get a whit box over my .png system logos. I don't get any black flashing though. This is an improvement over the previous build.
RE-USE THEME SVGS OFF - Now i get black flashes at every slide transition. It is better looking than the white flash, but not ideal. The SVG ON is definitely a very good setting to have! I still get white flashes over the .png logos the first time through after exiting a game.

Carbon Theme VRAM 100

RE-USE THEME SVGS ON - Runs very smooth. I do get a white box over the .svg logos after exting a game.
RE USE THEME SVGS OFF - I get a slight black flash moving screen to screen in slid transition. With 17 systems loaded, having SVGS ON is an improvement. With SVGS off - there is no white box over the .svg Logos after exiting a game that i can tell.

Menu - my ps4 controller hooked up over usb wireless dongle does not like the Other settings menu. It wants to skip over VRAM Limit and RE-USE THEME SVGS. It does not do this on any other menus or selection screens, but consistently on the Other Setting Menu. This just started with the introduction of the SVG ON/OFF. My keyboard performs fine with no issues. This is a minor detail, I wouldn't worry about it to much.

My thoughts. Honestly in both carbon theme and comic book theme i think that this is an improvement over where we started from. I think that the SVG ON option is a much smoother running system than with it off. The only downside is that the .svg logos get the white box. It appears that .png logos always get the white box after exiting a game in both the ON and OFF options. I think you did an exception job of making this a much better running system. To be able to run a heavy theme like comic book with 17 systems and get smooth transitions at 100 VRAM is unheard of! Nice work and thank you!

pjft

@TMNTturtlguy Thanks for the report, and as usual for the extensive testing.

I tend to agree that it is marginally better than where we started from, though I am still slightly unhappy about the status of things as a whole. I don't want to keep Re-Use SVGs ON as a default as it breaks previously functioning themes like Carbon, unless I can figure out how to prevent that from happening. So I'll keep looking into that, even if for a bit more. :|

Hopefully we'll be able to take this somewhere, but now it's more a matter of persistence and a bit of luck. Oh, and a tiny bit of masochism, I suppose! :)

I'll keep you posted. Thanks.

TMNTturtlguy

@pjft So i was laying in bed trying to get some rest last night and I thought of another test to run. As I noted above, the .png system logos flash after exiting a game with the RE-USE SVG in both the on and off setting. The .SVG system logos in Carbon only do so with SVG ON.

To test out my theory on .png logos flashing all the time I completely removed the .SVG outline boarder from the comic book theme. As you would expect the theme now runs smooth in with .SVG On and Off - no difference. However after exiting a game, the .PNG logos flash when scrolling through the first time with both .SVG ON and .SVG OFF. At this point I don't think it is a memory issue with the comic book theme that causes the .png to flash in both instances as I have removed the .SVG border. The .PNG logos are also very small and lite weight, so they shouldn't be causing any memory issues on their own. The comic theme slides with no "black" screens or flash, but that logos are an issue.

Second thought to this. Not sure that i know if the .png logos have always flashed or not? The entire screen flashed white in previous builds, so it would have been impossible to see. I don't have time before work to revert to the older build to test. But thought I would share some insight with you.

Sorry if this adds more complexity to the plate? I am unsure how many old themes use .png for system logos, but I think new ones in development will be. It appears that the theme ruckage is building has graphics for logos, so those will certainly be .jpg or .png. (i did not tag ruckage as there is no need to bother him with this post)

Again, thanks for all your hard work......Take some time to rest, no need to fret about this to much!

TMNTturtlguy

@pjft OK - so I had one more thing to test before work today, and I have done it! I have successfully found a way to create a crisp black border that is not an SVG file! I have successfully created a crisp clear PNG file to use as a border. It is still a separate file, but it is not an SVG. I can now run super smooth at VRAM 100 and the REUSE SVG OFF.

BUT...we still have issues with flashing after exiting a game. So i can run super smooth prior to entering a game. VRAM 100 SVG OFF. I enter a game, exit and when I scroll Right i get a slight black flash and a white box for the PNG system logo. The black flash and White box happen at the same time. This happens the first time through all 17 systems, once through the systems 1 time, they run smooth again. I can create a video to post later in the day if you would like to see this in action.

@pjft your continued efforts on this and other initiatives have kept pushing me to figure out how to fix my theme, and it appears we have done it! One small success today, let me know what i can do to assist you as you continue to trouble shoot. Thanks!

pjft

@TMNTturtlguy Hah. The moment you start thinking about this at night it's when you know you're in too deep :)

Yes, at this stage it's a problem with the logos in particular - in fact, I'll be more explicit: with the fact that we load two copies of the logos in memory - one for the selected version, the other for the unselected one. But it's all a bit convoluted.

I'm intrigued by your experiments with PNGs, though. Will you send that version of the theme my way for me to test? I'd love to see if it takes up less memory than the SVG (I doubt it, if it's 1920x1080), but in particular if it's being re-used in memory.

The Re-Use SVG option is not ideal, and I'm not happy with it as it brings all these other smaller side effects into play, which I'm not that keen on having to deal with, as it feels like the end result might be much worse than the original one.

A test for you, then, if you're keen on testing:

If your PNG overlay is 1920x1080 (which I imagine it may be), why not just having a single image, without overlay? i.e. having a single full res image for the system, with the border in it? I imagine (though I'd love to check that myself afterwards) that that overlay - if full res - will take up exactly the same amount of memory as any other full-res image. So, in effect, you're loading, for every system, a full-res overlay (that may perhaps be reused), and a HD image with the background.

Just a thought, if it's easy for you to test - though I think I had sent you earlier one such theme, with a single 1080p image as background.

Thanks for the tests here.

I don't think a video is necessary - I know exactly what you mean.

I am thinking that, if we get rid of the re-use SVG option (which I'd be in favor off), I might try to implement an option to "force reload all images" when coming back from a game - if, of course, that actually solves it. :)

It'll cause the return to ES after a game to take a bit more time, but it could perhaps make things work in heavier themes? What transition are you using these days - slide or fade?

Thanks.

TMNTturtlguy

@pjft The overlay is 1920x1080 - It works because I exported the SVG file to a PNG at 1080 resolution and have not opened it in an editing program. I tired 720 and it was to "Jaggy". I will try further testing, but when i try to combine it with the main 720 image it rasterizes the border and thus we loose clarity. I also found that if my backgrounds are 1080, i cannot compress the file size below 512kb without loosing color and clarity, what we worked so hard to fix with the dithering. When the files get over 512kb I have found that performance is really bad when running more than 8-10 systems.

As for testing. I don't have a full release ready, i would like to get you an advanced copy in some way without posting a link on here, and without either of us sharing personal contact info. Any thoughts?

Edit: i need to go to a work meeting, i will be back in 2 hours and will look into getting you the theme then.

josete2k

@pjft just tested your las build...

Is perfect for carbon theme! Thanks.

For heavy themes, you may add one flag in the theme.xml, so their authors could set the Re-Use SVG from theme...

It's just an idea...

pjft

@TMNTturtlguy No urgency.

I can download it from your GitHub, if you create a separate branch. No biggie. If you can't, it's also fine.

I am curious, however. Did you have a chance of trying out the 1080p version I sent you via MEGA earlier in the week - so, single image but 1080p?

I am intrigued by how much difference the file size/compression actually makes vs the resolution. If you could compare both and tell me your findings, I'd already be happy.

RedBatman

Wow I was gone a few days an already people are posting solutions. I'll try them out and post later.

TMNTturtlguy

@pjft I hadn't previously tested, but did download your files. I didn't test because a follow message said to ignore and try the SVG fixes you were working on in ES.

I did just test your 1080 version and it does not perform as well as the new option I created. I get long black screen pauses and still get the white flash on the logos. In my newest update the black screen is eliminated except for after exiting the game. Also, the 1080 theme is 320mb and my version is 50mb.

As for resolution and crispness. The line work in my new theme is much crisper than the line work in the 1080 version where the line work is merged into the background. Let me know when you are active on the forum. Thanks

pjft

Ok all.

New build in the same link.

Please test. Should be fixed, at least 99.9% :)

Would love your feedback and testing. So many iterations back and forth that maybe something is missing.

pjft

Hi all.

It seems this should now be available in the main Emulationstation release, so if you update from source in RetroPie-Setup, it should be good to go.

Do let me know if anything egregious comes up, but most changes were fairly self contained hopefully.

Hopefully this helps your bigger themes run smoother overall, even with the slide transition! And it should also get rid of those pesky ALSA buffer underruns that would sometimes crop up and freeze the CPU momentarily while navigating in ES.

Best.

josete2k

Just tested the newest ES.

PERFECT.

But the update isn't available yet, isn't it?

Now, only the N64 freeze problem remains... Can you take a look?

When I launch one N64 rom with mupenplus64 (not lr) the system "freezes" if I enter in the runcommand option. Not really a freeze, but the keyboard does nothing.

Enter intro launch options is not possible too.

I opened another post but it does not have any reply.

https://retropie.org.uk/forum/topic/10224/runcommand-freezes-after-mupen64plus/2

pjft

@josete2k It should be available if you updated from source. What makes you think it isn't in the main EmulationStation?

There are no options, it's just (supposedly) fixed.

In regards to the N64, I can't say I can help with that, as I have no such experience. Hopefully someone will be able to help!