Low framerate on main EmulationStation menu - Pi 4

bluestang

I'm starting to wonder if the SDL2 build script with the isPlatform "rpi" && conf_flags+=("--enable-video-rpi") is the cause of the framerate drop in ES for the Pi 4.

The Pi 4 should not be using any of the "legacy" VC libraries in /opt/vc/ because the Mesa libraries are the default now. The Pi 4 is adopting the standard Linux APIs so it should be mirroring what the other SBCs are doing. This is why omxplayer has been deprecated altogether.

The VC libraries are there for "backward compatibility" but should not be relied upon moving forward. There have been several threads on the official Raspberry Pi forums about this.

@mitu @dankcushions @BuZz

BuZz

@bluestang your assumptions are incorrect. Check the sdl2 module history and our sdl2 branch.

bluestang

@BuZz said in Low framerate on main EmulationStation menu - Pi 4:

@bluestang your assumptions are incorrect. Check the sdl2 module history and our sdl2 branch.

I did take a look and even though the VC headers are moved temporarily the --enable-video-rpi still triggers an option that is not necessary for the Pi 4. (https://github.com/RetroPie/SDL-mirror/blob/retropie-2.0.10/configure#L19829) - see setup log here --> lines 332 & 333. The standard Mesa libraries/drivers that come w/ RPiOS know which API to use based on the overlay driver configured - i.e. FKMS or KMS.

From my understanding via the official RPi forums, the Pi 4 should be built with the preferred Mesa headers/libraries rather than the Broadcom VC ones. Meaning, for the Pi 4 the options should be --enable-video-kmsdrm and --disable-video-rpi. I made this change here --> Setup Log and didn't see any negative issues. I need to do more testing to see if this fixes the framerate drop entirely but initial results look promising.

The differences in the two drivers are FKMS still uses the Broadcom proprietary Dispmanx API and KMS uses the standard Linux KMS/DRM API. (FKMS is just a wrapper on top of Dispmanx) Compatibility for Dispmanx will still be available for the Pi 4, but it seems that it will be deprecated at some point in the future. Especially once the KMS driver is fully developed.

In 64-bit RPiOS you've probably noticed that the Broadcom VC headers are now located in /usr/include but the Broadcom specific libraries no longer are present. Developers should be using the standard Linux libraries aka Mesa ones just as you would for standard Linux development.

BuZz

@bluestang said in Low framerate on main EmulationStation menu - Pi 4:

Developers should be using the standard Linux libraries aka Mesa ones just as you would for standard Linux development.

We are.

bluestang

@stoo @Darksavior what is your result if you pass the cmd line option emulationstation --vsync 1?

Darksavior

@bluestang No change. fkms or kms.

stoo

@bluestang yeah, seems exactly the same

bluestang

@stoo @Darksavior Can you confirm that you can get the 30 fps framerate drop immediately if you invoke the screensaver for 5 secs and then exit to go back to the main menu?

Also, I enabled the Enhanced option of the Power Saver Mode in Other Settings and although I still get the fps drop, ES will go back to 60 fps without having to go into RA separately.

This is a weird bug for sure - @BuZz

BuZz

It's primarily a driver/firmware issue or maybe an overlapping issue with SDL2 as the sdl2 Kms driver has some issues. I thought I got some improvements disabling the WSOD / dynamic gpu texture loading code (which does have one bug, in that 0 should be unlimited but isn't). But I'm actually looking into that currently as well as SDL2 due to upstream changes.

Our sdl2 includes the old driver, but isn't used for ES on the rpi4. As GL contexts don't work on the RPI4 on dispmanx, there wouldn't have been any display at all. They co-exist together in our SDL though.

Primarily for rpi3 etc. Our sdl2 will work in both old videocore via dispmanx or via kms on a rpi3.

Newest Mesa doesn't fix the slowdown btw. Before people said switching to the Kms driver sorted it. I can't remember but I'll retest.

bluestang

First off I'm sorry for bombarding your PRs with my nonsense. RA made it possible to use the developmental Vulkan driver post v1.8.8 and I was excited to try it out. Just in what I've been able to mess around with, it seems to provide a few improvements.

@BuZz said in Low framerate on main EmulationStation menu - Pi 4:

It's primarily a driver/firmware issue or maybe an overlapping issue with SDL2 as the sdl2 Kms driver has some issues. I thought I got some improvements disabling the WSOD / dynamic gpu texture loading code (which does have one bug, in that 0 should be unlimited but isn't). But I'm actually looking into that currently as well as SDL2 due to upstream changes.

Lowering the resolution to 720p does not exhibit the framerate drop but that is not the ideal solution. Would compiling the latest release of SDL2 with the neon optimizations help?

Our sdl2 includes the old driver, but isn't used for ES on the rpi4. As GL contexts don't work on the RPI4 on dispmanx, there wouldn't have been any display at all. They co-exist together in our SDL though.

Primarily for rpi3 etc. Our sdl2 will work in both old videocore via dispmanx or via kms on a rpi3.

Although the Pi 3 and Pi 4 share the same driver names, which probably leads to more confusion for everyone, they do not follow the same rules when it comes to 3D and video composition.

Newest Mesa doesn't fix the slowdown btw. Before people said switching to the Kms driver sorted it. I can't remember but I'll retest.

I can confirm that when KMS was first exposed in the 5.4 kernel it did fix the issue but that was many moons ago. An update at some point led to the regression, but that may have just been luck as the KMS driver was missing functionality and still does.

Swampy

@bluestang said in Low framerate on main EmulationStation menu - Pi 4:

I can confirm that when KMS was first exposed in the 5.4 kernel it did fix the issue but that was many moons ago. An update at some point led to the regression, but that may have just been luck as the KMS driver was missing functionality and still does.

I have had the same experience.

bluestang

I couldn’t find this issue at first but I did remember reading at some point. Seems relevant to the issues described in this thread.

https://github.com/raspberrypi/firmware/issues/1154#issuecomment-544428380

GreenHawk84

I am glad there is interest in this, thanks for the updates !

bluestang

Can anyone with a Pi 3 assist with some info?

With EmulationStation running, can you open an SSH session and run the following cmds and post the results?

cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
vcgencmd measure_clock v3d
vcgencmd dispmanx_list
tvservice -s

Appreciate the help.

mitu

@bluestang said in Low framerate on main EmulationStation menu - Pi 4:

cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
output:

ondemand
ondemand
ondemand
ondemand

vcgencmd measure_clock v3d
output:

frequency(43)=300000000

vcgencmd dispmanx_list
output:

display:2 format:RGB565 transform:0 layer:-127 1920x1080 src:0,0,1920,1080 dst:0,0,1920,1080 cost:889 lbm:0
display:2 format:RGBA32 transform:20000 layer:0 1920x1080 src:0,0,1920,1080 dst:0,0,1920,1080 cost:1156 lbm:0
display:2 format:UNKNOWN transform:20000 layer:10000 0x0 src:0,0,1920,1080 dst:0,0,1920,1080 cost:0 lbm:0
display:2 format:UNKNOWN transform:20000 layer:10000 0x0 src:0,0,1920,1080 dst:0,0,1920,1080 cost:0 lbm:0
display:2 format:UNKNOWN transform:20000 layer:10000 0x0 src:0,0,1920,1080 dst:0,0,1920,1080 cost:0 lbm:0

tvservice -s
output:

state 0xa [HDMI CEA (16) RGB lim 16:9], 1920x1080 @ 60.00Hz, progressive

stoo

@bluestang

cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

ondemand
ondemand
ondemand
ondemand

vcgencmd measure_clock v3d

frequency(43)=250000000

vcgencmd dispmanx_list

display:2 format:XRGB8888 transform:0 layer:-127 src:0,0,1920,1080 dst:0,0,1920,1080 cost:1156 lbm:0
display:2 format:UNKNOWN transform:20000 layer:10000 src:0,0,1920,1080 dst:0,0,1920,1080 cost:0 lbm:0
display:2 format:RGBA32 transform:20000 layer:10000 src:0,0,1920,1080 dst:0,0,1920,1080 cost:1156 lbm:0

tvservice -s

state 0x12000a [HDMI CEA (16) RGB lim 16:9], 1920x1080 @ 60.00Hz, progressive

bluestang

Thanks @mitu and @stoo!

Well looks like the 30 FPS issue for Pi 4s is related to the clock speed of v3d aka the GPU. The framerate drop is easily reproducible on the larger theme sets. (i.e. Cosmic Rise)

During video playback, a stock Pi 4's GPU clock speed will scale to 500 Mhz. On my Pi 4, I have a custom splash screen that plays prior to loading the ES main menu. During that video playback, the frequency reported is 500 Mhz. The GPU can technically be clocked higher with specific command line overclocks but for clarity lets eliminate overclocked use cases.

Whatever is telling the scaling governor to downclock the GPU is the cause of the 30 fps issue for Pi 4s. If I go to one of my collection menus where I also have video snaps played back, the GPU will scale its frequency to 500 Mhz, and I do not ever see the 30 FPS issue within those menus. It is only in the Main Menu in my case and I can induce the fps drop by invoking the screensaver where the GPU begins to downclock down to 300 Mhz or lower.

This also makes perfect sense that the issue "goes away" if you play a game and come back to the menu...its safe to say that is because the GPU stays scaled to 500 Mhz until the scaling governor downclocks the GPU.

I put v3d_freq_min=500 in /boot/config.txt and that has solved the issue for me. My SoC temp stays around 43/44* C and I'm using the Canakit case with a fan.

You could also change the scaling_governor to performance as well, but that will also keep the arm frequency to the max clock speed as well.

stoo

@bluestang I'll check your fix out soon.

I have no idea about the Pi4 arch, but in my completely ignorant opinion, this sounds like a firmware problem.

bluestang

@stoo Yea this could be an issue with how the kernel handles the scaling_governor on the ondemand setting. In fairness though, this could be expected behavior on this setting but I just asked the question on the official forums in the 5.4 Kernel thread.

In RetroPie's use case, it would make sense to scale the GPU to its highest allowable setting but that might not be the desired effect for users outside of RetroPie.

stoo

@bluestang How does it differ from the Pi3? How much does the Pi3 scale the GPU while ES is running?

The reason I ask is that while forcing the GPU to remain at 500MHz does indeed make ES much smoother on the Pi4, I can't help wondering whether we're just masking a deeper performance issue with ES...

...or a deeper performance issue somewhere else in the Pi4.