Add Vulkan Support to RetroPie for Pi 4?

bluestang

@dankcushions I compiled RetroArch from master using your gles flags —enable-gles3_1 and —disable-gles3_2 and using the gl driver results in some graphics anomalies with DuckStation. I think it has to do with geometry shaders as my log shows they are not supported which struck me as odd at first.
The caveat is that they are in upstream Mesa repo and have been for some time. The problem at large is that the GLES driver reports v3.1 which is technically correct because tessellation shader support was never implemented AFAIK to officially get 3.2 conformance.
So we are sort of stuck in limbo because we have a GLES V3.1+ implementation in current state.
I’m sure you are aware of this but I wanted to put this in this thread for future reference.
The Vulkan 1.0 driver doesn’t have this specific issue.

dankcushions

@bluestang there are known shader issues with the mesa gles driver: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3758. these manifest as visual bugs apparently.

however i don’t know about any geometry shaders showing as not supported, but being supported by mesa. can you show your log and where it shows that upstream supports them?

bluestang

@dankcushions said in Add Vulkan Support to RetroPie for Pi 4?:

@bluestang there are known shader issues with the mesa gles driver: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3758. these manifest as visual bugs apparently.

however i don’t know about any geometry shaders showing as not supported, but being supported by mesa. can you show your log and where it shows that upstream supports them?

Here is the DuckStation log - https://pastebin.com/n39ULv5c - line 487.

Here is the MR for GS feature in v3d in MESA - https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2731

dankcushions

@bluestang said in Add Vulkan Support to RetroPie for Pi 4?:

@dankcushions said in Add Vulkan Support to RetroPie for Pi 4?:

@bluestang there are known shader issues with the mesa gles driver: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3758. these manifest as visual bugs apparently.

however i don’t know about any geometry shaders showing as not supported, but being supported by mesa. can you show your log and where it shows that upstream supports them?

Here is the DuckStation log - https://pastebin.com/n39ULv5c - line 487.

Here is the MR for GS feature in v3d in MESA - https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2731

thanks. the check for geo shaders is here:
https://github.com/stenzek/duckstation/blob/c375d04bad1b19b41410398ec6291c438a71ab4a/src/core/gpu_hw_opengl.cpp#L261

m_supports_geometry_shaders = GLAD_GL_VERSION_3_2 || GLAD_GL_ARB_geometry_shader4 || GLAD_GL_ES_VERSION_3_2;

so it's looking for GL_ARB_geometry_shader4

as for support in mesa, note this:
https://gitlab.freedesktop.org/mesa/mesa/-/blob/master/docs/features.txt#L411-414

The following extensions are not part of any OpenGL or OpenGL ES version, and
we DO NOT WANT implementations of these extensions for Mesa.

  GL_ARB_geometry_shader4

i note that this is available as an EXT here:
https://gitlab.freedesktop.org/mesa/mesa/-/blob/master/docs/features.txt#L375

GL_EXT_geometry_shader4 commands (expose in GL 3.2)   DONE

however from that comment, it makes me think it's not available in GLES contexts. it's not clear to me if GL_OES_geometry_shader(which is in the GLES 3.2 features section) is the same as GL_ARB_geometry_shader4 - if it is, maybe the check is incomplete. you could ask stenzak on the duckstation discord? https://discord.gg/Buktv3t

dankcushions

looking at
https://www.khronos.org/registry/OpenGL/extensions/OES/OES_geometry_shader.txt
and
https://www.khronos.org/registry/OpenGL/extensions/ARB/ARB_geometry_shader4.txt

they seem the same to me! i'm going to attempt to integrate and see what happens. PR forthcoming.. maybe.

bluestang

@dankcushions said in Add Vulkan Support to RetroPie for Pi 4?:

looking at
https://www.khronos.org/registry/OpenGL/extensions/OES/OES_geometry_shader.txt
and
https://www.khronos.org/registry/OpenGL/extensions/ARB/ARB_geometry_shader4.txt

Yep, I just checked the Khronos website as well. Looks like GL_ARB_geometry_shader4 was first part of the 1.1 extensions and GL_OES_geometry_shader was later added as a core feature for GL/GLES 3.2 extensions. That's why the GLAD_GL_ES_VERSION_3_2 check is there, because in theory you can't be GL/GLES 3.2 conformant without geometry shaders, but it doesn't cover the Pi 4's use case since we have the feature implemented but don't have 3.2 conformance.

This makes sense since GLES followed GL's standards when it was created.

they seem the same to me! i'm going to attempt to integrate and see what happens. PR forthcoming.. maybe.

I added the following to gpu_hw_opengl.cpp

m_supports_geometry_shaders = GLAD_GL_VERSION_3_2 || GLAD_GL_ARB_geometry_shader4 || GLAD_GL_OES_geometry_shader || GLAD_GL_ES_VERSION_3_2;

And the error is gone now.

I believe a PR is indeed the right avenue to add the feature for the devices that are not fully GL/GLES 3.2 compliant but do in fact have geometry shaders implemented.

I still have graphical anomalies...

dankcushions

@bluestang

done (and additionally with another supported extension): https://github.com/stenzek/duckstation/pull/1121

I still have graphical anomalies...

ya, will be no change till we see a fix for https://gitlab.freedesktop.org/mesa/mesa/-/issues/3758

bluestang

@dankcushions said in Add Vulkan Support to RetroPie for Pi 4?:

@bluestang

done (and additionally with another supported extension): https://github.com/stenzek/duckstation/pull/1121

Kudos for the PR and also getting the other extension supported!

I still have graphical anomalies...

ya, will be no change till we see a fix for https://gitlab.freedesktop.org/mesa/mesa/-/issues/3758

I was thinking about submitting my specific issue to Mesa with the caveat that I don't get the same graphical glitches using the Vulkan driver or just provide more info to the existing issue. In my specific case, I'm getting sprite draw issues where it's unreadable.

I already have an issue opened on Mesa with lr-flycast using the Vulkan driver.

The developers are looking for more real-world use cases to bug test the driver with.

dankcushions

@bluestang said in Add Vulkan Support to RetroPie for Pi 4?:

I was thinking about submitting my specific issue to Mesa with the caveat that I don't get the same graphical glitches using the Vulkan driver or just provide more info to the existing issue.

it is already confirmed as not happening in the vulkan (v3dv) driver: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3758#note_701947

I already have an issue opened on Mesa with lr-flycast using the Vulkan driver.

you may want to try and get a backtrace with a debug build of flycast.DEBUG=1 buildflag, typically. that should make the trace a bit more useful.

bluestang

@dankcushions said in Add Vulkan Support to RetroPie for Pi 4?:

it is already confirmed as not happening in the vulkan (v3dv) driver: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3758#note_701947

I added some screenshots to the issue thread for visual references.

you may want to try and get a backtrace with a debug build of flycast.DEBUG=1 buildflag, typically. that should make the trace a bit more useful.

Thanks for the tip! I updated my issue with the new backtrace report. Seems like the issue appears to be in the GetPhysicalDeviceProperties2 struct returning a bunch of zeros where it should be returning the device limits in v3dv_device.c

bluestang

@dankcushions
I finally got lr-flycast working on the Pi4 Vulkan driver. The debug tips from you and Alejandro were extremely helpful.

Comment out lines 184-193 in vulkan_context.cpp and it should work now.
I did see performance gains in Vulkan vs GLES but I would like to see more feedback on this. I only tested a few games.

dankcushions

@bluestang nice work! i guess the issue is if that the line being commented out is important for flycast, as is presumably the case, but I guess flycast upstream would be able to confirm.

myzar

@bluestang I've recompiled mesa using latest trunk , i've recompiled retroarch master with vulkan and flycast with that workaround, but every game i've tried crashes always in this way:

https://pastebin.com/LuVpWF3N

what am i missing ?

mitu

@myzar You forgot to include the drivers for the RPI GPU, your Vulkan implementation is software only:

WARNING: lavapipe is not a conformant vulkan implementation, testing use only.
[INFO] [Vulkan]: Found GPU at index 0: llvmpipe (LLVM 7.0.1, 128 bits)
[INFO] [Vulkan]: Found GPU at index 1: V3D 4.2

bluestang

@myzar build the v3dv driver with following:

meson --prefix /usr -Dplatforms=x11 -Dvulkan-drivers=broadcom -Ddri-drivers= -Dgallium-drivers=kmsro,v3d,vc4 -Dbuildtype=release build
ninja -C build -j4
sudo ninja -C build install

There is a patch in the works to get the necessary extensions working to make lr-flycast work without changing the current source.

I will update this thread when it’s merged.

myzar

@bluestang wait do you need x running or xinit ? it works for me if i start retroarch from x

https://pastebin.com/W14Ss6xQ

it doesn't from framebuffer , i seem to understand that it should now using mesa master.

I've tried my two favorite games doa2 and soul calibur , no gfx glitches but both run alot slower for sure

my mesa build options https://pastebin.com/0aM0Dwvf

bluestang

@myzar said in Add Vulkan Support to RetroPie for Pi 4?:

@bluestang wait do you need x running or xinit ? it works for me if i start retroarch from x

https://pastebin.com/W14Ss6xQ

it doesn't from framebuffer , i seem to understand that it should now using mesa master.

I've tried my two favorite games doa2 and soul calibur , no gfx glitches but both run alot slower for sure

my mesa build options https://pastebin.com/0aM0Dwvfs far

Yes build from Mesa “master” - https://gitlab.freedesktop.org/mesa/mesa

It can be run from X11 or without. My tests have been without X11 thus far.

I’m not sure what your build settings are but your build is using the software driver llvmpipe so something is not right. The Vulkan driver has quite a few dependencies prior to building and install.

You need satisfy those dependencies and relook at your build settings.

myzar

@bluestang sorry for having wasted your time , i had some leftover of an old vulkan pkg and it was using that not the new one, purged and removed it and wow flycast is indeed faster , i'm testing daytona and i can see the difference

BuZz

@myzar are you comparing performance against the older packaged mesa driver? There are performance improvements with GLES in the latest mesa drivers so you should check with that also.

myzar

@buzz i'm switching live between the gles and vulkan driver with the fps counter enabled , in daytona is 6/7 fps higher and never dips below 30fps like the gles does, overall is a better experience. Yeah the gles driver is faster than it used to be for sure, daytona used to run like crap. Now it's okish with the gles driver and verygood with vulkan