I installed weekly build retropie-bullseye-4.8.5-rpi3_zero2w (24-Mar-2024 05:10) and added it to config.txt:

dtoverlay=vc4-kms-v3d dtoverlay=vc4-kms-DPI-35inch dtoverlay=waveshare-35dpi-3b-4b dtoverlay=waveshare-35dpi-3b dtoverlay=waveshare-35dpi-4b dtoverlay=waveshare-35dpi dtoverlay=waveshare-touch-35dpi

And I got a normal FPS!

Yust got it working with this Guide:
https://linustechtips.com/topic/1071052-pi-zero-w-154-hat-faster-driver/

@kristjan1111 said in I have problems with my 640x480 tft display:

@ballboff I got that done, but the text still flickers a little bit and also the lines flicker.

Don't know then. There may be some other setting that I'm missing, but you shouldn't need an adapter. The composite is designed for this purpose and you'd think the standard resolution would be suitable.