crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come)

ecto

@Riverstorm Hi, Can I use your script to mass-change the shader and resolution settings of my already installed Bezelproject game cfg files?

I couldn't look into the script because the download link has expired...

Thanks!

Riverstorm

@ecto - I don't think it's possible or at least not easy. You can generate a new set and it shouldn't be to hard doing a compare of old and new folders to pull a new set of cfgs named the same. If you're using custom resolutions on a per game basis for bezels I don't see any easy way.

Here's a new download link to the files for you to take a look though. I made a few more modifications.

One set for the crt-pi shaders and one for the zfast shaders. Also in the zip is the resolution_db files. You only need to install Python and they should work.

crt_pi_configs.bat
crt_pi_configs.py
zfast_configs.bat
zfast_configs.py

I modified the scripts so they could also be used for mame2003 or mame2003-plus. They are identical cores except for a few hundred games. Mainly mame2003-plus additions but there's also around a dozen or so that are mame2003 only.

I also added FBNeo but left FB Alpha for those still using the older core.

If a game is not in one of the resolution_db files (as they are getting older now) you can add an entry fairly easy by looking at what information is needed. It's a flat text file, one game entry per line and most of the fields are unused.

Another option is to find a clone or parent ROM .cfg file that has already been generated for the core and just copy/rename it. For example you can easily use the Donkey Kong (dkong.cfg) and modify it for Donkey Kong 3 (dkong3.cfg), etc.

The other 2 files in the archive are just batch files to create both a 1280x720, 1920x1080 and curvature set of cfgs, for all the cores at the same time and then zip them up. You can modify the batch file and add/remove specific cores and resolutions you're targeting for your setup.

The batch files aren't needed though. If you prefer to just run the python scripts from the command line similar to below which would not be zipped up but in a folder only.

python crt_pi_configs.py mame2003 1920 1080

For the other cores you can replace mame2003 with the specific core you want. For the x (horizontal resolution) replace 1920 for y (vertical resolution) replace 1080.

For curvature just remove the resolution values and add curvature 0 like so:

python crt_pi_configs.py mame2003 curvature 0.

Core Names:
MAME 2003 = mame2003
MAME-2003 Plus = 2003plus
FinalBurn Alpha = fbalpha
FinalBurn Neo = fbneo
Consoles = consoles

iainjh

Hi

Riverstorm would you mind checking that download location, as it timesout :( TIA!

also could someone please re confirm the correct folder names in retropie/configs/all/retroarch/config for mame2003plus and for fbneo.

thank you so much, I'm running 1280 x 1024 and these scanline improvements have been so appreciated

iain

ecto

@Riverstorm Thanks for uploading the scripts and explaining them once again!

Regarding my provious question, I think I may have come to a solution:
The Bezelproject just downloads cfg files for each game containing a path to the bezel (like here dondonpachi):

input_overlay = "/opt/retropie/configs/all/retroarch/overlay/ArcadeBezels/ddonpach.cfg"

I will now just take your generated zfast_curvature cfg files and merge them into one file with something like

#!/bin/bash

cd 'bezelproject_cfgs'
for bezel_file in *.cfg; do
  zfast_file="zfast_curvature_cfgs/$bezel_file"
  cat "$bezel_file" "$zfast_file" > "merged_cfgs/$bezel_file"
done

(Mind, I haven't tested the script, just googled away and found a solution to merge two textfiles!) Use at your own risk and only with backed up data!

Last thing worth noting is, that probably my Rpi3b+ isn't fast enough for handling bezels and a shader... That's at least what I learned reading through this thread. But that doesn't matter: It's alway fun to have new project going! Also, this paves the way for a future Rpi4! ;)

dankcushions

the scripts are all broken (again, but this time in a new way) with the latest version of retroarch :|

ecto

@dankcushions What happend?

Riverstorm

@iainjh - Doing a quick download test it seems to be working fine.

The cfg files need to be located in:

/opt/retropie/configs/all/retroarch/config/MAME 2003-Plus
/opt/retropie/configs/all/retroarch/config/FinalBurn Neo

@ecto - Thanks for a solution to merge. The cfg file generated is fairly small with 4 comment lines and only 7 "functional" lines but still it could be a lot of work without a script. :)

I would also be curious as to what's broke with the latest RetroArch. I mainly use RetroPie and when downloading the latest RetroPie script 4.5.1 (896b34e1) and updating RetroArch it's still on 1.7.6. Everything in 1.7.6 works still so I am guessing it's something in 1.7.7 but I can't find a thread on what's broke.

dankcushions

@Riverstorm https://www.libretro.com/index.php/retroarch-1-7-8-released/

i've not tried it, but looks like the shader changes will break them.

mitu

@dankcushions said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

i've not tried it, but looks like the shader changes will break them.

Yes, the video_shader parameter has disappeared, hence any usage of it in a .cfg file will be ignored.

Riverstorm

Some neat features in that version like the OCR, text to speech and CD support.

I use whatever RetroPie defaults to which I think is RA 1.7.6 which works well but it does look like when RetroPie goes to RA 1.7.8 some things will need tweaked in the script.

Example of new shader command line for RA 1.7.8:

retroarch --set-shader "D:\RetroArch\shaders\shaders_glsl\blurs\kawase_blur_5pass.glslp" -L <core> <content>

Example with relative path:

retroarch --set-shader "shaders_glsl\blurs\kawase_blur_5pass.glslp" -L <core> <content>

iainjh

thanks, I tried again and after a few goes it downloaded. thanks also re the paths!

I'll avoid updating retroarch.

:):):)

@Riverstorm said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

@iainjh - Doing a quick download test it seems to be working fine.

The cfg files need to be located in:

/opt/retropie/configs/all/retroarch/config/MAME 2003-Plus
/opt/retropie/configs/all/retroarch/config/FinalBurn Neo

@ecto - Thanks for a solution to merge. The cfg file generated is fairly small with 4 comment lines and only 7 "functional" lines but still it could be a lot of work without a script. :)

I would also be curious as to what's broke with the latest RetroArch. I mainly use RetroPie and when downloading the latest RetroPie script 4.5.1 (896b34e1) and updating RetroArch it's still on 1.7.6. Everything in 1.7.6 works still so I am guessing it's something in 1.7.7 but I can't find a thread on what's broke.

iainjh

Hi all

Stupidly, I've managed to lose the file "crt-pi-configs-master-prod.zip" and the download link requires a subscription. Could someone please share the file?

I have acquired a new monitor panel and need to output new .cfg's for its 1600x1200 resolution

Also, back over my normal pi (pi4, most recent retropie, 1280x1024 monitor and the .cfg's created above) it appears the optimum vertical mask and sizing works perfectly even though the last posts indicated some problems with recent retroarch builds. Are those issues now fixed?

But I have a new issue. The display on some games is now upside down! Donpachi for example. Has anyone else seen this or could suggest how to fix? ive added:

video_allow_rotate = "true"
video_rotation = "2"

with no change.

thank you very much for any pointers! scanlines FTW!:)

Riverstorm

@iainjh - Here's a link to the files. I haven't tested them yet on the Pi 4 with the latest RP image on Buster. I've mainly use the old version on the Pi 3 for gaming and have been just messing on the Pi 4 but will eventually migrate to the new board.

iainjh

thank you!!! superstar :)

@Riverstorm said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

link

edit: however now I've applied the whole set anew, I believe the issue remains that latest retroarch on Pi4/buster implements the scaling correctly but ignores the selection of shader and the vertical mask in the cfg. As was said above...

Some games are also rendered upside down.

I was wrong in my earlier post as i must have saved some overrides.

thanks again for the link, i'll save these properly this time:)

Riverstorm

@iainjh - It looks like you figured it out from the edit but I did some testing over the weekend and the scripts should still work fine running RetroPie 4.6.1 on a Pi 4 using Retroarch 1.8.8.

Riverstorm

I don't know how many folks still use it but I uploaded the shader script to Github here temporarily. The "free" file hosting sites seem to charge you after 30 days to download the file.

I consolidated the script down to one file for all cores and shaders.

A few things were broken but have been fixed.

If you used curvature it would generate zero byte files. It's been corrected.

There's no need to pad the command line when using curvature anymore (it was added but that commit was broke also). Just use curvature with no width or height.

The Scale Factor is included in the cfg files. Same reason as above, it was added after it was broke with a commit. If under 3 it's disabled in the cfg files. To see how it looks enabled check out Popeye or Rampage as good examples of games under a scale factor of 3. They don't look so good.

I also added a shader argument so you can use either the crt-pi or the zfast shader.

Usage:
python pi_shader_configs.py <core> <shader> <screen width> <screen height>

Parameters:
core (mame2003, 2003plus, fbalpha, fbneo or consoles)
shader (crtpi or zfast)
screen width (i.e. 1920 or curvature)
screen height (i.e. 1080)

NOTE: If using the curvature parameter don't add width or height to the command line.

Examples:

python pi_shader_configs.py mame2003 crtpi 1920 1080
python pi_shader_configs.py fbalpha crtpi curvature
python pi_shader_configs.py 2003plus zfast 1280 720
python pi_shader_configs.py fbneo zfast curvature

You can also use the bat file to auto-generate different target resolutions and have them zipped in one step. The .bat file has 1920x1080, 1280x720 and curvature for all cores using both shaders. It's pretty self-explanatory to modify it to your needs. It's easier to copy one zip file than thousands of cfg files! :)

There's a readme that explains most of the information.

I did download the .44 resolution database posted up further in the thread generated by UDb23. I know it's outdated but more current than the older fbalpha database. It needs a lot of corrections but once completed I'll modify the script to use it for fbneo and leave the fbalpha database for the older core.

dankcushions

@Riverstorm appreciate the help but please offer PRs to the original code rather than copy it, modify and rehost it under your own name, without the commit history from the other contributors, even if temporarily. if you’re going to that effort of putting it on git, why not just do a PR?

Riverstorm

@dankcushions - That was never my intention was for you to think I was out to copy your code. I only discuss the script here and it's never left my PC except to make its way to Github. I guess I'll try to explain why. Sorry, I don't have a short answer here and prefer to explain as I don't want you think I was out to do something shady with your script.

Up front I never meant to step on you toes and I most certainly wouldn't "steal" someones code and then do a turn about and post it in the same thread. That just sounds kind of silly. It was strictly to help out others with their cfgs as I am done.

The code hasn't seen any of the much needed updates for years. The script is broken in several areas in it's current state. It's been something like 2 1/2 years I think, or this fall 3 years since the last update. It's a broken script to the point of not functioning and needs maintenance in several areas to work correctly with the new versions of RetroPie on Buster and even the older versions on Stretch. From past comments it seems you have no desire or interest to do anything with it.

Almost 2-3 years ago Libretro changed an Index which broke the code and you stated you refused to make any further changes due to that event and linked the Github issue, several times. When someone asked for corrections you also linked that same issue.

Then after Andrew completely rewrote the engine effectively removing the "99 loop" and making the code much more efficient. He offered to make further enhancements to the script you stated the script's goal had been achieved and you didn't really want to add anything to it.

I know I've pinged you on certain issues and, I forget which member, stated zero byte curvature files were being generated by the code you ignored all those requests. For the most part you completely ignore all of my posts too. It's not easy when a global admin completely ignores you because of events, I assume, that occurred years ago.

I still think there could be some additional worthwhile enhancements. My first step was an attempt to post/create a Github issue on some of the problems but you blocked me way back then 2-3 years ago (on Github), in turn, feeling slightly torqued I did the same.

I'm not sure exactly when or why you blocked me, to be honest, and/if when or why you unblocked me. We just plain don't chat or have conversations in any of the same repos on Github, except for a very brief time during the interface discussions on the m3plus core several years ago. I would never waste my time talking to someone, if they don't want to talk. Without knowing I guess I just figured it was due to the long running history.

Anyway, being blocked and not sure if I needed to use another account or whatnot but I logged out so I could access your page "anonymously". When someone blocks you on Github you can't do much for a repo, no PR's, no commits, no comments, no activity feed updates, no collaborator, no sponsorship, no...no nothing.

So I started downloading and rolling back one commit at a time to find where it went off track. Then I had to spent some time creating extra variables to watch the output values, basically running/reading the code to figure out how the latter commits broke the code and then further spent some time on how to go about fixing it. Ugh...Python and referencing variables outside a local scope. ;\ I'm a rookie with Python.

So for the past few years I've been making small changes to the code here and there on my PC with Notepad++, then zipping/uploading them to hosting services for others to use that have expressed interest in the program. Updating paths, core names, indexes, adding a shader, external batch zip function, etc. I think they are clean changes but I have no doubt the more experienced coder could write tighter code. I feel the way I did it is fairly clean and straight forward if not simplistic.

It seems every once in a while the link expires and someone asks for a working program so I try to accommodate them by zipping/uploading when requested. I've always kept all this upfront and in this thread, nothing behind anyone's back.

I figured uploading on Github was the next logical step as you never complained about how I was doing it here for the past few years and you never said you should add those changes to the repo which is in essence the same thing I was doing already except I wasn't using Github, just my PC and Notepad++.

Uploading to Github, as you know, really only takes a minute and I've had the files on my PC for years. It was a drag and drop deal after the repo creation. Plus now I wouldn't have to zip/upload all the time (file website hosting links expire every 30 days or so). People would be able to grab a working copy whenever they wanted without having to ask every time. I figured it would just work better for others and allow me to continue making small tweaks here and there easier to keep it going, document the changes and right a decent README.MD with the directions and additional information I felt was pertinent on how to add entries to the DB files, what scaling factor under 3 means for shaders, etc. I feel like it's just a fork that I'm doing my own work on.

On the other-hand making corrections to the database files is proving to take hours. The mame2003 DB is pretty much static but I want to split the mame2003 DB into two add the m3plus games as a separate DB. For the older fbalpha and fbneo DB's I loaded them into Excel to sort them alphabetically, added missing fields to the fbneo database to match them up, exported them as comma delimited files and used WinMerge to compare and cross reference the differences. You need them as close as possible for the WinMerge step to be successful. Now there's some minor improvements to the original fbalpha file and hundreds of clone entries to the newer fbneo DB.

As for homebrew I've left them with 'E' designation as they can't be searched easily. Maybe the drivers are a good place to look. I might go back later as they will take a bit more time to research. Also games that have several monitors vertically or horizontally have weird ratios/resolutions (i.e. Super Arm Wrestling, Punch-Out!, Tecmo Bowl) but I just set them to 3:4 or 4:3. Vector games I left alone with 0,0,0,0 entries (I thought of removing those altogether as usually you wouldn't use shaders in conjunction with vector games).

I also thought about removing the maxResX, MaxResY, resX_Scaleinteger, resY_Scaleinteger and frequency (1440, 1080, 1196, 896, 60) altogether from the DB's. I don't know if they had an intended purpose at one point but they don't seem to serve any purpose now.

I also find the program useful for setting shaders for both vertical and horizontal games in one shot. I think if you do it through the RP menus you only get to set either one or the other, basically a vertical or horizontal shader or at least I don't know how if it can be done. So in a way it has a dual purpose. I think being able to generate vertical or horizontal cfgs only could be beneficial.

Basically, in a nutshell, the repo has several broken things from the index, paths, curvature files, added zfast, some timely DB work, etc. no one has been able to get a hold of a fully working copy for almost 2 1/2 to 3 years. We've been limping it along to keep a working version going.

I was blocked from your repo and didn't go back weekly or monthly to check if you unblocked me, plus you ignore 99% of my posts here anyway. You also mentioned and stated more than once you have no desire to further work on the project as the goal of the project has been completed. My hands feel a bit tied here it seems and any collaborative PR would most likely go down a rabbit hole, speaking from a historical perspective.

I guess people will just need to need to make the manual modifications from where you left off and I'll let you handle your script how you see fit. I managed to avoid any conflict with you over the past few years and I honestly don't want us to get into debate about how things unfolded now since things have been going well. So I'll change the repo to private and call it good. This is a small community where most everyone knows everyone and it seems wise to keep the peace. I don't want to war with you about the script.

I was already sharing with zips and Github seemed like a good choice. I was just trying to get the thing going again and share it with others since you have refused to do any updates starting with the Libretro index change. It was a very minor update, I think they changed a number from 22 to 23? Click, done! It hasn't changed in 3 years.

Libretro is a bit of juggernaut and a moving target so making a few concessions for the sake of keeping the wheels turning is ok. You can't stop every time you disagree with what upstream is doing which was basically changing a single number. You have to get creative with what they do sometimes even though this one was straight forward.

I agreed with some of Andrew's enhancements and think they would make fitting additions but you said you had no interest in them. Simple things like this I think were we simply don't see eye to eye upon would make it difficult to get things done. I just take that as you don't care to continue work on it until Libretro complies with your request so the program continues to grow stale going on 3 years.

As I write, I do agree and I do apologize as I should have put your name in the README.MD as the credit should have been given to you as the original author that started the script or fork it with a derivative work, I think, would have been more acceptable.

@RION I hope you wouldn't think I copy code for my own sake of code copying infamy of a small niche community...relatively speaking. My main goal was to contribute to the m3plus repo and you updates sounded like a fantastic way to start. I was doubly elated when I learned it was you with a different username. I read you posts for years here. I guess it's just that much sweeter when it's a fellow RetroPier that you like.

I was learning, I was close but I dropped the ball. Rather than stepping back, regrouping and taking another shot, frustration got the better part of me that day and I kind of let you down. I made a mistake but I did come back the next day and tried to make things right by submitting a single clean PR that wasn't full of a half dozen commits of garbage and apologized. It's unfortunate but ironic but I can actually contribute to a core with a proper PR at the end but it just happened to be my learning curve was with your catver. We all have to start somewhere. =/

Anyway I sincerely apologize for the script upload on Github and I honestly don't want you to think I was out to get your script for some personal gain or infamy. I have all my cfgs generated but occasionally add a game here and there but for the most part I'm done with them and was just trying to share a working version with others. This is your post and your script and you should run your railroad as you see fit. It was never meant to cause an issue but it's been removed and I'll digress from any further updates.

dankcushions

@Riverstorm

Up front I never meant to step on you toes and I most certainly wouldn't "steal" someones code and then do a turn about and post it in the same thread.

i have not accused you of that! however, maybe it's an open source faux pas (far, far worse :P )

Almost 2-3 years ago Libretro changed an Index which broke the code and you stated you refused to make any further changes due to that event and linked the Github issue, several times. When someone asked for corrections you also linked that same issue.

yes, i do not want to make fixes to deal with a moving target but I have no problem with others making said changes via a PR. in any case, i'm pleased to say that my whining about that particular issue had a positive effect - they no longer add to the aspect ratio index and are working on a work-around: https://github.com/libretro/RetroArch/issues/7536 - so now should be a fairly safe time to make the fix.

Then after Andrew completely rewrote the engine effectively removing the "99 loop" and making the code much more efficient. He offered to make further enhancements to the script you stated the script's goal had been achieved and you didn't really want to add anything to it.

it was certainly improved, and has a few bugs, which i would be more than happy to accept PRs for. i'm not totally against new features if i understand the change (as i clarified in this thread), but rehosting the script is not the way IMO. more than anything, we have a link to to the script in the first post, and it should be the right script.

i had forgotten i'd blocked you on github. i certainly got tired of the personal attacks and tagging me into endless debates, against my will. and even some linked in this thread: https://retropie.org.uk/forum/post/117248.

this is a hobby for all and should be fun, with a mutual level of respect given to all who contribute their time and skills to the project, even if we may occasionally disagree. i have never touched C, bash or python outside retropie, but they make a fun little challenge outside my development dayjob, and i'm proud of my small contributions, bugs aside...

anyways, all this crap is years ago and water under the bridge... so i've removed the block, and welcome any PRs!! (one per fix/feature, please!)

Riverstorm

@dankcushions - Yes, that's the conversation I was referring to. I tried to neatly bundle the whole thing into the "interface conversation" because you're the only that needed to understand the context and not to dig up a bunch of old blood was my thought. =/ I also removed any personal opinions from that last post to keep it on point. I just didn't want it to be a he said/she said, read here.

I don't mean to bring Grant into the conversation when he can't be here to add his thoughts but that's the thing though, all conversations have two sides. Rarely is one party a true victim and can contest "no fault". Cherry picking posts without framing the context of the entire thread really doesn't work. There was plenty of collateral damage on all sides. All parties in the conversation had been pushed to the limit and started taking pot shots and injecting strong personal opinions until Libretro came in and closed it for a while as a hot topic.

In all honesty you came completely unhinged on Grant by the end and he was pinging you constantly to get back into the conversation. You'd make a post, leave again and come back. It was full on. The meat of that conversation was between you and Grant on what's the "right" way for a layout but you kept "blanket" referring to the room at large to stop pinging you but not everyone was pinging you so I posted this:

@dankcushions - I've never tagged you, ever! I'm sorry but there's just as much good that has come from these conversations and I do learn quite a lot. I am trying here and also trying to add some useful input in a constructive way or at least be part of the community. No new idea is fully embraced and is usually met with resistance but thanks for the advice, again!

I tried to reiterate it again two posts later:

@dankcushions - short answer, just to be clear it wasn't me and I promise not to tag you, after this one. ;)

I know @ is pinging too but I only did it when you were part of the conversation and not "just because" after you left the conversation. I know that post above where I pinged you to say I never pinged you is a bit ironic but I thought it was a little funny! :)

On lighter note I did learn a good deal from the conversation about layouts. I really didn't care for any of them out of the box but was happy I could modify it to my needs.

I do appreciate you all your efforts and all the time put into things but I'm going to gracefully bow out of here of further updates as I really don't want to create issues. This conversation feels volatile and could go south quickly and I don't want to do that as it's water under the bridge now. I tried to keep that whole conversation from a few years ago from Github, on Github. We approach things differently, think differently or something and that's ok.

I do read many of your posts and mostly don't comment or there's weeks at a time where I only read what I follow through emails and don't visit the site at all. When a new version of RP or a new board is released it creates some buzz. Anyway hopefully the folks that do use it grabbed a copy with the additional shader off Github [when I had it up] or the link above is still active, which basically works the same just split into two scripts. If you prefer you can remove it as you commented you wanted to keep program links to the first post only.

It's pretty cool they are going to use a ratio vs an index that should make it worth fixing up now...hint, hint! ;)