crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come)

dankcushions

@andrewh i think i fixed it now, and possibly broke other things!

dankcushions

@andrewh said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

@dankcushions, I've done a (fair) bit more work and created an alternative version of your script - was wondering whether you'd consider including it in your git project.
Main features;

Checks for Python 3 and exits if not found
Creates a .csv 'log' file showing the viewport size and position created for each game (handy to check for odd values)
Includes additional information in the header of each .cfg generated, with details of game name, original resolution, aspect ratio, and whether either of the two new options detailed below were enabled. Also includes the target screen resolution.
Adds two new optional command line arguments which can be used individually or together;
- w - widen vertical games scaling factor by 1
- a - use only integer scaling on both axes
It also uses the approach that I had outlined previously (calculating the scale factors directly, rather than indexing them)

I've tested it pretty extensively using the more common resolutions - both widescreen and non-, and I'm happy enough with the output and stability.

crt_pi_configs_alt.py

# creates cfg files for crt-pi
# params are:
# * core (eg mame2003 or fbalpha)
# * screen width (eg 1920) OR curvature
# * screen height (eg 1080)
# * optional flags (w: widen, a: accurate) - use either or both, with no space between
#       widen : Increase horizontal scaling of vertical games by 1
#       accurate : Use only integer scaling for both horizontal and vertical dimensions
# example usage:
# python crt_pi_configs_alt.py mame2003 1920 1080
# python crt_pi_configs_alt.py fbalpha 1920 1080
# python crt_pi_configs_alt.py consoles 1920 1080
# python -c "import crt_pi_configs_alt; crt_pi_configs_alt.createZip(False,1920,1080)"

import sys
import os
import shutil

def generateConfigs(arg1, arg2, arg3, arg4):
    console = False
    if "mame2003" in arg1:
        fileName = "resolution_db/mame2003.txt"
        coreName = "MAME 2003"
    elif "fbalpha" in arg1:
        fileName = "resolution_db/fbalpha.txt"
        coreName = "FB Alpha"
    elif "consoles" in arg1:
        fileName = "resolution_db/consoles.txt"
        coreName = "Console"
        console = True

    if "w" in arg4:
        widen = "true"
        print("Widening of vertical games enabled")
    else:
        widen = "false"
    
    if "a" in arg4:
        accurate = "true"
        print("Accurate scaling enabled (may lead to black borders)")
    else:
        accurate = "false"
        
    if "curvature" in arg2:
        curvature = True
    else:
        curvature = False
        screenWidth = int(arg2)
        screenHeight = int(arg3)
        screenAspectRatio = screenWidth / screenHeight
        resolution = str(screenWidth) + "x" + str(screenHeight)
        outputLogFile = open(coreName + "-" + resolution + arg4 + ".csv", "w")
        outputLogFile.write("Widen : ,{}\n".format(widen))
        outputLogFile.write("Accurate : ,{}\n".format(accurate))
        outputLogFile.write("Name,X,Y,Orientation,Aspect1,Aspect2,ViewportWidth,ViewportHeight,HorizontalOffset,VerticalOffset\n")


    resolutionDbFile = open(fileName, "r" )
    print("opened database file {}".format(fileName))
    if not curvature:
        print("created log file {}".format(outputLogFile.name))
    # print('[', end='', flush=True)
    sys.stdout.write('[')
    sys.stdout.flush()
    gameCount = 0

    for gameInfo in resolutionDbFile:
        gameCount = gameCount+1
    	# strip line breaks
        gameInfo = gameInfo.rstrip()
        
        # parse info
        gameInfo = gameInfo.split(",")
        gameName = gameInfo[0]
        gameOrientation = gameInfo[3]
        gameWidth = int(gameInfo[1])
        gameHeight = int(gameInfo[2])
        aspectRatio = int(gameInfo[9]) / int(gameInfo[10])
        gameType = gameInfo[4]
        #integerWidth = int(gameInfo[7])
        #integerHeight = int(gameInfo[8])

        if console:
            coreName = gameName

        cfgFileName = gameName + ".cfg"

        # Create directory for cfgs, if it doesn"t already exist
        if curvature:
            path = "curvature" + "/" + coreName
        else:
            path = resolution + "/" + coreName
        if not os.path.isdir(path):
            os.makedirs (path)

        # create cfg file
        # print("creating {}/{}".format(path,cfgFileName))
        if (gameCount%100 == 0):
            # print('.', end='', flush=True)
            sys.stdout.write('.')
            sys.stdout.flush()
        newCfgFile = open(path + "/" + cfgFileName, "w")

        if "V" in gameType:
            # Vector games shouldn"t use shaders, so clear it out
            newCfgFile.write("# Auto-generated vector .cfg\n")
            newCfgFile.write("# Place in /opt/retropie/configs/all/retroarch/config/{}/\n".format(coreName))
            newCfgFile.write("video_shader_enable = \"false\"\n")

        else:
            if "V" in gameOrientation:
                if curvature:
                    shader = "crt-pi-curvature-vertical.glslp"
                else:
                    shader = "crt-pi-vertical.glslp"
                # flip vertical games
                gameWidth = int(gameInfo[2])
                gameHeight = int(gameInfo[1])
                # Calculate pixel 'squareness' and adjust gameHeight figure to make them square (keeping Width as-was to avoid scaling artifacts)
                pixelSquareness = ((gameWidth/gameHeight)/aspectRatio)
                gameHeight = int(gameHeight * pixelSquareness)

            elif "H" in gameOrientation:
                if curvature:
                    shader = "crt-pi-curvature.glslp"
                else:
                    shader = "crt-pi.glslp"
                # Calculate pixel 'squareness' and adjust gameWidth figure to make them square (keeping Height as-was)
                pixelSquareness = ((gameWidth/gameHeight)/aspectRatio)
                gameWidth = int(gameWidth / pixelSquareness)

            newCfgFile.write("# Auto-generated {} .cfg\n".format(shader))
            newCfgFile.write("# Game : {} , Width : {}, Height : {}, Aspect : {}:{}, Widen : {}, Accurate : {}\n".format(gameName, gameWidth, gameHeight, int(gameInfo[9]), int(gameInfo[10]), widen, accurate))
            if not curvature:
                newCfgFile.write("# Screen Width : {}, Screen Height : {}\n".format(screenWidth, screenHeight))
            newCfgFile.write("# Place in /opt/retropie/configs/all/retroarch/config/{}/\n".format(coreName))
            newCfgFile.write("video_shader_enable = \"true\"\n")
            newCfgFile.write("video_shader = \"/opt/retropie/configs/all/retroarch/shaders/{}\"\n".format(shader))

            if not curvature:
                # Check scale factor in horizontal and vertical directions
                vScaling = screenHeight/gameHeight
                hScaling = screenWidth/gameWidth
            	
                # Keep whichever scaling factor is smaller.  Also get the integer of that scaling factor.
                if vScaling < hScaling:
                    intScaleFactor = int(vScaling)
                    scaleFactor = vScaling
                else:
                    intScaleFactor = int(hScaling)
                    scaleFactor = hScaling

                # For vertical format games, width multiplies by an integer scale factor, height can multiply by the actual scale factor.
                # For horizontal games height multiplies by the integer scale factor, and width multiplies by the actual scale factor, 
                # but gets rounded up to the screen width if close enough.
                if "V" in gameOrientation:
                    if widen == "true":
                        viewportWidth = gameWidth * (intScaleFactor + 1)
                    else:
                        viewportWidth = gameWidth * intScaleFactor
                    if accurate == "true":
                        viewportHeight = gameHeight * intScaleFactor
                    else:
                        viewportHeight = screenHeight
                    newCfgFile.write("# To avoid horizontal rainbow artefacts, use integer scaling for the width\n")

                else:
                    if accurate == "true":
                        viewportWidth = gameWidth * intScaleFactor
                    else:
                        viewportWidth = int(gameWidth * scaleFactor)
                        if screenWidth - viewportWidth < 10:
                            viewportWidth = screenWidth
                    
                    viewportHeight = gameHeight * intScaleFactor
                    newCfgFile.write("# To avoid horizontal rainbow artefacts, use integer scaling for the height\n")

                viewportX = int((screenWidth - viewportWidth) / 2)
                viewportY = int((screenHeight - viewportHeight) / 2)                
                
                newCfgFile.write("aspect_ratio_index = \"22\"\n")
                newCfgFile.write("custom_viewport_width = \"{}\"\n".format(viewportWidth))
                newCfgFile.write("custom_viewport_height = \"{}\"\n".format(viewportHeight))
                newCfgFile.write("custom_viewport_x = \"{}\"\n".format(viewportX))
                newCfgFile.write("custom_viewport_y = \"{}\"\n".format(viewportY))
                
                outputLogFile.write("{},{},{},{},{},{},{},{},{},{}\n".format(gameInfo[0],gameInfo[1],gameInfo[2],gameInfo[3],gameInfo[9],gameInfo[10],viewportWidth,viewportHeight,viewportX,viewportY))

        newCfgFile.close()

    resolutionDbFile.close()
    if not curvature:
        outputLogFile.close()
    print("]\n")
    print("Done!")


def createZip(curvature=False, screenWidth=0, screenHeight=0):
    if curvature:
        outputFileName = "crt-pi-curvature_configs"
        path = "curvature"
    else:
        resolution = str(screenWidth) + "x" + str(screenHeight)
        outputFileName = "crt-pi_configs_" + resolution
        path = resolution
    outputFileName = outputFileName.replace(" ", "")
    outputFileName = outputFileName.lower()

    print("Creating zipfile {}".format(outputFileName))
    shutil.make_archive(outputFileName, "zip", path)

    # now delete config dirs
    print("Deleting temp directory: {}".format(path))
    shutil.rmtree(path)


if __name__ == "__main__":
    if sys.version_info[0] < 3:
        print ("\n\n*** This script must be run using Python 3 ***\n\n")
        exit()
    if len(sys.argv) < 5:
        adjust = ""
    else:
        adjust = sys.argv[4]
    generateConfigs(sys.argv[1], sys.argv[2], sys.argv[3], adjust)

oh, nice! unfortunately i was already fixing it myself!

as for the rest, well i'm afraid that i'm not too interested in new features - like i said, this script had a goal and i think i reached that. i don't want to maintain a bunch of extra stuff i didn't design.

i would have been happy to consider any fixes/improvements to current master if you submit via git (not here), so i can see the diffs and provide feedback, but generally each PR should be one change, not a whole re-write like this. but yeah i am not sold on new features. sorry!

Riverstorm

@andrewh said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

I've done a (fair) bit more work and created an alternative version of your script - was wondering whether you'd consider including it in your git project.

I think considering Dank's script complete is respectful as to keep it to one task it was meant to do and it does it well. It's what I currently use.

I would definitely be interested in using/testing the script with the expanded feature set. It would also be handy & welcome to have that additional information in the config header for quick reference if troubleshooting an odd resolution, PAR or what not. Do you have a Github account? It might be worth considering a new project or at least a new thread to start out so as to not muddy up this one with an alternate script.

I don't know if it's how you explained it or it's what I learned first but I follow the logic a little easier. I think a bit part is breaking down the scaling calculations from the original & also factoring the pixelSquareness.

AndrewH

@dankcushions said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

as for the rest, well i'm afraid that i'm not too interested in new features - like i said, this script had a goal and i think i reached that. i don't want to maintain a bunch of extra stuff i didn't design.

No worries - it's your project after all :-)

It's been an interesting journey figuring out what the deal was with the 1600 x 1200 output, and it seems to have spurred some interest from a few other members too, which is nice.
If I do spin off a separate project, are you ok with me using the resolution_db and re-used parts of your script?

@riverstorm said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

It might be worth considering a new project or at least a new thread to start out so as to not muddy up this one with an alternate script.

I've got a git account, but have absolutely no idea how it's all meant to work. But if I get time over the next week or two, I'll try to put something together, and rework it to be distinct enough not to cause confusion (hopefully).

dankcushions

@andrewh i wouldn't really like another project re-writing my idea, no ;)

but if you get git and do each feature/improvement as separate PRs to my repo i'm sure i can be convinced :) at least that way you can get credit for the bits you add, rather than me just copy/pasting code. let's collaborate!

git is a pain at first but there's lots of great guides out there, like http://rogerdudler.github.io/git-guide/.

AndrewH

@dankcushions alright then...

I'm not sure my scaling method can easily exist alongside yours in the same script, but I'd be happy to commit a couple of the other changes in the hope that they're useful.

Plus, if anyone really wants to give my script a go, it's in this thread..

Riverstorm

@andrewh said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

Plus, if anyone really wants to give my script a go, it's in this thread..

I grabbed a copy at work tonight to test next week. I was hoping you didn't decide to remove the post. We are heading out of town in the morning so the wife can take a medical class in another city while I go out sight seeing.

Honestly I almost look at it as a fork. Development is complete and it was clearly stated several times no further development would be done (unless others are doing the changes through PR's).

You rewrote the code making some fundamental computational changes, added features and corrections. It also gives you a chance to further develop it exactly like you want. It's not much different then MAME or any other open source software fork. The data is freely available as it's all data the MAME team compiled through the years. MAME is open source and can be forked anyway you can imagine and has been. This is the whole point. Creative freedom.

You rewrote it because you were reminded the project scope is complete and take it like it is or make the changes yourself if you don't like it and well you did! Any changes are coming from you anyway on a completed project. It just doesn't make sense to not setup your own project now and run with it. I think you have to give yourself the opportunity.

I'm not trying to be negative but I think Andrew already implemented some nice features and I'd love to see where he goes with it. I think you would be doing yourself a disservice to not at least try.

Eventually it will boil down to maintaining two scripts your own and adding features to an existing as your reply seems to indicate. I have no idea but I would take that bet your going to maintain your own script anyway and that would be a shame to be pressured into hiding it and sending small PR changes when you really feel like developing it your own way anyway.

dankcushions

@riverstorm you're injecting yourself into this conversation and i don't appreciate it.

open source development isn't just about forking whenever you want. you can do that, of course, but it's the nuclear option. i am happy to collaborate!

i changed my mind about introducing new features - i made the comment flippantly after being overwhelmed with the waffle in this thread. this script was a simple, fun thing for me to attempt some (terrible) python, that is definitely not the last word in scaling (actually i think the last word would be introducing better options in retroarch's aspect ratio settings... that's on my list). however i can't just copy and paste a complete re-write of my code - features have to be submitted as single PRs so i can understand them and evaluate them individually. that's the same for ALL open source projects.

in that context i would be likely to accept re-writes of the old scaling algorithm and all the rest - just one at a time :)

i'm going to lock this thread for a while because there's nothing more to say - @AndrewH, please don't take this to be anything about your work which is cool and I appreciate it! i'll see you on git! i'll unlock when we have something to show.

Riverstorm

@dankcusions - yes, mame uses the exact aspect ratio every time, because it doesn't care about integer scaling. what mame does is say, "ok, this is a 4:3 game, so I will do a 4:3 image - i will simply stretch the tiny 320x240 (say) image to the biggest 4:3 image that fits on the screen"

@AndrewH - It's not that you're not explaining yourself well it's my lack of understanding. I have little loose ends that probably aren't related but I feel it helps in understanding but also keeps me going in circles. I just need time to process the information. Let that subconscious do the work. :) I was up way to late reading. One thing drifted to another.

With that thought (Dank's comment above) it would be safe to say 100% of arcade games (due to lack of PAR calculations in upscaling) are stretched on x, y or both and we've just become accustomed or consider it correct because it's close enough for our memories?

If not integer scaling what is the MAME/Retroarch(?) process to stretch the image resolution to max 4:3 aspect ratio on modern displays (that have different resolutions)?

With integer scaling off and no custom config. How is MAME (set to defaults) filling the screen? This seems important to know what it does with no modifications. Basically what the developers decided for default settings.

I think of integer scaling as multiplying the original resolution by whole numbers (no fractions) until either the x or y exceeds the display resolution then it backs down 1 to fit the screen. Hence almost always leaving black borders all the way around the play area.

I know more pixels allow the shader to look better. Is it pixel density or an evenly divisible x scale (y on horizontal games) that is more important to looking better?

An observation is with Dank's script I didn't think he was taking pixel squareness into account and Andrew's does. What I find interesting is when using Andrews's "wide" flag both Dank's and Andrew's scripts produce the same viewport dimensions. How is that possible? (If Dank's isn't doing pixel squareness calculations). I just don't quite understand the sort array in Dank's script so maybe there's more happening there than I understand?

I was on one thread last night that was having Donkey Kong discussion that one guy said it was actually a 9:6, another guy said it was a 8:7, a 3rd guy says if you fill the screen horizontally (with h-sync?) it's 8:7 or if adjusted down a bit it's 9:6 so it can be either depending on the arcade operator settings, then a 4th guy says it's what you're used to and by changing what you're used to it can take hours to re-acclimate with the springs (elevators at higher levels) when changing the aspect ratio to something you're not used to playing on.

Others were deep into 31 to 15Khz conversions from hardware to software, display cards, etc. to get proper looking games. It kind of goes on and on.

I look forward to how the scripts turn out.

Yardley

Just found these cfg's, thanks for posting them! I placed them in the correct directory and they work fine. I also added a line of code to specify a bezel. I was curious to see what would happen if I made any changes through the RGUI so I changed the shader to the curvature variant for the game sf2 and then did a save game override. I then opened sf2.cfg in notepad++ and noticed the code wasn't right and my line specifying the bezel was gone. Any idea why this happens? I was expecting to see the change to the curvature shader with nothing else changed.

dankcushions

@riverstorm said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

@dankcusions - yes, mame uses the exact aspect ratio every time, because it doesn't care about integer scaling. what mame does is say, "ok, this is a 4:3 game, so I will do a 4:3 image - i will simply stretch the tiny 320x240 (say) image to the biggest 4:3 image that fits on the screen"

With that thought (Dank's comment above) it would be safe to say 100% of arcade games (due to lack of PAR calculations in upscaling) are stretched on x, y or both and we've just become accustomed or consider it correct because it's close enough for our memories?

PAR isn't neccesarily correct to the original, though. DAR is correct, but to get from original game image to display via DAR, you have to stretch.

if you run our example 320x224 image on a 1080p display, by default it will be 1440x1080 (4:3). 320 isn't an integer factor of 1440, nor is 224 an integer factor of 1080, so if you zoomed in you would see that (for example) the grid of 320x224 pixels from the original game are blurred slightly during the scaling process. you don't get clean edges. however, the aspect ratio of the image will be correct (4:3).

the blurring ("scaling artefacts") aren't usually a problem on a HD screen viewed at distance, but they are a problem if you use a scanline shader.

that's what the script tries to solve - it compromises on aspect ratio but reduces scaling artefacts. if you want to have a precise aspect ratio, don't use these .cfgs. don't do anything!

If not integer scaling what is the MAME/Retroarch(?) process to stretch the image resolution to max 4:3 aspect ratio on modern displays (that have different resolutions)?

it's bilinear, i believe.

I think of integer scaling as multiplying the original resolution by whole numbers (no fractions) until either the x or y exceeds the display resolution then it backs down 1 to fit the screen. Hence almost always leaving black borders all the way around the play area.

actually retroarch doesn't quite do it like that. it is mindful of the intended aspect ratio, so it will scale by whole numbers to get at close to the target aspect ratio as possible. so it might end up with something like X axis * 5, Y axis * 4, or whatever.

I know more pixels allow the shader to look better. Is it pixel density or an evenly divisible x scale (y on horizontal games) that is more important to looking better?

well, ideally you'd have an evenly divisible scale, but since retro gaming has so many different resolutions, that's impossible for one digital display. however, once you start going into 4k, then the pixels are so small that the artefacts are essentially invisible.

An observation is with Dank's script I didn't think he was taking pixel squareness into account and Andrew's does.

i do take it into account. it's just a different approach.

I was on one thread last night that was having Donkey Kong discussion that one guy said it was actually a 9:6, another guy said it was a 8:7, a 3rd guy says if you fill the screen horizontally (with h-sync?) it's 8:7 or if adjusted down a bit it's 9:6 so it can be either depending on the arcade operator settings, then a 4th guy says it's what you're used to and by changing what you're used to it can take hours to re-acclimate with the springs (elevators at higher levels) when changing the aspect ratio to something you're not used to playing on.

Others were deep into 31 to 15Khz conversions from hardware to software, display cards, etc. to get proper looking games. It kind of goes on and on.

i defer to what mame use in their drivers, here.

dankcushions

@yardley said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

Just found these cfg's, thanks for posting them! I placed them in the correct directory and they work fine. I also added a line of code to specify a bezel. I was curious to see what would happen if I made any changes through the RGUI so I changed the shader to the curvature variant for the game sf2 and then did a save game override. I then opened sf2.cfg in notepad++ and noticed the code wasn't right and my line specifying the bezel was gone. Any idea why this happens? I was expecting to see the change to the curvature shader with nothing else changed.

i already created curvature cfgs in the original post, so i would just use those as a base :) retroarch does strange things with regard to saving overrides, so i can't really guess the problem here.

Riverstorm

@dankcushions said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

PAR isn't neccesarily correct to the original, though. DAR is correct, but to get from original game image to display via DAR, you have to stretch.

I am not sure I am explaining myself correctly but this seems not quite what I was trying to say. If you have a 3:4 ratio and a resolution of 224 x 298 and another game that is a ratio of 3:4 and a resolution of 253 x 310. I thought the difference is the PAR not the DAR. They are both 3:4 DAR but the pixel width vs. height is what not the same between the two games.

The thought is most arcades games do not use square pixels. So when you display the original game on a modern display and force it to use square pixels it would be squished or stretched due forcing the pixels to be square.

it's bilinear, i believe.

I thought bilinear is a smoothing technique not scaling.

actually retroarch doesn't quite do it like that. it is mindful of the intended aspect ratio, so it will scale by whole numbers to get at close to the target aspect ratio as possible. so it might end up with something like X axis * 5, Y axis * 4, or whatever.

So integer scaling is per axis? If you have (x * 5 ) & (y * 4) wouldn't you destroy you 3:4 aspect ratio? I thought both axis where scaled by the same number and it has to be whole numbers or you will create an uneven stretch in pixel dimensions. To me doing it like that would be completely opposite of what it's trying to achieve?

i do take it into account. it's just a different approach.

Can you explain that a little more?

dankcushions

@riverstorm said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

@dankcushions said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

PAR isn't neccesarily correct to the original, though. DAR is correct, but to get from original game image to display via DAR, you have to stretch.

I am not sure I am explaining myself correctly but this seems not quite what I was trying to say. If you have a 3:4 ratio and a resolution of 224 x 298 and another game that is a ratio of 3:4 and a resolution of 253 x 310. I thought the difference is the PAR not the DAR. They are both 3:4 DAR but the pixel width vs. height is what not the same between the two games.

The thought is most arcades games do not use square pixels. So when you display the original game on a modern display and force it to use square pixels it would be squished or stretched due forcing the pixels to be square.

it’s stretched either way. either you’re stretching the raw image from its PAR (“square pixels”) to DAR (the correct aspect ratio), or you’re stretching away from the DAR to make your pixels square (PAR)

mame is DAR by default. the aspect ratio is right, but to get there it gets scaling artefacts.

it's bilinear, i believe.

I thought bilinear is a smoothing technique not scaling.

they’re the same thing. if you can’t apply an integer scale and still hit 4:3 with no (top and bottom) borders, then you have to use a scaling algorithm. bilinear or nearest neighbour are most common. even integer scaling is a scaling algorithm, i suppose - just a very ‘brute force’ one.

actually retroarch doesn't quite do it like that. it is mindful of the intended aspect ratio, so it will scale by whole numbers to get at close to the target aspect ratio as possible. so it might end up with something like X axis * 5, Y axis * 4, or whatever.

So integer scaling is per axis? If you have (x * 5 ) & (y * 4) wouldn't you destroy you 3:4 aspect ratio?

that depends on the raw image. street fighter 2 is a great example of this. if you scaled both axis by the same number, it will be the wrong aspect ratio (fat ryu).

I thought both axis where scaled by the same number and it has to be whole numbers or you will create an uneven stretch in pixel dimensions. To me doing it like that would be completely opposite of what it's trying to achieve?

not at all. again the raw pixel image does not have a direct connection to the aspect ratio on the CRT. CRTs don’t need square pixels. if you integer scale up from the raw pixel image you will often be off from the proper aspect ratio.

i do take it into account. it's just a different approach.

Can you explain that a little more?

it’s just fundamental. you must know the difference between your DAR and PAR to generate the CFGs

Yardley

@dankcushions thanks, I was using curvature just for testing. So I guess the verdict is RA is temperamental so don't try to save any new settings over your configs?

Someone over at the Libretro forum said that "shaders aren’t saved reliably in config overrides. There’s a separate per-core/per-game preset that you should use instead."

Trying to look at that although it sounds like I would have to make all the configs from scratch which defeats the purpose.

dankcushions

maybe it's best to describe this scaling stuff in pictures.

here's street fighter 2 - the raw 384x224 image sent from the game:

alt text

but wait! widescreen tvs weren't around in 1991. and how fat is guile now? BAD! the actual arcade cabinet looked like this:

alt text

a regular 4:3 tv. CRTs can receive images in a number of different horizontal resolutions, due to the way they work. rather than go into detail, let's just concentrate on the end result: SF2 is a 4:3 game.

so, to get the image displaying correctly on a 1080p HDTV, mame needs to stretch the original raw image vertically so it gets to 4:3, and also enlarge it ("upscale") so is 1080 pixels high, rather than 224.

we can work out what resolution the end image is going to be:

4/3 = 0.75
x/1080 = 0.75
x = 1440

so, mame knows our end image has to be 1440x1080 to fill the screen, but how does it get there? it uses a simple upscaling algorithm.you can repeat more or less what it does by just opening an image editor. eg, here's 'preview' on my mac:

0_1510178176748_Screen Shot 2017-11-08 at 21.55.59.png

notice i've unticked 'scale proportionally', because we know that the source image has the wrong proportions (it's not 4:3). i've also ticked 'resample image', because this will apply a (presumably bilinear) to obfuscate scaling artifacts - more on that after. here's what it produces:

alt text

great, 4:3! but if you look closely:

0_1510178668967_dca78989-6302-4781-9a94-6d235fa3ce5a-image.png

no pixel sharpness. actually mame's algorithm would be sharper than this, but you'll still get that blurring between pixel edges when you look closely, because the original image does not fit into the target resolution neatly (ie, by an integer scale).

what would it look like without resampling? here's a closeup of that output:

...well, dang - apple's default image editor doesn't let you turn off 'scale proportionally' when resampling is off - which makes sense i suppose. even nearest neighbour is a type of resampling.

hypothetically, nearest neighbour would have sharp pixel edges, but every so often you'd have a row of sf2 pixels that have a different width than the others. this looks especially obvious when the image is scrolling up and down. same story on the columns of pixels and horizontally scrolling images.

so, bilinear resampling is a good compromise for mame, but neither approach work very well when you're using a scanline shader, and that brings us to the start of this thread.

hopefully this makes a bit more sense!

Riverstorm

@dankcushions said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

it’s stretched either way. either you’re stretching the raw image from its PAR (“square pixels”) to DAR (the correct aspect ratio), or you’re stretching away from the DAR to make your pixels square (PAR)

I don't think you can stretch an image from it's PAR. Unless you just worded that incorrectly? The PAR is a static ratio of the of width in proportion to the height. You can with an old CRT by adjusting the v-sync/h-sync which does change the aspect ratio like they where discussing with Donkey Kong in a prior post.

I believe they get around those "oblong" 2:1-ish pixels by "concatenating pixels" on modern displays via doubling the x or y axis. I am sure there are more advanced techniques in displaying the DAR properly for old games.

not at all. again the raw pixel image does not have a direct connection to the aspect ratio on the CRT. CRTs don’t need square pixels. if you integer scale up from the raw pixel image you will often be off from the proper aspect ratio.

No, that's still not I am trying to say. I read that article too.

If I have three 16:9 monitors and display one is 1280×720, display two is 1600×900 and display three is 1920×1080 (these are real monitor examples). If I program a game that is 100 x 200 pixels using display one it's going to look different on display two and three keeping in mind they are all three 16:9. The difference? The pixel aspect ratio (PAR).

All 3 have different resolutions and pixel counts but maintain the 16:9 aspect ratio. The pixel size is changing. You can remove/add pixels all you want but to compensate and maintain the 16:9 AR the PAR needs to change. If you move from an old arcade monitor to a modern display it's going to display differently due to the PAR. That was the point I was trying to make.

If I create a monitor with a resolution of 1 pixel and the DAR/AR is 3:4. Then I decided to create one with 2 pixels but want to keep the 3:4 aspect ratio something needs to change. That would be the pixel aspect ratio again (PAR). Ratio is only a correlation between the width and height.

they’re the same thing. if you can’t apply an integer scale and still hit 4:3 with no (top and bottom) borders, then you have to use a scaling algorithm. bilinear or nearest neighbour are most common. even integer scaling is a scaling algorithm, i suppose - just a very ‘brute force’ one.

Ok it seems like you're doing quick reads & cutting and pasting? You're confusing 2 different things. It's a smoothing technique for textures when you scale up or down such as walking closer or further from an object in game. You can scale in either direction but it's a texture smoothing technique that is supposed to improve the image quality detail when scaling not for scaling. It's very common option in PC games when deciding if your PC has enough horse power to do bilinear, trilinear or anisotropic filtering. Each puts more demands on the ol' CPU cycles. I take it you're not a PC gamer or just use defaults.

it’s just fundamental. you must know the difference between your DAR and PAR to generate the CFGs

It was like I had originally written and I agree you are correct. It is fundamental when you understand it properly. I do appreciate your script and have fun. It's a neat project.

dankcushions

@riverstorm said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

@dankcushions said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

it’s stretched either way. either you’re stretching the raw image from its PAR (“square pixels”) to DAR (the correct aspect ratio), or you’re stretching away from the DAR to make your pixels square (PAR)

I don't think you can stretch an image from it's PAR. Unless you just worded that incorrectly? The PAR is a static ratio of the of width in proportion to the height. You can with an old CRT by adjusting the v-sync/h-sync which does change the aspect ratio like they where discussing with Donkey Kong in a prior post.

yes, the PAR is static. so to get a funky PAR image (eg sf2) to fit into a screen with 'square' pixels, we must stretch/distort that original image so it hits the right DAR.

I believe they get around those "oblong" 2:1-ish pixels by "concatenating pixels" on modern displays via doubling the x or y axis. I am sure there are more advanced techniques in displaying the DAR properly for old games.

who's "they"? mame? because that's not how it works in mame2003. they don't double axis, they just use an upscaling algorithm (like i describe in my next 'picture' post). doubling the x or y axis wouldn't give the proper DAR.

not at all. again the raw pixel image does not have a direct connection to the aspect ratio on the CRT. CRTs don’t need square pixels. if you integer scale up from the raw pixel image you will often be off from the proper aspect ratio.

No, that's still not I am trying to say. I read that article too.

If I have three 16:9 monitors and display one is 1280×720, display two is 1600×900 and display three is 1920×1080 (these are real monitor examples). If I program a game that is 100 x 200 pixels using display one it's going to look different on display two and three keeping in mind they are all three 16:9. The difference? The pixel aspect ratio (PAR).

yes - actually this is something neither script does. we don't take into account the aspect ratio of the target screen. we assume they have square pixels. for HDTVs this is almost always right. for PC monitors, yeah... it would be simple to add, though.

side note i wonder how PC games handle this kind of things? maybe pc monitors report their PAR through HDMI handshake. for all i know retroarch does this also, but that doesn't help our scripts. we'd have to get that info from the user.

they’re the same thing. if you can’t apply an integer scale and still hit 4:3 with no (top and bottom) borders, then you have to use a scaling algorithm. bilinear or nearest neighbour are most common. even integer scaling is a scaling algorithm, i suppose - just a very ‘brute force’ one.

Ok it seems like you're doing quick reads & cutting and pasting? You're confusing 2 different things. It's a smoothing technique for textures when you scale up or down such as walking closer or further from an object in game.

again, that's the same thing, more or less. if you show a thing of one resolution, into a thing with another, you have to resample (or upscale/downscale). to hide the artefacts of this, you use bilinear (or similar). if you like, you can consider the current frame of sf2 to be the 'texture', that you are standing very close to in a FPS game, to the extent that it is filling the screen.

You can scale in either direction but it's a texture smoothing technique that is supposed to improve the image quality detail when scaling not for scaling. It's very common option in PC games when deciding if your PC has enough horse power to do bilinear, trilinear or anisotropic filtering. Each puts more demands on the ol' CPU cycles. I take it your not a PC gamer?

*GPU cycles. you're confusing a lot of things here. just look at the code:
https://github.com/libretro/RetroArch/blob/b11620e1ebbc19937d675a48bdefc04c1b0b7421/libretro-common/gfx/scaler/scaler_filter.c#L202
retroarch's default upscaler with the GL driver is "point", but bilinear is right there as an alternative ('video smooth' option). point filtering is just a slightly less 'blurred' version. it's less expensive, also. there are lots of ways to skin this cat but bilinear is totally one of them.

AndrewH

@dankcushions said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

If I have three 16:9 monitors and display one is 1280×720, display two is 1600×900 and display three is 1920×1080 (these are real monitor examples). If I program a game that is 100 x 200 pixels using display one it's going to look different on display two and three keeping in mind they are all three 16:9. The difference? The pixel aspect ratio (PAR).

yes - actually this is something neither script does. we don't take into account the aspect ratio of the target screen. we assume they have square pixels. for HDTVs this is almost always right. for PC monitors, yeah... it would be simple to add, though.

For all of those displays, assuming they're advertised as 16:9, then the PAR should be the same for each (i.e. square).
1280 / 720 = 1.77777777777777
1600 / 900 = 1.77777777777777
1920 / 1080 = 1.77777777777777
and
16 / 9 = 1.77777777777777

The only thing that's changing is the size of the pixels to suit whatever the physical display size is. Similarly, for other displays;
1280 /1024 = 1.25
5/4 = 1.25

1600 / 1200 = 1.33333333
4 / 3 = 1.3333333333333

Overall, it's a very safe assumption that all current digital displays have square pixels.

Back to the earlier point - as Dank points out just above, it's possible to work out the degree of 'stretch' to apply in order to map from a display with one PAR to another. This is what I've done in the latest Pull Request that I submitted to Dank's script a few minutes ago, and interestingly, it gives results a little different, but more 'correct' than previous for some of these cases.

Dank's script currently produces the following output for Street Fighter 2 for a 1920 x 1080 display :

# Auto-generated crt-pi.glslp .cfg
# Game Title : sf2 , Width : 384, Height : 224, Aspect : 4:3
# Screen Width : 1920, Screen Height : 1080
# Place in /opt/retropie/configs/all/retroarch/config/MAME 2003/
video_shader_enable = "true"
video_shader = "/opt/retropie/configs/all/retroarch/shaders/crt-pi.glslp"
# To avoid horizontal rainbow artefacts, use integer scaling for the width
aspect_ratio_index = "22"
custom_viewport_width = "1536"
custom_viewport_height = "1080"
custom_viewport_x = "192"
custom_viewport_y = "0"

Here's the output using my method (again, SF2 on a 1920 x 1080 screen) :

# Auto-generated crt-pi.glslp .cfg
# Game Title : sf2 , Width : 298, Height : 224, Aspect : 4:3
# Screen Width : 1920, Screen Height : 1080
# Place in /opt/retropie/configs/all/retroarch/config/MAME 2003/
video_shader_enable = "true"
video_shader = "/opt/retropie/configs/all/retroarch/shaders/crt-pi.glslp"
aspect_ratio_index = "22"
custom_viewport_width = "1436"
custom_viewport_height = "1080"
custom_viewport_x = "242"
custom_viewport_y = "0"

You'll see that the viewport width in my output is closer to the 1440 manual calculation.
(I've just realised that I'm also showing the 'resampled' game width in the header - 298 vs. the original 384. I can fix this, although it's purely informational and has no bearing on the operation of the script)

Comparing across the wider range of games, I've managed to preserve a lot of the important stuff - on 1920 x 1080 displays, the vertical games should have the same width as using the original method, but for 1600 x 1200 displays they'll now be more proportional :-)
And in general, it looks like they should fill the screen a little better - I've re-purposed the 'tolerance' variable so that now, if the difference between the viewport width or height is less than that percentage of the original game width or height, we just stretch to fit. This means, that on a 1600 x 1200 display you get the following;

New method :

# Auto-generated crt-pi.glslp .cfg
# Game Title : sf2 , Width : 298, Height : 224, Aspect : 4:3
# Screen Width : 1600, Screen Height : 1200
# Place in /opt/retropie/configs/all/retroarch/config/MAME 2003/
video_shader_enable = "true"
video_shader = "/opt/retropie/configs/all/retroarch/shaders/crt-pi.glslp"
aspect_ratio_index = "22"
custom_viewport_width = "1600"
custom_viewport_height = "1200"
custom_viewport_x = "0"
custom_viewport_y = "0"

vs. old method :

# Auto-generated crt-pi.glslp .cfg
# Game Title : sf2 , Width : 384, Height : 224, Aspect : 4:3
# Screen Width : 1600, Screen Height : 1200
# Place in /opt/retropie/configs/all/retroarch/config/MAME 2003/
video_shader_enable = "true"
video_shader = "/opt/retropie/configs/all/retroarch/shaders/crt-pi.glslp"
# To avoid horizontal rainbow artefacts, use integer scaling for the width
aspect_ratio_index = "22"
custom_viewport_width = "1536"
custom_viewport_height = "1200"
custom_viewport_x = "32"
custom_viewport_y = "0"

caver01

@andrewh I may be missing the point reading along here, but I cannot help but notice that it looks like your new method the example doesn't integer scale the width. I thought that was the whole point: You integer scale the width so that you eliminate unsightly CRT-PI shader artifacts, then expand (or contract) the Y dimension to fit the display (whichever offers the least distortion). If you just ignore the integer scaling within a tolerance, why bother?