crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come)

AndrewH

Some more testing - I ran the script on my other system (1280 x 1024)

1941 - Old cfg file, probably downloaded from this thread;

# Auto-generated crt-pi-vertical.glslp .cfg
# Place in /opt/retropie/configs/all/retroarch/config/MAME 2003/
video_shader_enable = "true"
video_shader = "/opt/retropie/configs/all/retroarch/shaders/crt-pi-vertical.glslp"
# To avoid horizontal rainbow artefacts, use integer scaling for the width
aspect_ratio_index = "22"
custom_viewport_width = "672"
custom_viewport_height = "1024"
custom_viewport_x = "304"
custom_viewport_y = "0"

1941 - newly generated cfg file - looks too wide;

# Auto-generated crt-pi-vertical.glslp .cfg
# Place in /opt/retropie/configs/all/retroarch/config/MAME 2003/
video_shader_enable = "true"
video_shader = "/opt/retropie/configs/all/retroarch/shaders/crt-pi-vertical.glslp"
# To avoid horizontal rainbow artefacts, use integer scaling for the width
aspect_ratio_index = "22"
custom_viewport_width = "896"
custom_viewport_height = "1024"
custom_viewport_x = "192"
custom_viewport_y = "0"

dankcushions

i should probably have something in the script that checks for python version, sorry.

@andrewh said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

But for vertical games, some still appear wider than they should - almost square;

bear in mind that the script uses an integer scale on the X axis for vertical games to reduce the scanline scaling artifacts. in almost all cases they will be wider or narrower than before - that's just how it solves the problem.

so for example for bomb jack, it has a native X resolution of 224, and a native aspect ratio of 3:4. the script will pick the closest it can get to 3:4, whilst maintaining the integer scale, and using the full height of the screen. in your case, that's 1120(5x 224) x 1200.

896 (4x) x 1200 resolution will be further from 3:4 than what the script is generating.

AndrewH

1120/1200 = 0.9333
896/1200 = 0.7466
3/4 = 0.75

So, 896 should be the correct horizontal scaling, right?

dankcushions

@andrewh said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

1120/1200 = 0.9333
896/1200 = 0.7466
3/4 = 0.75

So, 896 should be the correct horizontal scaling, right?

yep, is this another python version thing again? are you running this on windows (i created it on windows and the IO might not be right on other OSs)? i dunno, i created a bunch of cfgs for common resolutions and they were all correctly calculated. you'd have to debug the script.

AndrewH

@dankcushions No - running it on the Pi itself.

AndrewH

Something I've noticed with a couple of the problematic games is that they don't have square pixels.
Take 1941 for example - its resolution is 384 x 224, which is a ratio of 5.1:3, so it would be 'squashed' vertically if displayed on a 4:3 ratio monitor.

Also, I think it should be possible to take a different approach than what you've done in order to compute optimum scaling factors. Here's what I've done in pseudocode, and then a hacked version of your script;

Calculate 'squareness factor' of pixels in original resolution : (width/height)/aspect ratio
- If it's a horizontal format game, scale the width by 1/squareness factor, leave the height as-is
- If it's a vertical format game, multiply the height by the squareness factor, leave the width as-is
Compute the horizontal & vertical scale factor between the game and the screen
- Use whichever works out lower (i.e. for a vertical format game, it'll always be height, but for horizontal format it could be either - or they could be the same)
For vertical games, multiply the game width by the integer value of the scaling factor (which will be the first whole number below the actual scaling factor. The height will be equal to the screen height.
For horizontal games, multiply the game height by the integer value of the scaling factor, and multiply the width by the actual scaling factor. If the viewport width is close enough to the screen width, just set it to the screen width.

As I'm not too familiar with Python, I just chopped out the sections of your script that weren't directly related to this, so the following will create files for MAME & FB Alpha, but not with curvature, and no zipping. It runs and produces output that matches what I calculated manually in Excel for a variety of resolutions / games.

If anyone wants to check it out, I'd be interested to hear what you think.

crt_pi_configs_AH.py

# creates cfg files for crt-pi
# params are:
# * core (eg mame2003 or fbalpha)
# * screen width (eg 1920) 
# * screen height (eg 1080)
# example usage:
# python crt_pi_configs_AH.py mame2003 1920 1080
# python crt_pi_configs_AH.py fbalpha 1920 1080

import sys
import os
import shutil

def generateConfigs(arg1, arg2, arg3):
    if "mame2003" in arg1:
        fileName = "resolution_db/mame2003.txt"
        coreName = "MAME 2003"
    elif "fbalpha" in arg1:
        fileName = "resolution_db/fbalpha.txt"
        coreName = "FB Alpha"

    screenWidth = int(arg2)
    screenHeight = int(arg3)
    resolution = str(screenWidth) + "x" + str(screenHeight)

    resolutionDbFile = open(fileName, "r" )
    print("opened file {}".format(fileName))

    for gameInfo in resolutionDbFile:
    	# strip line breaks
        gameInfo = gameInfo.rstrip()
        
        # parse info
        gameInfo = gameInfo.split(",")
        gameName = gameInfo[0]
        gameOrientation = gameInfo[3]
        gameWidth = int(gameInfo[1])
        gameHeight = int(gameInfo[2])
        aspectRatio = int(gameInfo[9]) / int(gameInfo[10])
        gameType = gameInfo[4]
        #integerWidth = int(gameInfo[7])
        #integerHeight = int(gameInfo[8])

        cfgFileName = gameName + ".cfg"

        # Create directory for cfgs, if it doesn"t already exist
        path = resolution + "_AH/" + coreName
        if not os.path.isdir(path):
            os.makedirs (path)

        # create cfg file
        print("creating {}/{}".format(path,cfgFileName))
        newCfgFile = open(path + "/" + cfgFileName, "w")
        newCfgFile.write("# {}\n".format(gameInfo))

        if "V" in gameType:
            # Vector games shouldn"t use shaders, so clear it out
            newCfgFile.write("# Auto-generated vector .cfg\n")
            newCfgFile.write("# Place in /opt/retropie/configs/all/retroarch/config/{}/\n".format(coreName))
            newCfgFile.write("video_shader_enable = \"false\"\n")

        else:
            if "V" in gameOrientation:
                shader = "crt-pi-vertical.glslp"
                # flip vertical games
                gameWidth = int(gameInfo[2])
                gameHeight = int(gameInfo[1])
                # Calculate pixel 'squareness' and adjust gameHeight figure to make them square (keeping Width as-was to avoid scaling artifacts)
                pixelSquareness = ((gameWidth/gameHeight)/aspectRatio)
                gameHeight = int(gameHeight * pixelSquareness)
                
            elif "H" in gameOrientation:
                shader = "crt-pi.glslp"
                # Calculate pixel 'squareness' and adjust gameWidth figure to make them square (keeping Height as-was)
                pixelSquareness = ((gameWidth/gameHeight)/aspectRatio)
                gameWidth = int(gameWidth / pixelSquareness)

            newCfgFile.write("# Auto-generated {} .cfg\n".format(shader))
            newCfgFile.write("# Place in /opt/retropie/configs/all/retroarch/config/{}/\n".format(coreName))
            newCfgFile.write("video_shader_enable = \"true\"\n")
            newCfgFile.write("video_shader = \"/opt/retropie/configs/all/retroarch/shaders/{}\"\n".format(shader))

            # Check scale factor in horizontal and vertical directions
            vScaling = screenHeight/gameHeight
            hScaling = screenWidth/gameWidth
            	
            # Keep whichever scaling factor is smaller.  Also get the integer of that scaling factor.
            if vScaling < hScaling:
                intScaleFactor = int(vScaling)
                scaleFactor = vScaling
            else:
                intScaleFactor = int(hScaling)
                scaleFactor = hScaling

            # For vertical format games, width multiplies by an integer scale factor, height can multiply by the actual scale factor.
            # For horizontal games height multiplies by the integer scale factor, and width multiplies by the actual scale factor, 
            # but gets rounded up to the screen width if close enough.
            if "V" in gameOrientation:
                viewportWidth = gameWidth * intScaleFactor
                viewportHeight = screenHeight
                newCfgFile.write("# To avoid horizontal rainbow artefacts, use integer scaling for the width\n")

            else:
                viewportWidth = int(gameWidth * scaleFactor)
                if screenWidth - viewportWidth < 10:
                    viewportWidth = screenWidth
                    
                viewportHeight = gameHeight * intScaleFactor
                newCfgFile.write("# To avoid horizontal rainbow artefacts, use integer scaling for the height\n")

            viewportX = int((screenWidth - viewportWidth) / 2)
            viewportY = int((screenHeight - viewportHeight) / 2)
            	

            newCfgFile.write("aspect_ratio_index = \"22\"\n")
            newCfgFile.write("custom_viewport_width = \"{}\"\n".format(viewportWidth))
            newCfgFile.write("custom_viewport_height = \"{}\"\n".format(viewportHeight))
            newCfgFile.write("custom_viewport_x = \"{}\"\n".format(viewportX))
            newCfgFile.write("custom_viewport_y = \"{}\"\n".format(viewportY))

        newCfgFile.close()

    resolutionDbFile.close()

if __name__ == "__main__":
    generateConfigs(sys.argv[1], sys.argv[2], sys.argv[3])

dankcushions

@andrewh said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

Something I've noticed with a couple of the problematic games is that they don't have square pixels.
Take 1941 for example - its resolution is 384 x 224, which is a ratio of 5.1:3, so it would be 'squashed' vertically if displayed on a 4:3 ratio monitor.

yes many arcade games don't have square pixels. a CRT has no 'resolution' as such. for the script, we don't care about the original ratio of the the pixels, only that of the original CRT (which is invariable 3:4 or 4:3). that's the ratio we need to get as close to as possible. if you displayed in the ratio of the pixels the image would be distorted.

like i said before, there's no problem with the script here. if you look at my generated zip files, i don't believe they will miscalculate anything. for example, 1080p cfg for bombjack:

# Auto-generated crt-pi-vertical.glslp .cfg
# Place in /opt/retropie/configs/all/retroarch/config/MAME 2003/
video_shader_enable = "true"
video_shader = "/opt/retropie/configs/all/retroarch/shaders/crt-pi-vertical.glslp"
# To avoid horizontal rainbow artefacts, use integer scaling for the width
aspect_ratio_index = "22"
custom_viewport_width = "896"
custom_viewport_height = "1080"
custom_viewport_x = "512"
custom_viewport_y = "0"

896/1080 = .8296
the next x scale down would be:
672/1080 = .6222

so it's picked the closest one to .75. can you find issues with my posted .cfgs?

btw, the script has a mild tolerance towards wider than narrower for vertical games. you can adjust that (tolerance var). maybe if you're using lower resolutions you should adjust it.

AndrewH

After a bunch more testing, I think I can summarize the problem as follows;

For a screen resolution of 1600 x 1200, the tolerance in your script needs to be dropped below 25, otherwise some vertical games will use a horizontal scaling that's too large.

Using 1941 for example, the optimal horizontal scaling should be a factor of 4, giving a viewport width of 896.

The screen vertical is stretched to 1200, so the aspect ratio becomes 896:1200, which is 0.7466.
Since this is slightly less than the original aspect ratio of 0.75, the script checks the next integer scale value.

Scale value 4 : 896/1200 = 0.7466
Scale value 5 : 1120/1200 = 0.9333

There's then a check to see whether the new scale is within the tolerance

if ((widerAspect - aspectRatio)/aspectRatio * 100) <= tolerance:
    viewportWidth = int(gameWidth * (scaleX + 1))

Using the figures above :
(0.9333-0.7466)/0.7466 * 100 = 25
So, the horizontal scale value of 5 gives a result that's bang on the tolerance, and therefore used, even though this gives an aspect ratio that's much further from the original.

Setting the tolerance to something less than 25 will yield more correct results.
It appears that this only affects 1600 x 1200 (that I've found so far), presumably because this resolution is actually a ratio of 4:3, whereas none of the others are.

Riverstorm

@andrewh said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

Setting the tolerance to something less than 25 will yield more correct results.
It appears that this only affects 1600 x 1200 (that I've found so far), presumably because this resolution is actually a ratio of 4:3, whereas none of the others are.

If we are using anything other than a 4:3 like regular ol' HD (1920x1080) do we gain any additional "more correct results" with your script?

Does bumping below a tolerance of 25 for 4:3 displays generates the same results?

Did you stumble upon the one exception being a 4:3 ratio? It sounds like you tested some other ratios but the results are the same?

AndrewH

@riverstorm : Based on the testing I've done, when run according to instructions (correct Python version, specifically), @dankcushions's script at the top of this thread works very well. It seems that only in the very specific circumstances of having a screen resolution that's exactly 4:3, there's the need to set the script's tolerance below 25. For all other resolutions that I'd tested, it works with no modification.

The script I posted is a different way of doing things, and it should produce 'reasonable' results, but should probably only be considered a dead-end in my efforts to troubleshoot the problem I had (vertical games being displayed too wide).

Riverstorm

@andrewh said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

The script I posted is a different way of doing things, and it should produce 'reasonable' results, but should probably only be considered a dead-end in my efforts to troubleshoot the problem I had (vertical games being displayed too wide).

Just to clarify now that you got to the bottom of the issue. Using Dank's script with a tolerance below 25 yields the same results as your script or if the results are different and you do end up using a 4:3 display your script might come in handy.

I don't know if I am over simplifying but I keep thinking modern displays have pretty much a square pixel aspect ratio so really you can only calculate using the DAR and discard the PAR information which will always produce some squishing in one of the dimensions?

Unless your emulator could emulate "PAR pixels" errrr....or something like that. :)

AndrewH

@riverstorm said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

Just to clarify now that you got to the bottom of the issue. Using Dank's script with a tolerance below 25 yields the same results as your script or if the results are different and you do end up using a 4:3 display your script might come in handy.

There's a small difference. My script will always keep the viewport within the bounds of the display, whereas Dank's may in some circumstances allow it to overspill a little in order to use more screenspace.

I don't know if I am over simplifying but I keep thinking modern displays have pretty much a square pixel aspect ratio so really you can only calculate using the DAR and discard the PAR information which will always produce some squishing in one of the dimensions?

For most displays, this is true. For a 1280 x 1024 display that's nominally 4:3, it's not though.

According to Dank, his script does effectively use the DAR, meaning that there shouldn't be squishing. Or to put it a different way, the scaling will be different for each axis as needed to ensure the aspect ratio as displayed matches the game aspect ratio as it would have been displayed.

There are a couple of tests you could do if you want to validate that things are working as expected;

Check in the resolution_db that the script uses to see what the game resolution and aspect ratio are (pick a game or two that you play)
Check the game's <name>.cfg file to see what the viewport height and width are set to, and check that they give an aspect ratio close to expected. You can work out the whole number multiples of the game resolution to see whether the next value up or down would give a ratio closer to the game's original.
You can also measure the game's aspect ratio on your screen (using a ruler or measuring-tape) to validate that it's suitably proportional.

dankcushions

the CPS2 fighting games are good examples of this. something like street fighter 2 has a resolution of 384x224, which is a wide image - close to 16:9. however it was on a regular 4:3 CRT just like most other arcade games, so should be 4:3 via the script/emulators.

Riverstorm

@andrewh said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

Take 1941 for example - its resolution is 384 x 224, which is a ratio of 5.1:3.

How are you calculating the ratio 5.1:3?

Here's the DB entry:

1941, 384, 224,V,R, 810, 1080, 576, 768, 3, 4, 60

Working through your example 1941. 384x224 with a ratio 5.1:3 on a display 1600x1200. Did I miss something or does that look correct? I am just working through an example more for better understanding.

pixelSquareness = (384 / 224) / 1.7 = 1.00
gameHeight = 224 * 1.00  =  224

vScaling = 5.36
hScaling = 4.16

intScaleFactor = 4

viewportWidth = 896
viewportHeight = 1200

viewportx = 704
viewporty = 0

caver01

@riverstorm I am just following along here, and I think your example looks good to me. For this one, you found the integer scaling of 4 will get you to 896 for the width. At a vertical of 1200, perfect 3:4 would have been 900, so fudging it just four pixels puts this game at a perfect integer in the X dimension. This is about as good as you can get. In other words, you have maximized the use of the display vertically, while preserving integer scale horizontally, all while trying to approximate a 3:4 AR for a game whose PAR is not 1:1. However, wouldn't the viewportx be 352? I thought you would take the display width minus the game width, and split that in half to center the viewport.

caver01

This game is harder on a 1280x1024 display, since a factor of 4 would be squished too much to fit into 1024 (about 17% distortion), and a 3 factor would stretch too high. It becomes somewhat subjective. In this case, the game would be stretched 128 vertical pixels beyond 3:4. That's 14% vertical distortion. This game is unfortunate because it falls very far from ideal either way. Obviously a 3 factor is better, but it might look too narrow. Still, it might be better than 64 pixel black bars on the top and bottom.

Sometimes, we have to take a subjective look and decide if it makes sense to preserve the look of a shader at the expense of a wonky AR. How much final distortion can you endure to preserve x-axis integer scaling before you simply add some black bars to the bottom and top? I wonder how many of us cared when the CRT was not tuned properly to fill the width and height? I expect we all played the arcade games when this was true and didn't even realize the AR was off!

Riverstorm

@caver01 said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

However, wouldn't the viewportx be 352? I thought you would take the display width minus the game width, and split that in half to center the viewport.

Ooops....yes I missed the /2! :)

Do you know how the 5.1:3 ratio is derived from the data above for 1941? That's the one piece I am missing.

This game is unfortunate because it falls very far from ideal either way.

Wow, almost smack dab in the middle. That's where you start expanding the script to make exceptions for thresholds beyond a certain percentage or pixel count. Hmmm...black borders or strreeetch!

caver01

@riverstorm said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

Do you know how the 5.1:3 ratio is derived

I think it is an approximation to simplify the ratio to get 1.7 like you noted. 384/224 is about 1.7, and 5.1/3 is exactly 1.7.

Riverstorm

@caver01 said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

I think it is an approximation to simplify the ratio to get 1.7 like you noted. 384/224 is about 1.7, and 5.1/3 is exactly 1.7.

Ok, that makes sense. Even rounded out it still carries a zero out to the thousandths.

How does Dank's example work when it's using the same width x height. Is this a really "skewed" PAR to pull off a 16:9?

something like street fighter 2 has a resolution of 384x224, which is a wide image - close to 16:9. however it was on a regular 4:3 CRT just like most other arcade games, so should be 4:3 via the script/emulators.

caver01

@riverstorm said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

Wow, almost smack dab in the middle. That's where you start expanding the script to make exceptions for thresholds beyond a certain percentage or pixel count. Hmmm...black borders or strreeetch!

Actually, the subjective decision is, black borders, stretch one way, stretch the other way, or simply live with a little bit of rainbow lines.

For me, I might live with some distortion, as I hate the rainbow lines, but I also like curvature which has its own problems with moire patterns and a lot of this careful scaling goes out the window.

Most vertical games for my setup end up in tate mode for me which introduces the same problems, just a different target display. It all becomes subjective on my box.