crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come)

[AndrewH]

In case anyone's still interested, here's a worked example for Street Fighter 2, which also has an odd PAR, as noted above:

From resolution_db

sf2,384,224,H,R,1440,1080,1196,896,4,3,60

pixelSquareness = (384 / 224) / (4 / 3)  = 1.285714   

gameWidth = int(384 / 1.285714) = 298    # This is the pixel width if we had a real 4:3 ratio.  You can also calculate it just by multiplying 224 by 4/3

vScaling = 1200/224 = 5.357                # How many times taller is the screen than the game (with corrected pixel size)?
hScaling = 1600/298 = 5.369                # How many times wider is the screen than the game?

intScaleFactor = 5                        # Take the integer value of the lower scale value

viewportWidth = int(298 * 5.369) = 1596   =~ 1600    # Round up to 1600, since it's less than 10 pixels smaller than the screen width
viewportHeight = 224 * 5 = 1120

viewportX = 0
viewportY = 40

In this example the pixelSquareness works out as the reciprocal of that for 1941. So the recalculated game width ends up the same as the recalculated game height for 1941.

[caver01]

@andrewh said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

apart from 1280 x 1024. That's a ratio of 5:4, but on a 4:3 screen.

Of course, this is what I run on my system! But I have not measured the physical dimensions. Is it 4:3? If so the pixels are wider than they are tall. Mine is a Sony, and I have seen specs like: "1280 x 1024 (5:4/Square Aspect Ratio)" which makes me think my display is not physically 4:3, but true to resolution with a 1:1 PAR.

[AndrewH]

@caver01 said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

@andrewh said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

apart from 1280 x 1024. That's a ratio of 5:4, but on a 4:3 screen.

Of course, this is what I run on my system! But I have not measured the physical dimensions. Is it 4:3? If so the pixels are wider than they are tall. Mine is a Sony, and I have seen specs like: "1280 x 1024 (5:4/Square Aspect Ratio)" which makes me think my display is not physically 4:3, but true to resolution with a 1:1 PAR.

You're right - it is a 5:4 screen, not 4:3 as I'd wrongly assumed. I've got a Dell 1907 in my bartop, which I've just measured as around 377 x 303mm, which also works out as 5:4.

So, thankfully, the pixels are square and no further messing about with figures is needed :-)

[Riverstorm]

Ok, I think I am square now! ;)

Can you explain one last thing. I don't quite understand the comment below. If it was a real 4:3 monitor wouldn't the dimensions be 224 x 384 scaled with whole numbers because you're actually running a real 4:3 monitor? Why the change from 384 to 298?

This is the pixel height if we had a real 4:3 ratio. Why 298?

Also any you guys explain what happens when not using custom configs? This seems like logical thing to do but I take it something different happens but still fills the screen. I've been using Dank's configs since I made the jump to mame2003.

[caver01]

@riverstorm said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

Also any you guys explain what happens when not using custom configs? This seems like logical thing to do but I take it something different happens but still fills the screen. I've been using Dank's configs since I made the jump to mame2003.

Scroll back up to the first message in this thread. Not using the configs with the CRT-PI shader and you can get rainbow lines because it is not integer scaling in the dimension it needs to in order to make the shader look nicer. It's not a bad problem, just distracting. I live with it actually, because I like fullscreen and curvature. I guess I don't notice it that much, but having the config option is really clever.

[AndrewH]

@riverstorm said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

Ok, I think I am square now! ;)

Can you explain one last thing. I don't quite understand the comment below. If it was a real 4:3 monitor wouldn't the dimensions be 224 x 384 scaled with whole numbers because you're actually running a real 4:3 monitor? Why the change from 384 to 298?

This is the pixel height if we had a real 4:3 ratio. Why 298?

It's because the pixels in the original game would not have been square. But on modern displays, the pixels are square. So if we don't account for the non-squareness of the pixels, we'll end up with a game that's stretched.

To test this, you can do the following;

Edit your 1941.cfg file and put in the following values;

custom_viewport_width = "448"
custom_viewport_height = "768"
custom_viewport_x = "0"
custom_viewport_y = "0"

Never mind that it's small and off-centre, this is just for testing purposes. You should see that the game appears vertically stretched when we do a simple 2x scaling on both axes. If instead you take the height as 298 rather than 384 and do the same 2x scaling, you'll see that it's correctly proportioned;

custom_viewport_width = "448"
custom_viewport_height = "596"
custom_viewport_x = "0"
custom_viewport_y = "0"

So, we're not really scaling the game vertically by 2x - we're scaling it vertically by 1.55 (384 x 1.55 = 596), and that corrects for the difference in 'squareness' of the pixels between the game and the modern screen.

[Riverstorm]

@andrewh said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

So, we're not really scaling the game vertically by 2x - we're scaling it vertically by 1.55 (384 x 1.55 = 596), and that corrects for the difference in 'squareness' of the pixels between the game and the modern screen.

Ah, nice ok, so the calculations are compensating for odd PARs. I didn't fully understand it but I think I'm on the train now. I appreciate you taking the time to explain that with the examples helped quite a lot. Kudo's to Dank for building the script. There's a bit of complexity there figuring all that out.

Not using the configs with the CRT-PI shader and you can get rainbow lines because it is not integer scaling in the dimension it needs to in order to make the shader look nicer.

So it basically it fills the screen disregarding any type of integer scaling which would makes sense for MAME. Being one of the most extensive and longest running projects on so many display types (traditional and modern) to not make assumptions on if to scale output and to what resolution.

There's a small difference. My script will always keep the viewport within the bounds of the display, whereas Dank's may in some circumstances allow it to overspill a little in order to use more screenspace.

Are you saying some games will be clipped (off the edge of the screen) vertically or horizontally?

[dankcushions]

my script will never overspill the edge of the screen. i thought about letting that happen to a certain tolerance but arcade games tend to have important stuff right to the edge of the image, since the cabinet CRT would be calibrated to each game, compared to console games which had to deal with varying consumer tv overscan capabilities.

[caver01]

@riverstorm

@AndrewH There's a small difference. My script will always keep the viewport within the bounds of the display, whereas Dank's may in some circumstances allow it to overspill a little in order to use more screenspace.

Are you saying some games will be clipped (off the edge of the screen) vertically or horizontally?

This confused me a bit too. I didn't think the original script from Dankcushions would actually crop the video, but that it still uses the display max to "squeeze" the non-integer dimension it back into view. This is no worse than being under by some pixels and stretching to the display limit, it is just the opposite distortion. The key is finding which one looks better: over/compressed or under/stretched.

It looked to me that one example could allow over and compressed because it fell within the tolerance, when an under/stretch was distorted less.

[caver01]

Seems like the best script would be one that measures the distortion with each integer scale and selects the one that distorts the least. No threshold necessary.

[dankcushions]

maybe it overspills the x axis with 4:3 displays since i never tested for that ;) with the widescreen hdtv i made it for it has plenty of room on the X axis!

[dankcushions]

@caver01 said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

Seems like the best script would be one that measures the distortion with each integer scale and selects the one that distorts the least. No threshold necessary.

i don’t agree. that’s how mine used to work but i changed it to pick the wider one (within a certain low tolerance) as it looks better to stretch than to squeeze on a widescreen display (up to a point, hence the tolerance variable). if you set tolerance to 0 then it won’t do that.

[caver01]

@dankcushions said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

i don’t agree. that’s how mine used to work but i changed it to pick the wider one (within a certain low tolerance) as it looks better to stretch than to squeeze on a widescreen display (up to a point, hence the tolerance variable). if you set tolerance to 0 then it won’t do that.

That does sound close to perfect then, as it takes into account some subjectivity while my idea (your original idea) is very rigid. However, the example above where the lower version is literally just 4 pixels smaller than perfect--it seems like it should have picked the lower, as it would be almost visually perfect, yet according to @AndrewH it picked the upper. Is the tolerance compared against the lower or the upper?

[dankcushions]

@caver01 said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

@dankcushions said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

i don’t agree. that’s how mine used to work but i changed it to pick the wider one (within a certain low tolerance) as it looks better to stretch than to squeeze on a widescreen display (up to a point, hence the tolerance variable). if you set tolerance to 0 then it won’t do that.

That does sound close to perfect then, as it takes into account some subjectivity while my idea (your original idea) is very rigid. However, the example above where the lower version is literally just 4 pixels smaller than perfect--it seems like it should have picked the lower, as it would be almost visually perfect, yet according to @AndrewH it picked the upper. Is the tolerance compared against the lower or the upper?

you'll have to show me the/an example in my own generated .cfgs from the original post. some of the examples above were generated via an older version of python, and/or a lower resolution 4:3 screen which my script needs a lowered tolerance value to work with. IMO you shouldn't bother with scanline shaders on sub-HD screens anyway!

some of the examples were generated with an edited version of the script!!

i have now definitely spent more time explaining this script than writing it ;)

[Riverstorm]

@dankcushions said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

i thought about letting that happen to a certain tolerance but arcade games tend to have important stuff right to the edge of the image, since the cabinet CRT would be calibrated to each game

That was my thought. There's nothing worse than playing a game and the top half of the score is "clipped".

i have now definitely spent more time explaining this script than writing it ;)

I have to say I've learned quite a bit from the whole discussion but continuing to beat on 1941. This statement below would actually be false and not true.

There's then a check to see whether the new scale is within the tolerance

>if ((widerAspect - aspectRatio)/aspectRatio * 100) <= tolerance:
    viewportWidth = int(gameWidth * (scaleX + 1))

Using the figures above :
(0.9333-0.7466)/0.7466 * 100 = 25
So, the horizontal scale value of 5 gives a result that's bang on the tolerance, and therefore used, even though this gives an aspect ratio that's much further from the original.

The actual value will be 25.006 (continuing on). That value would not be <= due to the fractional part so 896 should still be used?

Working with whole numbers 896 is exactly 25% less then 1120 which is exactly 224 pixels which is the original width of the game? Whether that's a coincidence or an actual percentage for tolerance I don't know.

[AndrewH]

@dankcushions said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

my script will never overspill the edge of the screen. i thought about letting that happen to a certain tolerance but arcade games tend to have important stuff right to the edge of the image, since the cabinet CRT would be calibrated to each game, compared to console games which had to deal with varying consumer tv overscan capabilities.

It does, I'm afraid - for non-widescreen aspect ratios (i.e. 4:3 and 5:4).

Below is a slightly modified version of your script - the only change is that it also creates a .csv file in the directory it is run in, which has the following columns;
Name,X,Y,Orientation,Aspect1,Aspect2,ViewportWidth,ViewportHeight,HorizontalOffset,VerticalOffset
All of the games, with their original and scaled sizes will be listed.

If it's run with the following command line, 361 of the 4646 games have a negative vertical offset - meaning they exceed the screen area.
python3 crt_pi_configs.py mame2003 1600 1200

Running for 1280 x 1024 gives 104 games with negative vertical offset.

Setting the tolerance to 0 doesn't reduce the number of games with negative offsets.

But running it for 1920 x 1080, 1280 x 720, or 1366 x 768 gives no games with negative vertical offset.

# creates cfg files for crt-pi
# params are:
# * core (eg mame2003 or fbalpha)
# * screen width (eg 1920) OR curvature
# * screen height (eg 1080)
# example usage:
# python crt_pi_configs.py mame2003 1920 1080
# python crt_pi_configs.py fbalpha 1920 1080
# python crt_pi_configs.py consoles 1920 1080
# python -c "import crt_pi_configs; crt_pi_configs.createZip(False,1920,1080)"

import sys
import os
import shutil


def generateConfigs(arg1, arg2, arg3):
    console = False
    if "mame2003" in arg1:
        fileName = "resolution_db/mame2003.txt"
        coreName = "MAME 2003"
    elif "fbalpha" in arg1:
        fileName = "resolution_db/fbalpha.txt"
        coreName = "FB Alpha"
    elif "consoles" in arg1:
        fileName = "resolution_db/consoles.txt"
        console = True

    if "curvature" in arg2:
        curvature = True
    else:
        curvature = False
        screenWidth = int(arg2)
        screenHeight = int(arg3)
        screenAspectRatio = screenWidth / screenHeight
        tolerance = 25
        resolution = str(screenWidth) + "x" + str(screenHeight)

    resolutionDbFile = open(fileName, "r" )
    print("opened file {}".format(fileName))
    
    outputFile = open(resolution + ".csv", "w")
    outputFile.write("Name,X,Y,Orientation,Aspect1,Aspect2,ViewportWidth,ViewportHeight,HorizontalOffset,VerticalOffset\n")

    for gameInfo in resolutionDbFile:
    	# strip line breaks
        gameInfo = gameInfo.rstrip()
        
        # parse info
        gameInfo = gameInfo.split(",")
        gameName = gameInfo[0]
        gameOrientation = gameInfo[3]
        gameWidth = int(gameInfo[1])
        gameHeight = int(gameInfo[2])
        aspectRatio = int(gameInfo[9]) / int(gameInfo[10])
        gameType = gameInfo[4]
        #integerWidth = int(gameInfo[7])
        #integerHeight = int(gameInfo[8])

        if console:
            coreName = gameName

        cfgFileName = gameName + ".cfg"

        # Create directory for cfgs, if it doesn"t already exist
        if curvature:
            path = "curvature" + "/" + coreName
        else:
            path = resolution + "/" + coreName
        if not os.path.isdir(path):
            os.makedirs (path)

        # create cfg file
        print("creating {}/{}".format(path,cfgFileName))
        newCfgFile = open(path + "/" + cfgFileName, "w")

        if "V" in gameType:
            # Vector games shouldn"t use shaders, so clear it out
            newCfgFile.write("# Auto-generated vector .cfg\n")
            newCfgFile.write("# Place in /opt/retropie/configs/all/retroarch/config/{}/\n".format(coreName))
            newCfgFile.write("video_shader_enable = \"false\"\n")

        else:
            if "V" in gameOrientation:
                if curvature:
                    shader = "crt-pi-curvature-vertical.glslp"
                else:
                    shader = "crt-pi-vertical.glslp"
                # flip vertical games
                gameWidth = int(gameInfo[2])
                gameHeight = int(gameInfo[1])

            elif "H" in gameOrientation:
                if curvature:
                    shader = "crt-pi-curvature.glslp"
                else:
                    shader = "crt-pi.glslp"

            newCfgFile.write("# Auto-generated {} .cfg\n".format(shader))
            newCfgFile.write("# Place in /opt/retropie/configs/all/retroarch/config/{}/\n".format(coreName))
            newCfgFile.write("video_shader_enable = \"true\"\n")
            newCfgFile.write("video_shader = \"/opt/retropie/configs/all/retroarch/shaders/{}\"\n".format(shader))

            if not curvature:
                # if not perfectly integer scaled, we will get scaling artefacts, so let"s fix that
                if screenAspectRatio > aspectRatio:
                    # games with an aspect ratio smaller than your screen should be scaled to fit vertically
                    newCfgFile.write("# To avoid horizontal rainbow artefacts, use integer scaling for the width\n")

                    # build list of potential aspect ratios with different integer scales
                    aspectRatios = []
                    for scaleX in range(1, 99):
                        aspectRatios.append((scaleX * gameWidth) / screenHeight)

                    # find closest integer scale to desired ratio
                    aspectRatios.reverse()
                    scaleX = 98-aspectRatios.index(min(aspectRatios, key=lambda x:abs(x-aspectRatio)))

                    viewportWidth = int(gameWidth * scaleX)
                    if console:
                        # consoles have overscan, so adjust viewportHeight to "Title Safe Area"
                        if "Nestopia" in coreName:
                            overscanV = 8
                        else:
                            overscanV = 0

                        # build list of potential aspect ratios with different integer scales
                        aspectRatios = []
                        for scaleY in range(1, 99):
                            aspectRatios.append(viewportWidth / (scaleY * gameHeight))

                        # find closest integer scale to desired ratio
                        aspectRatios.reverse()
                        scaleY = 98-aspectRatios.index(min(aspectRatios, key=lambda x:abs(x-aspectRatio)))

                        viewportHeight = screenHeight + (overscanV * scaleY)
                    else:
                        viewportHeight = screenHeight

                    # we prefer it to be wider than narrower, so do that, according to tolerance
                    newAspect = viewportWidth / viewportHeight
                    if newAspect < aspectRatio:
                        widerAspect = (gameWidth * (scaleX + 1)) / screenHeight
                        if ((widerAspect - aspectRatio)/aspectRatio * 100) <= tolerance:
                            viewportWidth = int(gameWidth * (scaleX + 1))

                    # centralise the image
                    viewportX = int((screenWidth - viewportWidth) / 2)
                    viewportY = int((screenHeight - viewportHeight) / 2)

                else:
                    # games with an aspect ratio larger than your screen should be scaled to fit horizontally
                    newCfgFile.write("# To avoid horizontal rainbow artefacts, use integer scaling for the height\n")
                    
                    # build list of potential aspect ratios with different integer scales
                    aspectRatios = []
                    for scaleX in range(1, 99):
                        aspectRatios.append(screenWidth / (scaleX * gameHeight))

                    # find closest integer scale to desired ratio
                    aspectRatios.reverse()
                    scaleY = 98-aspectRatios.index(min(aspectRatios, key=lambda x:abs(x-aspectRatio)))

                    viewportWidth = screenWidth
                    viewportHeight = int(gameHeight * scaleY)
                    
                    # centralise the image
                    viewportX = 0
                    viewportY = int((screenHeight - viewportHeight) / 2)

                newCfgFile.write("aspect_ratio_index = \"22\"\n")
                newCfgFile.write("custom_viewport_width = \"{}\"\n".format(viewportWidth))
                newCfgFile.write("custom_viewport_height = \"{}\"\n".format(viewportHeight))
                newCfgFile.write("custom_viewport_x = \"{}\"\n".format(viewportX))
                newCfgFile.write("custom_viewport_y = \"{}\"\n".format(viewportY))

                outputFile.write("{},{},{},{},{},{},{},{},{},{}\n".format(gameInfo[0],gameInfo[1],gameInfo[2],gameInfo[3],gameInfo[9],gameInfo[10],viewportWidth,viewportHeight,viewportX,viewportY))

        newCfgFile.close()

    resolutionDbFile.close()
    outputFile.close()

def createZip(curvature=False, screenWidth=0, screenHeight=0):
    if curvature:
        outputFileName = "crt-pi-curvature_configs"
        path = "curvature"
    else:
        resolution = str(screenWidth) + "x" + str(screenHeight)
        outputFileName = "crt-pi_configs_" + resolution
        path = resolution
    outputFileName = outputFileName.replace(" ", "")
    outputFileName = outputFileName.lower()

    print("Creating zipfile {}".format(outputFileName))
    shutil.make_archive(outputFileName, "zip", path)

    # now delete config dirs
    print("Deleting temp directory: {}".format(path))
    shutil.rmtree(path)


if __name__ == "__main__":
    generateConfigs(sys.argv[1], sys.argv[2], sys.argv[3])

[AndrewH]

Using the figures above :
(0.9333-0.7466)/0.7466 * 100 = 25
So, the horizontal scale value of 5 gives a result that's bang on the tolerance, and therefore used, even though this gives an aspect ratio that's much further from the original.

The actual value will be 25.006 (continuing on). That value would not be <= due to the fractional part so 896 should still be used?

There's some rounding going on with those figures - they're there for display purposes only :-)

Working with whole numbers 896 is exactly 25% less then 1120 which is exactly 224 pixels which is the original width of the game? Whether that's a coincidence or an actual percentage for tolerance I don't know.

That's the crux of the problem - the aspect ratio using a width of 896 is slightly smaller than the original, and by coincidence the next value up is 25% bigger, which is bang on the tolerance that's set in the script by default, so it gets used even though it's quite far from the original aspect ratio.

[Riverstorm]

@andrewh said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

There's some rounding going on with those figures - they're there for display purposes only :-)

That was kind of the point. By rounding it changed the statement from false to true? 25.006 is not <= 25 so the check is actually false and it would use the proper value of 896 which disregards the point for that example.

[dankcushions]

@andrewh

But running it for 1920 x 1080, 1280 x 720, or 1366 x 768 gives no games with negative vertical offset.

right - i didn’t test or design it for non-widescreen displays. if anyone wants to fix that please send a PR on git. i had a limited goal for the script and i reached that, but if anyone wants to add anything to it, that’s cool!

i don’t even use shaders anymore!

[Riverstorm]

@andrewh said in crt-pi shader users - reduce scaling artifacts with these configs in lr-mame2003, lr-fbalpha, lr-nestopia (and more to come):

That's the crux of the problem - the aspect ratio using a width of 896 is slightly smaller than the original, and by coincidence the next value up is 25% bigger, which is bang on the tolerance that's set in the script by default, so it gets used even though it's quite far from the original aspect ratio.

Setting the tolerance to 0 doesn't reduce the number of games with negative offsets.

Can you explain the issue a little more? If tolerance of 25 or 0 yields the same results. It doesn't sound like the tolerance check is what's failing?