maxSize Video only working in 1 direction (aspect)

[TMNTturtlguy]

I have been working on converting my Video Themes to the main branch of ES with the updated maxSize feature for videos. My original builds were on @fieldofcows /@pjft ES build with OMX player. The OMX player holds the videos original aspect ratio by default, so maxSize is not necessary. Because of this I know that i have mostly 4:3 videos, with a few 16:9 videos and several Vertical aspect ration videos like the 1942 arcade games and Nintendo DS games.

When working in the main branch I have found that max size will hold the 16:9 videos from stretching in the up down direction and keep their original aspect ratio bound to the larger width size I enter. However I have found that maxSize will not hold the aspect ratio of my vertical video like 1942. The vertical aspects always stretch. Has anyone else seen this happening? I know i am entering maxSize correctly, because when I only enter size, my 16:9 videos stretch, when I enter maxSize the 16:9 videos are correct. If anyone else has seen this please let me know, and if anyone else has vertical videos working correctly, please also let me know what you have done. Thanks!

[jdrassa]

@TMNTturtlguy I just retested this and maxSize is working properly for me with vertical videos.

[TMNTturtlguy]

@jdrassa can you post your video view text from your .xml? I can't figure out what I am doing wrong that it will correct my 16:9 as it should, but not the vertical. I am really stumped.

Thanks

[pjft]

@TMNTturtlguy while I had thought that vertical videos did look slightly thinner than expected, an easy video for me to replicate it in is Alien vs Predator. It looks fairly stretched to 16:9 when using max size, even though playing the video in a standalone video player suggests it's closer to 4:3.

@jdrassa Happy to take some screenshots if it helps, and/or take a look. Also happy to share the video in case it's related to the video itself, though it happened in more than one.

[TMNTturtlguy]

@pjft I will be on to test in about an hour and I will post some photos of what i find while testing as well. Thanks everyone for working on this!

[FlyingTomahawk]

All good here. Vertical or horizontal all works as i should.

[pjft]

@TMNTturtlguy just confirming: you are testing this on the main branch, correct?

Maybe upload one of the videos as well, for testing. I found it to be slightly orientation agnostic, but definitely something weird happening with max size on VLC. It could be related to the video as well, namely not reporting the right dimensions. But I'll check in the coming days.

Edit: obviously, could also be related to your theme, though I doubt that as max size should preserve the aspect ratio :)

[TMNTturtlguy]

@pjft I will test it on the main branch and then I will install your updated ES-SS-OMX and test it there as well. We will see what the night brings, but i hope to post my 4:3 version of the theme as well as an updated main branch 16:9 tonight, then everyone can test to see if it is my theme, or something else going on.

[pjft]

@TMNTturtlguy thanks.

Had a quick look at the code and from reading it it all seems to add up and to work as expected. I'll add a few log lines to it and set what may be happening there, and where the problem may be arising from.

https://github.com/RetroPie/EmulationStation/blob/master/es-core/src/components/VideoComponent.cpp

(See the setResize method)

[jdrassa]

@TMNTturtlguy said in maxSize Video only working in 1 direction (aspect):

@jdrassa can you post your video view text from your .xml? I can't figure out what I am doing wrong that it will correct my 16:9 as it should, but not the vertical. I am really stumped.

Thanks

I tested with the latest version of carbon.

[jdrassa]

@pjft Thanks for looking it over. The logic was mostly carried over directly from the resize method for images.

[TMNTturtlguy]

@pjft @jdrassa I have tested on both the main ES branch and the updated screensaver ES build.

@pjft - the changes you made worked on the screensaver build, the OMX player shows up exactly as it should, the VLC player shows correctly, except my vertical videos are still being stretched.

On the main ES i have the exact same results, my verticals are being stretched with maxSize. Here are some photos:

What it should look like (taken from the OMX player) Vertical Video

Here is what it looks like in Main Branch with <size>

Here is what it looks like in Main Branch with <maxSize>

But now here is a 16:9 aspect ratio video - OMX Player - correct

Main Branch with <size>

Main Branch with <maxSize> This one shows up correct! Why does this one show correctly but not vertical?

@jdrassa i copied the video lines out of the carbon .xml and pasted it into my theme, same results as shown. Also i get the same stretching in the carbon video theme. Could it be an issue with my videos?

Unfortunately I am having issues with my pi after switching between ES builds. Last night I was able to go to @pjft new build, the main branch and back to the old screensaver build with no issues. Everything worked perfectly and infact, my theme worked perfectly on the main ES with the vram set to 100. Today when i updated to the main branch, i started getting glget errors and white screens. I got the same on the new Screensaver build, and now when i go back to the original screensaver build, i get the same issues....Not sure what is happening. I restored by SD card from an image with the old screensaver build, worked perfectly. Updated ES - same issues.

@pjft Have you tested my comicbook theme on your updated ES build? Does it run ok? Any getGL errors?

I am now completely updating my entire system to see if that helps. Not sure why it worked last night, but today it breaks it? I don't want to post my theme in case it is something I did to it that is causing the problems. Hopefully the update will be the cure for everything!

[pjft]

@TMNTturtlguy Oh. That's certainly not how I observe it on my end. maxSize works as expected on my vertical videos - they just look slightly narrower.

Your theme works just fine on my end, no glGet errors. Have you tried to go back to your previous version of the theme to see what might be the cause? I do get some slight white screens every now and then, but it's just a second or so until the textures load. If you share the exact details of the error, maybe we can better identify the issue.

As for my situation in particular, I now understand what's happening, and technically, all implementations are "correct", even though they aren't exactly accurate :)

What's happening is that when we're rendering the videos to a texture, we're assuming the pixel aspect to be 1:1, and we create a texture of that size. However, the truth is that several games have different pixel aspect rations that - thankfully - are captured in the video metadata.

Case in point, for my Alien vs Predator video:

Width                                    : 384 pixels
Height                                   : 224 pixels
Display aspect ratio                     : 4:3
Original display aspect ratio            : 4:3

When we render it pixel by pixel in the texture, it will look stretched horizontally, as the pixel ratio is actually 1.714:1, even though the display aspect ratio is actually 4:3 (or 1.33:1).

When we render it in a video player, natively (either in the PC, or just letting OMX Player handling the rendering without forcing it to stretch) it will use the video's expected aspect ratio and preserve it, unlike our pixel-to-texture-size assumption.

@jdrassa : Now, we could perhaps base our maxSize assumptions on one (or two) of the following three instead:

• libvlc_video_get(mMediaPlayer)
• tracks[track]->video->i_sar_num and/or tracks[track]->video->i_sar_den

I haven't tested to know what the output of those are, but they might help solve the aspect ration issues on maxSize via VLC rendering.

Whether these are the same problems @TMNTturtlguy is experiencing, I don't know - can you share your 1943 video? That does look weird. The Altered Beast one seems to be similar to the case I'm describing, but without knowing the exact metadata of the video I'm just guessing.

[TMNTturtlguy]

@pjft I think I figured out what is happening to my theme. I completely did an update all and used my original posted theme that I know was working pefectly. At first it was going white and the only errors are glget errors. I tried upping my VRAM but it got worse, interestingly enough when I lower my vram to 60, it works super fast without any white flashing! So more investigation needed there. I also noticed that on the main ES build, my video and meta data are no longer showing up on games within folders....we worked on that issue a few months ago and both the screensaver and main branch had been updated to allow this to work. Again, more testing on this. A bunch of unexplained changes all happening at once.

I will try to post my videos of 1942 and altered beast, maybe to a Dropbox link?

More to come later. Thanks for all your support and help

[pjft]

@TMNTturtlguy Hm. Interesting.

I believe your findings on VRAM are accurate, as I've expressed several times without any particular empirical confirmation.

That being said, you are correct: the metadata changes for folders aren't yet approved to be merged on the main ES - I imagine they're waiting on someone to review them.

So that will only work on my fork, for the time being, until it's approved there.

Glad to hear you're getting things back to a workable place.

As for your videos, sure, a dropbox link would work!

Thanks.

[TMNTturtlguy]

@pjft I again tested max size settings in the updated main branch ES with no luck. Here is a link to videos on a dropbox folder. Let me know if this works. I have placed the 1942 video, the altered beast video and a nintendo DS video which is also a vertical format that stretches the same as the 1942 video.
Edit - Deleted Link - Thanks for trouble shooting @pjft!

[pjft]

@TMNTturtlguy Thanks.

Yes, for some reason, these videos are reporting a resolution that doesn't match the aspect ratio. This is what I get by running MediaInfo on them.

The correct aspect ratio, observed by a media player, is listed. The comments on actual computed ratio is just my note, from dividing width / height. For some reason those two vertical videos of yours are substantially wider than what I expected (as you see, they are more wide than they are tall).

That is what VLC will likely be using, as it tries to create a rendering surface proportional to the size being reported in the dimensions.

Did you convert/process them yourself using any particular tool?

I find Handbrake to be a good tool. If you want to fix your video collection, just run them all via Handbrake, and set "Anamorphic" to OFF. This should fix help match their pixels to the aspect ratio.

Or use OMX Player as an alternative, for the time being.

Altered Beast:

Width                                    : 480 pixels
Height                                   : 224 pixels
Display aspect ratio                     : 1.85:1
// [Actual computed ratio from pixels]   : 2.143:1 

1942: 

Width                                    : 480 pixels
Height                                   : 360 pixels
Display aspect ratio                     : 0.749
Original display aspect ratio            : 0.750
// [Actual computed ratio from pixels]   : 1.33:1 / 4:3

Atari:

Width                                    : 480 pixels
Height                                   : 346 pixels
Display aspect ratio                     : 0.680
// [Actual computed ratio from pixels]   : 1.387:1

[TMNTturtlguy]

@pjft You are my HERO! I did use handbrake on my videos to decrease their file size. I did this at your recommendation prior to the OMX player. I just left them all as smaller size because they play just fine at that size, the small amount of quality loss is ok with me. I will try to re-run the vertical videos as you have suggested later on today. Unfortunately i have work meetings all day :(

A side note. I have your newest build up and running beautifully at VRAM 60 - half of what i had it on with the older build....such weird things with VRAM! I noticed that with the OMX player and the updates you made, is the theme just says <size> it OMX will now stretch to the size, and if I have <maxSize> OMX will hold its aspect ratio correctly. So per your findings above, the OMX player will see the aspect correctly, VLC will not. In your prior build with OMX, the OMX player always played the correct aspect, but i suppose that was prior to <maxSize> being available.

Lastly, yes, my games within folders works perfectly in your Fork as well! Thanks to you, I have this really awesome arcade cabinet build that is beyond my wildest dreams when I started. I truly appreciate all your support and hard work on here, you make this community and retropie an even more amazing place.

I will post my updated theme with more systems added when I return home again. I already have ideas for my next theme using your build and carousel!

[pjft]

@TMNTturtlguy Thanks. Yes, likely some setting in Handbrake must have "messed" them up in the conversion (i.e. changed them in a way that, for implementation reasons, VLC doesn't handle properly in the current setup).

Run all videos, as even the horizontal one might have the wrong ratio. Just drop the folder in Handbrake, set Anamorphic to "OFF" and run it for 1 video. Then open it in VLC on your computer, or Media Info and see if the new reported resolution is more reasonable. :)

Yes, I now force the videos to stretch if the theme explicitly uses size, and to keep aspect ratio if it uses maxSize. I assumed that that is how it's meant to be. I previously used to have a setting in the menu to force stretch videos in the theme view, but realistically it doesn't do anything right now - I need to remove it from the menu!

@jdrassa do you confirm that the semantics I'm assuming for maxSize keeping aspect ratio, and size forcing to stretch the video to the available video element size?

Cheers.

[TMNTturtlguy]

@jdrassa thanks for your support as well. I appreciate all you do and your help in trouble shooting!