Please test: Using OMXPlayer as video renderer

[Nismo]

@fieldofcows +1 for vlc by default, theme developers can provide information if omx it's supported by the theme or if it requires vlc.

Or better suggestion, maybe adding a tag on main.xml ... Can we make ES autoselect default renderer based on theme?

<view name="video">
    <video name="md_video">
      <renderer>VLC</renderer>
    </video>
</view>

Of course we need also a tag on es_settings.cfg...

<bool name="OverrideThemeBasedRenderer" value="false" />

This one is for people who don't want the renderer be changed because of theme. Also maybe could be possible to add a ES menu entry for this one...

If doing the above commits we need to change this entry:

<bool name="VideoOmxPlayer" value="true" />

For this one:

<bool name="VideoPlayer" value="VLC" />

Where you can select VLC or OMX. This only will work in case you override the theme based renderer. If wrong value inserted here ES can override this entry with the default renderer (VLC).

What do you think?

[pjft]

@fieldofcows I wanted to have replied yesterday, but only had access to my phone and I didn't want to type it all there.

First of all, a big thank you for all the effort you're putting into this project. It has certainly helped revitalize ES, which was - up until a few months ago - a front-end that a lot of people in the RetroPie community were starting to give up on and even consider scrapping altogether in favor of others like AttractMode. You deserve a ton of credit for that, and I am positive that it can only get better from here!

A few questions you asked:

a) Maybe there isn't really a problem to fix?
b) Should people be fitting adequate cooling to their Pis if they run into over-temperature?
c) Leaving OMXplayer integration but set VLC to default?

So, in regards to a), I believe I was one of the first to escalate this issue. Truth be told, I've re-encoded most of my videos to 360p@30fps, and since then I've had no more problems. So I'm really involved in this at the moment because I feel it could be better for everyone, especially since with Kodi and other applications users are used to having videos of higher quality being rendered without overheating. That being said, it is perfectly functional as it stands, except for a few cases:

• 480p videos with official Pi case (or a case without a fan) will overheat. See here and here (even though @abodi claims these are the low quality from EmuMovies, which are 240p?).
• 480p videos at 60fps will have glitches when rendering.

This would not be a problem except that the EmuMovies HQ videos are 480p, and several are 60fps (I believe the SNES ones at least, and a few other systems).

So I can certainly envision some users attempting to get videos and running into these problems out of the box - as you can see even from the limited sample here in the forums. Certainly, people can re-encode videos or add cooling to it, but that's an additional step that not everyone may be equipped to do.

As for b), I've always seen mentions that the Raspberry Pi shouldn't need a heatsink or any cooling. After using mine for a while without any overclocking, I find that this is not always the case, but still I believe the recommendation stands in the sense that "it shouldn't be needed for the majority of situations". And certainly, if people can run videos of higher quality in other applications, there can certainly be the reasonable expectation that running videos in ES should not require (or have the recommendation) of adding cooling to the unit.

So, it all leads to c). I believe leaving the OMXplayer integration as it stands, as an option (you can mark it "experimental" in the option), provided that the video stops rendering when we open a menu (so that it doesn't overlay on top of menus and such), is probably a good compromise for the time being. Of course, all of this is assuming that @BuZz would be supportive of that, from the main ES branch perspective's, given that I expect that ultimately the goal is to have this merged back to it.

And, of course, assuming that it doesn't add a significant maintenance overhead for the project. If it does, I'm supportive of just halting this development at the moment and either try to integrate with HW accelerated VLC upstream, or wait for HW acceleration be the default in Raspbian, and then go from there. I certainly don't like to be defeated by a problem that feels solvable, but ultimately I'm not really the one facing these challenges hands-on, and it's comparatively easy to stand in the sidelines commenting, chiming in on things and supporting an idea.

As I said, I am currently fine with my particular setup, but would like to help solving the problem for all users - in particular new users out of the box. That is my key investment in this thread.

@Nismo : I think your ideas from a theme perspective have merit, and would provide some flexibility with the two players. From a long-term perspective, however, my personal take is that I fear they may add too many unnecessary changes and maintenance hooks for little benefit.

I believe we all agree at this stage that overlays are a key component of themes, and that the expectation is that in the future there will only be one single video player that has both good performance and allows for overlays. As such, temporarily changing the themes' schema just to accommodate a temporary situation with two players is likely bound to not add for a lot in the grand scheme of things.

If VLC is the default player, I expect that themes will work just fine for everyone, and if anyone needs to turn to OMXplayer, it'll be because they need the better performance/lower temperature, but accept the consequences of lack of overlays. But, truth be told, it might be that OMXplayer will not make it to the main branch as it stands at the moment - in that sense, @BuZz would be the main person to comment on what he'd consider the right take for the RetroPie ES project, given that ultimately he'll be the one approving any PR. Would having the two players as options be acceptable, even if it doesn't have overlays? Or is that something you'd not encourage in the RetroPie main ES branch?

My [extremely long] 2 cents here, and a big thank you to all of you for joining together and trying to tackle this challenge. I'd certainly love to hear about what the next steps are here.

Thanks.

[Nismo]

@pjft For videos without temp problems and glitches, I'm uploading optimized art and videos for Rpi here: https://retropie.org.uk/forum/topic/8019/oldroom-theme-w-i-p-media-packs?page=1

Videos are at max 360p 25fps, with very good compromise between quality and filesize. I don't have a rpi to test them, but I expect all will be ok with the videos now under VLC.

Sorry for the offtopic.

[pjft]

@Nismo Thank you sir! I had seen it earlier but I see you have added a few more now - I'll look into those and certainly update my OldRoom theme :)

[Nismo]

@pjft Yes I'm uploading art regularly, so keep tuned. Also a big update is coming for the theme, with some bugfixes and more systems.

Please if you try my videos, can you tell my if you have some temps or glitches problems?

Thanks.

[pjft]

@Nismo I certainly will. Truth be told, I haven't had much time with tinkering with my Pi recently, but I definitely want to do so. I will let you know how those go!

Thanks.

[MWGemini]

@pjft Awesome writeup, and I agree. Personally, temperature is not a concern for me, as I have a fan on my case, and I plan to add heatsinks in the near future (and then try overclocking). My main concern is can the videos all play without graphical artifacts. I'm a little OCD about things, so if given the choice, I would want the higher resolution, higher framerate videos in my personal collection.

@fieldofcows how difficult would it be to add a menu option (under UI options perhaps) for renderer switching? It would save me the trouble of having to exit ES and modify a config file every time I try to test, which saves me a ton of time, as ES takes about 8 minutes to load with my massive collection (we gotta get on that metadata fix... let's talk ;) ). It would also make it easy for non-technical users to switch, after they search for the "my videos look funky" problem (the new WSOD, since your fix eliminated that problem :) ) and discover that it's not really a bug in ES.

And to reiterate what @pjft said, thank you for all your hard work. As a developer, I know how painful, frustrating, and time consuming it can be to fix problems like this, and I know how frustrating it can be to feel "defeated" by what seems like it should be a simple fix. You've done awesome work, you've helped breathe new life into the community, IMHO, and I know I'm not the only one who recognizes that.

[fieldofcows]

Lots of things to reply to here but I'll keep this brief.

Firstly, thanks for all the recognition. However, I wouldn't be doing it if I didn't enjoy it so seeing people benefiting and enjoying the changes I make is a good reward.

With the video renderer selection - I can add a menu option to change the player. That's not a problem. I'll add that in and build another version for testing.

I think there is one more thing I would like to try before we put this to bed though. I'm going to see if the format conversion from the output of the codec to RGB in VLC is causing a significant CPU load (i.e. I'll comment it out and see if the temperatures stabilise) and if so I'll see if I can cobble together a shader to move the conversion into the GPU. That might just make enough difference to mean that the average pi doesn't need cooling.

@MWGemini - I've started looking again at the metadata changes. That is next in my queue.

@Nismo - thanks as always for your suggestions and support :)

[MWGemini]

@fieldofcows I'm happy to help with the development (and testing too of course) of the metadata changes- it's too big of a task to have to do it alone.

[Nismo]

@fieldofcows I have a little request for you "to do" list...

Option in ES UI to adjust the time to start scrolling description metadata. It can be values from 1 to 5, or values in milliseconds... actually it's very slow for my taste.

I have some ideas in my head, but I don't want to say anyone yet, you have enough work to do yet.

@MWGemini I really appreciate your interest in this project.

[pjft]

@fieldofcows Only today have I managed to try out your latest release, and it doesn't seem to show any video in OMX mode, only in VLC.

I tried building it with the latest code, and still the same - only now it has the video screensaver, which is great!

Maybe something went awry with the refactoring? Any way I can debug it? Your initial OMX-only build worked perfectly. I was looking at the code changes, but they're just too many for me to be able to thoroughly revert on my own :(

Hope you're all doing well.

[fieldofcows]

@pjft Where did you get 'the latest code'? I haven't had any time recently and haven't managed to look at this for a while. I use my dev branches in github as a convenient way to move my current working set between systems so you cannot rely on them to be working all the time.

I'm still not sure really where to go with this. I like the embedded omxplayer but having seen the (few) themes that are using video, they do mostly overlay images on top of the video. I don't want to break them.

I have been investigating shaders as I said in a previous post. However, ES uses OpenGL ES V1 so shaders aren't available without a bit of rework. I am playing around with this though just for fun.

[pjft]

@fieldofcows From your github video_process_dev branch - but even the one on the Releases, that's in the first topic of this thread, exhibits the same behavior.

It's fine, by all means - I was just going to test it out today as I had an hour or so.

I wasn't asking for any additional developments - even though, to be frank, your latest build with (theoretically) omxplayer and the screensaver would be ideal for me. Right now I'm running your latest VLC one with screensaver :)

Thanks anyway.

Do you have your new RPi3? :)

[fieldofcows]

@pjft said in Please test: Using OMXPlayer as video renderer:

Do you have your new RPi3? :)

Yep. Up and running again. And in a case this time :)

[pjft]

@fieldofcows Great to hear! ;)

[pjft]

@fieldofcows Well, after digging a bit into the code and troubleshooting things, it turns out it's because there's a particular video profile that OMXPlayer still does not support:

https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=142204

I noticed it when some videos actually worked where others didn't, and it stumped me.

I'm re-encoding my few videos that don't suit the profile, and should be good to go. I'm sticking to your latest OMX+Screensaver build for the time being :)

Thanks for everything here!

[pjft]

@fieldofcows I was going to submit a pull request for the OMXPlayer implementation, but in doing my changes I end up creating other problems. :)

Summary:

in VideoPlayerComponent.cpp , I had changed these two lines to the following:

const char* argv[] = { "", "--win", buf, "--layer", "10000", "--loop", "--no-osd", "-b", "--aspect-mode", "letterbox","", NULL };
argv[10] = mPlayingVideoPath.c_str();

It keeps the aspect ratio of the movies (noticeable for vertically-oriented videos) :) But, on the flipside, for the screensaver it doesn't hide the background (theme, screen, etc).

It was a brief attempt at helping out, but the new problem makes it worse than before - I'd rather have stretched video than screensavers that hide only part of the screen :)

Have a great weekend!

[fieldofcows]

@pjft I haven't tried this change to see what you mean about the screensaver but I'm guessing rendering a screen-sized black box behind the video in the screensaver will fix this.

[pjft]

@fieldofcows yes, that is correct, and it's what I ultimately am trying to do on my own, in a - what I imagine - very suboptimal manner :)

I'll share whatever I come up with. It's been a very long time since I've done any C++ programming, but this does not in any way strike me as a tremendous challenge. And I've been meaning to do something, however minor, for the project, so this sounds like an easy thing (famous last words).

I do have a very poor development workflow at the moment, though. I'm on a Mac, so I'm using Sublime (which is little more than a beefed up Notepad, kind of like the old UltraEdit) and then I scp my code to the Pi and attempt to compile there.

I could perhaps go back to Eclipse, which should be a lot better than what it was in 10 years ago, but would love to get some way to test the code consistency.

Any tips would be very much appreciated - even if just something that'd do some very basic error checking before I send it for compilation. Not syntax errors per se, but validating references, context, scope, etc.

Thanks!

[pjft]

@fieldofcows that's now sorted. :)

I'm actually adding a few extra tidbits here and there on my limited time, mostly quality of life changes, nothing really significant.

I'll send this one as a pull request for your review later, maybe in the coming days.

I'm still trying to get my head around GitHub. I was under the impression I had committed a couple of things so far but can't seem to find them on the web.

One thing at a time. :)

Thanks!