Please test: Using OMXPlayer as video renderer

MWGemini

@pjft With git, you have to do a commit (to the local repository) and then a push to get it to the remote repository. If you're used to things like CVS, SVN, or Perforce, it's a bit of a different workflow. This will be useful to you: https://git-scm.com/book/en/v2 .

pjft

@MWGemini Thanks! I have something sorted out now. :)

https://github.com/pjft/EmulationStation/tree/omx-fade-letterbox

I have submitted a PR to @fieldofcows - a very simple one, really - and am now looking into a few fun things. Will share if/when they become stable, in my limited time. Still, all credit to @fieldofcows 's work, as this is really all him. I'm just tweaking and adapting.

Would still like to have a proper development workflow on my Mac that'd allow me to quickly test the code and compile it. Right now I zip the code, copy it to the Pi, unzip, compile, and debug a lot via logs. Like the good old days of printf() debugging :P

Should have something interesting to share in the coming days, fingers crossed.

MWGemini

@pjft I can't help with the development workflow on a Mac, since I'm not a Mac user, and haven't ben able to find time in my schedule to actually do any development on ES. I keep hoping work will slack off a bit, but that never seems to happen...

fieldofcows

@pjft said in Please test: Using OMXPlayer as video renderer:

Would still like to have a proper development workflow on my Mac that'd allow me to quickly test the code and compile it. Right now I zip the code, copy it to the Pi, unzip, compile, and debug a lot via logs. Like the good old days of printf() debugging :P

How about installing VirtualBox on your Mac then creating an Ubuntu VM? You should then be able to build ES for Linux.

pjft

@fieldofcows That is actually a great suggestion. I tried QEmu with Raspbian, but that led me down a rabbit hole of which I couldn't come out of. I'll check out VirtualBox with Ubuntu.

I'm playing a bit with a few things here and there. Will share shortly when I feel things are minimally decent/presentable.

backstander

@pjft
I believe QEmu doesn't emulate the PRi GPU hardware and EmulationStation needs that to load:
https://retropie.org.uk/forum/topic/448/help-retropie-virtual-emulation-with-qemu/7

pjft

@backstander thanks for the heads up, I wasn't aware of that. I got in trouble even just compiling all things. I based it on a Wheezy image as I struggled with getting Jessie to work, but then even after getting all the libraries installed, from Boost to FreeType and in between, it seems to struggle with VLC. I suspect the required version for the current build isn't available for Wheezy. Which is fine.

I'll see if in Ubuntu the experience is smoother.

Thanks!

pjft

Yay! Release day, finally after doing this on a few minutes a day.

Very minor release for the moment, but thought I'd share. Also, my first release and overall experience with GitHub, so I'm quite happy. Most of the changes are minor/quality of life, but should help people on the RPi and PC have better temperatures while also getting screensavers and the few updates (@Dominus, @abodi, this will help your temperature issues based on @fieldofcows 's changes).

Just to make it clear: the extreme majority of this code is from @fieldofcows - I just added the following changes.

A few things in the release:

Keep aspect ratio for videos in OMXPlayer. Vertical games are no longer stretched. Haven't fixed it for VLC yet, but may look into it in the future.
Added game name and system on screensaver (@mattrixk , @CourierSS , @Dominus) . Very hacky, done via subtitles, but for the time being it's better than nothing (I hope). I will make these an option on the ES menu in the future. Please test on PC - the could should be fine, but VLC rendering performance on the Pi wasn't good in full screen resolution (the default video resolution made subtitles be very blocky, and stretched, and full screen just halted to a stop), which leads me to the next item.
Made OMXPlayer the default screensaver player for the RPi. No reason for that not to be the case, as there are no overlays for screensavers (yet, at least). There's still the option for having either VLC or OMXPlayer for the Video Gamelist View, of course, for overlay purposes. This way we can get a nice full-screen video scaling via the player rather than the texture stretching from VLC. PC should still be fully VLC-based, and have name and system as subtitle. If one day we get the game name as an overlay during the screensaver, this will be reverted - the code is all there, just commented out.
Pressing Select on the System Select screen will launch video screensaver. Why? Well... I have some plans that I'm sure @Zigurana will like :) I'm thinking of trying to use this as the random game selector, and if we exit the video screensaver with START, it will just select/launch that game. I will update if I end up succeeding, or just failing miserably. My several attempts at playing with the inputs so far resulted in the latter, so I'm not extremely hopeful, but I think it is a useful way to have a random game selector for the time being :P

I also did a few minor fixes on the screensaver via OMXPlayer, and when choosing a random video would select a path that does not exist (from @Nismo 's batch gamelist script, it happened on some BIOS roms in some folders for me, or ROMs I deleted but did not update the gamelist for).

To install this release, I'd follow the usual instructions from @fieldofcows :

wget https://github.com/pjft/EmulationStation/releases/download/v0.1-ss_title/emulationstation

Make executable and replace the original ES

chmod +x emulationstation
sudo cp emulationstation /opt/retropie/supplementary/emulationstation/

You may need to install dependencies, if they're not yet installed (I suspect they may be if you're on the latest RetroPie):

sudo apt-get install libvlc5 libvlccore8 vlc-nox

@fieldofcows if you feel any of these contributions are useful to your repository, I'm more than happy to send a pull request at this stage - I had sent out a smaller one earlier, but I certainly do not want to impose on your work. Thank you for taking ES this far!

Jolly good fun.

Nismo

@pjft I think that keep aspect ratio by default it's an error, what I mean it's that keep aspect ratio must be optional for the themer like we do actually with the images by using "size" or "maxSize" tags.

Example 1, not keeping the AR, just using a fixed size:

<video name="md_video">
			<pos>0 0</pos>
			<origin>0 0</origin>
			<size>0.35 0.40</size>
		</video>

Example 2, keeping AR, using maxSize instead:

<video name="md_video">
			<pos>0 0</pos>
			<origin>0 0</origin>
			<maxSize>0.30 0.40</maxSize>
		</video>

The problem it's maxsize it's not implemented for videos yet. Some themes like mine need a specific size for the video because it's "inside a box" and keeping AR will broken the theme or make it look "ugly".

Regards.

pjft

@Nismo thanks for the feedback! I believe the main system where this ends up being a problem is in the arcade, correct? Just out of curiosity, it doesn't change the fact that this is important for themes.

It's a fairly trivial thing to change on the player side - for OMX Player it's just a flag starting whether to stretch or letterbox it, though for VLC it'll be a bit more complex.

Given your previous feedback, though, I won't look into that on VLC for the time being, especially as your theme depends on VLC. When themes have that capability to distinguish max size vs size on video we can investigate that for both players.

The main reason I investigated it is because I think someone else had mentioned this earlier, and for the screensaver it made little sense to stretch the image unnecessarily as it ends up not looking like the actual game. :)

Is there any way I can provide you with a Windows build? I don't have a Windows machine to compile it on, but would love your feedback :/

jdrassa

@Nismo @pjft I have been working on adding maxSize support for video. I am planning on creating a pull request soon.

Nismo

@pjft Of course in arcades it's the main system with differents AR for games.

The main reason I investigated it is because I think someone else had mentioned this earlier

Yes it was mentioned before, and I know some people would love this, but i think that the right way to do this, it's by the same "size" and "maxSize" tags we are already using for images.

Another problem it's the not keeping the AR for images if you select to show image if no video in your theme, very noticiable if you use 3d boxes, that needs to be fixed:

<showSnapshotNoVideo>true</showSnapshotNoVideo>

Anyway, thank you very much to both @pjft and @jdrassa to help make this project better and better. Great work.

pjft

@jdrassa Thanks for the heads up.

I don't know where you forked it from (RetroPie vs fieldofcows) but anyway if you want to send it over to submit a PR for my repository as well, I'm more than happy to merge it so that it makes it easier in the end to converge everything.

Send it over to pjft/EmulationStation, the screensaver-control branch which is where I'm working on at the moment.

I'll try to merge things to my master soon, but can't promise.

Thanks!

Nismo

@pjft Oh! i'm not a programmer. I'm using the fieldofcows windows fork.

Thanks anyway.

Edit: would be nice to have a windows version from your fork to test.

pjft

Yeah. I don't have a windows machine to compile it to, but would love to send it your way.

If anyone uses windows and decides to build it from my repository, if you want to share the file it'd be perfect!

fieldofcows

@pjft - It looks like you've been doing some fantastic work here. I have had even less time than usual to devote to ES recently but I've been trying to keep up with what has been going on.

One thing that is worth noting is that the OMX player integration is still experimental. I'm not sure if that is the way forward for the retropie fork - do we really want the confusion of two different video renderers with different capabilities? I would much rather just pick one and stick with it and make it work the best we can.

I was all willing to go with embedded omxplayer as the only option for rpi but I think the overlay problem is a killer that is not fixable without some unacceptable hacks at the moment.

What does everyone else think? If the consensus is to keep the omxplayer work then I'm happy to collate the change/fixes into my dev branch before submitting a PR for consideration into retropie.

pjft

@fieldofcows Not at all, thanks for keeping up to date here.

My thoughts still stand as earlier - I agree that keeping two separate players makes no sense in the long run, particularly in the RetroPie main branch, and that it will be VLC. I don't have any expectation that this will ever be committed to the main branch - though I'd be more than happy to do whatever work is required to have it committed, should the project owners so decide to. The code I did works (hopefully) equally well on VLC and OMXPlayer (truth be told, the vast majority is yours - I just tweaked some minor things).

But truth is, the effort to have it running now is quite minimal, so that's what I've been doing, even if to get something running now for my own setup. If in 3-6 months the proper VLC HW support comes through, then surely we'll make that the main choice - but in the interim, everyone will be able to make the most of your work during these 90-180 days, and start developing their own things, be it their video setups or themes, while enjoying screensavers and video support in a way that doesn't fully tax the Pi processor. :)

I agree that overlays are critical for themers, and I don't like the idea of keeping both players simultaneously, so I'm fully behind VLC for the long run. But if this is the way it has to work now, I'm more than happy to live with that for the time being :) Who knows if in 3-6 months time you - or anyone - will have the time to make this work? Life takes several turns, so no point in delaying at least getting a minimum-viable-product out there for the community :)

Happy to add it to the experimental branch, if that would be the proper way to get it to reach more people until proper support comes.

Thanks!

fieldofcows

@pjft - That all makes sense, although if enough testing has gone into the omxplayer version then I'm coming around to the idea of keeping it and just using VLC by default. Tinkerers can then switch if they want to.

One of the things I've been playing with over the last couple of weeks which is just about working now is making a change to the VLC rendering pipeline to try to reduce CPU usage.

At the moment, everything is done in the CPU - the VLC library reads the video, prepares the frames and puts them through a colour conversion before calling a callback in ES which then puts this in an OpenGL texture. The colour conversion is to RGBA which actually seems particularly expensive in terms of CPU time and, in fact, in my testing it seems BGRA was less CPU intensive but I haven't proven that.

What I have been playing with though is getting VLC to skip the colour conversion and give ES the frames in their native format which is normally I420. I then put this straight into a texture and use an OpenGL ES 2 shader to perform the colour conversion in the GPU.

I got all this working on Linux but faltered when I tried it on the pi because ES is using OpenGL ES 1. So, of course for fun I went and switched all the ES code from fixed pipeline OpenGL to using shaders :)

This now works but I haven't performed any metrics on CPU performance, video playback performance or temperature yet. That is still todo.

Anyway I'll keep you all posted.

pjft

@fieldofcows I'm happy to run performance (i.e. temperature) tests on a proper Pi.

I can say that I have exclusively been using OMXPlayer, really - especially after long sessions using VLC, the temperature would always rise to a level that's higher than the ideal one. Also, the screensaver at full screen using VLC, being stretched via the texture, seems to look slightly off (but may be because of lack of aspect ratio, rather than anything else, as mathematically "it makes sense"). When I do stretch it via VLC, having VLC render the full screen resolution (in my case, 1080p), just renders a single frame or so and then halts, while the audio keeps playing. Can't blame it :)

If we're going in the VLC route then, why now attempt to use a HW accelerated VLC, even if we have to compile it ourselves? That could then be reused in the future, when the system gets its own version of HW-accelerated VLC, and it's easy to plug it into that, unless the versions are so different the API has changed significantly.

It could also serve to test properly whether HW-accelerated VLC would effectively improve the state of the nation.

What's the right step for us to do that? I imagine there'd be versions of it floating around - how do we link to it, rather than the current library?

Nismo

@pjft I vote for VLC as default renderer... people can use optimized videos for better temps.

@pjft I recommend to you to try my videos, re-encoded with rpi in mind, for better performance and temps: https://retropie.org.uk/forum/topic/8019/oldroom-theme-w-i-p-new-1-9-beta-media-packs?page=1

Of course I still need to upload a lot of systems , but i'm still working and uploading more optimized media for rpi (even when I don't have Rpi) just to contribute to the great work from @fieldofcows, @pjft, @jdrassa, @Zigurana and everybody who spend some of his free time to make this project going better.