[SCRIPT] RetroPie Convert Videos

Menion

The script should work with any vaapi host, but there is no detection of it, it is just hard coded

furnace

@hiulit
I've started to use your script and works like a charm. I've even modified it to support video scrapes attained by using Skyscraper on the retropie. I was just wondering though the following.

1- When the conversion is done, the converted videos are put into the converted directory, correct? can these then be copied back to the original directory to override the bad videos to reduce space usage?

2- Also if I use the --convert-system flag and let say I already converted a set of videos but then scrape new ones. Will re-running the conversion tool reconvert all the videos or just those that had not been done? If it's all the videos is there a why to just convert specific videos.

hiulit

@furnace Hey thanks! I'm glad you find it useful :D

It would be great if you can show me what modifications you've done, so maybe I can merge it with my version. Do you have a GitHub account to create a Pull request to my repository with you changes?

Now, I'm talking without looking at the code:

The script creates this converted folder so you can check if the videos are converted correctly or not. If not, they will a -failed suffix. The ones that are converted successfully are to be copied manually to the proper folder. I didn't code it in the script just in case anything goes wrong with the conversion.
I think it won't reconvert all the videos. Well, it depends, if you pass a from_color argument, the script will only search for the videos that have that particular "color encoding system". If you don't pass any from_color then yes, it will reconvert all the videos. I think.

In that latter case, I think I will try and add another option to the script to detect the "color encoding system" of the videos you want to encode.

furnace

@hiulit

Thanks for the feedback.

Sorry I don't have a github account. To modify the scripts though it was fairly simple. Steven Selph's Scraper will put the videos in a folder call video within the ROM folder. Skyscraper can be set to do the same, except the path will be $home/Retropie/roms/media/videos. So all I did was change the VIDEOS_DIR to the proper path.

Also your script looks for video file names that end in "-video.mp4", I replaced that to *.mp4 because the video file names are different between the two.

I had not set the from_color parameter before, so I'll test that out. Though I had tried to take a video that was not functional with the OMX enabled and pass it to AVprobe as per you script to see the encoding and it returned yuv420p. But once I convert them they are functional. I'm thinking the conversion might change something else which enables the functionality with OMX enabled. I'll keep investigating this and let you know.

hiulit

@furnace I can try to do something to "detect" the folders. And yeah, I think I can change the -video.mp4 for just .mp4. Thanks!

furnace

@hiulit
Or just as you force the customer to set the -to-color, you could just make the user to set a variable like
-source_scraper in the config to shelp or skyscraper and based on this set the video path. Cause I doubt someone would use various scrapers for different systems.

hiulit

@furnace Yeah, something like that could work. Thanks!

hiulit

@furnace Does this answer your question?

# From color (optional)
# Target only videos with this particular C.E.S.
# If left blank, it will target all videos, regardless of the C.E.S.
# (e.g. "yuv444p")

I haven't tried it yet, but looking at the code I found this.

hiulit

@furnace

~~This folder you mentioned $home/Retropie/roms/media/videos when using SkyScraper, does it contain folders for all the systems inside?~~

$home/Retropie/roms/media/videos/megadrive
$home/Retropie/roms/media/videos/nes

~~etc?~~

Forget that! I've found this table explaining where all the metadata and media are stored using different scrapers :)

https://retropie.org.uk/docs/Scraper/#where-are-my-scraped-media-and-metadata-saved

Is this information correct? (I'm guessing it is)

mitu

@hiulit said in [SCRIPT] RetroPie Convert Videos:

Is this information correct? (I'm guessing it is)

If you're using the scrapers through the RetroPie-Setup script, then it's correct.
Note that fi you're using the CLI parameters for folders or otherwise customize your configuration file, then the folders can be anywhere.

hiulit

@mitu Ok, thanks! I'll state that ROM folder for gamelists & media must be enabled. As well as I state the same thing when using Steven Selph's Scraper (use ROM folder for gamelists & images)

hiulit

@mitu Just out of curiosity, does the built-in EmulationStation scraper have the ability to download videos? It's been ages since I used it and maybe now it does. Thanks.

mitu

@hiulit No, it doesn't.

hiulit

@mitu Ok, thanks! Less things to take care then :P

hiulit

Making some progress on supporting both sselph and skyscraper :)

ERROR: There are no systems with videos!

You are using 'skyscraper' scraper.
Remember to use the 'ROM folder for gamelists & media' option.

Or maybe try using 'sselph'.

P.S. Which is better?

There are no systems with videos!
There aren't any systems with videos!

English is not my mother language :P

mitu

@hiulit Not an native English speaker also, but maybe start with videos:

No videos found for system's games/roms
No videos found in any system.

Here's an additional challenge - why look up the folder when you can read the video path from the gamelists ? This way you don't have to assume the usage of a particular scraper or a certain path.

hiulit

@mitu Thanks! I'll use your examples ;)

As for reading the path directly from the gamelist.xml, wouldn't I have the same problem? I mean, I have to know where the gamelist.xml is stored, and it can vary depending on what scraper is used and what options are set on where to store the gamelist.xml on that particular scraper, right?

ES will check three places for a gamelist.xml in the following order, using the first one it finds:

[SYSTEM_PATH]/gamelist.xml

~/.emulationstation/gamelists/[SYSTEM_NAME]/gamelist.xml

/etc/emulationstation/gamelists/[SYSTEM_NAME]/gamelist.xml

If you use another scraper (Steven Sselph's scraper or Lars Muldjord's `Skyscraper) you can choose whether your scraped data goes in

[SYSTEM_PATH]/gamelist.xml (the rom folder) or

~/.emulationstation/gamelists/[SYSTEM_NAME]/gamelist.xml

Taken from https://retropie.org.uk/docs/EmulationStation/#scraper.

If I have to know where the gamelist.xml is stored then I might just tell the user to use the option to store the media "where I want it" to be :P
Or "force" the user to have the gamelist.xml "where I want it" to be).

hiulit

Here it is! RetroPie Convert Videos v2.0.0

[2.0.0] - 2019-04-15

NOTE: This release contains breaking changes!

Deprecated

~~--convert-system~~ now is --convert-systems.
~~--from-color~~ now is --from-ces.
~~--to-color~~ now is --to-ces.

Added

Support for Lars Muldjord's Skyscraper.
New option --scraper to set the scraper. Available scrapers: sselph and skyscraper.
--convert-systems now accepts systems as (optional) arguments (e.g. --convert-systems "nes snes").
The script now detects the C.E.S of the video and if it is the same as the from_ces value, it won't convert the video.
New log system. Log files are stored in logs/.
Print script version with --version.
Added a progress bar!

Changed

Search for all .mp4 video files instead of just -video.mp4 so the script can work with many scrapers.

I've updated the first post with the new documentation :)

mitu

Great progress. Btw, you commented the actual conversion line in the convert_video function (the one that runs avconv), so I'm not sure that conversion would work.

hiulit

@mitu Thanks! Yeah, now it's in utils/progress-bar.sh ;)