[SCRIPT] RetroPie Convert Videos

[hiulit]

Hi everyone,

Inspired by a little script to convert videos that @Clyde made a while ago, I made RetroPie Convert Videos - A tool to convert videos for RetroPie.

RetroPie Convert Videos v3.2.0

A tool for RetroPie to convert videos.

WARNING: The Raspberry Pi doesn't have that much power and converting videos is very demanding. It takes about 35 seconds to convert a video, so if you have a lot of videos... Do the math ;)

Prerequisites

At this moment this script only works with videos downloaded using:

• Steven Selph's Scraper.
• Lars Muldjord's Skyscraper.

Use the ROM folder for gamelists & images option in Steven Selph's Scraper.

Use the ROM folder for gamelists & media option in Lars Muldjord's Skyscraper.

Installation

cd /home/pi/
git clone https://github.com/hiulit/RetroPie-Convert-Videos.git
cd RetroPie-Convert-Videos/
sudo chmod +x retropie-convert-videos.sh

Update

cd /home/pi/RetroPie-Convert-Videos/
git pull

Usage

./retropie-convert-videos.sh [OPTIONS]

If no options are passed, you will be prompted with a usage example:

USAGE: ./retropie-convert-videos.sh [OPTIONS]

Use './retropie-convert-videos.sh --help' to see all the options.

Before trying to convert any videos, --to-ces and --scraper must be set.

Once the conversion is finished, a new converted-<to_ces> folder with the resulting videos will be created in /home/pi/RetroPie/roms/<system>/images if you are using Steven Selph's Scraper or in /home/pi/RetroPie/roms/<system>/media/videos if you are using Lars Muldjord's Skyscraper. You can then replace the old videos with these ones.

Log files are stored in logs/.

Options

• --help: Print the help message.
• --path [OPTIONS]: Set the path to the ROMs folder. (optional)
• --from-ces [OPTIONS]: Set Color Encoding System (C.E.S) to convert from. (optional)
• --to-ces [OPTIONS]: Set Color Encoding System (C.E.S) to convert to. (mandatory)
• --scraper [OPTIONS]: Set the scraper. (mandatory)
• --convert-all: Convert videos for all systems. Warning! It can take a lot of time.
• --convert-systems [OPTIONS]: Select systems to convert videos.
• --gui [OPTIONS]: Start the GUI.
• --version: Print the script version.

Examples

--help

Print the help message.

Example

./retropie-convert-videos.sh --help

--path [OPTIONS] (optional)

Set the path to the ROMs folder.

By default it's /home/pi/RetroPie/roms.

Options

• path - The path to the ROMs folder.

Example

./retropie-convert-videos.sh --path "/path/to/your/roms/folder"

--from-ces [OPTIONS] (optional)

Set Color Encoding System (C.E.S) to convert from in the config file.

Target only videos with this particular C.E.S.
If left blank, it will target all videos, regardless of the C.E.S.

In most cases, you'd want to set it to yuv444p as this is the C.E.S that gives errors when using the OMX player.

Options

• C.E.S - Color Encoding System (C.E.S) to convert from.

Example

./retropie-convert-videos.sh --from-ces "yuv444p"

--to-ces [OPTIONS] (mandatory)

Set Color Encoding System (C.E.S) to convert to in the config file.

Convert videos to this particular C.E.S.

In most cases, you'd want to set it to yuv420p as this is the C.E.S that seems to work well with the OMX player.

Options

• C.E.S - Color Encoding System (C.E.S) to convert to.

Example

./retropie-convert-videos.sh --to-ces "yuv420p"

--scraper (mandatory)

Set the scraper in the config file.

Options

• sselph - Steven Selph's Scraper.
• skyscraper - Lars Muldjord's Skyscraper.

Example

./retropie-convert-videos.sh --scraper "sselph"

--convert-all

Warning! It can take a lot of time.

Convert videos for all systems.

Checks the config file to see if at least the to_ces key has a value.

Example

./retropie-convert-videos.sh --convert-all

--convert-systems [OPTIONS]

Select systems to convert videos.

If no options are passed, it displays a checklist from which one or more systems can be selected.

Checks the config file to see if at least the to_ces key has a value.

Options

• systems - Any of the system available in RetroPie's ROMS directory (e.g. nes, snes, megadrive, etc.). Systems must be quoted and comma-separated. See the example below.

Example (without options)

./retropie-convert-videos.sh --convert-systems

Example (with options)

./retropie-convert-videos.sh --convert-systems "nes, snes"

--gui [OPTIONS]

Start the GUI.

It lets you perform all the functions, but in a more friendly manner.

Options

• standalone - Launch the script without checking if RetroPie is installed, which it does by default.

Example (without options)

./retropie-convert-videos.sh --gui

Example (with options)

./retropie-convert-videos.sh --gui standalone

--version

Print the script's version.

Example

./retropie-convert-videos.sh --version

Config file

When setting the C.E.S using --from-ces or --to-ces, or when using --scraper or --path, the generated values are stored in retropie-convert-videos-settings.cfg.

# Settings for RetroPie Convert Videos.
#
# TIP: run the 'ffmpeg -pix_fmts' command to get a full list of Color Encoding Systems (C.E.S).

# From color (optional)
# Target only videos with this particular C.E.S.
# If left blank, it will target all videos, regardless of the C.E.S.
# (e.g. "yuv444p")

from_ces = ""

# To color (mandatory)
# Convert videos to this particular C.E.S.
# (e.g. "yuv420p")

to_ces = ""

# Scraper (mandatory)
# The scraper used to get the videos.
# Available scrapers:
# - "sselph" (Steven Selph's Scraper).
# - "skyscraper" (Lars Muldjord's Skyscraper).

scraper = ""

# ROMs folder path (optional)
# Set the path to the ROMs folder. By default it's "/home/pi/RetroPie/roms".
# (e.g. "/path/to/your/roms/folder")

roms_path = ""

You can edit this file directly instead of using --from-ces, --to-ces, --scraper or --path.

Authors

• Me 😛 @hiulit.

Credits

Thanks to:

• Clyde - For posting the code in the RetroPie forum that inspired this script.
• Dan Edwards - For helping with #3.
• furnace and mitu - For pointing out some issues on how to better support Lars Muldjord's Skyscraper.
• pruperting - For the code from which the progress bar is based on.
• Memodellocos - For fixing the Skyscraper's videos path in #5.
• Rion - For the idea of having the script work as a standalone app, without the need of RetroPie being installed.

License

MIT License.

[Alturis]

Thank you!

Some additional docs for the script could include the info that it only reads the first option, not a list of options.

Also just FYI in case you want bug reports the first video it converts gets put into a folder called "converted-" then the rest go into a folder called "converted-<COLORFORMAT>"

So you have to call it once to set the from-color in the .cfg file, then again to set the to-color in the cfg, then a third time to actually run it with either --convert-all or --convert-system.

Running it now and crossing my fingers that this allows the OMX accelerator to work with my preview videos....

Noticed a couple of videos errored out with something about "invalid next size" - will have to look into fixing them or finding replacement videos.

Works great thank you!

[hiulit]

@alturis I'm glad it's working! I'll take a look at your comments later on when I have more time and I'll get back to you :)

[meleu]

@hiulit I didn't have time to test it yet, but WOOHOOL! the code looks very clean! At least at a quick read on the github web interface it seems to be a nice job! :)

[Alturis]

@alturis said in [SCRIPT] RetroPie Convert Videos:

Noticed a couple of videos errored out with something about "invalid next size" - will have to look into fixing them or finding replacement videos.

I ran this in putty so I could take a close look at the output. Looks like its a known bug reported in the libav bugs db: (basically a crash)

https://lists.libav.org/pipermail/libav-bugs/2014-January/002969.html

I have actually quite a few of them that error out this way. Kind of a bummer because I would like to have all games with videos. But in order to see some without garbling and slowdowns i need the video accelerator running, and in order to see others I need yuv420p. I guess I will live with the slowdown and stick with yuv444p for the time being.

[Alturis]

@hiulit I have some changes to the script that fixes a few things

1. Fixes the "converted-" folder issue with the first converted video
2. Checks the result of avconv and if it failed it logs the result code and renames the file <rom>-video.mp4-failed so that you don't unknowingly stomp your valid video with a now corrupt one. ( which I certainly never did, just looking out for other folks ;) )

I can email it to you?

[hiulit]

@meleu It's needs a little love, but it's ok! Thanks! ;)

[hiulit]

@alturis Hi there! If you have a GitHub account, I would prefer if you could create a Pull Request https://github.com/hiulit/RetroPie-Convert-Videos. If not, you can find my email at https://github.com/hiulit/ ;) Thanks a lot!

[Alturis]

@hiulit Hmm.. I do have a github account and use it quite a lot to work on my own projects. But I dont have much experience using it in this respect. It looks to me like in order to create a pull request I first need to be able to create my own branch of it. Which it is saying I do not have permission to do when I try it.

Ah ok i see. I am able to create a fork and create a pull request that way. Done.

[hiulit]

I've just released Retropie-Convert-Videos v1.0.2.

Fixed

• Fixed some outputs.
• Fixed git clone URL in README.md.

Added

• Merged #3 - Thanks to Dan Edwards

[Menion]

@hiulit I have forked your repository here and modified your script so it uses ffmpeg and encode with VAAPI acceleration on a generic host machine (assuming that a VAAPI compatible GPU is present)
The results are pretty good speed wise (and also quality wise). The script takes the original video bitrate and set it for the encoded yuv420p. I am no expert of bash so there are few limitation, one is that the home directory is set in the script itself
Maybe you can/will merge everything in your script and adapt it so it can run on retropie or host and select the correct encoding?
Bye

[Clyde]

@Menion said in [SCRIPT] RetroPie Convert Videos:

I am no expert of bash so there are few limitation, one is that the home directory is set in the script itself

Just curious, why didn't you use the original home detection or the config variant you still have as a comment in your script?

[Menion]

@Clyde because the modified script shall run on a host pc without requiring to have Retropie

[Clyde]

@Menion Perfectly understandable, but reading the rom path from the cfg file or as an command line argument wouldn't be Retropie-dependant either, and yet more consistant with the other options.

Just some friendly feedback. And thanks for mentioning me in the credits. 😊

[Menion]

The script requires improvement, but I am very bad in bash script

[Clyde]

@Menion I would be very much inclined to help you, but my bash skills are extremely limited, too. My video conversion script is an example of the maximum level of my bash knowledge, and even for that I had to do some web research. Maybe @hiulit can give you some clues.

[Menion]

No problem. Meanwhile I need to figure out why the bitrate set seems to produce odd results. Without specifying anything the encoding goes up to double the bitrate of the original file, while setting it the result is 40% less. I know that with single pass enconding the bitrate cannot be precise, but this is a little bit too much

[hiulit]

@Menion Hey, thanks!

A couple of questions:

• Does your script work with a Raspberry Pi? Or does it only work on a "host" machine?
• Does your script only work with VAAPI compatible GPUs? And if so, how could we detect it with the script?

About merging your script with mine, I would be more inclined in creating a new branch and having two separate scripts, the "default" one and the "host version" one.

Also, right now i don't have that much time to take a deep look at your script to check it but I'll try in some time soon(ish).

PS. I'm with @Clyde about using home="$(get_config "home")" instead of hardcoding it into the script ;)

[Menion]

The script should work with any vaapi host, but there is no detection of it, it is just hard coded

[furnace]

@hiulit
I've started to use your script and works like a charm. I've even modified it to support video scrapes attained by using Skyscraper on the retropie. I was just wondering though the following.

1- When the conversion is done, the converted videos are put into the converted directory, correct? can these then be copied back to the original directory to override the bad videos to reduce space usage?

2- Also if I use the --convert-system flag and let say I already converted a set of videos but then scrape new ones. Will re-running the conversion tool reconvert all the videos or just those that had not been done? If it's all the videos is there a why to just convert specific videos.