[SCRIPT] RetroPie Convert Videos

hiulit

~~This folder you mentioned $home/Retropie/roms/media/videos when using SkyScraper, does it contain folders for all the systems inside?~~

$home/Retropie/roms/media/videos/megadrive
$home/Retropie/roms/media/videos/nes

~~etc?~~

Forget that! I've found this table explaining where all the metadata and media are stored using different scrapers :)

https://retropie.org.uk/docs/Scraper/#where-are-my-scraped-media-and-metadata-saved

Is this information correct? (I'm guessing it is)

mitu

@hiulit said in [SCRIPT] RetroPie Convert Videos:

Is this information correct? (I'm guessing it is)

If you're using the scrapers through the RetroPie-Setup script, then it's correct.
Note that fi you're using the CLI parameters for folders or otherwise customize your configuration file, then the folders can be anywhere.

hiulit

@mitu Ok, thanks! I'll state that ROM folder for gamelists & media must be enabled. As well as I state the same thing when using Steven Selph's Scraper (use ROM folder for gamelists & images)

hiulit

@mitu Just out of curiosity, does the built-in EmulationStation scraper have the ability to download videos? It's been ages since I used it and maybe now it does. Thanks.

mitu

@hiulit No, it doesn't.

hiulit

@mitu Ok, thanks! Less things to take care then :P

hiulit

Making some progress on supporting both sselph and skyscraper :)

ERROR: There are no systems with videos!

You are using 'skyscraper' scraper.
Remember to use the 'ROM folder for gamelists & media' option.

Or maybe try using 'sselph'.

P.S. Which is better?

There are no systems with videos!
There aren't any systems with videos!

English is not my mother language :P

mitu

@hiulit Not an native English speaker also, but maybe start with videos:

No videos found for system's games/roms
No videos found in any system.

Here's an additional challenge - why look up the folder when you can read the video path from the gamelists ? This way you don't have to assume the usage of a particular scraper or a certain path.

hiulit

@mitu Thanks! I'll use your examples ;)

As for reading the path directly from the gamelist.xml, wouldn't I have the same problem? I mean, I have to know where the gamelist.xml is stored, and it can vary depending on what scraper is used and what options are set on where to store the gamelist.xml on that particular scraper, right?

ES will check three places for a gamelist.xml in the following order, using the first one it finds:

[SYSTEM_PATH]/gamelist.xml

~/.emulationstation/gamelists/[SYSTEM_NAME]/gamelist.xml

/etc/emulationstation/gamelists/[SYSTEM_NAME]/gamelist.xml

If you use another scraper (Steven Sselph's scraper or Lars Muldjord's `Skyscraper) you can choose whether your scraped data goes in

[SYSTEM_PATH]/gamelist.xml (the rom folder) or

~/.emulationstation/gamelists/[SYSTEM_NAME]/gamelist.xml

Taken from https://retropie.org.uk/docs/EmulationStation/#scraper.

If I have to know where the gamelist.xml is stored then I might just tell the user to use the option to store the media "where I want it" to be :P
Or "force" the user to have the gamelist.xml "where I want it" to be).

hiulit

Here it is! RetroPie Convert Videos v2.0.0

[2.0.0] - 2019-04-15

NOTE: This release contains breaking changes!

Deprecated

~~--convert-system~~ now is --convert-systems.
~~--from-color~~ now is --from-ces.
~~--to-color~~ now is --to-ces.

Added

Support for Lars Muldjord's Skyscraper.
New option --scraper to set the scraper. Available scrapers: sselph and skyscraper.
--convert-systems now accepts systems as (optional) arguments (e.g. --convert-systems "nes snes").
The script now detects the C.E.S of the video and if it is the same as the from_ces value, it won't convert the video.
New log system. Log files are stored in logs/.
Print script version with --version.
Added a progress bar!

Changed

Search for all .mp4 video files instead of just -video.mp4 so the script can work with many scrapers.

I've updated the first post with the new documentation :)

mitu

Great progress. Btw, you commented the actual conversion line in the convert_video function (the one that runs avconv), so I'm not sure that conversion would work.

hiulit

@mitu Thanks! Yeah, now it's in utils/progress-bar.sh ;)

Menion

Hi all
first thank you for all the effort put in the script. I have a question: the script is still targetting Retropie host? Because I think that a lot of users have ROMs on a dedicated NAS, so it would be perfect if the script can run on a generic host.
In the end it needs to know only the ROMs root folder and little else
Bye

hiulit

@Menion Yes, right now it still targets only RetroPie in a Raspberry Pi, but I have in mind adding a parameter to set the path to the ROMS directory.
About running on a host machine, once this new parameter is added, it should just be a matter of where do you launch the script from, right? As long as it's a Linux machine.

Also, I'm not an expert on video conversion, bitrates, etc. so I don't know if the actual ffmpeg/avconv command is the best one. That could be improved I guess.

I looked at your code and there quite a few changes that I can't test because I don't have a VAAPI compatible GPU. So maybe the right think would be for you to wait until I add this new "add your own ROMS path" parameter, then merge your code with mine (or take mine, because there are many changes and recode it with your VAAPI changes), see if it works and if it does, then we can talk more about adding your changes to the my repository :)

hiulit

Already a new version! RetroPie Convert Videos v2.1.0.

[2.1.0] - 2019-04-17

Added

New option --path to set the path to the ROMs folder. By default it's /home/pi/RetroPie/roms.
Warning dialog when using the --convert-all option saying that it can take a lot of time to convert all the videos.
Better detection of dependencies.
Better error outputs.

I've updated the first post with the new documentation :)

ObscureUsername

Hi,
I'm having trouble getting this to work, any idea what I'm doing wrong? I'm trying to convert videos I've downloaded via sksyscraper that don't play due to using yuv444p.

If I run

./retropie-convert-videos.sh --convert-systems "fba"

... it says ...

"Starting video conversion ..."
*.mp4 --> Can't convert! Doesn't use the Color Encoding System (C.E.S set in 'from_ces': 'yuv444p'.
1 video was unsuccessful

...it only takes a few seconds for this to run so I don't think it could've checked every video in that time (there are thousands of them, which is why I only want it to convert yuv444p videos).

In the config files the following options are set:
from_ces = "yuv444p"
to_ces = "yuv420p"
scraper = "skyscraper"
roms_path = "" (I've left the roms path blank as I'm using the default folder).

Any ideas?
Thanks!

hiulit

@ObscureUsername Hi there! From the looks of your log file it seems like it's not getting any video file at all.

Are you using ROM folder for gamelists & media option? Here you can see where your downloaded videos should be (expected by the script) when using that option https://retropie.org.uk/docs/Scraper/#where-are-my-scraped-media-and-metadata-saved

Also, are the downloaded videos .mp4?

Rion

@hiulit

./retropie-convert-videos.sh --convert-systems "fba"

CLI mode
--------

Inputted systems: 'fba'.
Systems found: 'fba'.

Starting video conversion ...

fba
---
Converting "*.mp4" ... (1/1)
(standard_in) 1: syntax error
ERROR: '*.mp4' has 0 frames!


fba
---
> "*.mp4" --> FAILED!


1 video failed.

I am using skyscraper and i have configured everything correctly i think.

hiulit

@Rion Hmmm I'll take a look. It seems it has something to do with the videos path..

Did you you Use the ROM folder for gamelists & media option in Lars Muldjord's Skyscraper?
Did you set the scraper using the --scraper option? Or via the config file?

Rion

@hiulit said in [SCRIPT] RetroPie Convert Videos:

@Rion Hmmm I'll take a look. It seems it has something to do with the videos path..

Did you you Use the ROM folder for gamelists & media option in Lars Muldjord's Skyscraper?
Did you set the scraper using the --scraper option? Or via the config file?

@hiulit Hi again and sorry for the late reply.

I used Use the ROM folder for gamelists & media with Skyscraper and also set the scraper option to skyscraper
inside retropie-convert-videos-settings.cfg. But i did get it to work if a moved all my video files from

/home/pi/RetroPie/roms/fba/media/videos

to

/home/pi/RetroPie/roms/fba/media