Skyscraper / Emulation Station shows some videos, not others

Marky161

Hi all,

Newbie here, so hope I'm able to articulate the issue.

Pi Model or other hardware: Pi4 2gig
Power Supply used: Official supply
RetroPie Version Used: 4.7.11
Built From: From scratch using Retro Pie Buster (4.7.1)
Emulation Station theme: Carbon
Skyscraper: v3.6.9

I have created a new build for scratch for my newly purchased Atari Fightstick from Micro Centre, and can see light at the end of the tunnel, although am frustrated by one of my finishing touches; video previews.

I have copied across a few Arcade ROMS, in alphabetical order, and run SkyScraper from RetroPie-Setup Script (using Skyscraper as the source) to see if I liked the output. During the download I could see that I received the common "service is currently closed or full" error, but still built the gamelist.xml. When I opened EmulationStation I could see that all games had the info part of the game, but some were not playing videos, namely those that it skipped during the download.

I closed ES and rerun Skyscraper, setting the option to 'Scrape Only Missing', and it seemed to proceed further down the list after a few retries. I compiled the games list.xml once more but still couldn't see the videos.

Within the file Gameslist.xml (/home/pi/.emulationstation/gameslists/arcade) I can see the game entry to the correct path and filename with references to 'Image', 'Marquee' and 'Video' and can see the video in the correct location (/home/pi/.emulationstation/downloaded_media/arcade/videos).

To test there wasn't an encoder error with the video files I cloned one of the other games that worked, 1942, and renamed to a game which had no preview, yet still no preview (there was a correct file there, so renamed the original).

Is there another link in a file to the video preview other than gamelist.xml that needs a refresh? Ideally I'd like to manually edit where Skyscraper fails, but am at a loss where to begin?!

Any pointers much appreciated.

Mark

sleve_mcdichael

@marky161 said in Skyscraper / Emulation Station shows some videos, not others:

I closed ES and rerun Skyscraper, setting the option to 'Scrape Only Missing',

onlymissing will only scrape data for roms that don't have any piece of data from any scraper (screenscraper, thegamesdb, etc.) Useful to scrape those last few roms that aren't on your primary source, without duplicating all the data you've already downloaded for the bulk of your library. Default behavior is to only scrape files that don't have any data from the current scraper, which should be fine for your situation.

You say you see videos in the correct location, so my guess then would be that they are in a wrong format like yuv444p, while they need to be in yuv420p (or at least that is one that is known to work.) If so, there's a simple script that can convert them for you when downloaded. Here is a (mostly) copy-paste response from a similar post:

Check first: locate a game that doesn't show a video. Navigate to where the video should be, at:

/home/pi/.emulationstation/downloaded_media/<platform>/videos/<romname>.mp4

If the file exists, check it out with ffprobe (may require sudo apt install ffmpeg first):

ffprobe "filename.mp4" 2>&1 | grep Stream

Output will look something like:

    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv444p, 360x480 [SAR 1:1 DAR 3:4], 349 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)
    Stream #0:1(und): Audio: mp3 (mp4a / 0x6134706D), 24000 Hz, stereo, fltp, 64 kb/s (default)

See in the Video stream where it says yuv444p, 360x480? That's bad, we want it to say yuv420p instead.

Sounds like you? Good. This post on GitHub discusses the solution. TL;DR version after:

https://github.com/muldjord/skyscraper/issues/222

The quick version:

Edit your /home/pi/.skyscraper/config.ini to include the line:

videoConvertCommand="videoconvert.sh %i %o"

Script provided by jgcobra on the GitHub thread:

#! /bin/bash
# Script provided by jgcobra

echo Converting video...
echo Source: $1
echo Dest: $2

# Check if the video is already in 420p format
c=$(ffprobe "$1" 2>&1 | grep -c yuv420p)

if [ "$c" -eq 0 ]
then
    echo Incorrect source video format detected. Converting video to 420p format...
    ffmpeg -i "$1" -y -pix_fmt yuv420p -acodec copy -strict experimental "$2"
else
    echo Video is in correct 420p format. Copying as is...
    cp "$1" "$2"
fi

Place the script in your system path or in /home/pi/.skyscraper and make it executable with chmod +x videoconvert.sh. If you want to see the script's output when run by Skyscraper, I think you need to put verbosity="3" in your config.ini.

This script only runs on initial caching of the video, so re-scrape your roms with --cache refresh to force a re-download. Or you could scrape only the ones that don't work by specifying their filenames at the end of the command:

Skyscraper -p arcade -s screenscraper rom1.zip rom2.zip rom3.zip

(If you've installed Skyscraper through the RetroPie-Setup script then, before running it from the command line, it is recommended to create a symlink to the executable in your system path with:

sudo ln -s /opt/retropie/supplementary/skyscraper/Skyscraper /usr/local/bin/Skyscraper

This will allow you to run the command just by typing Skyscraper without having to use the full path every time.)

--cache refresh is not needed when specifying individual files, it is already implied.

This solved probably 90% of my problem videos. There were a handful that still didn't look right and I had to do a little extra fiddling with aspect ratios and such, but this should get you started.

Bonus tip: add symlink="true" to your config.ini to write shortcuts to the cached videos instead of duplicating them in your media folder, to save half the space.

Don't forget to rebuild your gamelist after.

Marky161

@sleve_mcdichael Many thanks for the comprehensive reply! I'll give that a go a bit later today.

I didn't think it would be the encoded files however as I tried renaming a file that worked? I wasn't sure if there was anywhere else that a user could edit to replace with their own files/paths etc. or is gameslist.xml the master file?

Regards,
Mark

Marky161

@sleve_mcdichael - Thank you for this - Having followed this process, and re-run Skyscraper with cache disabled, all video preview are now showing correctly! ;)

Happy days!

Mark

Retro80s

@marky161 I had this problem. Like mentioned, the issue is with the video encoding. Unfortunately, the scrape sources for Arcade games pull down videos that aren't encoded in a way that EmulationStation can run properly.

To get by this I use Skyscraper to download everything and build the Gameslist.xml file.

Then I go to: https://emumovies.com/files/file/2214-mame-video-snaps-pack-sq/ and download their MAME video snaps. They are encoded perfectly and work great. They are a good resource for everything really.

I would recommend creating an account, then you can use their FTP server to download everything easily. If you don't you can only download a set at a time and you have to wait longer.

Once you do that copy and replace your already downloaded videos with these. I literally just did all of this yesterday so I know it works and will help you out.

Marky161

@retro80s Thanks for this also excellent tip. I have LaunchBox / BigBox on a separate machine and sub’d to EmuMovies to get the additional snaps etc, so will replace where additional videos are missing.

Skyscraper is an excellent tool, and extremely handy, but is clearly now overused for the times you receive an error when updating.

Next step, custom loading screens to show joystick layout and screensaver image to show navigation (although might not need it if in kiosk mode).

sleve_mcdichael

@marky161 said in Skyscraper / Emulation Station shows some videos, not others:

Skyscraper is an excellent tool, and extremely handy, but is clearly now overused for the times you receive an error when updating.

I haven't received any errors updating Skyscraper. There was an update that broke a feature, but that was fixed within a day when reported.

Do you mean "updating" your downloaded media, i.e. the "system busy or overloaded" errors you sometimes get when just using Skyscraper? That's not Skyscraper's fault, that's on the scraping source that Skyscraper is getting its data from (screenscraper.fr is one of them, but there are like half a dozen others you can use. SKYscraper and SCREENscraper are not the same, despite similar-sounding names.) Sometimes this can be mitigated by signing up for a screenscraper account and adding your login credentials to config.ini; other times you will just have to wait.

Marky161

@sleve_mcdichael Indeed, I stand corrected. Screenscraper being the better resource (in my build) for description fields and image resources.

sleve_mcdichael

@marky161 said in Skyscraper / Emulation Station shows some videos, not others:

Screenscraper being the better resource

So you can see why it's popular ;)

You could always throw a donation their way, to help keep the servers running.

Marky161

@sleve_mcdichael Already there! ;). Keep the community alive!