Versatile C++ game scraper: Skyscraper

muldjord

@ikarith This actually seems to be a problem on ScreenScrapers end.

But first, let me clear some things up. The aliasMap.csv file is only used by filename based search modules. ScreenScraper uses checksum hashes or exact filename searches and therefore makes no use of the aliasMap.csv at all. So that won't help you in this case.

If ScreenScraper gives you back a faulty game, it is most likely because the checksum of your file or the exact filename, is connected to a wrong game inside of their database. This is not a rare thing to happen.

My suggestion is to log on to their website and find the specific checksum or filename that matches your files, and see if they are incorrectly connected to the wrong versions of those games. You can then proceed to correct that on their website as well. When the correction has been approved, you can then proceed to rescrape the games.

retro81

Hi @muldjord, first of all thanks for developing this amazing script for retropie!

I have a problem when fetching covers from screenscraper.fr - if the game has a flyer (like Samurai Shodown for NeoGeo) then the box is composed using the game "flyer" instead of the "box2D" art, any ideas why this happens and how to correct it? I'm using the default artwork.xml

Thanks!

retro81

I actually found this was a feature implemented in 3.4.3, any way to fine-tune skyscraper to either use flyers or actual box art? I use the retropie script.

retro81

Hi @muldjord, would it be possible to include the game hard/support media download as an option?

I like the way our ScreenScraper.fr friends do their "recalbox-mixes" including the game support media, here's a sample: https://www.screenscraper.fr/medias/113/176930/mixrbv2(sp).png

muldjord

@retro81 Hi, no, the ScreenScraper specific media won't be supported as they are unique to ScreenScraper and Skyscraper supports many different scraping sources. I don't want to turn Skyscraper into a ScreenScraper scraper.

But you can create a very similar result using Skyscraper's artwork compositor. It's documented here by using the already supported media types and add some custom frames and masks.

EDIT: A quick note: I am not currently active in developing Skyscraper beyond fixing bugs.

muldjord

@retro81 In the current version it will fetch "flyer" for all arcade platforms including "neogeo". But I see that that might not be optimal. So I've removed "neogeo" from that list. From 3.6.7 it will use "box-2D" instead. Thanks for letting me know.

EDIT: 3.6.7 out now with this change included. Please update your Skyscraper version and rescrape the neogeo platform with cache refresh enabled (important!).

retro81

@muldjord thank you for both responses!

Rion

@muldjord Would you consider adding support for Batocera?

muldjord

@rion Just did a Google search and I assume you mean Batocera Linux, right? I won't personally support it, but I don't see why it wouldn't work already. As long as it has Qt5 included and a build chain, you should be able to install it using the description at the Skyscraper github. How to get Qt5 on there is up to the user to figure out. Maybe it's in the Batocera package manager already (if there is such a thing, I have never heard of the distribution before just now).

Rion

@muldjord You should really try it out. It's a great distribution both for sbc's but also for an x86_64 PC.

Skyscraper is not yet available in the Batocera pacman repository. But could probably be in the future.

What i wanted to know is if you would think/consider to implement support for more systems supported by Batocera. I'm completely fine if i could just use it on my Linux machine and scrape the roms there.

Cause i just love how the artwork looks with Skyscraper and there is no better alternative out there! 😃

https://wiki.batocera.org/emulators

muldjord

@rion If they are major platforms that are related to consoles or home computers I will consider it. But I do have a lot of requests to add obscure "homebrew" and or / sub-platforms that I do not wish to support as they are often either too obscure or already supported by "arcade" or others.

Mainly Skyscraper was made to support the RetroPie naming scheme and platforms. I have, however, added some platforms besides that that I felt made sense. I am quite restricting when it comes to adding more, as I don't want to bloat Skyscraper too much.

But if you have a couple of platforms that you feel could fall under the RetroPie umbrella, I'd need the same info as found on the RetroPie supported platform pages, which is:

Platform name and shortname (eg. "Nintendo Entertainment System" / "nes")
File extensions (*.this, *.that)

But no promises I'll add them. :)

Obi-Wan

Not sure where to post this, but I felt it should be mentioned somewhere.

Scraping does not work on Windows with version 3.6.7, but works fine with the previous version, 3.5.8.

I was so happy to find Skyscraper because it seemed like a really nice tool, so I was a bit bummed out about the lack of Windows-support and even more so when I couldn't get it to work. But the second I tried the older 3.5.8 version scraping worked immediately.

I get the same behavior on two separate Windows systems. This reproduces the behavior on both my systems:

Download Skyscraper and unzip to C:\work\
Copy contents of the deploy folder to my user-folder
Open Powershell in Skyscraper-folder (cmd has the same behavior)
In PS, run: .\Skyscraper.exe -s thegamesdb -p snes -i "C:/work/ROMS"

3.6.7 returns Game not found
3.5.8 works fine.

I have no idea if I've missed something else here, but with 3.6.7 I got nowhere and with 3.5.8 everything seems to work as intended.

muldjord

@obi-wan I might look into this one of the coming days. And thank you for the detailed description, that is highly appreciated.

muldjord

@obi-wan Please redownload it from the same link. It was missing an OpenSSL dll that I've now included. I've tested it on my Windows dev machine and it works now. Let me know if it works for you too.

Obi-Wan

@muldjord Thanks a bunch! :) I was thrown off my emulator project for a while, but I'm back at it and 3.6.7 seems to work perfectly well now! Appreciate it, especially because the Windows version is not really supported!

muldjord

@obi-wan Yeah, I don't really support it beyond releasing it. But I at least want it to work, so thanks for reporting that. I don't really use it myself so... :D

muldjord

Quick show of hands: Would anyone be against me removing the "Simple mode" from Skyscraper? The questionnaire mode that it runs if you run Skyscraper without command line parameters. I haven't updated it in ages and it seems that it's currently broken. I never liked it, and I kind of want to remove it completely.

Given that the RetroPie Skyscraper script is much better I feel like "Simple mode" is deprecated.

Clyde

@muldjord Since I never noticed it until you mentioned it now, I don't mind. 😉

muldjord

Thanks guys. I will remove it in the next release, unless someone stops by within the next week with compelling arguments to keep it.

johnodon

I've been reading through the docs and haven't found my answer (or if it is possible)...

I like to save all of my media to a single folder using file suffixes (like sselph scraper does). For example in /roms/arcade/images/ I would have:

zaxxon-image.png
zaxxon-video.mp4
zaxxon-marquee.png
zaxxon-launching.png

Is this possible with Skyscraper?

TIA.

John