Versatile C++ game scraper: Skyscraper
-
@steptoe Somewhere in this thread there is a guide on how I made the "live view" but it is for Ubuntu Linux, which is what I run at home. I do not have a guide for Windows I'm afraid (and there won't be one as I don't use Windows).
You can compile it for Windows, but there will be tweaking to get it to work properly I think. I haven't done much testing of this myself, again, as I'm not really a Windows user.
But Qt5 is cross-platform, so with some tweaking it can be done. Just remember NOT to try and compile it through Cygwin, that will definitely fail. Instead use the Qt environment command prompt link that ships with the Qt5 installer.
Please read the documentation on the github page concerning locally caching data. Skyscraper already caches all data for later use, to significantly speed up the process of re-scraping. So whenever you have the artwork set up as you like, just scrape each platform with '-s localdb' and it will scrape using all cached data.
-
Would it be possible to have it use a secondary artwork.xml if some elements are missing? For example I'd like to have it output just a full-size box art if there's no screenshot available.
-
@wonderbread I thought about this at one point but decided against it. The reason being that I would have to basically do scripting inside the artwork.xml somehow. Since the artwork.xml is completely customizable I can't hardcode rules for it (because I can't know what the user wants), so it has to be user specified, and that would be tricky on several levels. It can be done of course, but it's a path I am not currently gonna walk down I'm afraid.
-
I've a question, I've installed retropie 4.4 n my ubuntu desktop and I recently today installed and started running Skyscraper against my Arcade directory. It seems to keep going through the same files over and over, it's on it's 4th recursion on the directory. Is this normal?
Thanks -
@brewskii If you run it in simple mode, then yes. Skyscraper scrapes with a different source each time, caches the data, and then does a local scrape with all stored data.
-
@analoghero Ah, I see that now, it's also mentioned in the dialog when working through simple mode questions. Thanks!
-
@muldjord I've just read this:
https://www.reddit.com/r/emulation/comments/8dqim8/announcing_thegamesdb_beta_site_api/
Maybe of your interest. Thanks!
-
@bleuge Thanks for the heads up, that is indeed interesting.
-
-
@muldjord From now i support you on patreon. I know its not much, but i hope it helps.
-
@analoghero Everything is appreciated mate :)
-
@muldjord For some strange reason patreon cant process my pledge. I tried several times adding and removing my mastercard my paypal account. Is there any other way i can send you money? Paypal maybe?
-
@analoghero I appreciate the sentiment but please don't bother too much with it. I appreciate the help you've done with bug hunting and helping other users a lot more than monetary support. :)
-
@muldjord I am currently using advmame for a few roms (mostly goldentee). I have the roms in the mame-advmame directory with the rest in the arcade dir. Is there a way to scrape the advmame games from the mame-advmame directory without messing up the db or already scraped info from the arcade directory?
-
@bighoffa said in Versatile C++ game scraper: Skyscraper:
arcade
Please ignore my question. I discovered how to use runcommands and specify the emulator to run for each rom so I can put everything in the arcade directory now.
-
@muldjord A few days ago i did a complete new retropie setup as a birthday present. Just a few classic systems without video previews. Did everything in simplemode and cant report any bugs or flaws.
-
@analoghero Glad to hear it :)
-
Working on improving the attractmode export module. Currently testing game descriptions implementation with user qqplayer. Requests for further attractmode improvements should come now, as this is what I am focused on. So let me know if you have anything that should be supported.
-
Skyscraper version 2.4.6 released: https://github.com/muldjord/skyscraper
- Added 'overview' support for AttractMode. It will now create the necessary cfg files to show the game descriptions
- Added '
<kidgame>bool</kidgame>' output to Emulationstation gamelist generation - Added 'ages' support in 'screenscraper' module. Will convert PEGI and ESRB to numeric
- Added 'ages' support in 'thegamesdb' module. Will convert PEGI and ESRB to numeric
The 'kid friendly release'. As per user request, I've implemented the
<kidgame>Emulationstation tag and added classification scraping support for both the 'thegamesdb' and 'screenscraper' modules. This means that you can now make use of the "kids" mode in Emulationstation, which will then hide all non-kid-friendly games from the lists.IMPORTANT! If you want to scrape the 'ages' on games you already have cached, you have to scrape using the '--updatedb' flag. Otherwise it will just scrape with the pre-2.4.6 cached data which doesn't contain the ages resources.
Lastly, I've also added 'description' support for the 'attractmode' frontend module. I'm not an Attract-Mode user myself, but I've done basic testing on it, and user 'qqplayer' has verified that it works as expected.
:)
-
@muldjord I've seen this very nice and complete MSX db : https://www.msxgamesworld.com/index.php
maybe you want to check it for scraping.
Thanks as usual!
Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.
Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.