Versatile C++ game scraper: Skyscraper

muldjord

@dylosaur That is actually a bug, and I think I've found the culprit. I'll try and fix it in a quick release as soon as I get the time. Thank you for notifying me about this.

muldjord

Skyscraper 2.0.5c released. Last Skyscraper release of 2017. Just a quick bugfix release: https://github.com/muldjord/skyscraper

Fixed a bug where the import module wouldn't look for all resource types during the same run. Thank you to 'dylosaur' for notifying me about this.

Merry christmas everyone, and happy new year!

dylosaur

@muldjord No problem ! I love your scraper. Wish I could use it with some of the unsupported systems because it makes changing the sizes of my screenshots a breeze. Keep it up.

muldjord

@dylosaur Let me know which ones you'd like me to include. Adding more systems is quite easy in Skyscraper, as long as the sources I currently support also support the platforms.

dylosaur

@muldjord Oh, cool. That would be great. I'm working on it for a gift.

Right now, the ones I'm looking to add that you don't support are:

Atari Lynx
GCE Vextrex
NeoGeo Pocket Color
Sega 32x

Was also looking to add, but might not get to (because of time):

Virtual Boy
Magnavox-Odyssey2
TurboGrafx16

How do I update the scraper, by the way? Just follow the install steps again?

muldjord

Skyscraper 2.0.6 released!

Ok, so THIS is the last Skyscraper release of 2017 (except bug fixes if something major pops up). Added a bunch of platforms: https://github.com/muldjord/skyscraper

Added support for atarilynx, vectrex, ngpc (Neo Geo Pocket Color), sega32x, virtualboy and videopac platforms.

Merry christmas everyone, and happy new year!

muldjord

@dylosaur I've added the platforms you've requested. Pc-Engine was already in there. Let me know if any of them don't work. I haven't tested them, just implemented them.

Yes, to get the newest release, just follow the steps again, that's the easiest way to do it. :)

dylosaur

@muldjord I just did the Lynx, Vectrex, NGPC, and Sega32x and can confirm they all work perfectly. This will make finishing this gift so much easier. I'll try the others tomorrow and let you know. Thanks so much!

muldjord

Just started working on the new 'scripting' mode ('scripter' branch on github). This will be easier to implement than I initially thought. Might even be out before new year! We'll see. :)

Anyways, I am off for the holidays, have fun everyone!

Rion

@muldjord God Jul/Glædelig Jul/ Merry Christmas

muldjord

@rion Lige over / right back at ya'!

AnalogHero

@muldjord Tried the new version with Atari7800 and it worked fine! Thanks again for this tool and Merry Christmas to all reading :)

muldjord

@analoghero Thank you, glad it works!

muldjord

Skyscraper 2.1.0 released! https://github.com/muldjord/skyscraper

Added 'simple mode'
Refactored and cleaned up all emulation platform specific code

'Simple mode' works by simply typing 'Skyscraper' after you've installed it. It will ask you a bunch of questions. Based on your answers it will generate a script and run it (if you allow it). When the script is done, you will have the optimal result for the chosen platform. This should help a lot of the newcomers to get the best results from the get-go. It basically does multiple scraping runs using the supported scraping modules for the selected platform, and finishes the job by scraping with 'localdb' to bind it all together.

Please try it out and let me know if you have any feedback or discover bugs. I've tested it a bit myself, but I've refactored a lot of code, so bugs can have snuck in through the process.

Happy scraping! And happy new year everyone!

AnalogHero

@muldjord Was about to test simple mode with the beta but then 2.1.0 was released and i updated to it.
It worked fine for me with sega32x. It did 2 web runs and the a localdb run.

muldjord

Skyscraper 2.1.1 released: https://github.com/muldjord/skyscraper

Added support for '3do', 'amstradcpc', 'coco', 'daphne', 'dragon32', 'dreamcast', 'fds', 'gameandwatch', 'intellivision', 'ngp', 'oric', 'pc88', 'pcfx', 'saturn', 'sg-1000', 'trs-80' and 'zmachine' platforms
Added 'forcefilename' option which forces the file name to be used in the frontend instead of the one returned by the scraping module
Changed 'gamelist.xml' to 'game list' in 'simple mode' question

Be sure to also check the notes for 2.1.0 if you haven't already (there's some cool new stuff in that one). Happy scraping!

SammyBoy

@muldjord Are you planning to add Fb Alpha as well allow ROM Hacks to be scraped in the future. Because I've got a lot of FB Alpha ROMS and one final question are the thumbnails/artworks that are scraped good quality because I know Steven Selph Scraper isn't in good quality. Sorry for the long message.

Rion

@sammyboy I think you can already scrape fba roms using the Arcade parameter because it's already incorporated by using adb.arcadeitalia.net?

muldjord

@sammyboy The artwork quality completely depends on what the scraping sources deliver. So I can't give you a clear answer to that as I am not the one creating them. I am planning a bit of a rewrite of the artwork part of Skyscraper, to allow for the largest possible artwork for those who needs it. Currently I resize them to save space, but I plan to allow users to disable this. Let me know what your thoughts are on this. Concerning 'fba' I will include it in 2.2.0. But as Rion suggests, it's already possible to scrape them with 'arcade'. But since you request it, of course I will implement it. :)

To all of you, 2.2.0 is just around the corner. Some big stuff in this release. Most importantly Skyscraper now also scrapes for 'wheel' and 'marquee' artwork. Not only that, but I also rewrote the artwork handler completely. A byproduct of this means you will be able to save the artwork directly "as is", completely disabling the artwork compositor. This has been requested quite a few times, so I hope this will be appreciated. It has been one helluwa rewrite. :D

Stay tuned... Need to test stuff, but it's almost ready for release.

SammyBoy

@muldjord Thank you for choosing to add FB Alpha and I will try Skyscraper out today and I know that it will be good. And one last question are you a able to choose what artwork gets scraped? And thank you for hard work and effort.