Versatile C++ game scraper: Skyscraper

[AnalogHero]

@muldjord Tried it. The interactive mode is awesome. I wish this would somehow work for screenscraper. But i understand its crc based...

All i can say is that this scraper has to be included with retropie.

[muldjord]

@analoghero Awesome! Well, I can enable it with screenscraper aswell, but I don't really see how it would be useful as there will always only be one entry to choose from since its checksum based. If you have any thoughts on this, I'd be happy to hear them.

[muldjord]

Wow, I just went to check out EmuMovies. The community on there is doing some pretty amazing work!

I just downloaded the wheels pack for the Sega Master System, plopped them onto my Pi in the '~/.skyscraper/import/wheels' folder and ran "Skyscraper -p mastersystem -s import" and then "Skyscraper -p mastersystem -s localdb" (-s localdb not really necessary as it is default if it's left out) to make use of the imported data and BAM! I now have wheels for about 70% or my Mastersystem roms. That is pretty amazing. All I had to do was to make sure that my rom naming conventions match that of the pack, which is often Hyperspin.

You guys should seriously check that out.

[dorkvader]

@muldjord OK, here are some images that better explain my intentions and my problem.

As you can see by the Atari 2600 Adventure "gamebox" with its spine created from a "spine image" resource, the sizing is off. (I used the wrong spine color intentionally to demonstrate the slight gap in the top and bottom, which I cannot fix).

For point of reference, I've included an Adventure gamebox that was generated with a wheel resource.

An Atlantis "gamebox" shows the other problem with using a "spine image" as the resource: the side and front colors don't match up.

Finally, I'm including a mixed-art image of what I'm creating for all the systems: screenshot, wheel, gamebox, and marquee (the cartridge).

Again, I love what you've done and cannot tell you enough how much I appreciate your help. Even if you tell me it's impossible, I'll be happy to end this quest for another!

[AnalogHero]

@muldjord Maybe there is a way to scrape not matching single roms with the screenscraper module by using a override flag and use a user provided checksum.

For example:

Skyscraper -p megadrive -s screenscraper /home/pi/RetroPie/roms/megadrive/Sonic.zip -crc AC400F6A

[muldjord]

@dorkvader Ah, yes, I see the problem. Let me think about this. I should be able to add some attributes to "gamebox" that will allow this to work. Great idea to use the marquee resource for cartridge btw, I really like that. Awesome!

@AnalogHero Not a bad idea, I'll give this some thought aswell and find a proper way to implement it. I'll probably just call it '--checksum [sum]' as it doesn't really matter if it's the sha1 or the md5 sum people input. Also, I'll disable it if the user tries to scrape more than 1 rom.

[AnalogHero]

@muldjord Exactly what i thought. Just for single roms.

Then you can scrape amiga .lhas which arent in their database using a checksum from an .adf entry :)

[muldjord]

@dorkvader 2.7.1 out now which can do what you are looking for. Just add the 'sidescaling="both"' attribute to the gamebox xml note, and it will autoscale the image to completely fit both width and height. That should do the trick. Please try it out and let me know if it does what you expect.

@AnalogHero Concerning the checksum option, I have decided against it for the simple reason that it is a screenscraper only option, and there's not a lot of payoff with it. For most roms you could just as easily just rename the input file, which would take just as long as getting the sha1's. With that said, for the LHA files that you suggest, I am currently investigating if I can get all of the Retroplay LHA's added to screenscraper. Then we can start linking them to the correct games. :)

[dorkvader]

THANK YOU SOOOOO MUCH, Muldjord! I am excited to try it out and will let you know the results ASAP!

[dorkvader]

@muldjord said in Versatile C++ game scraper: Skyscraper:

sidescaling="both"

Works BRILLIANTLY! Thought I'd really test it with a Vectrex box, and as you can see it worked beautifully, matching the graph lines as well as one can expect.

Thank you for your help and for your generosity in time and work. If you have some kind of "donation jar," I will gladly throw a tip your way.

[muldjord]

@dorkvader No need man, but I appreciate the sentiment :) Feel free to donate something to the RetroPie project though. Or, if you insist, I have a patreon at https://www.patreon.com/muldjord :)

Looks great btw! I love that idea and I am probably going to be doing the same thing for some of my platforms. Probably just using the marquee (with the spine graphics) for the spine instead, and then using a custom artwork.xml file for those platforms that I have them for. Thank you for the inspiration.

[AnalogHero]

@muldjord I see your point. Im also not sure if its worth the effort. But there are situations were renaming isnt a good option. (If you have already savestates or config files for some games).

Also you could not only do a checksum option but a general override option (- override {gameid OR gamename}). So it could be used for more than one scrapingmodule.

[muldjord]

@analoghero I'll consider it. It's definitely not a bad idea, I just have to make sure it makes sense for Skyscraper as a whole and that the implementation in the code doesn't bloat it. :S

[Used2BeRX]

@dorkvader Those are awesome man.

Any chance you could hook me up with your templates for those? I'd love to make 3D boxart for all of the Xtras I've been making. The only template I have though makes every game look like a generic DVD case. It still looks good, but it's miles away from what you're showing here.

[dorkvader]

@used2berx Sure thing. I'm finishing the Vectrex spines tonight and will be happy to zip my template and or/completed images (spine and boxes) or both. The system has about 25 officially released titles and I've more than tripled that with public domain homebrews. I'm also doing that with all the home consoles from the Golden Era o' Video Games. Just email me.

[Used2BeRX]

@dorkvader Thanks man. That would be great. :)

Not sure where to find your email though?

[easye9inches]

Can someone help me in installing Skyscraper to RetroPie on Ubuntu? muldjord mention that install is exactly the same as is on the PI. I have done the steps listed in the GitHub and I am getting this error:

$ wget -q -O - https://raw.githubusercontent.com/muldjord/skyscraper/master/update_skyscraper.sh | bash
bash: line 3: curl: command not found
--- Fetching Skyscraper v. ---
--2018-09-03 22:34:48-- https://github.com/muldjord/skyscraper/archive/.tar.gz
Resolving github.com (github.com)... 192.30.253.113, 192.30.253.112
Connecting to github.com (github.com)|192.30.253.113|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2018-09-03 22:34:48 ERROR 404: Not Found.

I am assuming that I should also install this in the RetroPie folder?

[mitu]

@easye9inches said in Versatile C++ game scraper: Skyscraper:

I am assuming that I should also install this in the RetroPie folder?

No, you can run it from anywhere you have write access. You need to install curl and then re-run the installation

sudo apt-get -y install curl

[easye9inches]

@mitu said in Versatile C++ game scraper: Skyscraper:

@easye9inches said in Versatile C++ game scraper: Skyscraper:

I am assuming that I should also install this in the RetroPie folder?

No, you can run it from anywhere you have write access. You need to install curl and then re-run the installation

sudo apt-get -y install curl

Thanks! got it installed

[easye9inches]

Do I have to edit the config file if all my roms are on a external HDD? Because nothing is scraping because it is the default home/pi/roms setting. Id rather scrape my games on the external for backup purpose, and than transfer them over to the pi.