Versatile C++ game scraper: Skyscraper

muldjord

@AlCzervik You can force it to use the genesis game data and artwork (if there is any, it will still fall back to Megadrive if none is found) by adding the following to /home/USER/.skyscraper/config.ini:

[genesis]
region="us"

But normally this is all up to the naming of your files. If they have region designations ((europe), (usa)) that match where they are from (and thereby Genesis vs Megadrive) they will get scraped correctly automatically.

But I must stress that i depends on whether the scraping module you are scraping from supports region and if it has the data for it in their database.

Read more here: https://github.com/muldjord/skyscraper/blob/master/docs/REGIONS.md

AlCzervik

@muldjord @mitu Thank you both. I just noticed that the payload wasn't passing "genesis" when running via Skyscraper. And yes, same folder for these ROMs so I assume the file name is driving the correct art per @muldjord's comment. And thanks, I'll attempt to force the the genesis game data/network. Thanks again!

muldjord

@AlCzervik Please also check my edits if you haven't seen it. I just added some more.

AlCzervik

@muldjord Got it. Thanks!

Retro

Is there a way to delete the cached information for an individual game? Sometimes the scraper fetches the wrong information for a game and I don't really wanna delete all my cached information for a system just to fix one error.

Cheers!

muldjord

@Retro Yes, you can easily delete the data for a single game with Skyscraper -p platform --cache edit FILENAME

If a game is connected to an incorrect game when using the screenscrapermodule, you should consider reporting it to them, so it gets fixed.

AlCzervik

@muldjord Again, sorry if this is the wrong forum, but are their any tricks to getting Daphne scrapes working properly? I've got Dragons Lair functional, but no art.

muldjord

@AlCzervik And you have tried scraping it with Skyscraper?

AlCzervik

@muldjord Yep, it found data but in Emulation Station it's not showing up. So I don't think the issue is with Skyscraper getting the data, but definitely Daphne stores ROMs differently than other platforms.

muldjord

@AlCzervik And you did generate the gamelist for it after grabbing the data, right?

KN4THX

I am having an issue where screenscraper says states that anonymous users only get 1 thread and I am locked to that but Screenscraper shows my account as having 6 threads and my scrape total on Screenscaper's site continually climbs while I am scraping on my Pi. I edited the config file to use 6 threads and obviously my user:pass is correct or it wouldn't show up on the site. Has anyone had the same trouble?

AlCzervik

@muldjord Yep. Tried it a few times.

muldjord

@KN4THX said in Versatile C++ game scraper: Skyscraper:

I edited the config file to use 6 threads and obviously my user:pass is correct or it wouldn't show up on the site. Has anyone had the same trouble?

You don't need to set threads=6 it will do this automatically when you've added:

[screenscraper]
userCreds="USER:PASS"

Please verify that you have it entered exactly like that in /home/USER/.skyscraper/config.ini. Also please verify what it says when it starts scraping. Something like Starting scraping run on X files using Y threads. Does it say 1 threads or 6 threadsthere?

AlCzervik

@muldjord A few more points. Per the RetroPie Daphne emulator site: https://retropie.org.uk/docs/Daphne/ the file structure is a bit different. From EmulationStation I just see 1 game entitled Lair. In the file structure under Daphne, you have two folders: lair.daphne and roms. The lair.daphne folder has the content (videos, screenshots, etc), the roms folder has the lair.zip file. So could that be the cause of the issue?

muldjord

@AlCzervik Oh, so it's not a file called lair.daphne it's a folder, I didn't know that. I don't have any Daphne games, so it would be cool if we could figure this out.

Well, basically I'm guessing, if you enter the roms folder inside the daphne folder, you will probably see the artwork and so on, right? It scrapes the lair.zip file from that folder and that's what's added to the gamelist.

What file do you select to start a game?

AlCzervik

@muldjord I think you're right. I'm sure it's stored 1 level lower as that's where the lair.zip is located. The file I select to start the game is the "Lair" file that is shown via EmulationStation so I assume we need to map the artwork somehow to that top level.

AlCzervik

@muldjord this is what it looks like:

muldjord

@AlCzervik I've been reading up on this and this is actually a bit tricky. EmulationStation actually launches the game when you select a gamename.daphne folder. This is quite unusual. No other platform I am aware of does this. They all launch using files. Skyscraper and other scrapers don't scrape folders, they scrape files, meaning that the gamename.daphne folder is not scraped, which is why you don't have any game information and artwork on it. But, the roms/lair.zip files is scraped, but this file is hidden inside of EmulationStation it would seem.

So I'll need to figure out how to deal with that.

AlCzervik

@muldjord Gotcha. Yeah, Daphne is quite different but once operational it works like a charm. Let me know how I can help troubleshoot/test!

muldjord

@AlCzervik This is now working in 3.5.0 which has just been released. Just update Skyscraper and run Skyscraper -p daphne and it should work without having to rescrape the data. Remember to exit EmulationStation first.