Versatile C++ game scraper: Skyscraper

muldjord

@AlCzervik You're welcome... I have to admit it frustrates the hell out of me that we can't get to the bottom of this. Especially if this is somehow a Skyscraper issue. But I guess I'll have to file this under "unsolved". Please let me know if you do end up getting it to work.

AlCzervik

@muldjord Last question, should the config.ini file be in that same directory? I assume not but I see the example in there so I was wondering.

muldjord

@AlCzervik Something I notice... arcadedb does not run https, just http. So that's a difference in ports from port 80 to 443. That might mean something. So it could still be some kind of network issue at your location (I kind of hope it is, it would explain the issue somewhat).

muldjord

@AlCzervik No, it is in /home/pi/.skyscraper/config.ini (which, when it's installed through RetroPie-setup is actually linked to the actual file inside /opt/retropie/configs/somethingsomething... I can't remember the exact location. But for all intends and purposes, it's located at /home/pi/.skyscraper/config.ini.

AlCzervik

@muldjord Just want to thank you again for all of your troubleshooting. Quick update is I'm fully functional but I needed a completely new install of RetroPie to get there. Not sure what was going on but I was looking to potentially do it for multiple reasons so all good. Skyscraper is awesome and working great!

AlCzervik

@muldjord One small hiccup. I'm attempting to load the Sega Genesis artwork vis Skyscraper. I select Genesis but I notice each payload is still seeking the artwork for platform=Megadrive. Obviously they are technically the same but I there are differences. Small issue but I assume it just isn't passing the right parameter when I select Genesis vs. Megadrive. Apologies if this is the wrong forum so please direct me elsewhere if needed. Thank you!

mitu

@AlCzervik said in Versatile C++ game scraper: Skyscraper:

I select Genesis but I notice each payload is still seeking the artwork for platform=Megadrive

In RetroPie, they're the same folder/platform. What differences do you see for scraping ?

muldjord

@AlCzervik You can force it to use the genesis game data and artwork (if there is any, it will still fall back to Megadrive if none is found) by adding the following to /home/USER/.skyscraper/config.ini:

[genesis]
region="us"

But normally this is all up to the naming of your files. If they have region designations ((europe), (usa)) that match where they are from (and thereby Genesis vs Megadrive) they will get scraped correctly automatically.

But I must stress that i depends on whether the scraping module you are scraping from supports region and if it has the data for it in their database.

Read more here: https://github.com/muldjord/skyscraper/blob/master/docs/REGIONS.md

AlCzervik

@muldjord @mitu Thank you both. I just noticed that the payload wasn't passing "genesis" when running via Skyscraper. And yes, same folder for these ROMs so I assume the file name is driving the correct art per @muldjord's comment. And thanks, I'll attempt to force the the genesis game data/network. Thanks again!

muldjord

@AlCzervik Please also check my edits if you haven't seen it. I just added some more.

AlCzervik

@muldjord Got it. Thanks!

Retro

Is there a way to delete the cached information for an individual game? Sometimes the scraper fetches the wrong information for a game and I don't really wanna delete all my cached information for a system just to fix one error.

Cheers!

muldjord

@Retro Yes, you can easily delete the data for a single game with Skyscraper -p platform --cache edit FILENAME

If a game is connected to an incorrect game when using the screenscrapermodule, you should consider reporting it to them, so it gets fixed.

AlCzervik

@muldjord Again, sorry if this is the wrong forum, but are their any tricks to getting Daphne scrapes working properly? I've got Dragons Lair functional, but no art.

muldjord

@AlCzervik And you have tried scraping it with Skyscraper?

AlCzervik

@muldjord Yep, it found data but in Emulation Station it's not showing up. So I don't think the issue is with Skyscraper getting the data, but definitely Daphne stores ROMs differently than other platforms.

muldjord

@AlCzervik And you did generate the gamelist for it after grabbing the data, right?

KN4THX

I am having an issue where screenscraper says states that anonymous users only get 1 thread and I am locked to that but Screenscraper shows my account as having 6 threads and my scrape total on Screenscaper's site continually climbs while I am scraping on my Pi. I edited the config file to use 6 threads and obviously my user:pass is correct or it wouldn't show up on the site. Has anyone had the same trouble?

AlCzervik

@muldjord Yep. Tried it a few times.

muldjord

@KN4THX said in Versatile C++ game scraper: Skyscraper:

I edited the config file to use 6 threads and obviously my user:pass is correct or it wouldn't show up on the site. Has anyone had the same trouble?

You don't need to set threads=6 it will do this automatically when you've added:

[screenscraper]
userCreds="USER:PASS"

Please verify that you have it entered exactly like that in /home/USER/.skyscraper/config.ini. Also please verify what it says when it starts scraping. Something like Starting scraping run on X files using Y threads. Does it say 1 threads or 6 threadsthere?