Scraping
-
@mitu Ignore my question about registering.. I've watched a tutorial on Skraper and understand now. Will give it a go.
-
I've been using Skraper for hours now.. rough estimate was 24 hours for 1500 roms.. does that sound right??
-
@marc81 I guess the site is busy/overloaded. Usually happens during the week-ends.
-
It looks like the ScreenScraper site has a few issues at the moment with their API, scraping is going to be either very slow or have lots of failed requests until they get it back. This was posted in their forum:
Hello ScreenScraper community,
we have a technical problem with the ScreenScraper clone server. This causes significant slowdowns on the site and the api and therefore on your scrapers. We recommend you to wait until both servers are working normally again to do your scrapes.
Thank you for your patience, the ScreenScraper team.
-
@mitu Thanks. Checked on it this morning.. 20 hours in and over a day to go.. not even half way 😅... I'm guessing this is why. If I stop it will it save what it's scraped up to now?
-
If I stop it will it save what it's scraped up to now?
If we're talking about Skraper, then yes, it keeps a local cache of the artwork already downloaded.
-
Not sure if my issue is related but using the built in ScreenScraper is yielding no scrapes. It always worked wonderfully too. Eternal sadness until I get scraping working on my Retropie.
Pi3 B+ no overclock, latest updates.
-
If we're talking about Skraper, then yes, it keeps a local cache of the artwork already downloaded.
Skyscraper also has a media cache, and Selph's Scraper at least keeps the already downloaded videos and pictures in the images directory, even if its gamelist mode is set to "overwrite".
Just stumbled upon your post and thought about this addendum. :)
-
Tried to scrape things using internal scraper via Screenscraper and had no results yet.
-
For about 2 weeks, ScreenScraper has had some problems - even after they've been back online following a maintenance, scraping for un-registered users is mostly a miss. It's not your setup, it's their API/site.
-
For about 2 weeks, ScreenScraper has had some problems - even after they've been back online following a maintenance, scraping for un-registered users is mostly a miss. It's not your setup, it's their API/site.
I am a registered user but I am not sure how this would apply to access their stuff from my Raspberry Pi while in Retropie vs. my Windows PC where I can login via Firefox.
-
You can use one of the external scrapers - Skyscraper or Steven Selph's scrapers - to make use of your user account when scraping. Each one has its way of using the account info, Skyscraper can read them from the configuration file, Steven Selph's scraper accepts the user/password as command line parameters when scraping.
EDIT: or, on Windows, you can use Skraper.
-
@mitu does Skraper allow me to scrape just one game at a time or does it look for whole systems?
-
From my brief usage, it scans whole systems (folders) of ROMs.
Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.
Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.