Built in scrapper hit or miss?
-
All jokes aside. I know at one point last year there was a problem with the internal scraper and even there was a “fix it/delete it” conversation (from what searches have told me.)
I never did figure out if full functionality ever got restored or if it’s still just limping along. The reason I ask is that I’ll be able to only randomly scrape a title here and there and only for a short while before everything just starts coming up “no games found”.
If there’s still something screwy with the scrapper I’ll look elsewhere for a way to scrap. Otherwise I will look for solutions to the problem.
-
@tackett1980 it doesn't seem like the built in scraper is working currently. It was working for a while but doesn't look that way anymore. I would use skyscraper, very customizable and allows scraping from multiple sources before generating your gameslist.xml
-
Roger that. That’s what I suspected, just wanted to confirm.
-
Well, that sucked
I installed skyscraper and it doesn’t work. It just gives me a red error message something about “non-registered users” and just goes back to the setup screen.
What gives?
-
@tackett1980 sign up for a free account on screenscrapers website. Then edit the skyscraper config.ini and add your new login info. When the screenscraper website is overloaded it will deny requests from unregistered users.
-
Is that screenscraper.fr?
Never mind I got it. Thanks.
Now to figure out how to get skyscraper to actually take my authentication. -
@tackett1980 go to the skyscraper config.ini and add
[screenscraper] userCreds="user:password"When you start scraping from screenscraper you should notice text pop up that you are logged in under your user name
-
Thanks sir. I got it figured out thanks to you.
Unfortunately, though it seemed like it was working and it ran all day long scraping without any errors, I don’t see where it actually did anything.
My games are still blank, there are no videos downloaded that I can see.
Just like most of the things with retropie.... I can say, “what did I screw up this time?”
Btw I used the etaprime tutorial video on YouTube to do this.
-
@tackett1980 did you generate your gameslist after you scraped?
-
Yeah and when I do it’s saying “game xxxx not found”
And it’s not like they obscure games.
One line says “game ‘pacman’ not found. I find that difficult to understand.
-
I think it has something to do with my sub folder setup.
When I scraped my NES roms directory it did every single one no issue.
-
I was actually able to scrape with the internal scraper last night via Screenscraper. I was beyond happy at that moment.
-
Actually it didn’t work.
It downloaded some descriptions but not videos.
I wonder if I’m out of space.
-
@tackett1980 said in Built in scrapper hit or miss?:
It downloaded some descriptions but not videos.
The built-in scraper doesn't download videos, only artwork and information (description, genre, title, rating, publisher, etc.)
-
I was referring to skyscraper. Sorry for the confusion.
-
@tackett1980 it doesn't download videos by default, did you turn the option on to download videos?
-
Yeah I did it seems like it only downloaded a few videos from my arcade folder up to about “P” and stopped.
I need to back up and punt here I think...
I need to get all these roms organized before I start worrying about scraping them .
-
@tackett1980 said in Built in scrapper hit or miss?:
It's not you. I just ran it myself and it did nothing, In fact it actually removed some of my artwork I had previously for games. lol
None of the scrapers have been working for me for a week now. I even tried to add manual artwork to my games but that wasn't working like it should have either.
-
Yeah I’m starting to get the feeling that this screenscraper database source is just spotty.
I ran skyscraper again last night and it looks like it just randomly decided which ones it wanted to do and which ones it didn’t.
Then I tried to run the internal scraper and now it’s not working again either. Everything just says no games found.
I’m going to start looking for a windows scraper that might use a more reliable database.
-
@tackett1980 are you using a login when scraping from screenscraper?
Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.
Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.