Having almost nothing but bad luck scraping

bfollowell

OK, so I posted yesterday about having no luck scraping data for my ROMs with the built-in scraper and the same with Selph's Scraper. Several people posted things to try, none of which helped. Finally, I switched to Skraper, and scraping the data from my PC and moving it to the Pi over the network. I thought that seemed to clear everything up.

Well, it's only cleared it up a little bit. Atari Lynx, PC Engine, Sega 32X, those all scraped every single game without a hitch. Everything else I've tried since, SNES, Genesis, Gameboy, Gameboy Color, Gameboy Advance, they all fail to find a single game. All of these ROMs are from No-Intro ROM sets and I've verified them with RomCenter. They all check as good. I have no idea why these are failing to pull back any results using Skraper.

From what little bit of good luck I've had, I really prefer the results coming back from Skraper, and would prefer to use it, but I have no idea why so many of my systems/ROMs are failing when, from everything I can tell, the ROMs are perfect in every way.

Does anyone have any idea what I may be doing wrong or why I'm not getting any results? Is there some other better way to verify my ROMs?

bfollowell

I should also have mentioned that I've checked a handful of these files on several of the problem systems and they all check out perfectly. I'd already verified using RomCenter, but I also verified the CRC of the zip file, and of the ROM inside it, against the CRC values shown for those ROMs on Screenscraper.fr. They're good, valid files that match by CRC and filename.

I'm at a complete loss.

quicksilver

@bfollowell What is your scraping source? Some websites limit the number of scrapes you can do per month.

bfollowell

@quicksilver said in Having almost nothing but bad luck scraping:

me websites l

Skraper uses Screenscraper.fr

I'd wondered if it may be something like that, but the weird thing is, I go to scrape one system and everything fails. Then I immediately go back and try one of the few that I've scraped successfully, and it will scrape again without any issues whatsoever, which it wouldn't do if it were limiting my scrapes.

I'm fairly certain Screenscraper.fr limits on a daily basis, not a monthly one, so I'll wait and try again before leaving for work in the morning with one of the systems that has failed me previously, and see if it makes a difference.

bfollowell

@quicksilver

It looks like your were spot on. Before I went to bed for the night, I finally found where I'm allowed 20,000 scrapes a day on Screenscraper.fr. I'd originally thought it was 50,000, but that was for paying supporters, which I'm not yet. Also, I was thinking, I've not scraped anywhere near that many games, but I guess I'm scraping numerous things for each title, and I'm sure each of those count as separate scrapes. So, anyway, I was over my daily allotment. I tried again before getting ready for work and everything took off just fine this morning.

So, until I pay to support, I'll need to keep a closer eye on my daily quota and scrape over several days.

Thanks.

Clyde

@bfollowell said in Having almost nothing but bad luck scraping:

So, until I pay to support, I'll need to keep a closer eye on my daily quota and scrape over several days.

Just ask yourself how much you're using screenscraper.fr and if you want it to continue. Apart from getting 50.000 scrapes/day, donating helps to ensure the latter.

Disclaimer: I'm not affiliated with SS in any way, just another user who decided one day that SS isn't the worst thing you can spend your money on. ;)

quicksilver

@bfollowell It's possible that the data you had already scraped was cached on your computer and that was why previously successful scrapes still appeared to scrape correctly and new scrapes did not.