EmulationStation Scraper NOT Working

keithk2581

Im missing for approx. 173 mega drive games. I have tried to scrape them individually and even changed the name in the input section. but still returned no games found. I even tried just typing one letter into the input field and searching and still returned no games found.

Here is the link for the error log after I scraped for genesis games:

mitu

@keithk2581 I don't know why it's not working for you, I just tried a few Megdrive games and they work fine when scraping from TheGamesDB. Maybe the names you have for the files are not recognized by the scraping site - what's the name of your ROMs, can you give some examples ?

keithk2581

Addams Family, The.gen
Arch Rivals - The Arcade Game.gen
The Aquatic Games starring James Pond and the Aquabats.gen

……...to name a few out of 173. I have even tried to rename them, the file name and by inputting different variants of the name into the search box, but no luck. Don't understand why its not working on mine. Any suggestions on what I could try next?

Thanks again to you all for you assistance, it is truly appreciated.

mitu

I think that the problem is with the names of the ROMs. If I try to scrape Addams Family, The, no result is found, but if I change the name of the ROM to The Addams Family, the game is found and scraped. It's odd that from over 100 titles, none of them is matched, but they seems to be named in a peculiar way.

As an alternative, you can try skyscraper or scraper - they're external scrapers that use mainly the file contents to scrape and the name of the ROM is not that important.

keithk2581

Yes it's very weird, I too, have tried changing the name, but nothing is found still, what's even more weird is that it was working right after the update to 4.4.12 but then suddenly stopped on both of my builds. I guess I'll have to try the other scrapers for now, manually enter each game it missing or format one of my builds and do a reinstall and see if that works. Thanks to all for your assistance, if you can think of anything else, my ears(eyes) are open

keithk2581

Other question I just thought of, is there any setting or file or something written in a file like the gamelist.xml file that would prevent the scraper from finding any games?

mitu

@keithk2581 There is a setting in for saving the metadata in Emulationstation (Save Metadata on Exit) which can be turned off - this way the gamelist.xml is not written, but that's not what happens in your case.
Is your disk full ?

keithk2581

Disk does not appear to be full:

https://imgur.com/gallery/63MCfwY alt text

keithk2581

So as an experiment, I decided to try a fresh install on a 3rd SD and after all was installed and updated, the scraper did work! Something must be corrupted on my other 2 builds, just don't know what or how or why! Any idea if I can fix this without formatting and starting from scratch or is that my only option?

mitu

@keithk2581 Try removing any files from ~/.emulationstation/scrapers. TheGamesDB scraper will download some files there on the 1st scrape, you'll have to wait a bit, then it will re-create the files.

keithk2581

I'll give it a shot when I get home tonight, thanks!

keithk2581

I deleted the files you said, and it works again! Thank you so much for your help!!

keithk2581

Only thing I notice now is that it still returns no games found for games I know are on the gamesdb for example, super battleship and wheel of fortune to name 2(Sega genesis), just an issue with the site or my scraper?

mitu

@keithk2581 Can you find the games on their site ? If not, then maybe there's an issue with the site. What's the complete ROM filename ?

keithk2581

Yes the games are on the site, but seems to only be a problem for mega drive/genesis games. Just did a bunch of super nes games with no issue and all of my rims are named in the same fashion. I have all of my genesis games in mega drive folder could that be an issue in regard to the scraping?

Rom names are in this format:
Wheel of Fortune(USA).gen
Adventures of Rocky and Bullwinkle And Friends,The(USA).gen

Both of those are on the games dB but won't scrape even if I manually change the input

mitu

@keithk2581 said in EmulationStation Scraper NOT Working:

I have all of my genesis games in mega drive folder could that be an issue in regard to the scraping?

It should not, unless they're in a different system on the scraping site. Given that the other ROMs are found, I don't think that's the case.

Wheel of Fortune(USA).gen
Adventures of Rocky and Bullwinkle And Friends,The(USA).gen

There are multiple games with this name - the first has one for Genesis and SegaCD, the 2nd one has one for multiple consoles of the time - NES, SNES, Genesis, GameBoy.

I can take a look to see what's confusing the scraper, but it could also be a bug on the server side.

keithk2581

Yeah, trying to figure that out is definitely above my current skill level :). Funny that its only the genesis(mega drive) games, gotta be something about them. My guess would be its on the server side, but who knows. Thanks again for all your help, and to all who have posted, I truly appreciate it.

mitu

It's because of the way TheGamesDB stores the information about platforms (consoles/systems).
The scraper filters the ROM search by Platform=Sega Mega Drive (and rightly so, since the same game is also available on NES/SNES and GameBoy), however the games are in platform=Sega Genesis, which is different (has a different identifier).

If you search on TGDB by Platform='Sega Mega Drive' and the name of the game - let's use The Adventures of Rocky and Bullwinkle and Friends as an example - there's nothing returned. However, if you search with Platform='Sega Genesis', there's one game returned (as it should).

I'm not sure right now how this should be handled in Emulationstation, I'm surprised there were only 2 games from your list. I think all US exclusive Genesis games will not be found by the TGDB scraper.

Now, the ScreeenScraper scraper would be able to find them, but right now it's a bit broken because of their SSL settings change.

keithk2581

Hmmm, I see. There were more than 2 games, was just giving you an example of the format of the file names of 2 of my roms that wouldn't scrape. Scraper actually worked for 611 of the 784 games i have, plus or minus a few name issues. I guess for now, ill just gradually add them over time manually, like a small project. Thank you so much for everything!

mitu

@keithk2581 You can try also Skyscraper, it should be at least as efficient as scraper.