EmulationStation Scraper NOT Working

keithk2581

Yeah, trying to figure that out is definitely above my current skill level :). Funny that its only the genesis(mega drive) games, gotta be something about them. My guess would be its on the server side, but who knows. Thanks again for all your help, and to all who have posted, I truly appreciate it.

mitu

It's because of the way TheGamesDB stores the information about platforms (consoles/systems).
The scraper filters the ROM search by Platform=Sega Mega Drive (and rightly so, since the same game is also available on NES/SNES and GameBoy), however the games are in platform=Sega Genesis, which is different (has a different identifier).

If you search on TGDB by Platform='Sega Mega Drive' and the name of the game - let's use The Adventures of Rocky and Bullwinkle and Friends as an example - there's nothing returned. However, if you search with Platform='Sega Genesis', there's one game returned (as it should).

I'm not sure right now how this should be handled in Emulationstation, I'm surprised there were only 2 games from your list. I think all US exclusive Genesis games will not be found by the TGDB scraper.

Now, the ScreeenScraper scraper would be able to find them, but right now it's a bit broken because of their SSL settings change.

keithk2581

Hmmm, I see. There were more than 2 games, was just giving you an example of the format of the file names of 2 of my roms that wouldn't scrape. Scraper actually worked for 611 of the 784 games i have, plus or minus a few name issues. I guess for now, ill just gradually add them over time manually, like a small project. Thank you so much for everything!

mitu

@keithk2581 You can try also Skyscraper, it should be at least as efficient as scraper.

mitu

I submitted a bug report - https://github.com/RetroPie/EmulationStation/issues/565 - for this problem, maybe the scraper can be easily modified to search simultaneously in the Genesis and Megadrive systems.

MrJordaaany

Is this fixed now and in the main build? Or do you still need to source build the experimental release of Emulationstation?

mitu

@MrJordaaany It's what fixed ?

MrJordaaany

@mitu The EmulationStation built in scraper. Or do you have to build the experimental release from source. Will this new version be included in 4.5?

mitu

@MrJordaaany If you use the built in scraper is should have a functional TGDB scraper. Did you scrape from TheGamesDB or from ScreenScraper ? Scraping from ScreenScraper only works on the -dev branch, due to a change on their site.

MrJordaaany

It wasn't working for me so I installed the experimental one from source. If I do a fresh 4.4 install, it should be ok now you reckon? Or still need to update EmulationStation under core packages?

Cheeeers

mitu

@MrJordaaany What's the exact Emulationstation version you see when you look at the ES menu (Start) ?

MrJordaaany

@mitu v2.7.6. I could be wrong though, im not at home so can't check right now.

mitu

@MrJordaaany This is not the latest version of Emulationstation. If you're using the 4.3 or older image (based on Raspbian Jessie) and you're not using the 4.4 image, then you're not going to get the latest version.

MrJordaaany

Yeah probably not going to update my own Retropie setup, I know emulator updates will break my setup.

However when setting up new units for mates or whatever, the built in EmulationStation scraper should be good right off the bat on 4.4?

mitu

@MrJordaaany said in EmulationStation Scraper NOT Working:

However when setting up new units for mates or whatever, the built in EmulationStation scraper should be good right off the bat on 4.4?

If you start with the 4.4 image you'll get the same 2.7.6RP version, you need to do at least an update after a fresh 4.4 image for the packages included.

AssPunchMan

Hi

I have a similar problem with version 2.8.3RP.

Yesterday it stopped working and you can not find any more games. There is no way to make it work again?

It tells me ScraperHttpRequest error .. SSL conection fail or something similar.

Pi 3B+ all updated to latest version -with downgrade of kernel for fix bluethod controllers bug-.

Thx

mitu

@AssPunchMan said in EmulationStation Scraper NOT Working:

Yesterday it stopped working and you can not find any more games. There is no way to make it work again?

What scraper source are you using ? TheGameDB or ScreenScraper ? TheGamesDB should work, for the latter you need to wait until the fix for the SSL issues (caused by changes to the site) are added in the stable branch.

As this topic has become an old dumpster bin for all issues regarding scraping (whatever they are), I'll be locking this down - it's not providing any useful solutions for anyone looking at it right now and the error that started the issue is long gone.