Having trouble Scraping games

lesk

Re: Can't scrape from thegamesdb.net - known problem?

I found this topic and I updated the ES package. But the scraper still dosen't work.
Is there a way to change the scraper to use another source then thegamesdb?

Raspberry pie 3
retropie version: v4.4
ES version: v2.7.6RP
Connected to internet (updating ES etc)

mitu

@lesk said in Having trouble Scraping games:

Is there a way to change the scraper to use another source then thegamesdb?

Not at the moment, there isn't another scraping source implemented in Emulationstation. What ROM are you trying to scrape - can you give some examples ?

lesk

@mitu said in Having trouble Scraping games:

@lesk said in Having trouble Scraping games:

Is there a way to change the scraper to use another source then thegamesdb?

Not at the moment, there isn't another scraping source implemented in Emulationstation. What ROM are you trying to scrape - can you give some examples ?

The list is super long, but some examples are:
Wrecking Crew (nes)
Gremlins (nes)
Crash Team Racing (psx)

I also tried installing the Selphs scraper. It worked, for some games. But it renamed the games in the list as well.
Instead of King of Dragons , it now says King of Dragons (USA). On some ROMs it now says REV. Or 1.1. Or (JAPAN), (EUROPE) etc.

I did enjoy the old scraper, but I guess it's not working anymore?
Does anyone know when the default scraper is going to be working again?

I like how I could use it via the UI and then get a choice if there was multiple choices.

simpleethat

There are a couple of stand-alone scrapers that have worked pretty well for me. Check out Universal XML Scraper and Skraper

They both run in a Windows environment over your network and Skraper has a Linux distro as well. They will both require you to sign up for a free account with screenscraper.fr but I've found them to both work well and be fairly accurate (if not for some of the metadata of games being off (ie genre))

mitu

@lesk I just checked and the legacy API server that Emulationstation uses has been shutdown yesterday by the thegamesdb.net team yesterday. That's why it's not working anymore. For now, you'd have to use other scrapers (external) to scrape for data.

omen_peter

I agree with @lesk, I like how the internal scraper works, allowing you to manually pick if there are multiple choices (basically I like having total control - ha!).

Until I updated RetroPie to v4.4 I never really used it, but recently I have been using it to scrape a few games every now and then. Hopefully a future update to EmulationStation will get it working again! In the meantime I'm happy to wait.

che_don_john

@simpleethat said in Having trouble Scraping games:

There are a couple of stand-alone scrapers that have worked pretty well for me. Check out Universal XML Scraper and Skraper

They both run in a Windows environment over your network and Skraper has a Linux distro as well. They will both require you to sign up for a free account with screenscraper.fr but I've found them to both work well and be fairly accurate (if not for some of the metadata of games being off (ie genre))

Can you advise me on how I can scrape roms individually in Skraper? So far I can only get it to scrape all the roms for each system, meanign that it keeps overwriting previously scraped roms. I liked how Selph's would ignore already scraped roms - is that an option with Skraper?

simpleethat

@che_don_john I don't think it can natively do that but you can do what I do when I add games to my already scraped collections:

rename the default folder for the system I want to add games to (ie snes to snes2)
create a new folder with the default name (a new snes)
add the games I want to scrape to the new folder (snes)
open the gamelist.xml file in a text editor and copy all the info between <gameList> and </gamelist>
paste the copied data to the old gamelist.xml above </gameList>
copy the roms from the snes folder to snes2
delete snes
rename snes2 back to snes

It isn't the most elegant method and there might be a more efficient way but it works for me. One thing that I do really like about skraper is that it adds all the media and the gamelist folder to the selected system's roms folder instead of inside configs\emulationstation so when I want to build a different setup I only have to copy the roms folder over instead of having to move a bunch.

I hope that is clear enough, I'm not always the best at typing these things out. Let me know if you have any further questions.

lesk

@mitu said in Having trouble Scraping games:

@lesk I just checked and the legacy API server that Emulationstation uses has been shutdown yesterday by the thegamesdb.net team yesterday. That's why it's not working anymore. For now, you'd have to use other scrapers (external) to scrape for data.

Sorry for replying on a old thread now.
But it seems that they are giving out API keys now again. Maybe the developers of Retropie could get one?
Or is it possible to change the key yourself inside Retropie?

mitu

@lesk said in Having trouble Scraping games:

But it seems that they are giving out API keys now again. Maybe the developers of Retropie could get one?

I don't think they stopped 'giving keys' at any moment.

Or is it possible to change the key yourself inside Retropie?

No, it's not possible because the version of the scraper that didn't work was not using any key for scraping.

In the mean time, a new version of the scraper has been added to Emulationstation, if you update now your package it should work again.

lesk

@mitu said in Having trouble Scraping games:

@lesk said in Having trouble Scraping games:

But it seems that they are giving out API keys now again. Maybe the developers of Retropie could get one?

I don't think they stopped 'giving keys' at any moment.

Or is it possible to change the key yourself inside Retropie?

No, it's not possible because the version of the scraper that didn't work was not using any key for scraping.

In the mean time, a new version of the scraper has been added to Emulationstation, if you update now your package it should work again.

Sorry for digging up old post. But haven't been able to play with my pie until now xD

I tried updating but the scraper is still not working. What version of Emulationstation is working? I went into config and press update. I also went into manage packages and selected Emulationstation and updated that. Still not able to scrape.

mitu

@lesk said in Having trouble Scraping games:

What version of Emulationstation is working?

Versions 2.8.2 should work - this is the latest.

RyanT

@mitu

Just updated to the latest version 2.8.3RP

Still not working. With gamesDB or screenscraper

Any ideas?

mitu

@RyanT Please give more details about your setup - https://retropie.org.uk/forum/topic/3/read-this-first. What game/system are you trying to scrape and it's not working ? Give me a concrete example.