Scraping no longer working due to old api

Smeltie

As the title might suggest scraping is since 01-04-2019 no longer functional in retropie.
I as many others found this out the hard way by not being able to scrape.
We initially thought that it might be on thegamesdb's end like a server being down.

After having spoken one of the developers at thegamesdb it turns out they turned off their old/legacy api.
And switched to a new api.
The new api was released in august.

The developer from thegamesdb had this to say:
"Hi, we shutdown the legacy api"
"New api has been out since August
You'd need ask the scraper maintainer to update his code"
"As far as I can tell, retro pie guys knew about the new api (well the fact that they used the legacy api, means they knew, since legacy domain was changed when the new api was introduced)
So they either don't plan on supporting the new api, or should have work in place already
"

So my support question is:
is it being worked on? and can we expect it anytime soon.
i have alot of games that need to be scraped and would love to do that.

My retropie is the latest version and has ben updated yesterday 22:15 gmt+1 time

Thanks in advance.

(feel free to remove, disable or close this topic if not correct or not allowed)
(this is my first forum post here)

mitu

https://retropie.org.uk/forum/topic/20613/emulationstation-internal-scraper-problems - it's the first post in the Help & Support forum.

Smeltie

@mitu thanks for sharing, altho i thought i’d still make this with the question if this is going to get patched, or updated. Since i have had issues with those other scrapers before.

BuZz

@Smeltie I don't know anyone working on it. I'm more inclined to just remove the scraping from ES.

Smeltie

@BuZz
Oh, uhh you mean all scraping or just the built in one?

MosaicDragon

@Smeltie I'd hope(and presume) he's just refering to the built in scraper - which is pretty useless tbh.
I got skyscraper working it's a little effort but it's definitely more than adequate

BuZz

@Smeltie the built in one. Why would I remove the working ones? :-)

quicksilver

@BuZz that would be a shame. It was useful for scraping individual games or ones that don't match a specific list (dos games).

BuZz

@quicksilver I have zero interest in fixing it. Never used it. Patches welcome if you want to update it to use the new API.

quicksilver

@BuZz sorry I wasn't trying to come across demanding or anything. I appreciate all you do to let us end users enjoy retropie.

mitu

@BuZz I have something working with screenscraper.fr. Their API is a bit similar to the TheGamesDB's API, so it's simple to re-use the existing code and add a new scraper. It just needs more testing and I can submit a PR for review.

naddel81

being able to scrape games manually is crucial for the roms that have translations etc. and are not found through automatic scraping. I am currently missing several covers due to this on every system. would really appreciate the inbuilt scraper to be working again for its manual approach! that would be awesome!

Smeltie

@MosaicDragon i tried skyscraper but no matter what i try the output keeps outputting a compiled image consisting of the cover, screenshot and the wheel.
I have tried the different included xml files but those all give error’s so i don’t know what now 😅.
I’d just like to have covers only just like the built in scraper.

mitu

@Smeltie You could have asked in the Skyscraper topic and I'm sure somebody would have helped. Have you looked in the documentation to see how the artwork.xml works ?

Smeltie

@mitu have tried.
To anyone still looking i tried a tool called "Skraper"
And it works phenominally.
it can really do it all, from combined art, to video's to 2d boxart it does it all.
i highly reccomend everyone try it.
i found it to work best when running on a windows computer with the configs and roms folders shared and added as drives under windows 10.

naddel81

but skraper can't do it manually for missing games. :(

naddel81

@mitu said in Scraping no longer working due to old api:

@BuZz I have something working with screenscraper.fr. Their API is a bit similar to the TheGamesDB's API, so it's simple to re-use the existing code and add a new scraper. It just needs more testing and I can submit a PR for review.

that would be awesome!

BuZz

@mitu sounds great thanks. I think recalbox includes support for screenscraper.fr in their ES fork - might be useful for reference.

Failrunner

Sweet it will be harder to use now. I haven't cursed at this os enough. I need to blow my voice screaming! 🤒

naddel81

@Failrunner it will soon work again and maybe better than before with gamesDB.