Scraping no longer working due to old api
-
@BuZz sorry I wasn't trying to come across demanding or anything. I appreciate all you do to let us end users enjoy retropie.
-
@BuZz I have something working with screenscraper.fr. Their API is a bit similar to the TheGamesDB's API, so it's simple to re-use the existing code and add a new scraper. It just needs more testing and I can submit a PR for review.
-
being able to scrape games manually is crucial for the roms that have translations etc. and are not found through automatic scraping. I am currently missing several covers due to this on every system. would really appreciate the inbuilt scraper to be working again for its manual approach! that would be awesome!
-
@MosaicDragon i tried skyscraper but no matter what i try the output keeps outputting a compiled image consisting of the cover, screenshot and the wheel.
I have tried the different included xml files but those all give error’s so i don’t know what now 😅.
I’d just like to have covers only just like the built in scraper. -
@Smeltie You could have asked in the Skyscraper topic and I'm sure somebody would have helped. Have you looked in the documentation to see how the
artwork.xmlworks ? -
@mitu have tried.
To anyone still looking i tried a tool called "Skraper"
And it works phenominally.
it can really do it all, from combined art, to video's to 2d boxart it does it all.
i highly reccomend everyone try it.
i found it to work best when running on a windows computer with the configs and roms folders shared and added as drives under windows 10. -
but skraper can't do it manually for missing games. :(
-
@mitu said in Scraping no longer working due to old api:
@BuZz I have something working with screenscraper.fr. Their API is a bit similar to the TheGamesDB's API, so it's simple to re-use the existing code and add a new scraper. It just needs more testing and I can submit a PR for review.
that would be awesome!
-
@mitu sounds great thanks. I think recalbox includes support for screenscraper.fr in their ES fork - might be useful for reference.
-
Sweet it will be harder to use now. I haven't cursed at this os enough. I need to blow my voice screaming! 🤒
-
@Failrunner it will soon work again and maybe better than before with gamesDB.
-
Can someone please make instructions on how to get the internal scraper "THEGAMEDB" working again? I have one game I need to scrape and I tried Skyscraper and Steven Selph's Scraper and it just didn't work out for me. I kept getting a bash error in the terminal with Skyscaper. With Steven Selph's Scraper I kept getting "It appears that thegamesdb.net isn't up. If your are sure it is use -skip_check to bypass this error. No sources available
-
@Nikorasu081 just look at the help and support section. Every other post today has been about scraping - in summary the in-built scraper isn't currently working, other alternatives are available, and there is a will it/won't it be fixed debate.
Personally, I'm surprised I've not seen it called "scrapping" today. Always laugh when I see that someone has tried to "scrap" games. At the end of the day fighting never solved anything.
-
@omen_peter As you can see in my above post nothing worked for me.
-
we have to wait for @mitu to fix the scraper. really looking forward to it.
-
This post is deleted! -
As simple mortal end user, I just wanted to add my 2 cents in and say that the work of developers is much appreciated!
I also wanted to chime in and say that I also appreciate the built-in scraper for being able to manually search and add individual games. I usually use the Selph scraper to take care of the bulk, then manually add the ones it doesn't find.
Thank you!
-
@BuZz While I understand that the internal scraper may be far from the best tool, it is the only one that I've found that works (worked) fantastic for scraping just a single ROM that I've added to my list. As someone who is a bit less on the technical side, having this tool has been a great help in many situations. Even if it is not high on the priority list to fix, I for one would be greatly appreciative of it was somewhere on the list down the road. Thanks for all the time and hard work put into this!
-
Since there seem to be a lot of people interested in this feature I would suggest waiting to see what mitu is able to come up with. He stated he is working on a fix to have the internal scraper switched to a different website. If that fix doesn't come then those that want it should probably consider putting together a bounty to get it fixed. This is a good way for those of us with no coding skills to motivate those who have the skills.
-
@quicksilver For now, I don't think a bounty is needed. A couple of patches have been posted on the ES RetroPie repository to enable scraping again - one of them is mine -, they're just waiting for review and merge. I avoided adding an update on the forum so that the github page wouldn't be paged with unnecessary 'please fix/me too' messages.
Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.
Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.