Unable to scrape, unable to update
-
Hi,
I'm extremly new on the whole topic and have nearly zero experiences with linux-systems and/or retropie. So I come here, hoping for some help.A few days ago I set up retropie on the raspberry pi 4 according to the tutorial of "ETA PRIME" (https://www. youtube. com/watch?v=L5Pz60iHtL8) and so far it went well. However, recently the problems have been accumulating.
After I have successfully scraped hundreds of games, it seems that the scraper hasn't worked for about three or four days. The following error message is displayed: "lvl0: ScreenScraperRequest - Error Parsing XML. No document element found"It doesn't matter if I use the internal scraper or Selph's scraper, the result is the same. Strangely enough, the gamelist. xml files are also not renewed itself when I rename or move them out of the folders.
Besides, I don't seem to be able to update retropie. When I try to update via the configuration the following error messages appear:
- Error running 'git checkout master' - returned 128
- Error running 'git pull' - returned 128
- Error running 'git submodule update --init -- recursive' - returned 128
The whole topic is really new to me and I can't understand anything of the error messages.
I really thank you for every help.About my pi:
Pi Model or other hardware: Pi 4 Model B
Power Supply used: Case and power supply https://smile.amazon.de/gp/product/B07Z3T58BF/ref=ppx_yo_dt_b_asin_title_o02_s03?ie=UTF8&psc=1
RetroPie Version Used: 4.5.18
Built after: ETA PRIME Tutorial https://www. youtube. com/watch?v=L5Pz60iHtL8
USB Devices connected: XBOX 360 Controller, mouse, keyboard -
Ive tried scraping games tonight and nothing has worked for me! Not sure whats going on.
-
The tutorial you followed is not a supported method to install RetroPie on a Pi4 - it's actually giving now a wrong installation method.
As noted in the https://retropie.org.uk/forum/topic/3/read-this-first topic, work on the PI4 is ongoing and problems might arise from time to time.If you'd like to try RetroPie on the Pi4, then follow the manual installation method in the Docs and install using a Raspbian Lite image.
-
@ElrasM said in Unable to scrape, unable to update:
Hi,
I'm extremly new on the whole topic and have nearly zero experiences with linux-systems and/or retropie. So I come here, hoping for some help.
A few days ago I set up retropie on the raspberry pi 4 according to the tutorial of "ETA PRIME" (https://www. youtube. com/watch?v=L5Pz60iHtL8) and so far it went well.I'm running raspberry pi 3b + on the latest patch level. Skyscraper and Scraper stopped working for me about 2 days ago as well. I get the same error message. I noticed it said their API's are not available or has too many threads. I ran skyscraper to update the videos for a few new games I added and it wiped out the game list for some of my emulators because it didn't pull back anything. I don't think it has anything to do with your setup, I think the issue is their servers are having problems. It had been working for a long time for me until about 2 days ago. I have 2 builds and it doesn't work on either anymore.
-
@ULTRA-MAGNUSS
I've been unable to scrape too. I've read on other forums that the servers are having issues. Hopefully, this will be corrected soon. -
Thank you for your replies.
Good to hear it's not just me.
Maybe the scrapers won't actually work at the moment. I'll might try again in a few days.Unfortunately, I've already spent several hours working on the Pi, configuring emulators, mildly overclock, making settings, setting up Raspian, etc.
Wouldn't a new installation after https://retropie. org. uk/forum/topic/3/read-this-first lead to the fact that everything would be lost?
Does anyone know a reason for the error messages when trying to update? Maybe a solution as well.Thanks again.
-
i have also been unable to scrape with the built-in scrapers for the past few days. hopefully these do start working again soon but in the meantime i've used this method to manually add box art:
https://www.reddit.com/r/RetroPie/comments/3n032p/how_can_i_manually_add_cover_art_to_my_games/worked great!
-
@ElrasM Re-installing would wipe out everything. You can backup your current image which would save everything you currently have (all images, settings, etc). However, if you are re-installing for the purpose of the scraper, I wouldn't for now since everyone is having the same problem with the scraper.
-
Ok, thanks again.
It's a little bit frustrating that all the effort put in the retropie until now should be for nothing. But you may be right, a clean install might be a better start for future issues.
I will try the tutorial posted here.
And for the scraper i maybe will wait or try the reddit tutorial.
-
@ElrasM said in Unable to scrape, unable to update:
Wouldn't a new installation after https://retropie. org. uk/forum/topic/3/read-this-first lead to the fact that everything would be lost?
You can back-up your games/configs/artwork - https://retropie.org.uk/docs/Updating-RetroPie/#making-a-backup-option-2 - and transfer back after the re-installation.
Does anyone know a reason for the error messages when trying to update? Maybe a solution as well.
Well, you have a solution, though it's not the one you expected. Follow the manual installation method outlined in the Docs and don't follow - grossly - outdated and incorrect videos.
-
@mitu said in Unable to scrape, unable to update:
You can back-up your games/configs/artwork - https://retropie.org.uk/docs/Updating-RetroPie/#making-a-backup-option-2 - and transfer back after the re-installation.
That's a great idea. I'm on it.
Well, you have a solution, though it's not the one you expected. Follow the manual installation method outlined in the Docs and don't follow - grossly - outdated and incorrect videos.
Yeah, you are right. But in this case YouTube brought me to the idea of using retropie. So it was my first source.
Btw: It's crazy that some people sell whole images for the Ras Pi 4 with pre installed roms on ebay for a plenty amount of money. Giving the feeling that setting up retropie is a money worth work.
-
@ULTRA-MAGNUSS said in Unable to scrape, unable to update:
I ran skyscraper to update the videos for a few new games I added and it wiped out the game list for some of my emulators because it didn't pull back anything
This makes no sense to me. Skyscraper doesn't wipe a gamelist. It has no mode that would do that unless it runs from an empty cache. And the cache can only be cleared / purged by the user.
-
Screenscraper currently is only letting registered users scrape. Retropie's scraper does not pass Screenscraper credentials to their website, so you'll get an error. This has all happened within the last couple days.
-
use the pc version of the scraper it is 100% faster,it can scrape 2k roms in 30minutes or 20minutes and if you scraped in raspberry pi it will take half a day or whole day.
-
@simonc said in Unable to scrape, unable to update:
use the pc version of the scraper it is 100% faster,it can scrape 2k roms in 30minutes or 20minutes and if you scraped in raspberry pi it will take half a day or whole day.
For both Skyscraper and probably SSelph's scraper, this is incorrect. This might be true when using the EmulationStation built-in scraper compared to Skraper on Windows when using ScreenScraper as the source. For Skyscraper at least it is way faster when you have entered your ScreenScraper credentials correctly in the configuration. We're talking about 2000 roms in 30 minutes or maybe even less.
-
I got the skyscraper to work by registering a username and password on https://www.screenscraper.fr/ (i had to use IE, not google chrome). Then I edited my /opt/retropie/configs/all/skyscraper and edited my config.ini file. I edited the lines with the userCreds with my username and password and the line above it.
[screenscraper]
userCreds="MyUser:MyPassword"Make sure you remove the # that's in front. Save it. You can only run the SkyScraper from command line. The scraper in Emulation Station still doesn't work.
-
Referring to my original post: The new install as descriped in docs fixed my problems and almost all errors. (I will make a seperate post for that)
Scraping seems to work with skycraper although, as mentioned here, GamesDB gives me a limited amount of scrapes per month and screenscraper.fr needs a registration.
Thank you for your help.
-
@ULTRA-MAGNUSS I followed your instructions, got an account and modified the config.ini file with my userId and password, and uncommented out the line, plus added the [screenscaper] line above it.
I'm running skyscraper from the retropie-setup.sh.
But when the job is running, I'm getting the error "The screenscaper serviced is currently closed or too busy to handle requests from unregistered and inactive users..."
Is there something else that I am missing?
Thanks.
-
@corminos said in Unable to scrape, unable to update:
Is there something else that I am missing?
Can you post exactly the what you entered in the
config.inifile (minus the password) ? -
@mitu
I found the problem. I had misspelled "screenscraper". I was looking over what I had pasted just to make sure I didn't look like an idiot...and realized I'm an idiot.The bottom section is below for anyone wanting to look, where XXXXX is my user name, and YYYYYY is my password. I uncommented out the videos and set the value to "true".
#[<SCRAPING MODULE, eg 'screenscraper'>]
[screenscraper]
userCreds="XXXXX:YYYYYYY"
#threads="1"
#minMatch="0"
#maxLength="10000"
#interactive="false"
#cacheCovers="true"
#cacheScreenshots="true"
#cacheWheels="true"
#cacheMarquees="true"
videos="true"
#videoSizeLimit="42"
Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.
Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.