Versatile C++ game scraper: Skyscraper
-
@muldjord said in Versatile C++ game scraper: Skyscraper:
You need to run the game list generation pass after gathering the resources, please read the docs, it's explained there. This was also the case with 3.0.1.
Okay, that's weird. I haven't even seen "gamelist generation" mentioned in the RetroPie-Setup script menu from where I initiate the scraping. I though I'd be handled automatically? With Skyscraper 3.0.1 I could delete gamelist.xml, enter RetroPie-Setup, perform the scraping and the gamelist.xml would be generated.
But I'll read the docs and see if I can get a handle on it.
I'll check that one example you mention and see if I can find out why it won't work.
EDIT: Checked the Double Dragon file you mentioned and that particular file does not exist in their database for the NES platform (at leas I can't find it, so please provide a direct link to the NES version of this exact file if you did find it). So that is the reason. Whatever UXS does I can't tell you why it recognizes it. From the screenscraper APIv2, I can't get a hit on that file when it doesn't exist. I'm guessing this is the case with all of the remaining files you have trouble with.
No idea why you can't get a hit on that file. Just search from the screenscraper.fr front page with the MD5 hash and it'll list a single result for NES: https://www.screenscraper.fr/rominfos.php?romid=265259
-
@Brunnis said in Versatile C++ game scraper: Skyscraper:
With Skyscraper 3.0.1 I could delete gamelist.xml, enter RetroPie-Setup, perform the scraping and the gamelist.xml would be generated
If you chose the "generate game list(s)" option yes. Not while scraping.
I see the file now. It is odd that it won't recognize it. I even tried renaming it to match the same name, but still no luck. I have no explanation for this currently, but I'll dig a bit deeper. Thanks.
-
@muldjord said in Versatile C++ game scraper: Skyscraper:
@Brunnis said in Versatile C++ game scraper: Skyscraper:
With Skyscraper 3.0.1 I could delete gamelist.xml, enter RetroPie-Setup, perform the scraping and the gamelist.xml would be generated
If you chose the "generate game list(s)" option yes. Not while scraping.
I had apparently not updated the RetroPie-Setup script in a few weeks. I see now that the whole menu has been redone. Problem solved.
I see the file now. It is odd that it won't recognize it. I even tried renaming it to match the same name, but still no luck. I have no explanation for this currently, but I'll dig a bit deeper. Thanks.
Okay, thanks.
-
@muldjord I figured out why now. Stay tuned...
EDIT: Yes, this is because of un-escaped amperands in the returned xml. The file actually returns a result, but my parser quits because it doesn't see valid xml. I've reported this to the Screenscraper team, hopefully they will be interested in fixing it. If they do it will work without updating Skyscraper.
-
@muldjord said in Versatile C++ game scraper: Skyscraper:
@muldjord I figured out why now. Stay tuned...
EDIT: Yes, this is because of un-escaped amperands in the returned xml. The file actually returns a result, but my parser quits because it doesn't see valid xml. I've reported this to the Screenscraper team, hopefully they will be interested in fixing it. If they do it will work.
Nice that you found it! That's a pretty major issue... Yep, let's hope they'll fix it.
-
@Brunnis I've worked around the invalid xml by simply fixing it internally before I parse it. I've just released Skyscraper 3.0.3 where this is fixed. Please try it out with the roms you are having trouble with and let me know how it goes. I've already tested it with the Battletoads file and that worked brilliantly.
-
@muldjord Awesome, now it found every single ROM I have! :-D For some reason, it missed a lot of games when running the first time, but then I immediately ran it again and it found all of them. Perhaps communication issues with screenscraper.fr?
Anyway, thanks for fixing this so quickly! With these issues fixed, I guess a lot of people will notice an improvement.
-
@Brunnis Yes, I am currently going over some of my previously skipped files aswell, and a lot of them are now getting recognized. This is really cool, so thank you for being persistent in getting me to look at this.
EDIT: Btw, for anyone using the 'screenscraper' module in Skyscraper, do consider supporting them on patreon or by contributing data to their database. They are doing an amazing job with the new APIv2 (which Skyscraper uses). Same goes for the RetroPie project, please consider supporting them aswell.
-
@muldjord No problem, glad I could help! :-)
-
I find that I can't rescrape certain games that previously worked with screenscraper. For example:Never mind, it looks like it was a temporary problem with screenscraper. It seems to be working ok for me now.
-
@muldjord Is Skyscraper down ? I had been using it via command line and tried to update it today to the lastest version and got an error, then I uninstalled it and tried to re-install it and the same error popped up not allowing it to install from scratch.This is the error:
compilation terminated.
Makefile:748: recipe for target 'skyscraper.o' failed
make *** (skyscraper.o) Error 1I then tried to install it via Retropie Setup and received the error
Could not successfully build skyscraper -Scraper for Emulationstation by Lars Muldjord.
(/home/pi/Retropie Setup/tmp/build/skyscraper/Skyscraper not found) -
@ByteThis Can you post the complete error log on pastebin.com - you should find the complete log file in
~/RetroPie-Setup/logs. -
@mitu Sure here is the one from trying to install via Retropie Setup : https://pastebin.com/r3Sc4CXX
I deleted the one from the command line but I will reproduce the error and paste it here.
Here is the second log : https://pastebin.com/TeGYj0bD
-
@ByteThis Looks really odd. Seems like it is missing a qt5 dependency. It can't find the QStorageInfo headers. This works just fine on my own Pi 3B+. Just for the heck of it, try running:
$ sudo apt-get install qt5-defaultAnd then try installing it again.
-
-
@mitu This build still on jessie. I will try muldjord's suggestion.
@muldjord This is what the output after running that command :
https://pastebin.com/UqrXU2bB -
@ByteThis I don't think it will help - the install script already installs that package and it's probably already installed from the first time you added Skyscraper.
-
-
@ByteThis Yeah, I have no idea why this would happen... Only thing that springs to mind is a failing SD card. I haven't seen this error before and, as I mentioned, it compiles just fine on my own Pi.
-
@muldjord Weird..my sd card is fine as it is only about a month old and have been adding roms and such fine with it. Is your newest version of Skyscraper compatible with jessie ? (I imagine it is)
Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.
Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.