Versatile C++ game scraper: Skyscraper

muldjord · 26 Dec 2019, 22:37

@steptoe said in Versatile C++ game scraper: Skyscraper:

Just wondering if you have any plans to add support for pinball in your scraper

No current plans, no.

I'm aware not many sites have pinball data on their videos or images. emumovies seems to be about the only one that has some videos/images

Skyscraper doesn't support emumovies as they don't have an open API. At least not to my knowledge. I've been looking for one but never found one. It has to be some kind of REST-based API, not just an ftp with images or something like that.

muldjord · 26 Dec 2019, 22:38

@Clyde said in Versatile C++ game scraper: Skyscraper:

@muldjord Happy holidays! I recently noticed a wrongly scraped game. Would it help if I reported this to you? If so, how should I do it?

That should be reported to the module that delivered the wrong data. I suggest registering with them and fixing it yourself if you can. That's the power of community based game info databases.

muldjord · 26 Dec 2019, 22:44

@steptoe said in Versatile C++ game scraper: Skyscraper:

I'm trying to compiler the latest Skyscraper 3.3.5 for Windows

The howto describes how to compile it under Windows. You shouldn't use QT Creator to do that, just the Qt command prompt as the text you pasted also tells you to. But you're basically on your own here. I don't use Windows myself and quite frankly don't have much interest in that system. Sorry.

Remember that Skyscraper is licensed under the GPL if you end up using any of the Skyscraper code for any of your projects. :) Good luck with it!

Clyde · 27 Dec 2019, 14:38

@muldjord said in Versatile C++ game scraper: Skyscraper:

That should be reported to the module that delivered the wrong data. I suggest registering with them and fixing it yourself if you can. That's the power of community based game info databases.

Ah, I thought that Skyscraper mistook one game for another (but on the second thought, only getting the video wrong then is illogical). I didn't think about checking the data on screenscraper.fr, though. And there it is: the wrong video (from Dangerous Xmas instead of Zettaizetsumei Dangerous Jiisan). I'll report it immediately, I have already an account there. edit: Done.

Thanks for pointing me to something obvious, I plead insanity because of a heavy cold I'm having at the moment. 🥴

Clyde · 1 Feb 2020, 16:36

FYI: TheGamesDB is back online, but it needs a prefix /v1/ to work now.

(Thanks to @maserspark for reporting this.)

muldjord · 1 Feb 2020, 20:01

@Clyde Thanks for the update. It seems they are running from an old backup, but I don't quite understand what they mean by

the API is also back online,
but using the prefix /v1/ which is both to get devs to explicitly make sure old data doesn't just crash their apps,
but this prefix will remain as our main endpoint for this api going forward

Sounds to me like it's only the old API that's online but it doesn't say so specifically.

Either way, great to see that they are slowly getting back online.

Clyde · 2 Jan 2020, 21:42

@muldjord Indeed, it is great. Do you have to update SS to include the prefix?

muldjord · 1 May 2020, 11:09

@Clyde Probably not if it's just the old API (which I think it is). I think we'll just have to wait this out until it's back online in v2.

EDIT: After re-reading their message a couple of times I think I get it. The 'v1' will stay, even when they get a more recent backup imported. So I've added the 'v1' prefix, and release 3.3.6.

muldjord · 5 Jan 2020, 11:08

Skyscraper 3.3.6 released: https://github.com/muldjord/skyscraper

Skipped file now includes platform in the filename (Thanks to 'parker-hemphill' for suggesting this)
Now handles the '****' error message from ScreenScraper (Thank you to 'GiulioCentorame' for reporting this)
Updated 'thegamesdb' to include the 'v1' prefix (Thank you to 'Clyde' for letting me know)

This release fixes the 'thegamesdb' module, but be aware that their database is currently running on an old backup due to the December server shutdown. Thank you to @Clyde for letting me know about the 'v1' prefix. In other words: Updating to this release will make 'thegamesdb' work with Skyscraper again, but with old data. They are planning to get back to a more recent version soon, but it will still require the v1 prefix as I understand it, so it will work regardless.

steptoe · 12 Jan 2020, 08:55

@muldjord said in Versatile C++ game scraper: Skyscraper:

Skyscraper 3.3.6 released: https://github.com/muldjord/skyscraper

Skipped file now includes platform in the filename (Thanks to 'parker-hemphill' for suggesting this)

Now handles the '****' error message from ScreenScraper (Thank you to 'GiulioCentorame' for reporting this)

Updated 'thegamesdb' to include the 'v1' prefix (Thank you to 'Clyde' for letting me know)

This release fixes the 'thegamesdb' module, but be aware that their database is currently running on an old backup due to the December server shutdown. Thank you to @Clyde for letting me know about the 'v1' prefix. In other words: Updating to this release will make 'thegamesdb' work with Skyscraper again, but with old data. They are planning to get back to a more recent version soon, but it will still require the v1 prefix as I understand it, so it will work regardless.

Your unofficial Windows version is on 3.30. I'm trying to compile the latest versions but not 100% what I need to compile. You give basic compile instructions for OSX and Linux but not Windows so can't really try SkyScraper fully on the PC. Thanks

muldjord · 1 Dec 2020, 10:46

@steptoe It's not unofficial, it's just not officially supported (I don't use Windows). I know it's old, for that very reason. If I get the time today, I will create a 3.3.7 version of the Windows version.

EDIT: Windows version now updated to 3.3.7. Get it here

o0alucard0o · 12 Jan 2020, 10:54

@muldjord

Thanks so much for this fantastic scraper ;-)

I'm trying to scrape igdb, but with no results

For example

Skyscraper -p steam -i /Volumes/Games/Roms/Steam -g "/Volumes/Games/Attract-Mode/romlists_b" -o "/Volumes/Games/Attract-Mode/scraper/Steam_b" -f attractmode -e "/Users/matteo/Games/skyscript-nuc-one/Steam.cfg" --addext "*.bat" --nosubdirs --nobrackets --unattendskip -s igdb -u IGDBKEY --pretend --verbosity 3

--- The following is a list of skipped games ---
'Cuphead', No returned matches
'GRIS', No returned matches
'Hollow Knight', No returned matches
'Monster Boy And The Cursed Kingdom', No returned matches
'My Friend Pedro', No returned matches
'Neon Chrome', No returned matches
'Nex Machina', No returned matches
'Ori and the Blind Forest Definitive Edition', No returned matches
'PinballArcade', No returned matches
'TrackMania Nations Forever', No returned matches

I'm able to find game through web interface

(sorry for my english)

muldjord · 1 Dec 2020, 11:13

@o0alucard0o I just tried with "My Friend Is Pedro" and indeed something seems to be wrong... I'll look into it, stay tuned.

o0alucard0o · 12 Jan 2020, 11:17

@muldjord Unfortunately I get the same result with "pc".

muldjord · 1 Dec 2020, 11:27

@o0alucard0o I've found the error, I'll release 3.3.8 with this fixed in a minute.

EDIT: 3.3.8 now released, please update and try again.

o0alucard0o · 12 Jan 2020, 11:58

@muldjord now works! thank you :-)

muldjord · 12 Jan 2020, 12:04

@o0alucard0o And thank you for reporting it! :)

steptoe · 1 Dec 2020, 17:24

@muldjord said in Versatile C++ game scraper: Skyscraper:

@steptoe It's not unofficial, it's just not officially supported (I don't use Windows). I know it's old, for that very reason. If I get the time today, I will create a 3.3.7 version of the Windows version.

EDIT: Windows version now updated to 3.3.7. Get it here

Any chance of explaining briefly how you compiled it, if you don't use windows, that way you don't have to compile a version for us windows users for every release. Now you are on 3.3.8 already

Thanks for the quick response and 'unofficial' windows version

muldjord · 12 Jan 2020, 18:21

@steptoe It's way too much to cover I'm afraid. On Linux and macOS it's quite simple due to their relatively standardized nature. Not so much for Windows. I will try to update it semi-regularly though. 3.3.8 is on there now.

jrbbib · 1 Dec 2020, 23:58

i have a question as to the location of all the stored files. would it be possible to be able to move it to where the roms ,bios, and themes folders are? when i connect to pi with windows pc this folder is what i can access. so when i make a backup it would be right there with the main files and would make transfer easier.