RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login

    Versatile C++ game scraper: Skyscraper

    Scheduled Pinned Locked Moved Ideas and Development
    skyscraperscrapergamelist.xmlscrapinggithub
    1.6k Posts 113 Posters 1.7m Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • muldjordM
      muldjord @iKarith
      last edited by muldjord

      @ikarith This actually seems to be a problem on ScreenScrapers end.

      But first, let me clear some things up. The aliasMap.csv file is only used by filename based search modules. ScreenScraper uses checksum hashes or exact filename searches and therefore makes no use of the aliasMap.csv at all. So that won't help you in this case.

      If ScreenScraper gives you back a faulty game, it is most likely because the checksum of your file or the exact filename, is connected to a wrong game inside of their database. This is not a rare thing to happen.

      My suggestion is to log on to their website and find the specific checksum or filename that matches your files, and see if they are incorrectly connected to the wrong versions of those games. You can then proceed to correct that on their website as well. When the correction has been approved, you can then proceed to rescrape the games.

      1 Reply Last reply Reply Quote 0
      • R
        retro81
        last edited by

        Hi @muldjord, first of all thanks for developing this amazing script for retropie!

        I have a problem when fetching covers from screenscraper.fr - if the game has a flyer (like Samurai Shodown for NeoGeo) then the box is composed using the game "flyer" instead of the "box2D" art, any ideas why this happens and how to correct it? I'm using the default artwork.xml

        Thanks!

        R muldjordM 2 Replies Last reply Reply Quote 0
        • R
          retro81 @retro81
          last edited by

          I actually found this was a feature implemented in 3.4.3, any way to fine-tune skyscraper to either use flyers or actual box art? I use the retropie script.

          1 Reply Last reply Reply Quote 0
          • R
            retro81
            last edited by

            Hi @muldjord, would it be possible to include the game hard/support media download as an option?

            I like the way our ScreenScraper.fr friends do their "recalbox-mixes" including the game support media, here's a sample: https://www.screenscraper.fr/medias/113/176930/mixrbv2(sp).png

            muldjordM 1 Reply Last reply Reply Quote 0
            • muldjordM
              muldjord @retro81
              last edited by muldjord

              @retro81 Hi, no, the ScreenScraper specific media won't be supported as they are unique to ScreenScraper and Skyscraper supports many different scraping sources. I don't want to turn Skyscraper into a ScreenScraper scraper.

              But you can create a very similar result using Skyscraper's artwork compositor. It's documented here by using the already supported media types and add some custom frames and masks.

              EDIT: A quick note: I am not currently active in developing Skyscraper beyond fixing bugs.

              R 1 Reply Last reply Reply Quote 0
              • muldjordM
                muldjord @retro81
                last edited by muldjord

                @retro81 In the current version it will fetch "flyer" for all arcade platforms including "neogeo". But I see that that might not be optimal. So I've removed "neogeo" from that list. From 3.6.7 it will use "box-2D" instead. Thanks for letting me know.

                EDIT: 3.6.7 out now with this change included. Please update your Skyscraper version and rescrape the neogeo platform with cache refresh enabled (important!).

                1 Reply Last reply Reply Quote 2
                • R
                  retro81 @muldjord
                  last edited by

                  @muldjord thank you for both responses!

                  1 Reply Last reply Reply Quote 1
                  • RionR
                    Rion
                    last edited by

                    @muldjord Would you consider adding support for Batocera?

                    FBNeo rom filtering
                    Mame2003 Arcade Bezels
                    Fba Arcade Bezels
                    Fba NeoGeo Bezels

                    muldjordM 1 Reply Last reply Reply Quote 0
                    • muldjordM
                      muldjord @Rion
                      last edited by muldjord

                      @rion Just did a Google search and I assume you mean Batocera Linux, right? I won't personally support it, but I don't see why it wouldn't work already. As long as it has Qt5 included and a build chain, you should be able to install it using the description at the Skyscraper github. How to get Qt5 on there is up to the user to figure out. Maybe it's in the Batocera package manager already (if there is such a thing, I have never heard of the distribution before just now).

                      RionR 1 Reply Last reply Reply Quote 0
                      • RionR
                        Rion @muldjord
                        last edited by

                        @muldjord You should really try it out. It's a great distribution both for sbc's but also for an x86_64 PC.

                        Skyscraper is not yet available in the Batocera pacman repository. But could probably be in the future.

                        What i wanted to know is if you would think/consider to implement support for more systems supported by Batocera. I'm completely fine if i could just use it on my Linux machine and scrape the roms there.

                        Cause i just love how the artwork looks with Skyscraper and there is no better alternative out there! 😃

                        https://wiki.batocera.org/emulators

                        FBNeo rom filtering
                        Mame2003 Arcade Bezels
                        Fba Arcade Bezels
                        Fba NeoGeo Bezels

                        muldjordM 1 Reply Last reply Reply Quote 0
                        • muldjordM
                          muldjord @Rion
                          last edited by muldjord

                          @rion If they are major platforms that are related to consoles or home computers I will consider it. But I do have a lot of requests to add obscure "homebrew" and or / sub-platforms that I do not wish to support as they are often either too obscure or already supported by "arcade" or others.

                          Mainly Skyscraper was made to support the RetroPie naming scheme and platforms. I have, however, added some platforms besides that that I felt made sense. I am quite restricting when it comes to adding more, as I don't want to bloat Skyscraper too much.

                          But if you have a couple of platforms that you feel could fall under the RetroPie umbrella, I'd need the same info as found on the RetroPie supported platform pages, which is:

                          • Platform name and shortname (eg. "Nintendo Entertainment System" / "nes")
                          • File extensions (*.this, *.that)

                          But no promises I'll add them. :)

                          1 Reply Last reply Reply Quote 0
                          • O
                            Obi-Wan
                            last edited by Obi-Wan

                            Not sure where to post this, but I felt it should be mentioned somewhere.

                            Scraping does not work on Windows with version 3.6.7, but works fine with the previous version, 3.5.8.

                            I was so happy to find Skyscraper because it seemed like a really nice tool, so I was a bit bummed out about the lack of Windows-support and even more so when I couldn't get it to work. But the second I tried the older 3.5.8 version scraping worked immediately.

                            I get the same behavior on two separate Windows systems. This reproduces the behavior on both my systems:

                            • Download Skyscraper and unzip to C:\work\
                            • Copy contents of the deploy folder to my user-folder
                            • Open Powershell in Skyscraper-folder (cmd has the same behavior)
                            • In PS, run: .\Skyscraper.exe -s thegamesdb -p snes -i "C:/work/ROMS"

                            3.6.7 returns Game not found
                            3.5.8 works fine.

                            I have no idea if I've missed something else here, but with 3.6.7 I got nowhere and with 3.5.8 everything seems to work as intended.

                            muldjordM 2 Replies Last reply Reply Quote 2
                            • muldjordM
                              muldjord @Obi-Wan
                              last edited by

                              @obi-wan I might look into this one of the coming days. And thank you for the detailed description, that is highly appreciated.

                              1 Reply Last reply Reply Quote 0
                              • muldjordM
                                muldjord @Obi-Wan
                                last edited by

                                @obi-wan Please redownload it from the same link. It was missing an OpenSSL dll that I've now included. I've tested it on my Windows dev machine and it works now. Let me know if it works for you too.

                                O 1 Reply Last reply Reply Quote 4
                                • O
                                  Obi-Wan @muldjord
                                  last edited by

                                  @muldjord Thanks a bunch! :) I was thrown off my emulator project for a while, but I'm back at it and 3.6.7 seems to work perfectly well now! Appreciate it, especially because the Windows version is not really supported!

                                  muldjordM 1 Reply Last reply Reply Quote 1
                                  • muldjordM
                                    muldjord @Obi-Wan
                                    last edited by

                                    @obi-wan Yeah, I don't really support it beyond releasing it. But I at least want it to work, so thanks for reporting that. I don't really use it myself so... :D

                                    1 Reply Last reply Reply Quote 1
                                    • muldjordM
                                      muldjord
                                      last edited by

                                      Quick show of hands: Would anyone be against me removing the "Simple mode" from Skyscraper? The questionnaire mode that it runs if you run Skyscraper without command line parameters. I haven't updated it in ages and it seems that it's currently broken. I never liked it, and I kind of want to remove it completely.

                                      Given that the RetroPie Skyscraper script is much better I feel like "Simple mode" is deprecated.

                                      ClydeC 1 Reply Last reply Reply Quote 4
                                      • ClydeC
                                        Clyde @muldjord
                                        last edited by

                                        @muldjord Since I never noticed it until you mentioned it now, I don't mind. 😉

                                        1 Reply Last reply Reply Quote 1
                                        • muldjordM
                                          muldjord
                                          last edited by muldjord

                                          Thanks guys. I will remove it in the next release, unless someone stops by within the next week with compelling arguments to keep it.

                                          1 Reply Last reply Reply Quote 0
                                          • johnodonJ
                                            johnodon
                                            last edited by

                                            I've been reading through the docs and haven't found my answer (or if it is possible)...

                                            I like to save all of my media to a single folder using file suffixes (like sselph scraper does). For example in /roms/arcade/images/ I would have:

                                            zaxxon-image.png
                                            zaxxon-video.mp4
                                            zaxxon-marquee.png
                                            zaxxon-launching.png
                                            

                                            Is this possible with Skyscraper?

                                            TIA.

                                            John

                                            muldjordM 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.