RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login

    Versatile C++ game scraper: Skyscraper

    Scheduled Pinned Locked Moved Ideas and Development
    skyscraperscrapergamelist.xmlscrapinggithub
    1.6k Posts 113 Posters 1.6m Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • LineOf7sL
      LineOf7s
      last edited by

      I've never had much luck scraping my Amiga games despite all the improvements made recently. With the release of v3.0.0, what's the best way for me to remove anything I've scraped in the past and start again from scratch? FS-UAE Arcade on Windows also scrapes from OpenRetro.org and I'm hoping for something approaching that level of success (which I'm sure SkyScraper can do).

      muldjordM 1 Reply Last reply Reply Quote 0
      • muldjordM
        muldjord @LineOf7s
        last edited by

        @LineOf7s Best approach is simply to run the following commands:

        $ Skyscraper -p amiga -s openretro --refresh
        $ Skyscraper -p amiga
        

        The openretro module has been improved a bit, so you might have luck with that. With that said, I would strongly recommend using the RetroPlay lha files with Amiberry. It turns the Amiga into almost-a-console in that it uses the WHDLoad files. And Skyscraper scrapes these really well with OpenRetro since I've worked behind-the-scenes with some really cool guys to make use of OpenRetro's uuid's for these games.

        megaJekanM LineOf7sL 2 Replies Last reply Reply Quote 2
        • megaJekanM
          megaJekan @muldjord
          last edited by

          Finally figured out how to use skyscrapper in command line mode. Unfortunately, when I scrap single roms, it always writes in log that "Pretend is set" (event if I set it to false in config.ini). So my custom query works, the game is found, but gamelist.xml is not updated. When I don't specify a filename, it goes through all roms per platform and works fine.

          mituM muldjordM megaJekanM 3 Replies Last reply Reply Quote 0
          • mituM
            mitu Global Moderator @megaJekan
            last edited by

            @megaJekan The new 3.0.0 version no longer updates your gamelist.xml after gathering the info, you need to run the scraper again to generate the gamelist(s) - as indicated by @muldjord above - adding just the platform parameter.

            1 Reply Last reply Reply Quote 1
            • muldjordM
              muldjord @megaJekan
              last edited by

              @megaJekan And you can just remove the pretend from your config again. It isn't relevant.

              HalvhjearneH 1 Reply Last reply Reply Quote 0
              • megaJekanM
                megaJekan @megaJekan
                last edited by

                @mitu said in Versatile C++ game scraper: Skyscraper:

                @megaJekan The new 3.0.0 version no longer updates your gamelist.xml after gathering the info, you need to run the scraper again to generate the gamelist(s) - as indicated by @muldjord above - adding just the platform parameter.

                Thanks. I am still using the old version. The problem was that I haven't read the documentation for scraping single roms carefully enough: "It DOES NOT update it in your game list. To do so you need to regenerate the game list by simply leaving out the -s option entirely like so Skyscraper -p [platform]."

                1 Reply Last reply Reply Quote 0
                • HalvhjearneH
                  Halvhjearne @muldjord
                  last edited by

                  @muldjord

                  i just updated and the first thing i noticed is that the menu in retropie setup is now missing the refresh and purge options and it seems it is just set to always refresh the db.

                  also my config.ini did not migrate propper and i had to manually edit it after first run.

                  other than that, it seems to work as expected so far, alltho i would love to see a skipped file for each console, so it does not overwrite and only shows last run, if i choose to use the menu or automated bach script.

                  mituM muldjordM 2 Replies Last reply Reply Quote 0
                  • mituM
                    mitu Global Moderator @Halvhjearne
                    last edited by

                    @Halvhjearne said in Versatile C++ game scraper: Skyscraper:

                    i just updated and the first thing i noticed is that the menu in retropie setup is now missing the refresh and purge options and it seems it is just set to always refresh the db.

                    They're not missing, they have their sub-menu now and the purge options have been moved in the Cache options sub-menu.
                    The refresh part depends on your settings - it has also been moved in the Cache options sub-menu, but if you set it to 'always refresh', then the configuration is kept.

                    HalvhjearneH 1 Reply Last reply Reply Quote 0
                    • HalvhjearneH
                      Halvhjearne @mitu
                      last edited by

                      @mitu

                      ahh, i see ... it just almost crashed, when i went there first time, so i killed it and thought it was a menu error or something.

                      i guess its processing the db, when you enter that menu.

                      mituM 1 Reply Last reply Reply Quote 0
                      • mituM
                        mitu Global Moderator @Halvhjearne
                        last edited by

                        @Halvhjearne said in Versatile C++ game scraper: Skyscraper:

                        i guess its processing the db, when you enter that menu.

                        It's just calculating the space occupied by the cache - how large is your cache folder ?

                        HalvhjearneH 1 Reply Last reply Reply Quote 0
                        • HalvhjearneH
                          Halvhjearne @mitu
                          last edited by

                          @mitu

                          its about 52gb. :)

                          1 Reply Last reply Reply Quote 0
                          • muldjordM
                            muldjord @Halvhjearne
                            last edited by

                            @Halvhjearne said in Versatile C++ game scraper: Skyscraper:

                            also my config.ini did not migrate propper and i had to manually edit it after first run.

                            What exactly didn't migrate properly? This would be pretty useful to me.

                            HalvhjearneH 1 Reply Last reply Reply Quote 0
                            • robertvb83R
                              robertvb83
                              last edited by

                              @muldjord i entered my Login and Password for screenscraper in the config.ini and defined 5 threads. Nevertheless during gathering only 1 thread is used
                              (i started gathering via the gui)

                              My full size arcade cabinet Robotron vs. Octolyzer

                              muldjordM 1 Reply Last reply Reply Quote 0
                              • HalvhjearneH
                                Halvhjearne @muldjord
                                last edited by

                                @muldjord

                                i think it was brackets, hints and Resize options, but im unsure right now.

                                maybe its cause i did not use the default layout and instead of just uncommenting lines i added extra lines, cause i like to see the what the default option/line looked like.

                                muldjordM 1 Reply Last reply Reply Quote 0
                                • muldjordM
                                  muldjord @robertvb83
                                  last edited by

                                  @robertvb83 Just set the login, it will automatically set it to the max number of threads that are allowed using your credentials. No need to set the number of threads yourself.

                                  You did set it like so I assume:

                                  [screenscraper]
                                  userCreds="USERID:PASSWD"
                                  
                                  robertvb83R 1 Reply Last reply Reply Quote 0
                                  • muldjordM
                                    muldjord @Halvhjearne
                                    last edited by muldjord

                                    @Halvhjearne It should replace all instances of noHints to hints and reverse the bool and the same for the others no matter what line they are on. You can check if it has automigrated it by looking for a config.ini.oldfile which contains the old config.

                                    1 Reply Last reply Reply Quote 0
                                    • robertvb83R
                                      robertvb83 @muldjord
                                      last edited by

                                      @muldjord said in Versatile C++ game scraper: Skyscraper:

                                      @robertvb83 Just set the login, it will automatically set it to the max number of threads that are allowed using your credentials. No need to set the number of threads yourself.

                                      You did set it like so I assume:

                                      [screenscraper]
                                      userCreds="USERID:PASSWD"
                                      

                                      this is what i did…
                                      config.ini.JPG

                                      how can i cancel the gathering?

                                      My full size arcade cabinet Robotron vs. Octolyzer

                                      muldjordM 1 Reply Last reply Reply Quote 0
                                      • muldjordM
                                        muldjord @robertvb83
                                        last edited by

                                        @robertvb83 Try ctrl+c or ctrl+d

                                        1 Reply Last reply Reply Quote 1
                                        • robertvb83R
                                          robertvb83
                                          last edited by

                                          i entered this to my config.ini (with my userID and PW obviously)

                                          [screenscraper]
                                          userCreds="USERID:PASSWD"
                                          

                                          still gathering is with 1 thread only, i checked the Homepage and it says 7 threads

                                          My full size arcade cabinet Robotron vs. Octolyzer

                                          muldjordM 1 Reply Last reply Reply Quote 0
                                          • muldjordM
                                            muldjord @robertvb83
                                            last edited by muldjord

                                            @robertvb83 Please paste your entire config.ini file (but leave out the id and password again of course). Something must be wrong with how you are doing this. Also please reverify that you've typed your userid and password correctly. I am absolutely sure this works, I use it almost everyday, so it has to be something in your config or your setup in general.

                                            EDIT: I assume you remember to choose the "screenscraper" module as the gathering source, right?

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.