RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login

    Versatile C++ game scraper: Skyscraper

    Scheduled Pinned Locked Moved Ideas and Development
    skyscraperscrapergamelist.xmlscrapinggithub
    1.6k Posts 113 Posters 1.6m Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • B
      Brewskii
      last edited by

      I've a question, I've installed retropie 4.4 n my ubuntu desktop and I recently today installed and started running Skyscraper against my Arcade directory. It seems to keep going through the same files over and over, it's on it's 4th recursion on the directory. Is this normal?
      Thanks

      AnalogHeroA 1 Reply Last reply Reply Quote 0
      • AnalogHeroA
        AnalogHero @Brewskii
        last edited by

        @brewskii If you run it in simple mode, then yes. Skyscraper scrapes with a different source each time, caches the data, and then does a local scrape with all stored data.

        B 1 Reply Last reply Reply Quote 1
        • B
          Brewskii @AnalogHero
          last edited by

          @analoghero Ah, I see that now, it's also mentioned in the dialog when working through simple mode questions. Thanks!

          1 Reply Last reply Reply Quote 1
          • B
            bleuge
            last edited by

            @muldjord I've just read this:

            https://www.reddit.com/r/emulation/comments/8dqim8/announcing_thegamesdb_beta_site_api/

            Maybe of your interest. Thanks!

            muldjordM 1 Reply Last reply Reply Quote 2
            • muldjordM
              muldjord @bleuge
              last edited by

              @bleuge Thanks for the heads up, that is indeed interesting.

              B AnalogHeroA 2 Replies Last reply Reply Quote 0
              • B
                bleuge @muldjord
                last edited by

                @muldjord https://www.reddit.com/r/emulation/comments/8eip7n/if_you_need_your_collection_scraped_for

                Some public exposure 😏

                1 Reply Last reply Reply Quote 1
                • AnalogHeroA
                  AnalogHero @muldjord
                  last edited by AnalogHero

                  @muldjord From now i support you on patreon. I know its not much, but i hope it helps.

                  muldjordM 1 Reply Last reply Reply Quote 1
                  • muldjordM
                    muldjord @AnalogHero
                    last edited by

                    @analoghero Everything is appreciated mate :)

                    1 Reply Last reply Reply Quote 0
                    • AnalogHeroA
                      AnalogHero
                      last edited by

                      @muldjord For some strange reason patreon cant process my pledge. I tried several times adding and removing my mastercard my paypal account. Is there any other way i can send you money? Paypal maybe?

                      muldjordM 1 Reply Last reply Reply Quote 1
                      • muldjordM
                        muldjord @AnalogHero
                        last edited by

                        @analoghero I appreciate the sentiment but please don't bother too much with it. I appreciate the help you've done with bug hunting and helping other users a lot more than monetary support. :)

                        B 1 Reply Last reply Reply Quote 2
                        • B
                          bighoffa @muldjord
                          last edited by

                          @muldjord I am currently using advmame for a few roms (mostly goldentee). I have the roms in the mame-advmame directory with the rest in the arcade dir. Is there a way to scrape the advmame games from the mame-advmame directory without messing up the db or already scraped info from the arcade directory?

                          B 1 Reply Last reply Reply Quote 0
                          • B
                            bighoffa @bighoffa
                            last edited by

                            @bighoffa said in Versatile C++ game scraper: Skyscraper:

                            arcade

                            Please ignore my question. I discovered how to use runcommands and specify the emulator to run for each rom so I can put everything in the arcade directory now.

                            1 Reply Last reply Reply Quote 0
                            • AnalogHeroA
                              AnalogHero
                              last edited by

                              @muldjord A few days ago i did a complete new retropie setup as a birthday present. Just a few classic systems without video previews. Did everything in simplemode and cant report any bugs or flaws.

                              muldjordM 1 Reply Last reply Reply Quote 1
                              • muldjordM
                                muldjord @AnalogHero
                                last edited by

                                @analoghero Glad to hear it :)

                                1 Reply Last reply Reply Quote 0
                                • muldjordM
                                  muldjord
                                  last edited by

                                  Working on improving the attractmode export module. Currently testing game descriptions implementation with user qqplayer. Requests for further attractmode improvements should come now, as this is what I am focused on. So let me know if you have anything that should be supported.

                                  1 Reply Last reply Reply Quote 1
                                  • muldjordM
                                    muldjord
                                    last edited by muldjord

                                    Skyscraper version 2.4.6 released: https://github.com/muldjord/skyscraper

                                    • Added 'overview' support for AttractMode. It will now create the necessary cfg files to show the game descriptions
                                    • Added '<kidgame>bool</kidgame>' output to Emulationstation gamelist generation
                                    • Added 'ages' support in 'screenscraper' module. Will convert PEGI and ESRB to numeric
                                    • Added 'ages' support in 'thegamesdb' module. Will convert PEGI and ESRB to numeric

                                    The 'kid friendly release'. As per user request, I've implemented the <kidgame> Emulationstation tag and added classification scraping support for both the 'thegamesdb' and 'screenscraper' modules. This means that you can now make use of the "kids" mode in Emulationstation, which will then hide all non-kid-friendly games from the lists.

                                    IMPORTANT! If you want to scrape the 'ages' on games you already have cached, you have to scrape using the '--updatedb' flag. Otherwise it will just scrape with the pre-2.4.6 cached data which doesn't contain the ages resources.

                                    Lastly, I've also added 'description' support for the 'attractmode' frontend module. I'm not an Attract-Mode user myself, but I've done basic testing on it, and user 'qqplayer' has verified that it works as expected.

                                    :)

                                    1 Reply Last reply Reply Quote 2
                                    • B
                                      bleuge
                                      last edited by

                                      @muldjord I've seen this very nice and complete MSX db : https://www.msxgamesworld.com/index.php
                                      maybe you want to check it for scraping.
                                      Thanks as usual!

                                      1 Reply Last reply Reply Quote 0
                                      • muldjordM
                                        muldjord
                                        last edited by muldjord

                                        This post is deleted!
                                        1 Reply Last reply Reply Quote 0
                                        • muldjordM
                                          muldjord
                                          last edited by muldjord

                                          [deleted, makes no sense]

                                          AnalogHeroA Used2BeRXU 2 Replies Last reply Reply Quote 0
                                          • AnalogHeroA
                                            AnalogHero @muldjord
                                            last edited by

                                            @muldjord Thats bad news. Good that there is still some other good sources left.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.