RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login
    Please do not post a support request without first reading and following the advice in https://retropie.org.uk/forum/topic/3/read-this-first

    (Another) skyscraper help topic!

    Scheduled Pinned Locked Moved Help and Support
    skyscraperretropie 4.5screenscraper
    22 Posts 3 Posters 1.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      showstopper @mitu
      last edited by

      @mitu Thank you so much - i was looking everywhere to try and find the MD5 for the files!!

      Is there a way i can put the missing files in an xml and then run a check - or is this a one-at-a-time process?

      mituM 1 Reply Last reply Reply Quote 0
      • mituM
        mitu Global Moderator @showstopper
        last edited by

        @showstopper You can query the scraping source any time you want, the results will be added to the cache and stay there, then later on when you generate the gamelists for Emulationstation they'll be written to the gamelist and media folders.

        S 1 Reply Last reply Reply Quote 0
        • S
          showstopper @mitu
          last edited by showstopper

          @mitu OK really sorry but i am trying this with my atari lynx game 'Ninja Gaiden III - Ancient Ship of Doom (USA, Europe).7z' that is coming up as unmatched.

          I have tried searching with query and MD5 which is failing for me. This is what i am getting:

          pi@retropie:~ $ Skyscraper -p atarilynx -s screenscraper --query 'md5=B263E945DA412BDCE0DC88697B629412' ./Ninja\ Gaiden\ III\ \(USA\).zip
          ------------------------------------------
          Running Skyscraper v3.3.1 by Lars Muldjord
          ------------------------------------------
          Filename: './Ninja Gaiden III (USA).zip' requested either on command line or with '--fromfile' not found!
          
          Please verify the filename and try again...
          pi@retropie:~ $
          

          Here is the link to the MD5

          Sorry for the noob questions

          EDIT - OK so i need the full path to my rom and then it worked. Thanks for all your help.

          Skyscraper -p atarilynx -s screenscraper --query 'md5=B263E945DA412BDCE0DC88697B629412' "/home/pi/RetroPie/roms/atarilynx/Ninja Gaiden III - Ancient Ship of Doom (USA, Europe).7z"
          
          1 Reply Last reply Reply Quote 0
          • S
            showstopper
            last edited by

            OK so i am now scraping my GBA roms and i have a few that dont scrape on screenscraper. I am trying to run a query in thegamesdb but it says it cant find it. Could someone tell me what i am doing wrong please?!

            pi@retropie:~ $ Skyscraper -p gba -s thegamesdb --noscreenshots --nowheels --nom                                                                                                                                                          arquees --query 'Cartoon Network Block Party / Cartoon Network Speedway' "/home/                                                                                                                                                             pi/RetroPie/roms/gba/2 Games in 1 - Cartoon Network Block Party + Cartoon Networ                                                                                                                                                             k Speedway (USA).7z"
            ------------------------------------------
            Running Skyscraper v3.1.4 by Lars Muldjord
            ------------------------------------------
            Platform:           'gba'
            Scraping module:    'thegamesdb'
            Input folder:       '/home/pi/RetroPie/roms/gba'
            Game list folder:   '/home/pi/RetroPie/roms/gba'
            Covers folder:      '/home/pi/RetroPie/roms/gba/media/covers'
            Screenshots folder: '/home/pi/RetroPie/roms/gba/media/screenshots'
            Wheels folder:      '/home/pi/RetroPie/roms/gba/media/wheels'
            Marquees folder:    '/home/pi/RetroPie/roms/gba/media/marquees'
            Cache folder:       'cache/gba'
            
            DID YOU KNOW: Skyscraper caches all scraped data in the resource cache. This sav                                                                                                                                                             es bandwidth for the scraping sources as any rescaping will make use of this cac                                                                                                                                                             he instead of pounding the servers again.
            
            Reading and parsing resource cache, please wait...
            Successfully parsed 9869 resources!
            
            Looking for optional 'priorities.xml' file in cache folder... Not found, skippin                                                                                                                                                             g...
            
            
            Starting scraping run on 1 files using 4 threads.
            Sit back, relax and let me do the work! :)
            
            #1/1 (T1) Pass 1 ---- Game '2 Games in 1 - Cartoon Network Block Party + Cartoon                                                                                                                                                              Network Speedway (USA)' not found :( ----
            
            
            'thegamesdb' requests remaining: 1497
            
            0/1/1
            Elapsed time   : 00:00:00
            Est. time left : 00:00:00
            
            ---- Resource gathering run completed! YAY! ----
            Writing 9869 (0 new) resources to cache, please wait... Success!
            
            ---- And here are some neat stats :) ----
            Total completion time: 00:00:01
            
            Total number of games: 1
            Successfully processed games: 0
            Skipped games: 1 (Filenames saved to '~/.skyscraper/skipped-thegamesdb.txt')
            
            pi@retropie:~ $
            
            1 Reply Last reply Reply Quote 0
            • S
              showstopper
              last edited by showstopper

              I have issues with the code tags and getting spammed!!

              This is what i am typing in:

              Skyscraper -p gba -s thegamesdb --noscreenshots --nowheels --nomarquees --query 'Cartoon Network Block Party / Cartoon Network Speedway' "/home/pi/RetroPie/roms/gba/2 Games in 1 - Cartoon Network Block Party + Cartoon Network Speedway (USA).7z"
              
              1 Reply Last reply Reply Quote 0
              • mituM
                mitu Global Moderator
                last edited by

                I don't see any actual errors in your scraping command output. Maybe the game doesn't exist on the source site ?

                S 1 Reply Last reply Reply Quote 1
                • S
                  showstopper @mitu
                  last edited by showstopper

                  @mitu It does exist as i copied the wording from thegamesdb exact since they dont seem to have CRC numbers.
                  No matter i just manually downloaded images and edited the db.xml. All good now.

                  1 Reply Last reply Reply Quote 0
                  • muldjordM
                    muldjord
                    last edited by muldjord

                    @showstopper, Just a side-note, you are using a really old version of Skyscraper. I would strongly suggest you update it. Also, I would like to point you to the Skyscraper documentation. It has examples for pretty much all of the available options. https://github.com/muldjord/skyscraper/blob/master/docs/CLIHELP.md

                    If you want to use the --query option with the thegamesdb module, you'll need to know their query format. I believe it basically needs you to simplify things a bit so "Rick Dangerous" would become --query "rick+dangerous". It is a bit of an expert option and sometimes it is quite hard to get it right, especially with the name searched based modules such as TheGamesDb. So it expects users to know a bit more about the underlying API. But it is very useful to fill in the missing roms.

                    S 1 Reply Last reply Reply Quote 1
                    • S
                      showstopper @muldjord
                      last edited by showstopper

                      @muldjord Thank you for this great application! I did an update today and so i'm surprised it is an old version. I'll try and update again.

                      EDIT - I did an update from within the app - looks like i need to do it outside the application. Thanks again for your hard work with this scraper.

                      muldjordM 1 Reply Last reply Reply Quote 1
                      • muldjordM
                        muldjord @showstopper
                        last edited by

                        EDIT - I did an update from within the app - looks like i need to do it outside the application. Thanks again for your hard work with this scraper.

                        You're welcome! Yes, you need to stick to one or the other (the one installed from the RP script or the one installed manually as describe on Github). Mixing them can cause problems.

                        1 Reply Last reply Reply Quote 1
                        • S
                          showstopper
                          last edited by

                          I didnt want to make a new thread so....

                          I have this working for all my systems but i noticed that the Mame (Arcade) scrape for option Covers seems to get the artwork (from ArcadeDB) Flyer. Is there anyway to change this?

                          For example Super Street Fighter II uses the Flyer image and i would like it to use the Decal image (or other).

                          Is that possible?

                          muldjordM 1 Reply Last reply Reply Quote 0
                          • muldjordM
                            muldjord @showstopper
                            last edited by muldjord

                            @showstopper No, it's hardcoded to first try for the flyer and then the title artwork if it doesn't find a flyer. That's the problem with the arcade artworks, there are so many different ones and none of them match up well with an actual game box cover. So it comes down to a subjective thing and I personally like the flyer artwork best.

                            If you want, you can import them into the cache yourself by using the import module. Read more about that here.

                            S 1 Reply Last reply Reply Quote 1
                            • S
                              showstopper @muldjord
                              last edited by showstopper

                              @muldjord Thanks for replying. I totally agree that it is a subjective preference. Some of them look good as flyer. I manually checked a few and Decal seems to be quite a good one (for me).
                              Just reading your KB - i expect that i am reading it wrong but it looks like it would be a manual individual import - is that right? Is there no way to add 'Decal' as a bulk import option?

                              muldjordM 1 Reply Last reply Reply Quote 0
                              • muldjordM
                                muldjord @showstopper
                                last edited by muldjord

                                @showstopper Well, I guess you could compile Skyscraper from source and just change the hardcoded "flyer" to "decal" in src/arcadedb.cpp. That would probably do it.

                                The import module is manual to the degree that you have to prepare each file. But once all files are placed correctly and named correctly, it's a bulk import.

                                S 1 Reply Last reply Reply Quote 1
                                • S
                                  showstopper @muldjord
                                  last edited by

                                  @muldjord do you have a compile KB by any chance? I'm happy to give it a go.

                                  muldjordM 1 Reply Last reply Reply Quote 0
                                  • muldjordM
                                    muldjord @showstopper
                                    last edited by

                                    @showstopper I don't know what a KB is, but the compile process is described in the documentation. :)

                                    S 1 Reply Last reply Reply Quote 1
                                    • S
                                      showstopper @muldjord
                                      last edited by showstopper

                                      @muldjord KB = Knowledge Base (or documentation!)
                                      I am running this on a Raspberry Pi so not sure how i would compile it on there. Can i change the aracdedb.cpp file on the Pi already - or does the app need compiling first with that change?

                                      EDIT - Actually could i do this by editing the artwork.xml file?

                                      EDIT 2 - Ah no looks like it would need compiling

                                      1 Reply Last reply Reply Quote 0
                                      • S
                                        showstopper
                                        last edited by

                                        Is there anyway to download a single game artwork for a different region? For example 'Another World' on the Amiga is currently set to 'Out of this World'. I see you can change region in the config file but i just want to manual scrape this game with the European artwork and name.

                                        muldjordM 1 Reply Last reply Reply Quote 0
                                        • muldjordM
                                          muldjord @showstopper
                                          last edited by

                                          @showstopper Yes, you can scrape a single file by adding it to the end of the command-line. And you can set a temporary region with with the --region option. Check the region documentation. Check it all here: https://github.com/muldjord/skyscraper/tree/master/docs

                                          Remember to regenerate the game list afterewards and be aware that not all sources have regional support.

                                          1 Reply Last reply Reply Quote 2
                                          • First post
                                            Last post

                                          Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                          Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.