RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login
    Please do not post a support request without first reading and following the advice in https://retropie.org.uk/forum/topic/3/read-this-first

    Skyscraper not scraping some systems

    Scheduled Pinned Locked Moved
    Help and Support
    skyscraper cache scrape error
    2
    17
    1.5k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • G
      gordonfreemanjr
      last edited by gordonfreemanjr

      Exactly what I've been looking for. Thanks. And all I need to do is to correctly edit the line in the .ini file?

      Also, how do I delete the cache from the Pi directory afterwards? Manually?

      RPi 4 4GB
      Retropie v 4.7.1

      1 Reply Last reply Reply Quote 0
      • G
        gordonfreemanjr @mitu
        last edited by gordonfreemanjr

        @mitu Doesn't work. I've updated the .ini file:

        [main]
        ##Setting input, gamelist or media folder here, will automatically append '/[platform]' to path.
        ##If you need better control, use the same variables under a [<platform>] section instead.
        #inputFolder="/media/usb/retropie-mount/roms"
        #excludeFiles="[BIOS]"
        #includeFiles="Super*"
        #gamelistFolder="/home/pi/RetroPie/roms"
        #mediaFolder="/media/usb/retropie-mount/roms"
        #cacheFolder="/media/usb/retropie-mount/skyscrapercache"
        #cacheResize="false"
        #nameTemplate="%t [%f], %P player(s)"
        #jpgQuality="95"
        #cacheCovers="true"
        #cacheScreenshots="true"
        #cacheWheels="true"
        #cacheMarquees="true"
        #importFolder="/home/pi/.skyscraper/import"
        #unpack="false"
        #frontend="emulationstation"
        #emulator=""
        #launch=""
        #videos="false"
        #videoSizeLimit="42"
        #symlink="false"
        #brackets="true"
        #maxLength="10000"
        #threads="2"
        #pretend="false"
        #unattend="false"
        #unattendSkip="false"
        #interactive="false"
        #forceFilename="false"
        #verbosity="1"
        #skipped="false"
        #maxFails="30"
        #lang="en"
        #region="wor"
        #langPrios="en,de,es"
        #regionPrios="eu,us,ss,uk,wor,jp"
        #minMatch="0"
        #artworkXml="artwork.xml"
        #relativePaths="false"
        #addExtensions="*.zst"
        #hints="false"
        #subdirs="true"
        #spaceCheck="true"
        #scummIni="/full/path/to/scummvm.ini"

        #[<PLATFORM, eg 'amiga'>]
        #inputFolder="/media/usb/retropie-mount/roms/amiga"
        #excludeFiles="[BIOS]"
        #includeFiles="Super*"
        #gamelistFolder="/home/pi/RetroPie/roms/amiga"
        #mediaFolder="/media/usb/retropie-mount/roms/amiga/media"
        #cacheFolder="/media/usb/retropie-mount/skyscrapercache/amiga"
        #cacheResize="false"
        #nameTemplate="%t [%f], %P player(s)"
        #jpgQuality="95"
        #cacheCovers="true"
        #cacheScreenshots="true"
        #cacheWheels="true"
        #cacheMarquees="true"
        #importFolder="/home/pi/.skyscraper/import/amiga"
        #unpack="false"
        #emulator=""
        #launch=""
        #videos="false"
        #videoSizeLimit="42"
        #symlink="false"
        #brackets="true"
        #lang="en"
        #region="wor"
        #langPrios="en,de,es"
        #regionPrios="eu,us,ss,uk,wor,jp"
        #minMatch="0"
        #maxLength="10000"
        #threads="2"
        #startAt="filename"
        #endAt="filename"
        #pretend="false"
        #unattend="false"
        #unattendSkip="false"
        #interactive="false"
        #forceFilename="false"
        #verbosity="1"
        #skipped="false"
        #artworkXml="artwork.xml"
        #relativePaths="false"
        #extensions="*.zip *.uae .adf"
        #addExtensions="        .zst *.rom"
        #subdirs="true"

        #[<FRONTEND, eg 'pegasus'>]
        #excludeFiles="[BIOS]"
        #includeFiles="Super*"
        #artworkXml="artwork.xml"
        #emulator=""
        #launch=""
        #gamelistFolder="/home/pi/RetroPie/roms/amiga"
        #mediaFolder="/home/pi/RetroPie/roms/amiga/media"
        #skipped="false"
        #brackets="true"
        #videos="false"
        #symlink="false"
        #startAt="filename"
        #endAt="filename"
        #unattend="false"
        #unattendSkip="false"
        #forceFilename="false"
        #verbosity="1"
        #maxLength="10000"

        #[<SCRAPING MODULE, eg 'screenscraper'>]
        #userCreds="user:password"
        #threads="1"
        #minMatch="0"
        #maxLength="10000"
        #interactive="false"
        #cacheResize="false"
        #jpgQuality="95"
        #cacheCovers="true"
        #cacheScreenshots="true"
        #cacheWheels="true"
        #cacheMarquees="true"
        #videos="false"
        #videoSizeLimit="42"

        And apparently Skyscraper DID create the media folder in the FDS roms folder on the USB stick, but when I tried to scrape GB, GBC and GBA the prompt was the same: "can't create folder". Eventually I created the MEDIA folders and subfolders in every rom folder myself, but it did nothing. Still won't scrape.

        Also, the "disk is full" prompt comes back, which seems to indicate, that despite the modifications to the .ini file (media/usb/retropie-mount/skyscrapercache) the cache is still being saved to the Pi - which, by the way, is 100% full. I don't know why.

        I'm lost.

        EDIT: I haven't uncommented the lines in the .ini file. Sorry. Retrying.

        RPi 4 4GB
        Retropie v 4.7.1

        mituM 1 Reply Last reply Reply Quote 0
        • mituM
          mitu Global Moderator @gordonfreemanjr
          last edited by

          @gordonfreemanjr said in Skyscraper not scraping some systems:

          EDIT: I haven't uncommented the lines in the .ini file. Sorry. Retrying.

          Well, duh. Use code tags (``` on a line before and after the code ), this will make it easier to view/find stuff in configuration/code fragments.

          1 Reply Last reply Reply Quote 0
          • G
            gordonfreemanjr
            last edited by

            OK, here's the thing: GB library is now officially scrapped, so it works. Also, both, the cache and GB media files, are on the USB stick, so good on the double.

            The problem is, the roms and media files are also at the same time on the Pi. I don't get it. I installed the usbrom service, created the retropie-mount folder, everything has copied over nicely. Then, I deleted all the files from the copied over rom folders and replaced them with new libraries. Put the stick back into Pi, it works, roms are there and the run.

            Why is everything doubled? This was I'm barely conserving any space on the Pi...

            RPi 4 4GB
            Retropie v 4.7.1

            mituM 1 Reply Last reply Reply Quote 0
            • mituM
              mitu Global Moderator @gordonfreemanjr
              last edited by

              @gordonfreemanjr said in Skyscraper not scraping some systems:

              Why is everything doubled? This was I'm barely conserving any space on the Pi...

              How are you checking that ?

              1 Reply Last reply Reply Quote 0
              • G
                gordonfreemanjr
                last edited by gordonfreemanjr

                Via the file manager while on the Emulation Station. Wait... unless going to RetroPie/roms/xxx actually sends me over to retropie-mount/roms/xxx on the USB stick, while connected?
                Also, the Pi is 97% full, according to the loading screen.

                RPi 4 4GB
                Retropie v 4.7.1

                mituM 1 Reply Last reply Reply Quote 0
                • mituM
                  mitu Global Moderator @gordonfreemanjr
                  last edited by

                  @gordonfreemanjr said in Skyscraper not scraping some systems:

                  Via the file manager while on the Emulation Station. Wait... unless going to RetroPie/roms/xxx actually sends me over to retropie-mount/roms/xxx on the USB stick, while connected?

                  If you're looking at the ROM file share, then you'll see the files on the USB disc, not what's on the sdcard, since the ROM folder is not on the USB disc.

                  Also, the Pi is 97% full, according to the loading screen.

                  If the root partition (/dev/root) is the one at 97%, then that's the SD card space, you should check what's left there - did you delete the Skyscraper cache ? Make sure you haven't left the ROMs still there - you can stop EmulationStation, un-mount the USB stick and check again th ROMs folder for files.

                  G 1 Reply Last reply Reply Quote 0
                  • G
                    gordonfreemanjr
                    last edited by

                    So, if I disconnect the USB stick, start up the Pie and go to the file manager to discover, that any roms or skyscraper cache is still on the SD card, I can delete them freely?

                    RPi 4 4GB
                    Retropie v 4.7.1

                    1 Reply Last reply Reply Quote 0
                    • G
                      gordonfreemanjr @mitu
                      last edited by

                      @mitu You were right. It’s working.
                      Thank you!

                      One last question: if I’m done scraping with, say, ScreenScraper, and I have like 5% of the roms missing - if the “force cache refresh” option is OFF and I run another scraper on the same system, does it find only the missing roms without overwriting the existing cache? I find the description bit unclear.

                      RPi 4 4GB
                      Retropie v 4.7.1

                      mituM 1 Reply Last reply Reply Quote 0
                      • mituM
                        mitu Global Moderator @gordonfreemanjr
                        last edited by

                        @gordonfreemanjr said in Skyscraper not scraping some systems:

                        • if the “force cache refresh” option is OFF and I run another scraper on the same system, does it find only the missing roms without overwriting the existing cache? I find the description bit unclear.

                        Foce refresh will re-process all entries. You don't need that to find things that are missing. Every scrape will add to the cache, but will not delete from it, so you can re-run the scraping for the same system and it will try to get the missing items.
                        If you're running from the command line, then you can scrape just the missing files, without going through the whole system.

                        1 Reply Last reply Reply Quote 0
                        • G
                          gordonfreemanjr
                          last edited by

                          So, if the scraping process for a platform is completed, and I'd like to use the collected media with the same rom set, but on another Pi (instead of scraping the same files all over again), is this OK to copy the cache folder onto the another USB stick, set correct cache location, set the cache folder as the local scraping source and run the Skyscraper again on the new Pi, but locally, importing from the cache folder?

                          RPi 4 4GB
                          Retropie v 4.7.1

                          1 Reply Last reply Reply Quote 0
                          • mituM
                            mitu Global Moderator
                            last edited by

                            Yes, it should work.

                            1 Reply Last reply Reply Quote 0
                            • G
                              gordonfreemanjr
                              last edited by

                              Shouldn't I be able to just generate the game lists? If Skyscraper generates them from the cache, and the cache is already gathered....

                              RPi 4 4GB
                              Retropie v 4.7.1

                              mituM 1 Reply Last reply Reply Quote 0
                              • mituM
                                mitu Global Moderator @gordonfreemanjr
                                last edited by

                                @gordonfreemanjr said in Skyscraper not scraping some systems:

                                Shouldn't I be able to just generate the game lists? If Skyscraper generates them from the cache, and the cache is already gathered....

                                It all depends on what ROMs the other system has - if there's a checksum mismatch, the ROM will be scraped again.

                                G 1 Reply Last reply Reply Quote 0
                                • G
                                  gordonfreemanjr @mitu
                                  last edited by

                                  @mitu It's going to be exactly the same set copied over.

                                  RPi 4 4GB
                                  Retropie v 4.7.1

                                  1 Reply Last reply Reply Quote 0
                                  • First post
                                    Last post

                                  Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                  Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.