RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login
    Please do not post a support request without first reading and following the advice in https://retropie.org.uk/forum/topic/3/read-this-first

    Scraping

    Scheduled Pinned Locked Moved Help and Support
    scraping newbie
    17 Posts 4 Posters 1.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      marc81 @marc81
      last edited by

      I've been using Skraper for hours now.. rough estimate was 24 hours for 1500 roms.. does that sound right??

      mituM 1 Reply Last reply Reply Quote 0
      • mituM
        mitu Global Moderator @marc81
        last edited by

        @marc81 I guess the site is busy/overloaded. Usually happens during the week-ends.

        1 Reply Last reply Reply Quote 0
        • mituM
          mitu Global Moderator
          last edited by

          It looks like the ScreenScraper site has a few issues at the moment with their API, scraping is going to be either very slow or have lots of failed requests until they get it back. This was posted in their forum:

          Hello ScreenScraper community,

          we have a technical problem with the ScreenScraper clone server. This causes significant slowdowns on the site and the api and therefore on your scrapers. We recommend you to wait until both servers are working normally again to do your scrapes.

          Thank you for your patience, the ScreenScraper team.

          M 1 Reply Last reply Reply Quote 0
          • M
            marc81 @mitu
            last edited by

            @mitu Thanks. Checked on it this morning.. 20 hours in and over a day to go.. not even half way 😅... I'm guessing this is why. If I stop it will it save what it's scraped up to now?

            mituM 1 Reply Last reply Reply Quote 0
            • mituM
              mitu Global Moderator @marc81
              last edited by

              @marc81 said in Scraping:

              If I stop it will it save what it's scraped up to now?

              If we're talking about Skraper, then yes, it keeps a local cache of the artwork already downloaded.

              ClydeC 1 Reply Last reply Reply Quote 0
              • GreenHawk84G
                GreenHawk84
                last edited by

                Not sure if my issue is related but using the built in ScreenScraper is yielding no scrapes. It always worked wonderfully too. Eternal sadness until I get scraping working on my Retropie.

                Pi3 B+ no overclock, latest updates.

                1 Reply Last reply Reply Quote 0
                • ClydeC
                  Clyde @mitu
                  last edited by

                  @mitu said in Scraping:

                  If we're talking about Skraper, then yes, it keeps a local cache of the artwork already downloaded.

                  Skyscraper also has a media cache, and Selph's Scraper at least keeps the already downloaded videos and pictures in the images directory, even if its gamelist mode is set to "overwrite".

                  Just stumbled upon your post and thought about this addendum. :)

                  1 Reply Last reply Reply Quote 0
                  • GreenHawk84G
                    GreenHawk84
                    last edited by

                    Tried to scrape things using internal scraper via Screenscraper and had no results yet.

                    1 Reply Last reply Reply Quote 0
                    • mituM
                      mitu Global Moderator
                      last edited by

                      For about 2 weeks, ScreenScraper has had some problems - even after they've been back online following a maintenance, scraping for un-registered users is mostly a miss. It's not your setup, it's their API/site.

                      GreenHawk84G 1 Reply Last reply Reply Quote 0
                      • GreenHawk84G
                        GreenHawk84 @mitu
                        last edited by GreenHawk84

                        @mitu said in Scraping:

                        For about 2 weeks, ScreenScraper has had some problems - even after they've been back online following a maintenance, scraping for un-registered users is mostly a miss. It's not your setup, it's their API/site.

                        I am a registered user but I am not sure how this would apply to access their stuff from my Raspberry Pi while in Retropie vs. my Windows PC where I can login via Firefox.

                        1 Reply Last reply Reply Quote 0
                        • mituM
                          mitu Global Moderator
                          last edited by mitu

                          You can use one of the external scrapers - Skyscraper or Steven Selph's scrapers - to make use of your user account when scraping. Each one has its way of using the account info, Skyscraper can read them from the configuration file, Steven Selph's scraper accepts the user/password as command line parameters when scraping.

                          EDIT: or, on Windows, you can use Skraper.

                          1 Reply Last reply Reply Quote 0
                          • GreenHawk84G
                            GreenHawk84
                            last edited by

                            @mitu does Skraper allow me to scrape just one game at a time or does it look for whole systems?

                            1 Reply Last reply Reply Quote 0
                            • mituM
                              mitu Global Moderator
                              last edited by

                              From my brief usage, it scans whole systems (folders) of ROMs.

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post

                              Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                              Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.