RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login
    Please do not post a support request without first reading and following the advice in https://retropie.org.uk/forum/topic/3/read-this-first

    Having trouble Scraping games

    Scheduled Pinned Locked Moved Help and Support
    scrapehelpraspberry pi 3
    14 Posts 6 Posters 2.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • L
      lesk
      last edited by

      Re: Can't scrape from thegamesdb.net - known problem?

      I found this topic and I updated the ES package. But the scraper still dosen't work.
      Is there a way to change the scraper to use another source then thegamesdb?

      Raspberry pie 3
      retropie version: v4.4
      ES version: v2.7.6RP
      Connected to internet (updating ES etc)

      mituM 1 Reply Last reply Reply Quote 0
      • mituM
        mitu Global Moderator @lesk
        last edited by

        @lesk said in Having trouble Scraping games:

        Is there a way to change the scraper to use another source then thegamesdb?

        Not at the moment, there isn't another scraping source implemented in Emulationstation. What ROM are you trying to scrape - can you give some examples ?

        L 1 Reply Last reply Reply Quote 0
        • L
          lesk @mitu
          last edited by lesk

          @mitu said in Having trouble Scraping games:

          @lesk said in Having trouble Scraping games:

          Is there a way to change the scraper to use another source then thegamesdb?

          Not at the moment, there isn't another scraping source implemented in Emulationstation. What ROM are you trying to scrape - can you give some examples ?

          The list is super long, but some examples are:
          Wrecking Crew (nes)
          Gremlins (nes)
          Crash Team Racing (psx)

          I also tried installing the Selphs scraper. It worked, for some games. But it renamed the games in the list as well.
          Instead of King of Dragons , it now says King of Dragons (USA). On some ROMs it now says REV. Or 1.1. Or (JAPAN), (EUROPE) etc.

          I did enjoy the old scraper, but I guess it's not working anymore?
          Does anyone know when the default scraper is going to be working again?

          I like how I could use it via the UI and then get a choice if there was multiple choices.

          1 Reply Last reply Reply Quote 0
          • simpleethatS
            simpleethat
            last edited by

            There are a couple of stand-alone scrapers that have worked pretty well for me. Check out Universal XML Scraper and Skraper

            They both run in a Windows environment over your network and Skraper has a Linux distro as well. They will both require you to sign up for a free account with screenscraper.fr but I've found them to both work well and be fairly accurate (if not for some of the metadata of games being off (ie genre))

            C 1 Reply Last reply Reply Quote 0
            • mituM
              mitu Global Moderator
              last edited by

              @lesk I just checked and the legacy API server that Emulationstation uses has been shutdown yesterday by the thegamesdb.net team yesterday. That's why it's not working anymore. For now, you'd have to use other scrapers (external) to scrape for data.

              L 1 Reply Last reply Reply Quote 0
              • O
                omen_peter
                last edited by

                I agree with @lesk, I like how the internal scraper works, allowing you to manually pick if there are multiple choices (basically I like having total control - ha!).

                Until I updated RetroPie to v4.4 I never really used it, but recently I have been using it to scrape a few games every now and then. Hopefully a future update to EmulationStation will get it working again! In the meantime I'm happy to wait.

                Raspberry Pi3 Model B V1.2
                16 GB Micro SD Card (OS), 4TB WD Elements Portable HDD (Roms)
                5V 3.0A Power Supply
                RetroPie V4.7.7, EmulationStation V2.9.6RP, RetroArch V1.8.8
                1 x Sonnics i8 Mini Keyboard (Wireless), 4 x Wii U Pro Controllers (Bluetooth)

                1 Reply Last reply Reply Quote 0
                • C
                  che_don_john @simpleethat
                  last edited by

                  @simpleethat said in Having trouble Scraping games:

                  There are a couple of stand-alone scrapers that have worked pretty well for me. Check out Universal XML Scraper and Skraper

                  They both run in a Windows environment over your network and Skraper has a Linux distro as well. They will both require you to sign up for a free account with screenscraper.fr but I've found them to both work well and be fairly accurate (if not for some of the metadata of games being off (ie genre))

                  Can you advise me on how I can scrape roms individually in Skraper? So far I can only get it to scrape all the roms for each system, meanign that it keeps overwriting previously scraped roms. I liked how Selph's would ignore already scraped roms - is that an option with Skraper?

                  simpleethatS 1 Reply Last reply Reply Quote 0
                  • simpleethatS
                    simpleethat @che_don_john
                    last edited by

                    @che_don_john I don't think it can natively do that but you can do what I do when I add games to my already scraped collections:

                    1. rename the default folder for the system I want to add games to (ie snes to snes2)
                    2. create a new folder with the default name (a new snes)
                    3. add the games I want to scrape to the new folder (snes)
                    4. open the gamelist.xml file in a text editor and copy all the info between <gameList> and </gamelist>
                    5. paste the copied data to the old gamelist.xml above </gameList>
                    6. copy the roms from the snes folder to snes2
                    7. delete snes
                    8. rename snes2 back to snes

                    It isn't the most elegant method and there might be a more efficient way but it works for me. One thing that I do really like about skraper is that it adds all the media and the gamelist folder to the selected system's roms folder instead of inside configs\emulationstation so when I want to build a different setup I only have to copy the roms folder over instead of having to move a bunch.

                    I hope that is clear enough, I'm not always the best at typing these things out. Let me know if you have any further questions.

                    1 Reply Last reply Reply Quote 0
                    • L
                      lesk @mitu
                      last edited by

                      @mitu said in Having trouble Scraping games:

                      @lesk I just checked and the legacy API server that Emulationstation uses has been shutdown yesterday by the thegamesdb.net team yesterday. That's why it's not working anymore. For now, you'd have to use other scrapers (external) to scrape for data.

                      Sorry for replying on a old thread now.
                      But it seems that they are giving out API keys now again. Maybe the developers of Retropie could get one?
                      Or is it possible to change the key yourself inside Retropie?

                      mituM 1 Reply Last reply Reply Quote 0
                      • mituM
                        mitu Global Moderator @lesk
                        last edited by

                        @lesk said in Having trouble Scraping games:

                        But it seems that they are giving out API keys now again. Maybe the developers of Retropie could get one?

                        I don't think they stopped 'giving keys' at any moment.

                        Or is it possible to change the key yourself inside Retropie?

                        No, it's not possible because the version of the scraper that didn't work was not using any key for scraping.

                        In the mean time, a new version of the scraper has been added to Emulationstation, if you update now your package it should work again.

                        L 1 Reply Last reply Reply Quote 0
                        • L
                          lesk @mitu
                          last edited by

                          @mitu said in Having trouble Scraping games:

                          @lesk said in Having trouble Scraping games:

                          But it seems that they are giving out API keys now again. Maybe the developers of Retropie could get one?

                          I don't think they stopped 'giving keys' at any moment.

                          Or is it possible to change the key yourself inside Retropie?

                          No, it's not possible because the version of the scraper that didn't work was not using any key for scraping.

                          In the mean time, a new version of the scraper has been added to Emulationstation, if you update now your package it should work again.

                          Sorry for digging up old post. But haven't been able to play with my pie until now xD

                          I tried updating but the scraper is still not working. What version of Emulationstation is working? I went into config and press update. I also went into manage packages and selected Emulationstation and updated that. Still not able to scrape.

                          mituM 1 Reply Last reply Reply Quote 0
                          • mituM
                            mitu Global Moderator @lesk
                            last edited by

                            @lesk said in Having trouble Scraping games:

                            What version of Emulationstation is working?

                            Versions 2.8.2 should work - this is the latest.

                            R 1 Reply Last reply Reply Quote 0
                            • R
                              RyanT @mitu
                              last edited by

                              @mitu

                              Just updated to the latest version 2.8.3RP

                              Still not working. With gamesDB or screenscraper

                              Any ideas?

                              mituM 1 Reply Last reply Reply Quote 0
                              • mituM
                                mitu Global Moderator @RyanT
                                last edited by

                                @RyanT Please give more details about your setup - https://retropie.org.uk/forum/topic/3/read-this-first. What game/system are you trying to scrape and it's not working ? Give me a concrete example.

                                1 Reply Last reply Reply Quote 0
                                • First post
                                  Last post

                                Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.