RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login
    Please do not post a support request without first reading and following the advice in https://retropie.org.uk/forum/topic/3/read-this-first

    EmulationStation Scraper NOT Working

    Scheduled Pinned Locked Moved Help and Support
    scraperemulstationnot working
    114 Posts 25 Posters 89.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • K
      keithk2581
      last edited by

      So as an experiment, I decided to try a fresh install on a 3rd SD and after all was installed and updated, the scraper did work! Something must be corrupted on my other 2 builds, just don't know what or how or why! Any idea if I can fix this without formatting and starting from scratch or is that my only option?

      mituM 1 Reply Last reply Reply Quote 0
      • mituM
        mitu Global Moderator @keithk2581
        last edited by

        @keithk2581 Try removing any files from ~/.emulationstation/scrapers. TheGamesDB scraper will download some files there on the 1st scrape, you'll have to wait a bit, then it will re-create the files.

        1 Reply Last reply Reply Quote 0
        • K
          keithk2581
          last edited by

          I'll give it a shot when I get home tonight, thanks!

          1 Reply Last reply Reply Quote 0
          • K
            keithk2581
            last edited by

            I deleted the files you said, and it works again! Thank you so much for your help!!

            1 Reply Last reply Reply Quote 0
            • K
              keithk2581
              last edited by

              Only thing I notice now is that it still returns no games found for games I know are on the gamesdb for example, super battleship and wheel of fortune to name 2(Sega genesis), just an issue with the site or my scraper?

              mituM 1 Reply Last reply Reply Quote 0
              • mituM
                mitu Global Moderator @keithk2581
                last edited by

                @keithk2581 Can you find the games on their site ? If not, then maybe there's an issue with the site. What's the complete ROM filename ?

                1 Reply Last reply Reply Quote 0
                • K
                  keithk2581
                  last edited by

                  Yes the games are on the site, but seems to only be a problem for mega drive/genesis games. Just did a bunch of super nes games with no issue and all of my rims are named in the same fashion. I have all of my genesis games in mega drive folder could that be an issue in regard to the scraping?

                  Rom names are in this format:
                  Wheel of Fortune(USA).gen
                  Adventures of Rocky and Bullwinkle And Friends,The(USA).gen

                  Both of those are on the games dB but won't scrape even if I manually change the input

                  mituM 1 Reply Last reply Reply Quote 0
                  • mituM
                    mitu Global Moderator @keithk2581
                    last edited by

                    @keithk2581 said in EmulationStation Scraper NOT Working:

                    I have all of my genesis games in mega drive folder could that be an issue in regard to the scraping?

                    It should not, unless they're in a different system on the scraping site. Given that the other ROMs are found, I don't think that's the case.

                    Wheel of Fortune(USA).gen
                    Adventures of Rocky and Bullwinkle And Friends,The(USA).gen

                    There are multiple games with this name - the first has one for Genesis and SegaCD, the 2nd one has one for multiple consoles of the time - NES, SNES, Genesis, GameBoy.

                    I can take a look to see what's confusing the scraper, but it could also be a bug on the server side.

                    1 Reply Last reply Reply Quote 0
                    • K
                      keithk2581
                      last edited by

                      Yeah, trying to figure that out is definitely above my current skill level :). Funny that its only the genesis(mega drive) games, gotta be something about them. My guess would be its on the server side, but who knows. Thanks again for all your help, and to all who have posted, I truly appreciate it.

                      1 Reply Last reply Reply Quote 0
                      • mituM
                        mitu Global Moderator
                        last edited by mitu

                        It's because of the way TheGamesDB stores the information about platforms (consoles/systems).
                        The scraper filters the ROM search by Platform=Sega Mega Drive (and rightly so, since the same game is also available on NES/SNES and GameBoy), however the games are in platform=Sega Genesis, which is different (has a different identifier).

                        If you search on TGDB by Platform='Sega Mega Drive' and the name of the game - let's use The Adventures of Rocky and Bullwinkle and Friends as an example - there's nothing returned. However, if you search with Platform='Sega Genesis', there's one game returned (as it should).

                        I'm not sure right now how this should be handled in Emulationstation, I'm surprised there were only 2 games from your list. I think all US exclusive Genesis games will not be found by the TGDB scraper.

                        Now, the ScreeenScraper scraper would be able to find them, but right now it's a bit broken because of their SSL settings change.

                        c03457ad-8f48-485f-b51e-154cb064e814-image.png

                        1 Reply Last reply Reply Quote 0
                        • K
                          keithk2581
                          last edited by

                          Hmmm, I see. There were more than 2 games, was just giving you an example of the format of the file names of 2 of my roms that wouldn't scrape. Scraper actually worked for 611 of the 784 games i have, plus or minus a few name issues. I guess for now, ill just gradually add them over time manually, like a small project. Thank you so much for everything!

                          mituM 1 Reply Last reply Reply Quote 0
                          • mituM
                            mitu Global Moderator @keithk2581
                            last edited by

                            @keithk2581 You can try also Skyscraper, it should be at least as efficient as scraper.

                            1 Reply Last reply Reply Quote 0
                            • mituM
                              mitu Global Moderator
                              last edited by

                              I submitted a bug report - https://github.com/RetroPie/EmulationStation/issues/565 - for this problem, maybe the scraper can be easily modified to search simultaneously in the Genesis and Megadrive systems.

                              1 Reply Last reply Reply Quote 1
                              • MrJordaaanyM
                                MrJordaaany
                                last edited by

                                Is this fixed now and in the main build? Or do you still need to source build the experimental release of Emulationstation?

                                mituM 1 Reply Last reply Reply Quote 0
                                • mituM
                                  mitu Global Moderator @MrJordaaany
                                  last edited by

                                  @MrJordaaany It's what fixed ?

                                  MrJordaaanyM 1 Reply Last reply Reply Quote 0
                                  • MrJordaaanyM
                                    MrJordaaany @mitu
                                    last edited by MrJordaaany

                                    @mitu The EmulationStation built in scraper. Or do you have to build the experimental release from source. Will this new version be included in 4.5?

                                    mituM 1 Reply Last reply Reply Quote 0
                                    • mituM
                                      mitu Global Moderator @MrJordaaany
                                      last edited by

                                      @MrJordaaany If you use the built in scraper is should have a functional TGDB scraper. Did you scrape from TheGamesDB or from ScreenScraper ? Scraping from ScreenScraper only works on the -dev branch, due to a change on their site.

                                      1 Reply Last reply Reply Quote 0
                                      • MrJordaaanyM
                                        MrJordaaany
                                        last edited by

                                        It wasn't working for me so I installed the experimental one from source. If I do a fresh 4.4 install, it should be ok now you reckon? Or still need to update EmulationStation under core packages?

                                        Cheeeers

                                        mituM 1 Reply Last reply Reply Quote 0
                                        • mituM
                                          mitu Global Moderator @MrJordaaany
                                          last edited by

                                          @MrJordaaany What's the exact Emulationstation version you see when you look at the ES menu (Start) ?

                                          MrJordaaanyM 1 Reply Last reply Reply Quote 0
                                          • MrJordaaanyM
                                            MrJordaaany @mitu
                                            last edited by

                                            @mitu v2.7.6. I could be wrong though, im not at home so can't check right now.

                                            mituM 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.