RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login

    Improving the SCRAPE function...

    Scheduled Pinned Locked Moved Ideas and Development
    scrapingsearch engine
    8 Posts 4 Posters 1.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • K
      KevinsHope
      last edited by

      I love the scrape function in Retropie. Very handy. But the way the search works is way too specific. I had a : missing from a title, and the search engine returned no results. You can not use part of a title and get search results. If it has extra things in it, like numbers, the search engine returns no results. Please tell me this is an area targeted for improvement in the next release for retropie. Please understand. I'm not unhappy with RetroPie, but being the completionist I tend to be, I'd love to see some improvements in the way the search engine works when trying to find your game you are scraping.

      herb_fargusH 1 Reply Last reply Reply Quote 0
      • herb_fargusH
        herb_fargus administrators @KevinsHope
        last edited by

        @kevinshope sselphs scraper uses hashes which renders the need for searching names irrelevant

        If you read the documentation it will answer 99% of your questions: https://retropie.org.uk/docs/

        Also if you want a solution to your problems read this first: https://retropie.org.uk/forum/topic/3/read-this-first

        K 1 Reply Last reply Reply Quote 0
        • muldjordM
          muldjord
          last edited by muldjord

          I'd also like to point you to https://github.com/muldjord/skyscraper. It also uses hashes depending on what source you choose to scrape from with '-s'

          1 Reply Last reply Reply Quote 0
          • K
            KevinsHope @herb_fargus
            last edited by

            @herb_fargus I am aware of sselphs. I used it to eliminate most of the roms I needed to scrape. However, for those that did not scrape using sselphs scraper, I am hoping for a more functional retropie bound scraper.

            1 Reply Last reply Reply Quote 0
            • HurricaneFanH
              HurricaneFan
              last edited by

              The built in scraper in EmulationStation definitely has issues with certain names.
              It seems to fail on really short names and also names with special characters like - (dash).

              These all fail:
              NFL 2K
              F-Zero
              Ms. Pac-man

              1 Reply Last reply Reply Quote 0
              • K
                KevinsHope
                last edited by

                Yes. And then...later...with a similar name, it list the exact one it could not find Ms. Pac-Man...right there in the list...spelled exactly that way, hyphen and all! LOL

                1 Reply Last reply Reply Quote 1
                • HurricaneFanH
                  HurricaneFan
                  last edited by

                  Is it possible to add another datasource to the built in EmulationStation scraper. TheGamesDB is getting really bad lately and the built in scraper doesn't have any other datasource to switch to.

                  I know there is command line based scrapers, but I like the ability to scrape 1 game through the gui if I add a new rom.

                  K 1 Reply Last reply Reply Quote 0
                  • K
                    KevinsHope @HurricaneFan
                    last edited by KevinsHope

                    @hurricanefan No...or...I guess I should say...not from within retropie...but your best solution for your woes is probably using sselphs scraper to do batch scraping. I'm not going to tell you how many Arcade roms I have, but the sselphs scraper automatically identified and added images and data for all but 10% of the roms...so maybe 10 roms out of 100 wouldn't identify using this method...with the arcade. On the other hand, sselphs only recognized about 40% of the Genesis roms I have, so...it can be hit and miss too. Anyway, it is not really hard to access this scraper. Follow the tutorial here:

                    1 Reply Last reply Reply Quote 0
                    • First post
                      Last post

                    Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                    Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.