• Recent
  • Tags
  • Popular
  • Home
  • Docs
  • Register
  • Login
RetroPie forum home
  • Recent
  • Tags
  • Popular
  • Home
  • Docs
  • Register
  • Login

FYI Major update to sselph/scraper

Scheduled Pinned Locked Moved Ideas and Development
scrapersselph scraper
10 Posts 7 Posters 2.1k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • S
    sselph
    last edited by 23 Apr 2017, 02:13

    A while back I updated retropie to download a more modern version of Go, the language I write my scraper in. I did this so I could switch to using more modern features. This was needed for various reasons but tonight I switched my code to require it. Please let me know if there are issues at https://github.com/sselph/scraper/issues or you could reply here and I will probably see it.

    The notes for this release are at https://github.com/sselph/scraper/releases/tag/v1.3.0

    Auto-scraper: https://github.com/sselph/scraper
    Donate to Extra-Life 2018 and help save lives: https://goo.gl/diu5oU

    B V 2 Replies Last reply 23 Apr 2017, 02:24 Reply Quote 7
    • B
      BuZz administrators @sselph
      last edited by 23 Apr 2017, 02:24

      @sselph thanks!

      To help us help you - please make sure you read the sticky topics before posting - https://retropie.org.uk/forum/topic/3/read-this-first

      1 Reply Last reply Reply Quote 2
      • B
        bleuge
        last edited by 25 Apr 2017, 08:15

        Best scraper around, thanks you very much for updates.

        1 Reply Last reply Reply Quote 0
        • V
          vretro @sselph
          last edited by vretro 5 Apr 2017, 00:03 1 May 2017, 13:09

          Thank you for your continued efforts with scraper @sselph

          I have a feature request I'd like to run past you.

          I've found many sources of data available in scraper provide inaccurate player counts. Is it possible to have an option to always append a higher player count if a higher value exists, when you initiate additional scrapes from any source?

          This way inaccurate values will more likely be replaced with accurate values. Also values which are already correct, are less ikely to be replaced with smaller inaccurate values.

          Thanks

          R 1 Reply Last reply 3 May 2017, 13:14 Reply Quote 0
          • D
            DarkWolf
            last edited by 1 May 2017, 13:12

            Omg thank you! It was a pain for me to use the Universal XML Scraper with ScreenScraper to get the mix images.

            System: Raspberry 3 Model B, RetroPie 4.2.1
            Storage: 16gb Micro SD, 64gb USB Drive
            Theme: Updated NBBA

            Documentation solves many problems: https://retropie.org.uk/docs/

            1 Reply Last reply Reply Quote 0
            • R
              robertybob @vretro
              last edited by 3 May 2017, 13:14

              @vretro Basically, if player count = 0, then make it a 1 ? I'd like this too, although I guess it's not too difficult to find and replace within the XML.

              V 1 Reply Last reply 3 May 2017, 23:08 Reply Quote 0
              • V
                vretro @robertybob
                last edited by 3 May 2017, 23:08

                @robertybob not exactly. I've edited my post to now read as follows to make it clearer “Is it possible to have an option to always append a higher player count if a higher value exists”

                This option presumes the higher value to be the truth, so should overwrite lower values. It seems 1 is often used as a default, when the actual number isn't known.

                /cc @sselph

                S 1 Reply Last reply 4 May 2017, 04:39 Reply Quote 0
                • S
                  sselph @vretro
                  last edited by 4 May 2017, 04:39

                  @vretro This isn't something that would be trivial but it is essentially an option to merge multiple sources of information. Feel free to open an issue on github but I have to warn you that I have been busy and it might take some time to get to it.

                  Auto-scraper: https://github.com/sselph/scraper
                  Donate to Extra-Life 2018 and help save lives: https://goo.gl/diu5oU

                  V 1 Reply Last reply 14 Jul 2017, 19:16 Reply Quote 1
                  • A
                    alaindesjardins
                    last edited by 6 May 2017, 14:44

                    Your scraper was super helpful for my needs. I went through all the command line arguments thoroughly and was able to get the results i wanted. I made a feature suggestion on the repo in hopes that you would consider adding options that would have made it even better. https://github.com/sselph/scraper/issues/146

                    Let me know if you have seen those features as they would be a boon to many users out there. Mainly, I am looking for options to add my own data to the gamelist.xml using the scraper. Anyways, if you get a chance to look at it I hope you consider adding those options :)

                    1 Reply Last reply Reply Quote 0
                    • V
                      vretro @sselph
                      last edited by 14 Jul 2017, 19:16

                      Thanks @sselph - it took a while, but I added a feature request over on Github:
                      https://github.com/sselph/scraper/issues/167

                      1 Reply Last reply Reply Quote 0
                      • First post
                        Last post

                      Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                      Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.

                        This community forum collects and processes your personal information.
                        consent.not_received