RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login
    Please do not post a support request without first reading and following the advice in https://retropie.org.uk/forum/topic/3/read-this-first

    Has Selph's Scraper been updated and now broken?

    Scheduled Pinned Locked Moved
    Help and Support
    scraping selphs scraper
    2
    11
    3.3k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • hansolo77H
      hansolo77
      last edited by hansolo77

      I've added a bunch of new systems to my setup, including Amstrad CPC, Atari 5200, Atari 800, Atari St, and Sega SG-1000. The last couple of days I've been trying to scrape with UXS but there is a lot of stuff missing. So I was about to go to step #2 and scrape with Selph's. Then I'll last-ditch try the internal EmulationStation scraper. When I tried to scrape Amstrad CPC in Selph's though, it ran through the ROMs really quick and came back saying it was done. I did it again and watched and it was actually saying a bunch of stuff like this:

      2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blackbeard (1988)(Topo Soft)(es).dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blackbeard (1988)(Topo Soft)(es)[a].dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blackbeard (1988)(Topo Soft)[t +2].dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blade Runner (1986)(CRL Group).dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blade Warrior (1989)(Codemasters).dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blade Warrior (1989)(Codemasters)[t +2].dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blagger (1985)(Alligata Software - Amsoft).dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blasted Squares (19xx)(-)(de).dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blasteroids (1987)(Image Works)[cr Mc Spe].dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blasteroids (1987)(Image Works)[t].dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blazing Thunder (1990)(Hi-Tec Software)[cr GPA][t GPA].dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blip! (1989)(Silverbird Software).dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Block Invaders (2002)(Richard Wilson).dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Block Invaders (2002)(Richard Wilson)[a].dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blockbusters (1984)(Macsen Software).dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blockhaus (1986)(P. Balhade)(fr).dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blokker (1985)(CT).dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blood Brothers (1988)(Gremlin Graphics Software).dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blood Brothers (1988)(Gremlin Graphics Software)[a][cpm version].dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blood Brothers (1988)(Gremlin Graphics Software)[cpm version].dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blood Valley (1987)(Gremlin Graphics Software)(M3).dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Bloodwych (1990)(Image Works)[cr CBS].dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blue Crystal, The (1985)(Rainbow Arts)(Disk 1 of 2)[cr NPS].dsk
2016/12/14 22:25:58 Can't find original path: /home/pi/RetroPie/roms/amstradcpc/Blue Crystal, The (1985)(Rainbow Arts)(Disk 2 of 2)[cr NPS].dsk

      I checked the path, and it IS correct... I then tried to run it against Atari 5200, and initially had the same type of results:

      2016/12/14 22:26:52 Can't find original path: /home/pi/RetroPie/roms/atari5200/QIX (USA).zip
2016/12/14 22:26:52 Can't find original path: /home/pi/RetroPie/roms/atari5200/Quest for Quintana Roo (USA).zip
2016/12/14 22:26:52 Can't find original path: /home/pi/RetroPie/roms/atari5200/RealSports Baseball (USA).zip
2016/12/14 22:26:52 Can't find original path: /home/pi/RetroPie/roms/atari5200/RealSports Basketball (USA) (82-11-05) (Proto).zip
2016/12/14 22:26:52 Can't find original path: /home/pi/RetroPie/roms/atari5200/RealSports Basketball (USA) (83-10-31) (Proto).zip
2016/12/14 22:26:52 Can't find original path: /home/pi/RetroPie/roms/atari5200/RealSports Basketball (USA).zip
2016/12/14 22:26:52 Can't find original path: /home/pi/RetroPie/roms/atari5200/RealSports Football (USA).zip
2016/12/14 22:26:52 Can't find original path: /home/pi/RetroPie/roms/atari5200/RealSports Soccer (USA).zip
2016/12/14 22:26:52 Can't find original path: /home/pi/RetroPie/roms/atari5200/RealSports Tennis (USA).zip
2016/12/14 22:26:52 Can't find original path: /home/pi/RetroPie/roms/atari5200/Rescue on Fractalus! (USA).zip
2016/12/14 22:26:52 Can't find original path: /home/pi/RetroPie/roms/atari5200/Road Runner (USA) (Proto).zip

      Then it started a 2nd pass and was actually finding stuff. So I'm confused. Before starting this pass, I ran the RetroPie-Setup updates and also updated Selph's Scraper script. So I'm using the latest version (as of the writing of this message). Any ideas what might have happened?

      Who's Scruffy Looking?

      1 Reply Last reply Reply Quote 0
      • hansolo77H
        hansolo77
        last edited by hansolo77

        I think I might be on to something when figuring this out on my own.... So I restarted the system and exited EmulationStation. What happened next was shocking. My screen went black like it should, then started reading off a whole bunch of

        lvl10:   Could not find <gameList> node in gamelist "/home/pi/RetroPie/roms/amstradcpc"
lvl10:   Could not find <gameList> node in gamelist "/home/pi/RetroPie/roms/atari2600"
lvl10:   Could not find <gameList> node in gamelist "/home/pi/RetroPie/roms/atari5200"
lvl10:   Could not find <gameList> node in gamelist "/home/pi/RetroPie/roms/atari7280"
lvl10:   Could not find <gameList> node in gamelist "/home/pi/RetroPie/roms/atari800"
lvl10:   Could not find <gameList> node in gamelist "/home/pi/RetroPie/roms/atarilynx"
lvl10:   Could not find <gameList> node in gamelist "/home/pi/RetroPie/roms/atarist"

        It did that for every system on my Pi. This comes right after scraping with UXS. So maybe UXS is to blame?

        Who's Scruffy Looking?

        hansolo77H 1 Reply Last reply Reply Quote 0
        • hansolo77H
          hansolo77
          last edited by hansolo77

          I'm still having problems with this. I erased all instances of gamelist.xml I have on my SD card. I then installed the latest version of UXS (I like it's MIX artwork). After spending 1 Day and 20 Hours scraping my entire system (yeah, I have a lot of systems lol), I then loaded up Selph's to do an append on anything UXS missed. Once I started the scan on the first system (Amstrad CPC), it started doing that whole big "Can't find original path: /home/pi/RetroPie/roms/amstradcpc/" stuff listing every single file. If I let it run through the whole thing, it starts over and then does a bunch of "skipping already in gamelist". The path it shows is the exact same. So I'm not sure what's going on here. Can anybody help?

          For the record, I am storing gamelist.xml files in the /opt/retropie/configs/all/emulationstation/gamelists path, with each system in it's own folder. This is actually symlinked from /home/pi/.emulationstation/gamelists So like:

          • /opt/retropie/configs/all/emulationstation/gamelists/amstradcpc/gamelist.xml
          • /opt/retropie/configs/all/emulationstation/gamelists/atari2600/gamelist.xml
          • /opt/retropie/configs/all/emulationstation/gamelists/nes/gamelist.xml
          • etc

          I checked, and there are no other gamelist.xml files hidden elsewhere (like in the roms path).

          Who's Scruffy Looking?

          1 Reply Last reply Reply Quote 0
          • vbsV
            vbs
            last edited by

            According to the scraper's website here:
            https://github.com/sselph/scraper

            It does only support this systems:

            An auto-scraper for EmulationStation written in Go using hashes. This currently works with NES, SNES, N64, GB, GBC, GBA, MD, SMS, 32X, GG, PCE, A2600, LNX, MAME/FBA(see below), Dreamcast(bin/gdi), PSX(bin/cue), ScummVM, SegaCD ROMs.
            1 Reply Last reply Reply Quote 0
            • hansolo77H
              hansolo77
              last edited by

              Ah, so you think all the problems are because it doesn't actually have the ability to scan those systems?

              Who's Scruffy Looking?

              vbsV 1 Reply Last reply Reply Quote 0
              • vbsV
                vbs @hansolo77
                last edited by

                @hansolo77
                Hm, I don't know, probably only some of your problems are related to systems not supported by sselph's.

                vbsV 1 Reply Last reply Reply Quote 0
                • vbsV
                  vbs @vbs
                  last edited by vbs

                  @hansolo77
                  Have you tried to shutdown ES, then delete all files named "gamelist.xml" (everywhere) and then scrape a system using UXS?

                  1 Reply Last reply Reply Quote 0
                  • hansolo77H
                    hansolo77
                    last edited by

                    Yup. It's after I scrape with UXS that I start having the problems. I think I'm on to something though. I've clearly narrowed it down to UXS, as using Selph's alone works fine. Using UXS though causes really slow boot ups, and when ES closes I get those errors about the "<gameList> node". I did a comparison of the way Selph creates the xml vs UXS, and it looks like UXS is making them differently. So I'm experimenting now.

                    Who's Scruffy Looking?

                    vbsV 1 Reply Last reply Reply Quote 0
                    • vbsV
                      vbs @hansolo77
                      last edited by

                      @hansolo77
                      I had that error once when ES was complaining about the gameList-node. In my case I had a 0 byte sized gamelist.xml that ES tried to read.
                      I can't believe that your ES cannot read the gamelist.xml generated by UXS, I mean it does for everyone else so I doubt it is a general error related to the format of the gamelist.xml that UXS generates.

                      1 Reply Last reply Reply Quote 0
                      • hansolo77H
                        hansolo77 @hansolo77
                        last edited by

                        I don't get it for just 1 file though, it's all of them:

                        @hansolo77 said in Has Selph's Scraper been updated and now broken?:

                        lvl10:   Could not find <gameList> node in gamelist "/home/pi/RetroPie/roms/amstradcpc"
lvl10:   Could not find <gameList> node in gamelist "/home/pi/RetroPie/roms/atari2600"
lvl10:   Could not find <gameList> node in gamelist "/home/pi/RetroPie/roms/atari5200"
lvl10:   Could not find <gameList> node in gamelist "/home/pi/RetroPie/roms/atari7280"
lvl10:   Could not find <gameList> node in gamelist "/home/pi/RetroPie/roms/atari800"
lvl10:   Could not find <gameList> node in gamelist "/home/pi/RetroPie/roms/atarilynx"
lvl10:   Could not find <gameList> node in gamelist "/home/pi/RetroPie/roms/atarist"

                        I agree though, it is strange that I'm the only one who notices this. Unless I'm the only one who's being OCD about metadata and trying to scrape from multiple utilities.

                        Who's Scruffy Looking?

                        vbsV 1 Reply Last reply Reply Quote 0
                        • vbsV
                          vbs @hansolo77
                          last edited by

                          @hansolo77
                          I would bet money that you have somewhere a broken (empty?) gamelist.xml.
                          Try to delete all instances of gamelist.xml on your system.
                          Or you could try to ES manually using parameter "--debug", then you should see the full path of the gamelist.xml ES is trying to read.

                          1 Reply Last reply Reply Quote 0
                          • First post
                            Last post

                          Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                          Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.