RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login
    Please do not post a support request without first reading and following the advice in https://retropie.org.uk/forum/topic/3/read-this-first

    Steven Selph scraper ignor scraping PSX bin files

    Scheduled Pinned Locked Moved Help and Support
    15 Posts 4 Posters 6.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R
      ruxton_bruce
      last edited by

      Is there a way to ignor the .bin files being scraped. I have 1400 PSX roms in .cue and .bin files. I have removed .bin and .BIN from the esconfig file but when I use scraper it is adding double entries as it scrapes both the .cue and .bin files.

      Any help much appreciated.

      1 Reply Last reply Reply Quote 0
      • FloobF
        Floob
        last edited by

        Which scraper are you using? Can you include more info?
        https://retropie.org.uk/forum/topic/3/read-this-first

        Please read the Docs before asking a new question.
        RetroPie Help Guides: https://goo.gl/3gcNsT

        R 1 Reply Last reply Reply Quote 0
        • R
          ruxton_bruce @Floob
          last edited by

          @Floob I thought I made it clear in the title that I was using the Steven Selph scraper.

          Anyway I run it by typing sudo ./retropie-setup.sh

          It is the scraper found within the retropie setup.

          1 Reply Last reply Reply Quote 0
          • FloobF
            Floob
            last edited by

            The .bin extension should already be removed - which version of RetroPie are you using?
            https://retropie.org.uk/forum/topic/3/read-this-first

            https://github.com/retropie/retropie-setup/wiki/Playstation-1

            Please read the Docs before asking a new question.
            RetroPie Help Guides: https://goo.gl/3gcNsT

            R 1 Reply Last reply Reply Quote 0
            • R
              ruxton_bruce @Floob
              last edited by

              @Floob Thanks again for the quick reply. I have updated to the latest version just now and am about to scrape my PSX folder again using Stephen Selphs Scraper. Will let you know how it turns out.

              Cheers

              1 Reply Last reply Reply Quote 0
              • S
                sselph
                last edited by

                My scraper reads every .cue file and then every .bin file listed in the .cue is skipped. When hashing it hashes the cue then every bin until it finds a match. Is it possible that the .bin file has been renamed so that it no longer matches what is listed in the .cue file?

                Auto-scraper: https://github.com/sselph/scraper
                Donate to Extra-Life 2018 and help save lives: https://goo.gl/diu5oU

                R 2 Replies Last reply Reply Quote 0
                • R
                  ruxton_bruce @sselph
                  last edited by

                  @sselph Thanks very much for the reply Stephen. Yes you are spot on. I have renamed the .cue name to more display friendly name. What do you think is the best way to revert from this situation.

                  Thanks very much @sselph

                  1 Reply Last reply Reply Quote 0
                  • R
                    ruxton_bruce @sselph
                    last edited by

                    @sselph Hi Steven. After receiving your messaged I cancelled a scrape that was half way through. I had just updated to the latest version of Retropie and thought i'd try to scrape again. The scrape had got down to G in my PSX roms and after cancelling I thought I would see if it had done the same thing. It looks like it has only added the .cue files this time?? Is this something that may have been fixed in the latest update??

                    S 1 Reply Last reply Reply Quote 0
                    • S
                      sselph @ruxton_bruce
                      last edited by

                      @ruxton_bruce Cool. My scraper's logic hasn't changed in a while, but I haven't been keeping up with RetroPie changes so it could be fixed on their end.

                      Auto-scraper: https://github.com/sselph/scraper
                      Donate to Extra-Life 2018 and help save lives: https://goo.gl/diu5oU

                      R 1 Reply Last reply Reply Quote 0
                      • R
                        ruxton_bruce @sselph
                        last edited by

                        @sselph Hi Steven. Thanks for your help yesterday mate. I still had problems with having the .cue files named differently so I took the plunge and reverted all the names back to match the .bin file name. After doing this I re ran your scraper. It has worked this time except any game which has multiple audio tracks (which are the named "game title (Track 02).bin) etc. have a second entry in the gamelist.xml file. If the game has 20 tracks it still only has two entries, one for the .cue file and one for the .bin file. Is there some way I can remove the second entries from showing as there are a lot of games with multiple tracks.

                        Cheers mate

                        S 1 Reply Last reply Reply Quote 0
                        • S
                          sselph @ruxton_bruce
                          last edited by

                          @ruxton_bruce

                          My theory as to why this is happening, like I mentioned before, was that the bin files don't match the contents of the cue file. The cue file if you open it in a text editor looks something like the following. My scraper parses that to read each FILE entry and uses that to consolidate things. Can you check what one of the cue file looks like and what the bins are named and let me know. This code is the first time I've worked with these types of files so there might be something I'm not considering(case sensitivity, etc). I'll also try and take a look and see if I can reproduce it if there is an error.

                          $ cat "Die Hard Trilogy (USA) (v1.0).cue"
                          FILE "Die Hard Trilogy (USA) (v1.0) (Track 01).bin" BINARY
                            TRACK 01 MODE2/2352
                              INDEX 01 00:00:00
                          FILE "Die Hard Trilogy (USA) (v1.0) (Track 02).bin" BINARY
                            TRACK 02 AUDIO
                              INDEX 00 00:00:00
                              INDEX 01 00:02:00
                          FILE "Die Hard Trilogy (USA) (v1.0) (Track 03).bin" BINARY
                            TRACK 03 AUDIO
                              INDEX 00 00:00:00
                              INDEX 01 00:02:00
                          ...
                          

                          Auto-scraper: https://github.com/sselph/scraper
                          Donate to Extra-Life 2018 and help save lives: https://goo.gl/diu5oU

                          R 1 Reply Last reply Reply Quote 0
                          • G
                            geneworld
                            last edited by geneworld

                            Quick and easy method would be to move all the *.bin files to another directory outside of the psx one, run the scraper, then just move them back. Not the best way but its something to get you by.

                            On looking at the parameters for the scraper (https://github.com/RetroPie/RetroPie-setup/wiki/scraper) it doesn't seem like you can isolate an extension otherwise you can easily run it from the command line with something like

                            pi@raspberrypi:~ $ /opt/retropie/supplementary/scraper/scraper -output_file="/home/pi/emulationstation/gamelists/gamelist.xml" -missing="/home/pi/.emulationstation/gamelists/missing.txt" -image_dir="/home/pi/.emulationstation/downloaded_images" -image_path="/home/pi/.emulationstation/downloaded_images" -rom_dir="/home/pi/RetroPie/roms/psx" -rom_path="." -thumb_only -workers 4

                            You'd need something like extension=".cue" (Is that an option? @sselph ) If you run it this way also include append=true so it doesn't wipe out your other system's data.

                            R 1 Reply Last reply Reply Quote 0
                            • R
                              ruxton_bruce @sselph
                              last edited by

                              @sselph Hi Steven. Thanks again for the reply. Upon closer inspection you are once again correct. In the example below:-

                              FILE "Air Combat (USA) (Track 01).bin" BINARY
                              TRACK 01 MODE2/2352
                              INDEX 01 00:00:00
                              FILE "Air Combat (USA) (Track 02).bin" BINARY
                              TRACK 02 AUDIO
                              INDEX 00 00:00:00
                              INDEX 01 00:02:00
                              FILE "Air Combat (USA) (Track 03).bin" BINARY
                              TRACK 03 AUDIO
                              INDEX 00 00:00:00
                              INDEX 01 00:02:00
                              FILE "Air Combat (USA) (Track 04).bin" BINARY
                              TRACK 04 AUDIO
                              INDEX 00 00:00:00
                              INDEX 01 00:02:00
                              FILE "Air Combat (USA) (Track 05).bin" BINARY
                              TRACK 05 AUDIO
                              INDEX 00 00:00:00
                              INDEX 01 00:02:00

                              The .bin and .cue file were actually named Air Combat (USA).bin and Air Combat (USA).cue without the (Track 01) as shown in the contents of the .cue file. Once again my stupid mistake for renaming the .cue and .bin files. I did the laborious task of editing the gamelist.xml file and removing all entries with .bin in them (all 450 of them) using Notepad++. Everything looks great now in Emulation Station but I'm wandering if I'm going to experience issues with the .bin file not having the (Track 01) in the filename as is listed in the contents of the .cue file. They seem to run fine though

                              Thanks again for the replies.

                              Cheers

                              1 Reply Last reply Reply Quote 0
                              • R
                                ruxton_bruce @geneworld
                                last edited by

                                @geneworld I wish I had thought of that before manually editing the gamelist.xml file. Would have saved me a tonne of time.

                                We live and learn :)

                                Cheers mate

                                S 1 Reply Last reply Reply Quote 0
                                • S
                                  sselph @ruxton_bruce
                                  last edited by

                                  @ruxton_bruce I'm glad you have it sorted out. I've opened an issue to track adding a check for bin files named the same as the cue. I imagine that it is common enough that the emulator handles it. https://github.com/sselph/scraper/issues/100

                                  Auto-scraper: https://github.com/sselph/scraper
                                  Donate to Extra-Life 2018 and help save lives: https://goo.gl/diu5oU

                                  1 Reply Last reply Reply Quote 0
                                  • First post
                                    Last post

                                  Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                  Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.