RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login
    Please do not post a support request without first reading and following the advice in https://retropie.org.uk/forum/topic/3/read-this-first

    Scraping PSX with 6 threads dreadfully slow!

    Scheduled Pinned Locked Moved Help and Support
    scrapingscreenscraperskraper
    26 Posts 4 Posters 5.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • simpleethatS
      simpleethat @Dinsdale
      last edited by

      @Dinsdale I'm not sure that scraping while they're compressed is going to work for PSX that might be part of your problem.

      1 Reply Last reply Reply Quote 0
      • D
        Dinsdale
        last edited by

        I see... uncompressing them is going to make the folder very messy, and there may be conflicting names for the bin files... hmmm.

        simpleethatS 1 Reply Last reply Reply Quote 0
        • D
          Dinsdale
          last edited by

          Actually, just saw that 7z can extract each archive into it's own folder... but question is, will Retropie/Emulationstation then read down the extra level to find the files?

          simpleethatS 1 Reply Last reply Reply Quote 0
          • simpleethatS
            simpleethat @Dinsdale
            last edited by

            @Dinsdale you're going to have to uncompress them if you want to use the emulators included in retropie. That or go track down the games in PBP format.

            1 Reply Last reply Reply Quote 0
            • D
              Dinsdale
              last edited by

              Never heard of PBP, will research that - but should be noted the games do run in Retropie fine in 7z.

              simpleethatS 1 Reply Last reply Reply Quote 0
              • simpleethatS
                simpleethat @Dinsdale
                last edited by

                @Dinsdale I believe it will read subfolders but it's not something I've played with

                1 Reply Last reply Reply Quote 0
                • simpleethatS
                  simpleethat @Dinsdale
                  last edited by

                  @Dinsdale interesting as the docs don't list 7z as an accepted format.

                  there's also this method to consolidate you PSX games

                  1 Reply Last reply Reply Quote 1
                  • D
                    Dinsdale
                    last edited by

                    Thanks! I've done a test and I'm going to try this... means I need to stop scraping so the risk is ultimately it doesn't work and I have to repeat the last 24 hours, but this isn't the end of the world.

                    simpleethatS 1 Reply Last reply Reply Quote 0
                    • simpleethatS
                      simpleethat @Dinsdale
                      last edited by simpleethat

                      @Dinsdale Just a reminder, if you're going to scrape bin/cue format games move the bin files elsewhere. If you don't Skraper will try to scrape every file in the directory and it will be a huge mess

                      EDIT: unless they've done something about that in a recent update.

                      1 Reply Last reply Reply Quote 1
                      • D
                        Dinsdale
                        last edited by

                        Funny you say that, but I found that with my SEGA CD collection and put that to the side as something to investigate later, BUT, this CHDMAN actually does the Sega CD games as well, so I'm actually trying that right now first... and once it's done, I'll try the much larger PSX collection.

                        1 Reply Last reply Reply Quote 0
                        • D
                          Dinsdale
                          last edited by

                          Alright, so scraping is now ultra fast on the CHD files. Converting to CHD has also allowed me to clean up the Sega CD list. There's just one problem, the roms now don't run.

                          For the Sega CD, I know picodrive doesn't support that format, but even with genesis-plus, it doesn't work.

                          For PSX, psx-reamred doesn't work either...

                          Help!

                          1 Reply Last reply Reply Quote 0
                          • simpleethatS
                            simpleethat
                            last edited by simpleethat

                            Did you use the CHDMAN from the download link in the docs? They indicate for PSX that it has to be CHD V5. Sega CD indicates V1-5. Not really sure why they aren't working. I use bin/cue and have converted the bins to only be on file.

                            EDIT: It looks like only the libretro (lr-) emulators support CHD for PSX

                            1 Reply Last reply Reply Quote 0
                            • D
                              Dinsdale
                              last edited by

                              Yeah, and it’s lr-pcsx-rearmed that I’m using for PSX so it should work.

                              Yes, I used CHD from the link you provided.

                              There must be something I’m missing. I added chd to the es_systems.cfg file for PSX, so the file shows up fine - it just doesn’t load, I get the loading screen then it goes right back to the menu.

                              I am really keen to get this to work because now the ROMs are nice and tidy!

                              I’m on RetroPie 4.4. I can’t use 4.5 as it causes another issue I have with Bluetooth.

                              mituM 1 Reply Last reply Reply Quote 0
                              • mituM
                                mitu Global Moderator @Dinsdale
                                last edited by

                                @Dinsdale said in Scraping PSX with 6 threads dreadfully slow!:

                                There must be something I’m missing. I added chd to the es_systems.cfg file for PSX, so the file shows up fine - it just doesn’t load, I get the loading screen then it goes right back to the menu.

                                You're probably using an older version of lr-pcsx-rearmed, support for it was added last year in May, so it's relatively recent.

                                1 Reply Last reply Reply Quote 0
                                • D
                                  Dinsdale
                                  last edited by

                                  Thanks. What’s the correct way of updating just thag emulator? I should probably do genesis-plus-gx too.

                                  Reason I ask is yesterday I tried this and my keyboard stopped responding. Saw a thread on this very forum where updating something was the culprit - I obviously want to avoid that - as the keyboard is key during the setup phase.

                                  1 Reply Last reply Reply Quote 0
                                  • mituM
                                    mitu Global Moderator
                                    last edited by

                                    What’s the correct way of updating just thag emulator? I should probably do genesis-plus-gx too.

                                    You can update individual emulators from the RetroPie-Setup menu - https://retropie.org.uk/docs/Updating-RetroPie/#updatinginstalling-individual-packages - but you should also have the latest RetroPie-Setup image.

                                    Reason I ask is yesterday I tried this and my keyboard stopped responding.

                                    That's weird, but I don't think there's something in the update scripts that would stop the keyboard during the update.

                                    1 Reply Last reply Reply Quote 0
                                    • D
                                      Dinsdale
                                      last edited by

                                      I've updated lr-pcsx-rearmed from source. Now PSX games work with CHD.

                                      I did the same for lr-genesis-plus-gx, unfortunately Sega CD games still aren't working with CHD.

                                      Here's the thread about the keyboard: https://retropie.org.uk/forum/topic/5296/help-keyboard-not-working/16 - though this may be a red herring.

                                      I'll try Retropie-Setup-Script and get back to you.

                                      mituM 1 Reply Last reply Reply Quote 0
                                      • mituM
                                        mitu Global Moderator @Dinsdale
                                        last edited by

                                        @Dinsdale said in Scraping PSX with 6 threads dreadfully slow!:

                                        Here's the thread about the keyboard: https://retropie.org.uk/forum/topic/5296/help-keyboard-not-working/16 - though this may be a red herring.

                                        A single 3 years old post is hardly an indication of a recurring/general problem with the update process.

                                        1 Reply Last reply Reply Quote 0
                                        • D
                                          Dinsdale
                                          last edited by Dinsdale

                                          Maybe not, but I just wanted to point out the reasoning for my theory, and that's all it was based on. Anyway, I've updated the RetroPie Script (but not the kernel) and the keyboard still works.

                                          Sega CD though, still nothing... any other suggestions? I've set the Sega CD emulator to lr-genesis-plus-gx and it still isn't working.

                                          Here's what the runcommand.log says.

                                          Parameters: 
                                          Executing: /opt/retropie/emulators/retroarch/bin/retroarch -L /opt/retropie/libretrocores/lr-genesis-plus-gx/genesis_plus_gx_libretro.so --config /opt/retropie/configs/segacd/retroarch.cfg "/home/pi/RetroPie/roms/segacd/Terminator, The (USA).chd" --appendconfig /dev/shm/retroarch.cfg
                                          

                                          EDIT: Ah, may need a different BIOS for genesis-plus-gx

                                          1 Reply Last reply Reply Quote 0
                                          • mituM
                                            mitu Global Moderator
                                            last edited by

                                            Enable verbose logging from the Runcommand launch menu when running the game, then post the output of /dev/shm/runcommand.log to pastebin.com.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.