RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login

    [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture

    Scheduled Pinned Locked Moved Projects and Themes
    scrapescraperuxs
    728 Posts 111 Posters 739.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • K
      kbronctjr
      last edited by

      @screech sorry to bother you, but do you know why the scraper doesn't detect some roms even when the names are registered in the list of rom names for that game in screenscraper?

      E 1 Reply Last reply Reply Quote 0
      • E
        EctoOne @kbronctjr
        last edited by

        @kbronctjr The games you listed before contain special characters (i had the same problems with Chip's Challenge because of the ') and there always was a problem with them. The only thing you can try is to use the hash method instead of just names.

        K 1 Reply Last reply Reply Quote 0
        • K
          kbronctjr @EctoOne
          last edited by

          @EctoOne The problem is that the psx games are compressed in PBP format and the crc is not the same I guess.

          The GBA splinter cell is a regular version but still is undetected with hash method

          1 Reply Last reply Reply Quote 0
          • screechS
            screech
            last edited by

            I still have problem with Filename Match with the '.
            It's not only on my side... I ask the API with an URL request, so I can put ' directly in the url or urlencode it.. but in both case, the API don't find it... I need to work with the API dev but he (and me) are really busy these last time... can't manage to find some time to check that.. Sorry

            Still the Hash method who is the best to be sure match is OK, but with PBP it's a problem (every PBP will be different so we can't add all PBP Hash in the DB, it's a nonsense. They're some, but it's because they are coming from "kown" romset)...

            K 1 Reply Last reply Reply Quote 1
            • K
              kbronctjr @screech
              last edited by

              @screech don't worry, you already have done a lot of work by providing this application.

              For one of the games I tried using and original .cue and hashing it and it worked, then I rename the rom name to the pbp one in the gamelist. Hashing bin takes a lot of time so if I find .cue for the remaining roms I will use that method

              1 Reply Last reply Reply Quote 0
              • hansolo77H
                hansolo77
                last edited by

                So I finished scraping all my systems. The new version is a vast improvement from the older one I was using before. It's still a little buggy though. When I looked at the game lists, specifically CD games that have multiple disks, it doesn't display things like "disk 1", "disk 2", etc. So I'm kind of lost there. I'm going to try rescraping that system (psx) with a different rom naming setting.

                Something else I'd like to ask the community in general... Has anybody experienced weird playback of the videos inside RetroPie? I'm on a Raspberry Pi 3 using the Carbon theme (because it's the least cpu/gpu intensive theme). All of my videos are playing back green, super pixelated, and laggy. Is this something that I can fix, or is it something that's just going to have to be lived with until better support for video comes out? If I have to wait on a solution, is there a way to have it display the image art first?

                Who's Scruffy Looking?

                AlexMurphyA 1 Reply Last reply Reply Quote 0
                • AlexMurphyA
                  AlexMurphy Banned @hansolo77
                  last edited by

                  @hansolo77 I don't use videos but everyone says that the res has to be shrunk to at least 320X240 @ 30fps for them to run correctly.

                  hansolo77H 1 Reply Last reply Reply Quote 0
                  • hansolo77H
                    hansolo77 @AlexMurphy
                    last edited by

                    @AlexMurphy Ah.. is there a utility to batch process this?

                    Who's Scruffy Looking?

                    AlexMurphyA 1 Reply Last reply Reply Quote 0
                    • AlexMurphyA
                      AlexMurphy Banned @hansolo77
                      last edited by

                      @hansolo77 Again, handbrake is often spoken of.

                      1 Reply Last reply Reply Quote 0
                      • hansolo77H
                        hansolo77
                        last edited by

                        Thanks I'll give it a try.

                        Who's Scruffy Looking?

                        1 Reply Last reply Reply Quote 0
                        • hansolo77H
                          hansolo77 @screech
                          last edited by hansolo77

                          @screech where do I find the settings for this "RomsExcluded" functionality? Is it built into the program, or is it something I have to manually add to the ini? When I scrape SegaCD I still get multiple entries for the same game. Here is an example of the artwork it creates:

                          https://pastebin.com/UVQY7egb

                          Who's Scruffy Looking?

                          RionR 1 Reply Last reply Reply Quote 0
                          • RionR
                            Rion @hansolo77
                            last edited by Rion

                            @hansolo77 Your img files are split in to multiple bin files. Rebuild your Redump games by mounting the image. Select your cue file and mount it. Make new image with imgburn.

                            That will produce one bin/img file and one cue file.

                            FBNeo rom filtering
                            Mame2003 Arcade Bezels
                            Fba Arcade Bezels
                            Fba NeoGeo Bezels

                            1 Reply Last reply Reply Quote 0
                            • hansolo77H
                              hansolo77
                              last edited by hansolo77

                              That sounds too easy. :) Will the emulators for Sega read bin/cue? I thought I read somewhere the files had to be bin/cue/wav.

                              Who's Scruffy Looking?

                              1 Reply Last reply Reply Quote 0
                              • screechS
                                screech
                                last edited by

                                @hansolo77
                                Normally the RomsExcluded is already in the Retropie.xml Profil.
                                But maybe it won't work, I haven't so much tested this :S

                                The best way still to exclude *.wav from the scrape or like Rion Say : Create a cue+bin and scrape the cue.

                                1 Reply Last reply Reply Quote 0
                                • hansolo77H
                                  hansolo77
                                  last edited by

                                  Yeah, my xml has it listed, but it didn't catch it. I'm trying to rebuild bin/cue images now.

                                  Who's Scruffy Looking?

                                  1 Reply Last reply Reply Quote 0
                                  • hooperreH
                                    hooperre
                                    last edited by

                                    @screech Good news. Was able to download the new XML update without any firewall/Windows defender hits.

                                    4B ~ RPi PSU 5.1V / 3.0A ~ 32GB SanDisk microSD ~ 128GB USB

                                    1 Reply Last reply Reply Quote 0
                                    • hansolo77H
                                      hansolo77
                                      last edited by

                                      Ok, so mounting the images and remaking them in bin/cue doesn't work. For some reason, I'm now only able to scrap about 3 games. :(

                                      Who's Scruffy Looking?

                                      1 Reply Last reply Reply Quote 0
                                      • hansolo77H
                                        hansolo77
                                        last edited by

                                        :( I'm having all kinds of problems now. I managed to get my SegaCD to scrape with only 1 entry for each finally. Now I'm trying to re-scrape my PSX path. It goes through the "Please wait testing server.." then says it will kill EmulationStation. I click "OK" then it gives me an error "Roms path can't be reached". It's ubsurd. I don't know why it's giving me this error. I checked, and I can access the path fine.

                                        In Putty:

                                        pi@retropie:~ $ cd /home/pi/RetroPie/roms/psx
                                        pi@retropie:~/RetroPie/roms/psx $
                                        

                                        With a keyboard connected to the PI:

                                        pi@retropie:~ $ cd /home/pi/RetroPie/roms/psx
                                        pi@retropie:~/RetroPie/roms/psx $
                                        

                                        Through SFTP:
                                        /home/pi/RetroPie/roms/psx

                                        Everything works. So why is UXS telling me it can't access the path?! Argh!

                                        Who's Scruffy Looking?

                                        1 Reply Last reply Reply Quote 0
                                        • AndersHPA
                                          AndersHP
                                          last edited by

                                          Is Universal XML working when you boot roms off a USB?
                                          I've connected to my Pie via Windows 10, and can see that the scraper makes images in the home/pi/.emulationstation/downloaded_images folder, and the corresponding home/pi/.emulationstation/gamelist links to the above mentioned images, but the images does not show, and it seems that it's still the old scraping from the built-in / sselph scraper metadata that's showing underneath the games.

                                          I can see that in my other games folders, the gamelist is in the roms folder, but for this particular folder, it's not there. So where does it get the old metadata from? And how do I reset all this old metadata and start using Universal XML with my roms running off a usb?

                                          My "Bubble Bobble" Themed Bartop Arcade
                                          My Gameboy

                                          K 1 Reply Last reply Reply Quote 0
                                          • K
                                            kbronctjr @AndersHP
                                            last edited by

                                            @AndersHP I have my roms on a usb and have a symlink from the usb0 to the roms folder and the scraping is working well

                                            AndersHPA 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.