• Recent
  • Tags
  • Popular
  • Home
  • Docs
  • Register
  • Login
RetroPie forum home
  • Recent
  • Tags
  • Popular
  • Home
  • Docs
  • Register
  • Login

[SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture

Scheduled Pinned Locked Moved Projects and Themes
scrapescraperuxs
728 Posts 111 Posters 741.1k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • D
    Darksavior @Rion
    last edited by 23 May 2017, 10:25

    @Rion said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:

    @Darksavior Have you tried to whitelist screenscraper.fr in your ad block script/db?

    No, since I can access the website on my browser and with sselph's scraper.

    1 Reply Last reply Reply Quote 0
    • D
      DynaMight
      last edited by 23 May 2017, 16:04

      Is there any chance you can make it so when you upgrade to a new version it keeps the settings from the previous version? I had everything setup but now upgraded to the latest version (Copied the new .exe into the directory where the prev .exe etc was) but it started up fresh again and started asking to be setup again, which normally brings up an error about not being able to find the roms folder then I have to end the task and restart it.

      I dont personally use it to connect directly to my Pi so all my scraping is completed with roms on my PC and this program is a little tough to use for this unless you get the correct settings done.

      It is an awesome program though.

      1 Reply Last reply Reply Quote 1
      • K
        kbronctjr
        last edited by kbronctjr 23 May 2017, 17:50

        The scraper can't find the following psx rom name:

        Disney Lilo & Stitch en Problemas en el Paraiso (Spain)

        This name is registered in the screenscraper database.

        Another ROM name that doesn't detect is:

        "Tom Clancy's Splinter Cell (Europe) (En,Fr,De,Es,It,Nl)"

        Also registered in screenscraper

        @screech

        1 Reply Last reply Reply Quote 0
        • H
          hansolo77
          last edited by 24 May 2017, 14:02

          What is the best way to scrape systems like PSX that have multiple files for a single game? Meaning (for instance)...

          Alien Resurrection (USA) (Track 1).bin
          Alien Resurrection (USA) (Track 2).bin
          Alien Resurrection (USA) (Track 3).bin
          Alien Resurrection (USA) (Track 4).bin
          Alien Resurrection (USA) .cue
          

          This returns 5 copies of the same file, and 5 entries in my gamelist.xml. I just need one. Also, for SegaCD, I have things like this....

          Wing Commander (1994)(Electronic Arts)(NTSC)(US)(Track 1 of 6)[!].iso
          Wing Commander (1994)(Electronic Arts)(NTSC)(US)(Track 2 of 6)[!].wav
          Wing Commander (1994)(Electronic Arts)(NTSC)(US)(Track 3 of 6)[!].wav
          Wing Commander (1994)(Electronic Arts)(NTSC)(US)(Track 4 of 6)[!].wav
          Wing Commander (1994)(Electronic Arts)(NTSC)(US)(Track 5 of 6)[!].wav
          Wing Commander (1994)(Electronic Arts)(NTSC)(US)(Track 6 of 6)[!].wav
          Wing Commander (1994)(Electronic Arts)(NTSC)(US)[!].cue
          

          Which, inevitably generates 7 copies of all the artwork, and 7 entries for the same game in the gamelist.xml. It's really quite annoying. So what is the best way to scrape this?

          Who's Scruffy Looking?

          K 1 Reply Last reply 24 May 2017, 14:05 Reply Quote 0
          • K
            kbronctjr @hansolo77
            last edited by kbronctjr 24 May 2017, 14:05

            @hansolo77 go to the scraper settings and remove the .bin and .wav extensions from the list of file extensions to look for

            1 Reply Last reply Reply Quote 0
            • H
              hansolo77
              last edited by 24 May 2017, 14:05

              Yeah that's what I'm trying now.. I removed *.wav and *.bin and put them in the excluded line... seems to be working.

              Who's Scruffy Looking?

              K 1 Reply Last reply 24 May 2017, 14:07 Reply Quote 0
              • K
                kbronctjr @hansolo77
                last edited by kbronctjr 24 May 2017, 14:07

                @hansolo77 Also I think that retropie by default doesn't show rom files with .bin extension for psx, so only the entry of the .cue should appear in the front end

                1 Reply Last reply Reply Quote 0
                • S
                  screech
                  last edited by 24 May 2017, 15:05

                  @hansolo77
                  I added in the last version this :

                  • New Element Type : RomExcluded

                  Exemple :

                  <Element Type="RomExcluded">
                  	<Source_Type>Variable_Value</Source_Type>
                  	<Source_Value>%AutoExclude%</Source_Value>
                  	<AutoExcludeEXT>bin|img|iso|ccd|sub</AutoExcludeEXT>
                  	<AutoExcludeValue>(Track |[Bios]|(Bios)</AutoExcludeValue>
                  </Element>
                  

                  Mean :
                  When a duplicate name file (without extension) is found:

                  • it check the extension. If in list (bin|img|iso|ccd|sub) it won't scrape it
                  • it check if the file contain a value ("(Track ","[Bios]","(Bios)"). If yes, it won't scrape it

                  normaly it works (on my test)...
                  You can add in <AutoExcludeEXT> the WAV format. have you the last one ?

                  H 1 Reply Last reply 28 May 2017, 02:13 Reply Quote 0
                  • K
                    kbronctjr
                    last edited by 24 May 2017, 16:06

                    @screech sorry to bother you, but do you know why the scraper doesn't detect some roms even when the names are registered in the list of rom names for that game in screenscraper?

                    E 1 Reply Last reply 24 May 2017, 17:57 Reply Quote 0
                    • E
                      EctoOne @kbronctjr
                      last edited by 24 May 2017, 17:57

                      @kbronctjr The games you listed before contain special characters (i had the same problems with Chip's Challenge because of the ') and there always was a problem with them. The only thing you can try is to use the hash method instead of just names.

                      K 1 Reply Last reply 24 May 2017, 19:42 Reply Quote 0
                      • K
                        kbronctjr @EctoOne
                        last edited by 24 May 2017, 19:42

                        @EctoOne The problem is that the psx games are compressed in PBP format and the crc is not the same I guess.

                        The GBA splinter cell is a regular version but still is undetected with hash method

                        1 Reply Last reply Reply Quote 0
                        • S
                          screech
                          last edited by 25 May 2017, 10:10

                          I still have problem with Filename Match with the '.
                          It's not only on my side... I ask the API with an URL request, so I can put ' directly in the url or urlencode it.. but in both case, the API don't find it... I need to work with the API dev but he (and me) are really busy these last time... can't manage to find some time to check that.. Sorry

                          Still the Hash method who is the best to be sure match is OK, but with PBP it's a problem (every PBP will be different so we can't add all PBP Hash in the DB, it's a nonsense. They're some, but it's because they are coming from "kown" romset)...

                          K 1 Reply Last reply 25 May 2017, 10:23 Reply Quote 1
                          • K
                            kbronctjr @screech
                            last edited by 25 May 2017, 10:23

                            @screech don't worry, you already have done a lot of work by providing this application.

                            For one of the games I tried using and original .cue and hashing it and it worked, then I rename the rom name to the pbp one in the gamelist. Hashing bin takes a lot of time so if I find .cue for the remaining roms I will use that method

                            1 Reply Last reply Reply Quote 0
                            • H
                              hansolo77
                              last edited by 25 May 2017, 13:37

                              So I finished scraping all my systems. The new version is a vast improvement from the older one I was using before. It's still a little buggy though. When I looked at the game lists, specifically CD games that have multiple disks, it doesn't display things like "disk 1", "disk 2", etc. So I'm kind of lost there. I'm going to try rescraping that system (psx) with a different rom naming setting.

                              Something else I'd like to ask the community in general... Has anybody experienced weird playback of the videos inside RetroPie? I'm on a Raspberry Pi 3 using the Carbon theme (because it's the least cpu/gpu intensive theme). All of my videos are playing back green, super pixelated, and laggy. Is this something that I can fix, or is it something that's just going to have to be lived with until better support for video comes out? If I have to wait on a solution, is there a way to have it display the image art first?

                              Who's Scruffy Looking?

                              A 1 Reply Last reply 26 May 2017, 20:51 Reply Quote 0
                              • A
                                AlexMurphy Banned @hansolo77
                                last edited by 26 May 2017, 20:51

                                @hansolo77 I don't use videos but everyone says that the res has to be shrunk to at least 320X240 @ 30fps for them to run correctly.

                                H 1 Reply Last reply 26 May 2017, 23:04 Reply Quote 0
                                • H
                                  hansolo77 @AlexMurphy
                                  last edited by 26 May 2017, 23:04

                                  @AlexMurphy Ah.. is there a utility to batch process this?

                                  Who's Scruffy Looking?

                                  A 1 Reply Last reply 26 May 2017, 23:24 Reply Quote 0
                                  • A
                                    AlexMurphy Banned @hansolo77
                                    last edited by 26 May 2017, 23:24

                                    @hansolo77 Again, handbrake is often spoken of.

                                    1 Reply Last reply Reply Quote 0
                                    • H
                                      hansolo77
                                      last edited by 27 May 2017, 13:12

                                      Thanks I'll give it a try.

                                      Who's Scruffy Looking?

                                      1 Reply Last reply Reply Quote 0
                                      • H
                                        hansolo77 @screech
                                        last edited by hansolo77 28 May 2017, 02:13

                                        @screech where do I find the settings for this "RomsExcluded" functionality? Is it built into the program, or is it something I have to manually add to the ini? When I scrape SegaCD I still get multiple entries for the same game. Here is an example of the artwork it creates:

                                        https://pastebin.com/UVQY7egb

                                        Who's Scruffy Looking?

                                        R 1 Reply Last reply 28 May 2017, 10:20 Reply Quote 0
                                        • R
                                          Rion @hansolo77
                                          last edited by Rion 28 May 2017, 10:20

                                          @hansolo77 Your img files are split in to multiple bin files. Rebuild your Redump games by mounting the image. Select your cue file and mount it. Make new image with imgburn.

                                          That will produce one bin/img file and one cue file.

                                          FBNeo rom filtering
                                          Mame2003 Arcade Bezels
                                          Fba Arcade Bezels
                                          Fba NeoGeo Bezels

                                          1 Reply Last reply Reply Quote 0
                                          517 out of 728
                                          • First post
                                            517/728
                                            Last post

                                          Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                          Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.

                                            This community forum collects and processes your personal information.
                                            consent.not_received