• Recent
  • Tags
  • Popular
  • Home
  • Docs
  • Register
  • Login
RetroPie forum home
  • Recent
  • Tags
  • Popular
  • Home
  • Docs
  • Register
  • Login

[SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture

Scheduled Pinned Locked Moved Projects and Themes
scrapescraperuxs
728 Posts 111 Posters 741.1k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • A
    AlexMurphy Banned @ortsac
    last edited by 20 May 2017, 19:49

    @ortsac Hmmm, the connection is actually timing out on me now. But when it does work there is a menu up top including media. You can also click on the little US flag on the left to get a (mostly) translated version of the site.

    1 Reply Last reply Reply Quote 0
    • O
      ortsac
      last edited by 20 May 2017, 19:51

      OK...I got the site in English. which area in the site contains games video for download?

      E 1 Reply Last reply 20 May 2017, 20:09 Reply Quote 0
      • E
        EctoOne @ortsac
        last edited by 20 May 2017, 20:09

        @ortsac the videos are found in (some) game details. Like https://screenscraper.fr/gameinfos.php?plateforme=2&gameid=754 it's listed under All Media. If you want to scrape them with UXS you need to pick the right profil or enable an option (can't remember what works now and actually never really tried it).

        O 1 Reply Last reply 20 May 2017, 20:31 Reply Quote 0
        • S
          screech
          last edited by 20 May 2017, 20:30

          @ortsac I just forced your validation ;) (sometime, email don't want to be send :S )

          For information, we've got some problem with the server today... And normally it's ok now ;)

          O 1 Reply Last reply 20 May 2017, 20:32 Reply Quote 0
          • O
            ortsac @EctoOne
            last edited by 20 May 2017, 20:31

            This post is deleted!
            H T 2 Replies Last reply 20 May 2017, 20:32 Reply Quote 0
            • O
              ortsac @screech
              last edited by 20 May 2017, 20:32

              @screech Thanks!!! I'll give it a try again.

              1 Reply Last reply Reply Quote 0
              • S
                screech
                last edited by 20 May 2017, 20:32

                You can't download rom ^^ it's only a DB to scrape your own rom.
                And to Scrape, the best way is to use a scraper ;) like the one from sselph or UXS ;)

                1 Reply Last reply Reply Quote 0
                • H
                  herb_fargus administrators @ortsac
                  last edited by 20 May 2017, 20:32

                  @ortsac you don't download roms. thats illegal. its only the metadata.

                  If you read the documentation it will answer 99% of your questions: https://retropie.org.uk/docs/

                  Also if you want a solution to your problems read this first: https://retropie.org.uk/forum/topic/3/read-this-first

                  1 Reply Last reply Reply Quote 0
                  • T
                    TMNTturtlguy @ortsac
                    last edited by 20 May 2017, 20:32

                    @ortsac just an FYI it is not ok to share details or even ask about obtaining or downloading roms. You may want to quickly edit your post and keep the topic about scraping art and video. Thanks

                    1 Reply Last reply Reply Quote 0
                    • M
                      mattrixk
                      last edited by 21 May 2017, 12:59

                      @screech: I've come across some oddness when scraping NeoGeo.

                      I have a folder with ~130 NeoGeo roms, and when I scrape them about 45 of them come back as "2020 Super Baseball". They also download the images for 2020 Super Baseball.

                      You can see a copy of the gamelist.xml here.

                      I've had a look at a few of the affected titles on screenscraper.fr and they all seem to have correct data and images, so I don't know what could be causing it.

                      I've deleted the gamelist.xml and the images and tried scraping again, but the same thing happened again to the same games.

                      Any thoughts?

                      My ES themes: MetaPixel | Spare | Io | Indent

                      1 Reply Last reply Reply Quote 0
                      • S
                        screech
                        last edited by 21 May 2017, 16:52

                        wow... that's really weird :S

                        Can you send me your log : screech[@]free.fr ? Just to check ?
                        (My neogeo folder scrape well ^^)

                        For information : 2.2.0.2 is out :
                        https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/releases/tag/2.2.0.2

                        Corrected :

                        • Full Scrape don't neverend anymore.
                        • SSH kill works now on scrape demande.
                        • Date with only a 'Year' will work now (the date will be 'Year/01/01')
                        • When adding Missing Rom, Name were "(Clone, Beta, Demo, ...)". It's OK now ;)
                        • When you force JPG or PNG, Video stay in MP4.

                        Modification :

                        • New ProgressBar changing color (Green = OK, Red = Timeout or error download, Yellow = Not found)

                        Added :

                        • New, in case UXS Hang (it never happen ^^), when you re-launch a Scrape, it will ask you if you want to generate a gamelist.xml with the temporary file found.
                        • New Shortcut was created at the first launch : 'Silent_UXS'. You can now run UXS silently ;)
                        • New Visual info about Engine Use. Some check box corresponding to the Thread Number will tell you if the Engine is used or not.
                        • 2 New Advanced Menu : Reset Autoconfiguration Path and Alt Autoconfiguration Path (only for Retropie, it put the Rom's Folder Path)
                        • New Element Type : RomExcluded

                        Exemple :

                        	<Element Type="RomExcluded">
                        		<Source_Type>Variable_Value</Source_Type>
                        		<Source_Value>%AutoExclude%</Source_Value>
                        		<AutoExcludeEXT>bin|img|iso|ccd|sub</AutoExcludeEXT>
                        		<AutoExcludeValue>(Track |[Bios]|(Bios)</AutoExcludeValue>
                        	</Element>
                        

                        Mean :
                        When a duplicate name file (without extension) is found:
                        - it check the extension. If in list (bin|img|iso|ccd|sub) it won't scrape it
                        - it check if the file contain a value ("(Track ","[Bios]","(Bios)"). If yes, it won't scrape it

                        M 1 Reply Last reply 22 May 2017, 02:05 Reply Quote 0
                        • H
                          hooperre
                          last edited by hooperre 21 May 2017, 18:29

                          @screech I used to open this file using the source code zip download by running "Universal_XML_Scraper64.exe" included in the 2.1.0.6 download. I noticed that file is no longer included in the source download and when I try to run either "Scraper.exe" or "Scraper 64.exe" nothing happens.

                          How do I open the Scraper?

                          4B ~ RPi PSU 5.1V / 3.0A ~ 32GB SanDisk microSD ~ 128GB USB

                          1 Reply Last reply Reply Quote 0
                          • S
                            screech
                            last edited by 21 May 2017, 18:40

                            @screech said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:

                            https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/releases/tag/2.2.0.2

                            just download the Universal_XML_Scraper.exe or Universal_XML_Scraper64.exe that's all.. The other stuff are the source code...

                            And scraper.exe or scraper64.exe are the scrape engine and can't work as standalone...

                            H 1 Reply Last reply 21 May 2017, 18:43 Reply Quote 0
                            • H
                              hooperre @screech
                              last edited by 21 May 2017, 18:43

                              @screech Thanks. For some reason I can't get those to download because my browsers detect a 'virus.' I'll just have to disable it. I think you're aware of this and it was covered previously in the thread.

                              4B ~ RPi PSU 5.1V / 3.0A ~ 32GB SanDisk microSD ~ 128GB USB

                              1 Reply Last reply Reply Quote 0
                              • S
                                screech
                                last edited by 21 May 2017, 19:35

                                Yep :( sorry for that :( the language I use (Autoit) is well know by antivirus because in the past a lots of people create malicious software with it...
                                So every AV block every Autoit software...
                                I contacted some of the main AV compagny to tell them it's a false positive.
                                Some respond and it's ok now with some of them...
                                Some never answer... And False positive is still here :'(

                                so you need to "accept" UXS in your AV. sorry...
                                (In case of doubt if there is malicious code, all sources are on the github ^^)

                                H 1 Reply Last reply 21 May 2017, 19:44 Reply Quote 0
                                • H
                                  hooperre @screech
                                  last edited by hooperre 21 May 2017, 19:44

                                  @screech Ya. I sent it to Microsoft alerting a false positive as their Defender AV is picking it up and really not even letting me open it in any fashion. Really not even download it. I added the download URL to the exclusion list, was able to download it to Edge's temp folder. Would it be possible to include the .exe with the source code .zip in the future? Seemed to circumvent the issue.

                                  4B ~ RPi PSU 5.1V / 3.0A ~ 32GB SanDisk microSD ~ 128GB USB

                                  1 Reply Last reply Reply Quote 0
                                  • M
                                    mattrixk @screech
                                    last edited by 22 May 2017, 02:05

                                    @screech said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:

                                    wow... that's really weird :S
                                    Can you send me your log : screech[@]free.fr ? Just to check ?
                                    (My neogeo folder scrape well ^^)

                                    I'm at work at the moment, so it will have to wait until later tonight. I might try scraping with a different version (maybe the new one) and see if that makes a difference.

                                    On that note, congrats on getting the next version out. I know I bug you a bit, but UXS is seriously one of the best programs I use on a regular basis.

                                    My ES themes: MetaPixel | Spare | Io | Indent

                                    1 Reply Last reply Reply Quote 0
                                    • M
                                      mattrixk
                                      last edited by 22 May 2017, 12:12

                                      @screech: I emailed the log files to you.

                                      My ES themes: MetaPixel | Spare | Io | Indent

                                      1 Reply Last reply Reply Quote 0
                                      • S
                                        screech
                                        last edited by 22 May 2017, 12:44

                                        Great ^^

                                        I just received and studied your logs...

                                        What I see :
                                        Your hash weren't found in the DB...
                                        So UXS try to find a matching name. And can't....
                                        But you have the "In Zip" Option activated. So it unzip the file, and try to match at least 1 of the file in the ZIP.
                                        And he can !!! but, the problem on Neogeo is some eprom are exactly the same from a game to another.
                                        Like "000-lo.lo"... (I think it's a Neogeo Bios File)

                                        It found this file every time. And UXS/Screenscraper API take the first match every time :(

                                        That's explain your duplicate :'(

                                        My advice :
                                        Try in GENERAL OPTION to

                                        • uncheck the "Auto select your system"
                                        • uncheck the "Unzip File"
                                        • Select in Research Mode : "2 - Filename"

                                        So when you launch the scrape it will ask you wich system it is.
                                        Select "Neogeo"
                                        If you haven't renamed the file (or it's a standard name) like I see in your log, it will work ;)

                                        K M 2 Replies Last reply 22 May 2017, 20:16 Reply Quote 0
                                        • K
                                          kbronctjr @screech
                                          last edited by 22 May 2017, 20:16

                                          @screech there are some psx games that the scraper can't find, I even changed their names so they match the name in screenscraper

                                          A 2 Replies Last reply 22 May 2017, 20:19 Reply Quote 0
                                          450 out of 728
                                          • First post
                                            450/728
                                            Last post

                                          Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                          Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.

                                            This community forum collects and processes your personal information.
                                            consent.not_received