[SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture

kbronctjr · 31 May 2017, 20:28

@AndersHP When I scraped with the visual builtin scraper I had also the images an game list inhome/pi/.emulationstation/

AndersHP · 6 Jan 2017, 07:07

@kbronctjr so it must be the sselph scraper that makes a gameslist and images folder in the roms folder. I'll try uninstalling sselphs scraper then...

SonPluck · 2 Jun 2017, 18:45

Where do the images come from? for the rom's.

EctoOne · 2 Jun 2017, 22:31

@SonPluck from screenscraper.fr

ghostlywindmill · 3 Jun 2017, 23:30

I am having an issue trying to get UXs to work. I'm using a Raspberry Pi 3B and attempting to use USC on a Windows 8.1 laptop. The issue is that UXS does not seem to connect via SSH. I out in the correct host and username/password and try to stop ES and reboot as a test, but nothing happens...no comma d line pops up or anything, assuming it is supposed to. This is using UXS 2 though I also tried an older UXS (Not 2) release that didn't work either, though I did get a command line window popping up in that case. With that one though, the SSH options were entirely greyed out.

I am able to successfully SSH into my Pi with PuTTY and can connect via Filezilla to FTP files over as well. So my PC can connect to the Pi, it just won't work with UXS. Any thoughts on this? Am I missing something painfully obvious?

AlexMurphy · 3 Jun 2017, 23:41

@ghostlywindmill No command line pops up on your PC... It simply uses SSH to kill emulationstation and to read / transfer the files to the stipulated path. What is it you are expecting to see?

AlexMurphy · 3 Jun 2017, 23:45

@ghostlywindmill Download the correct version from here. https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/releases
Set up your profile and what you want. Choose a system at a time, rather than all, then scrape.

ghostlywindmill · 6 Apr 2017, 01:00

@AlexMurphy

Thanks for the replies. The latest version from that link is the one I am using. The problem is that the SSH commands do not actually shut down ES or reboot my Pi, and whenever I select the raspberry pi as the ROM location it says it can't find ROMs in the autoconfigured path.

It's strange because when my Pi is off and I try an SSH command, it says the SSH command had a problem (tineout); when the Pi is on, I don't get any errors, but nothing actually happens.

Edit: I am only trying with one system, which is NES.

AlexMurphy · 4 Jun 2017, 01:58

@ghostlywindmill Well, first of all, you won't be able to SSH into any computer that isn't powered up or on the network. I hate to say it but it it sounds to me like "user error" ie YOU are doing something wrong. Explain what you are doing and maybe someone will point out your error.

ghostlywindmill · 4 Jun 2017, 03:19

@AlexMurphy

I only tried SSH when off as a troubleshooting step. As I already stated, with my Pi on I can SSH into it using PuTTY on a PC and can remotely perform any terminal command. I can also successfully connect to it via Filezilla and add files. Here are my steps:

Turn on my Raspberry Pi and connect it to the same Wi-Fi network as my PC.
SSH into it via PuTTY and confirm terminal commands work.
Open UXS and select the wizard. When I select the Raspberry Pi as the ROM location, it errors out saying it can't find the autoconfigured location.
I then check the SSH settings and confirm the host, username and password are correct.
Try to kill ES or reboot the Pi via the SSH menu and nothing happens. No error message is given. If the Pi is off when I try this, there IS an error message, but no message when it is on. This tells me that it is recognizing the Pi but can't seem to access it for some reason.

My problem may very well be used error, but it's not obvious to me what it is since I am only seeing the problem in UXS--no issues with SSH otherwise.

AlexMurphy · 6 Apr 2017, 04:36

@ghostlywindmill One thing jumps out... what "wizard"? Are you using some odd set-up? Sounds like it to me. Have you configured it properly?

AlexMurphy · 4 Jun 2017, 03:38

@ghostlywindmill
You should be getting images like this:

ghostlywindmill · 4 Jun 2017, 03:58

@AlexMurphy in UXS, I click Configuration -> Launch wizard. This prompts me to select my IS (I select RetroPie), the visual type (I select the three image setup as shown in your screenshot) and then the ROM location. If I choose Raspberry Pi, I receive 'Roms Path for autoconfiguration can't be reached. If I select my PC and perform a scrape, I do successfully get images, but it puts them on my PC in a configuration folder. I run scrapes on PC no problem, the issue is when I select the Pi.

The above mentioned error message is what led me to try the SSH options to try to isolate the problem. I'm just using the latest build from the github site, nothing abnormal I don't think.

AlexMurphy · 4 Jun 2017, 04:03

@ghostlywindmill I feel bad, man. I am trying to help you. BUT the problem seems to be on your end, specific to you. I can't help you with that. Sorry. It works fine for me.

ghostlywindmill · 4 Jun 2017, 04:25

@AlexMurphy no worries, thanks for trying to help. I'll keep fiddling with it and report back if I can get it to work. Or I might just run it on my PC ROM files and move the ikayes/.xml over.

AlexMurphy · 4 Jun 2017, 04:29

@ghostlywindmill I mean it, if SSH is working and you have the path configured then the problem is on your end. It could be an issue with your Firewall, if you use such a thing. If you get it sorted please post the resolution here. Don't forget those that come after you...

hansolo77 · 5 Jun 2017, 01:38

Is the server having issues right now or something? I noticed a bunch of my NES roms were missing, and checked and saw they weren't included in the xml because there were no matches. I'm trying to rescrape and build a new xml with the unmatched roms added, but UXS keeps crashing. It does it at different stages of progress, like 10% in, 90% in, 50%, etc. Been trying to do it for the last 2 hours and haven't gotten it to complete yet. Just thought maybe the problem is with the server, because I've been able to scrape 100% this system before with the same settings. The logs just show the progression, but have no indication to a cause of a crash.

ghostlywindmill · 6 Jun 2017, 01:40

@hansolo77

I am having the same issue...cant get a complete scrape done, and the crashing points are random.

hansolo77 · 6 Jun 2017, 02:51

I've narrowed down the problem to being related to the 2.2.0.2 version of the scraper. Appears to be some kind of memory leak. Funny thing is, when it crashes, it actually keeps scraper.exe running in your task manager. I had 4 copies of it running earlier. Then I saw @screech mention in another thread about the version .3 coming with new MIX profiles, and that .2 was "really" buggy. So I gave it a try and was able to get a full scrape with 2.2.01. Downgrade to the 2.2.0.1 version and you should be good.

@screech - Can I make an observed suggestion? When you scrape ROMs that aren't in the server's database, and you have it set to "Add missing roms in XML", the gamelist.xml file creates game names that are not helpful at all.. meaning:

<name> (Clone, Beta, Demo, Proto, Trad, Hack, Unl)</name>

I actually get a list with about 30 of these, and because of the parentheses these are at the top of all my lists inside EmulationStation. Can you change this so it creates the name as simply the ROMFILE instead?

AndersHP · 6 Jun 2017, 08:25

I have just tried this scraper for the first time, and I'm having the following issues:

I have 40 games in my FBA folder, but the program only recognizes 32.
It then says that it's scraping all 32 with succes, but only 5-6 of these show up with images in ES.
When scrolling through the remaining games, the system starts to become very slow, essentially crashes.

It seems there's PNG's made for all the recognized 32 games in the downloaded_images folder, though...