[SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture

[ghostlywindmill]

@AlexMurphy

Thanks for the replies. The latest version from that link is the one I am using. The problem is that the SSH commands do not actually shut down ES or reboot my Pi, and whenever I select the raspberry pi as the ROM location it says it can't find ROMs in the autoconfigured path.

It's strange because when my Pi is off and I try an SSH command, it says the SSH command had a problem (tineout); when the Pi is on, I don't get any errors, but nothing actually happens.

Edit: I am only trying with one system, which is NES.

[AlexMurphy]

@ghostlywindmill Well, first of all, you won't be able to SSH into any computer that isn't powered up or on the network. I hate to say it but it it sounds to me like "user error" ie YOU are doing something wrong. Explain what you are doing and maybe someone will point out your error.

[ghostlywindmill]

@AlexMurphy

I only tried SSH when off as a troubleshooting step. As I already stated, with my Pi on I can SSH into it using PuTTY on a PC and can remotely perform any terminal command. I can also successfully connect to it via Filezilla and add files. Here are my steps:

1. Turn on my Raspberry Pi and connect it to the same Wi-Fi network as my PC.
2. SSH into it via PuTTY and confirm terminal commands work.
3. Open UXS and select the wizard. When I select the Raspberry Pi as the ROM location, it errors out saying it can't find the autoconfigured location.
4. I then check the SSH settings and confirm the host, username and password are correct.
5. Try to kill ES or reboot the Pi via the SSH menu and nothing happens. No error message is given. If the Pi is off when I try this, there IS an error message, but no message when it is on. This tells me that it is recognizing the Pi but can't seem to access it for some reason.

My problem may very well be used error, but it's not obvious to me what it is since I am only seeing the problem in UXS--no issues with SSH otherwise.

[AlexMurphy]

@ghostlywindmill One thing jumps out... what "wizard"? Are you using some odd set-up? Sounds like it to me. Have you configured it properly?

[AlexMurphy]

@ghostlywindmill
You should be getting images like this:

[ghostlywindmill]

@AlexMurphy in UXS, I click Configuration -> Launch wizard. This prompts me to select my IS (I select RetroPie), the visual type (I select the three image setup as shown in your screenshot) and then the ROM location. If I choose Raspberry Pi, I receive 'Roms Path for autoconfiguration can't be reached. If I select my PC and perform a scrape, I do successfully get images, but it puts them on my PC in a configuration folder. I run scrapes on PC no problem, the issue is when I select the Pi.

The above mentioned error message is what led me to try the SSH options to try to isolate the problem. I'm just using the latest build from the github site, nothing abnormal I don't think.

[AlexMurphy]

@ghostlywindmill I feel bad, man. I am trying to help you. BUT the problem seems to be on your end, specific to you. I can't help you with that. Sorry. It works fine for me.

[ghostlywindmill]

@AlexMurphy no worries, thanks for trying to help. I'll keep fiddling with it and report back if I can get it to work. Or I might just run it on my PC ROM files and move the ikayes/.xml over.

[AlexMurphy]

@ghostlywindmill I mean it, if SSH is working and you have the path configured then the problem is on your end. It could be an issue with your Firewall, if you use such a thing. If you get it sorted please post the resolution here. Don't forget those that come after you...

[hansolo77]

Is the server having issues right now or something? I noticed a bunch of my NES roms were missing, and checked and saw they weren't included in the xml because there were no matches. I'm trying to rescrape and build a new xml with the unmatched roms added, but UXS keeps crashing. It does it at different stages of progress, like 10% in, 90% in, 50%, etc. Been trying to do it for the last 2 hours and haven't gotten it to complete yet. Just thought maybe the problem is with the server, because I've been able to scrape 100% this system before with the same settings. The logs just show the progression, but have no indication to a cause of a crash.

[ghostlywindmill]

@hansolo77

I am having the same issue...cant get a complete scrape done, and the crashing points are random.

[hansolo77]

I've narrowed down the problem to being related to the 2.2.0.2 version of the scraper. Appears to be some kind of memory leak. Funny thing is, when it crashes, it actually keeps scraper.exe running in your task manager. I had 4 copies of it running earlier. Then I saw @screech mention in another thread about the version .3 coming with new MIX profiles, and that .2 was "really" buggy. So I gave it a try and was able to get a full scrape with 2.2.01. Downgrade to the 2.2.0.1 version and you should be good.

@screech - Can I make an observed suggestion? When you scrape ROMs that aren't in the server's database, and you have it set to "Add missing roms in XML", the gamelist.xml file creates game names that are not helpful at all.. meaning:

<name> (Clone, Beta, Demo, Proto, Trad, Hack, Unl)</name>

I actually get a list with about 30 of these, and because of the parentheses these are at the top of all my lists inside EmulationStation. Can you change this so it creates the name as simply the ROMFILE instead?

[AndersHP]

I have just tried this scraper for the first time, and I'm having the following issues:

• I have 40 games in my FBA folder, but the program only recognizes 32.
• It then says that it's scraping all 32 with succes, but only 5-6 of these show up with images in ES.
• When scrolling through the remaining games, the system starts to become very slow, essentially crashes.

It seems there's PNG's made for all the recognized 32 games in the downloaded_images folder, though...

[screech]

Hi guys, the 2.2.0.3 is out with correction :

2.2.0.3
Corrected :

• Random Error making UXS crash. (Changing a part of MAIN <-> ENGINE communication system)
• Changing color progressbar is now only green (Maybe a raison of random crash)
  SSH finally will work (sorry for that)
• Apostrophes in filename now works... (Thanks Marblemad)

Added :

• ProgressBar when creating the XML File

(And of course correction of the 2.2.0.2 like the missing rom name ;) )

[hansolo77]

SSH kill of EmulationStation still doesn't work.

[screech]

I haven't Retropie installed :( can someone give me a kill command to kill ES from SSH ?

I use "killall emulationstation" can someone test it from putty for exemple ?

It works on recalbox no reason it don't works on retropie :(

[hansolo77]

I just tried something else, and it is working... Apparently if you have it set to create the gamelist.xml locally (on the computer) then it doesn't kill EmulationStation. If you have it create on the pi, then it does. It wasn't even prompting before.

[brunocedup]

Yesterday I scraped some roms to test and I choice the video download in YES. For some systems, how FBA and Master Systems the videos run very well, but for SNES, for example, the videos don't run well on my Raspberry.

I think that the problem can be the power supply. When I surf at SNES roms, show every time a yellow ray on right top corner TV.

[ghostlywindmill]

@screech Thanks for the update. I have done some testing and can confirm that SSH now works for me, and I still haven't had another crash either.

Wondering if you (or anyone else) can help me figure out what i'm doing wrong with my autoconfig path, though--I still can't get it to save images to my Pi, it just says it detected a problem with the autoconfiguration path. I tried resetting it to default but still doesn't work.

Here is a link to what it looks like:
https://ibb.co/bG6ZtF

Right now it's just saving it to my C:\configs folder, which is fine I suppose (can still move it to the Pi manually via FTP) but I'm sure I'm doing something wrong.

Edit: I should say that I have not changed any default settings for rom, imge or XML location; roms are in RetroPie/roms and the gameslist and download_images folders are on my Pi in /opt/retropie/configs/all/emulationstation

[mattrixk]

Hi @screech, I know it was over 2 weeks ago that you gave me advice about scraping NeoGeo, but I finally got around to testing it properly. What I ended up doing was scraping NeoGeo normally, which found ~60 out of ~120 games, then I turned scrape mode to "update xml", left research mode on "crc+filename" and turned off "auto-select your system". Then when I went to scrape, I selected Mame instead of NeoGeo and it found the rest of the games.