[SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture

hansolo77

I'm using the plain old RetroPie (MIX) profile. But I think I might have fixed it. I changed the Autoconfiguration line Path to the roms in your XML file to ./. That little change now enables Selph's to work with the existing xml just fine. I wonder if the problem I'm having is limited to my setup, because I have my ROMs stored on an external HD instead of the SD card. In either case, further testing is commencing, but I think that fixed it.

paradadf

Use the new profile I told you, I'm pretty confident it will work.

hansolo77

I was using one of the RetroPie [ROM FOLDER] profiles before, but that's not what I wanted. I didn't want to have the artwork saved next to my ROMs. This method I'm using now seems to be working just fine.

screech

@hansolo77 you are using a pretty "strange" configuration.... Gamelist.xml in rom folder and downloaded_image in an other place... Rom on an external device... Not matching the "standard" Retropie configuration nether sselph's configuration... So it's normal if you scrape with another scraper that's not working :S

For information, XML generated by UXS are legit...

<?xml version="1.0"?> : isn't necessary (But I can add it if you want)
indentation : is just a "visual" thing and change nothing.
<playcount> and <lastplayed> are added by ES... so not generated by UXS (but it's possible with empty value if you want)
Path Rom are just "preset" with standard path... You can change them, but you must change them "correctly" . You can choose relative path or full path.
But if you make "strange" things, don't expect to have a working things...

screech

@mattrixk I found why >< (after a long time searching)...

Your file in the zip must be at the root, not in an folder.

When you open the zip file you must see directly "config.xml"
Not "Standard(test)/config.xml"

^^

chuzzwuzzer

I've tried the latest version of this software and it kept locking up on my pc so had to revert back to the earlier version. So now I have 'downloaded images' folders in the roms folder of some systems and other images in some of the 'downloaded images' folder of configs>all>emulationstation. All I need to know is how do I create or edit the gamelist.xml for each system to direct it to the correct folder so the images show up on ES. Sorry if this is a stupid question but I've spent ages trying to fix this and getting no where fast !

hansolo77

@screech said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:

@hansolo77 you are using a pretty "strange" configuration.... Gamelist.xml in rom folder and downloaded_image in an other place... Rom on an external device...

That's not how I have it setup at all.. :/ All I have is the ROMs on an external device. Instructions for doing so are on the RetroPie wiki. I have it this way because I have SegaCD and PlayStation games that are large. The gamelist.xml files are NOT in the ROM folders... they are located in the /opt/retropie/configs/all/emulationstation/gamelists/"system"/ path, which is the default. The artwork is located in /opt/retropie/configs/all/emulationstation/downloaded_images/"system"/ which is also the default. I don't see how my setup is a "strange" configuration. This is how everything was setup originally. The only difference is I'm storing ROMs on the external drive, but as far as the system knows, everything is still the way it should be. When you're browsing via sftp, samba, or even in the terminal (ssh or local), you don't even know you switched drives. So how is my setup "strange"?

screech

So I don't understand why you are "changing" your path :S

what I read from the wiki (https://github.com/retropie/retropie-setup/wiki/scraper ) is

/home/pi/RetroPie/roms/%system%/
/home/pi/.emulationstation/downloaded_images/%system%/

and

/opt/retropie/configs/all/emulationstation/gamelists/%system%/
/opt/retropie/configs/all/emulationstation/downloaded_images/%system%/

will work the same...

but don't expect it will work with Sselph's scraper.... It don't use at all these path...

Sselph put "all things" in the Rom folder directly...

So if you want to "update" don't use 2 differents soft who don't work the same...
Or, like @paradadf say : Use the same configuration as Sselph. The profil are already created and works : Retropie [Rom folder] (MIX) for exemple.

hansolo77

They are the same paths! lol... They are simply symbolic links. Like "Genesis" actually goes to "MegaDrive".

It doesn't really matter though. It's working correctly.

mattrixk

@screech said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:

Your file in the zip must be at the root, not in an folder.

It was that simple? Argh, thank you so much! It's like wondering why your js/php/css doesn't work and then finding out you've missed a semi-colon somewhere.

herb_fargus

@screech sselphs scraper is modular enough that you can specify the ROM path, image folder etc. Retropie just integrated it as a module with specific file paths. They can really be set to anything you want if you know what you're doing.

chuzzwuzzer

In each rom folder I have my gamelist.xml files plus a downloaded images folder in each rom folder. Only problem is each of my gamelist.xml files are empty which is why I'm not seeing the images. Any idea why they are empty ?

paradadf

@chuzzwuzzer you need to kill emulationstation before scraping. I believe the latest version of UXS has a bug within the ssh commands. So for the time being, do it through other way.

chuzzwuzzer

But I already have the scraped images in the downloaded images folder. Are you saying I need to rescrape the entire system again ?

paradadf

Yes.

daeks

Feature request:

(Optional) Scrape only roms with missing images.
(Optional) Cleanup image folders for missing roms

screech

@chuzzwuzzer If you rescrape and keep your image in your folder, UXS won't re-download them... And just create the XML.

@daeks you already have 2 scrape options :
New rom, who scrape only roms who aren't in your XML
Picture only, who only scrape picture without touching your XML.

And for the cleanup it's already in my todo list ;)
( https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/issues/28 )

chuzzwuzzer

@screech - Just trying this. Should it still be reading : 'Hashing please wait.... Time left 2d, 22h, 41 mins 516/17651 '

Seems a long time just to rewrite the gamelist.xml if its not redownloading the images or is it normal for it to take this long ?

screech

have you the last version (I don't think)...
17651 roms ? it's pretty huge...
How many thread have you ? just 1 ?
So maybe it's normal... But 2 day seems to me a little too much :S

hansolo77

It takes my setup 2 days and 20 hours to scan EVERY system automatically, using 3 threads. I have 25 systems (10,895 roms) and then an extra system MAME that has about 8,000 or so ROMs. Hitting the CD image systems takes the longest time, and I only have like 20 PSX games and 8 SegaCD ones.