[SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture

ShawnS

@paradadf I don't have Universal_XML_Scraper.exe it has the .au3 extension. I'm not sure how to run that "out of memory".

paradadf

@ShawnS I don't know what you've downloaded, but the files are here:
https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/releases

That is also the only launcher.

ShawnS

@paradadf doh! The only link I saw in the original post was to the GitHub repository for the code. I totally missed the launcher in there. Thank you!!

screech

Sorry to be late ;)

Thank you @paradadf to answer ;)

yep, just launch "Universal_XML_Scraper.exe" or "Universal_XML_Scraper64.exe"
"Scraper.exe" and "scraper64.exe" are just the "Scrape Engine" not the main core software... So they do nothing if you launch them directly (that's why they normally be hiden by the soft ;) )

captainvelvet

Ok guys, I've fixed it. I deleted the xml file and images, then pressed F4 in emuletionstation and ran the scraper. Everything ran ok. Beautiful PI that I have now!

hansolo77

Latest version (2.0.0.8) changed RetroPi Scraping again. Not sure where it saves all the metadata/artwork... but it doesn't save it in the \\RetroPie\configs\all\emulationstation\downloaded_images\%system% path anymore. Actually, it looks like it's saving them into \\RetroPie\roms\%system%\downloaded_images now. This is a brand new setup.

Autoconfiguration path configuration is set as follows:

Root System directory
\\retropie\roms
XML file name
%Source_RootPath%\%system%\gamelist.xml
Path to the roms in your XML file
./
Folder Name where pictures will be do
%Source_RootPath%\%system%\downloaded_images
Path to the picture in your XML file
./downloaded_images/

So why this change? I don't want the artwork saved like this. I want the artwork saved on my SD card (the way it was before) and the ROMs located on the external USB HD. The problem with this too, is when I go to scrape for the missing stuff UXS doesn't find, and I use Selph's or the ES one. They default to saving in the other path. This just sucks.

vbs

@hansolo77
There is a new profile named Retropie [Rom folder] now which behaves as you described. Did you choose that maybe? Make sure to choose Retropie if you want the old behavior.

hansolo77

I don't remember what I chose. I just changed it and thought to myself "wait a sec, this was a NEW install, with a new config. So it's probably just using a default setting". I'll try and that [Rom Folder] option and see if that's what I needed.

hansolo77

Ok I erased the the ini file and restarted UXS. I chose Retropie [Rom Folder] MIX and it still has the same %Source_RootPath%\%system%\gamelist.xml designation.

vbs

@hansolo77 said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:

Retropie [Rom Folder] MIX

Sorry, stupid question: You don't want the gamelist.xml created it in the ROM folder but you chose the profile Retropie [Rom Folder] MIX?
I don't want that either so I choose Retropie MIXand for me it saves the gamelist.xml to \\RetroPie\configs\all\emulationstation\gamelists\%system%

hansolo77

Correct. I want it saved in the \\RetroPie\configs\all\emulationstation\gamelists\%system%\ path and the art in the \\RetroPie\configs\all\emulationstation\downloaded_images\%system%\ path. So I don't want the [Rom Folder] option, I want the Regular option?

hansolo77

..side note..
This sucks, I wish I had more than 1 thread. But I don't want to have to pay MONTHLY for the access. 10 is a lot of money for somebody who works part time. Especially for just artwork/metadata. After all the "beta" testing I've been doing, it should be a given. LOL. Having it run through 5k ROMs takes HOURS.

vbs

@hansolo77 said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:

So I don't want the [Rom Folder] option, I want the Regular option?

So, what happens if you choose Retropie MIX and then scrape a system? For me it works.

..side note..
This sucks, I wish I had more than 1 thread. But I don't want to have to pay MONTHLY for the access

Huh, do you have a link/source for that? I always thought it was free but accepts donations?

hansolo77

I'll let you know.... in an hour. :)

And yeah, when you go into the settings, there is a place for you to provide your username/password. That is the login information you create on the screenscraper.fr website. Depending on your donation level, they grant you more threads. But you don't get ANYTHING until you donate at least 5 bucks, and that's only 1 thread. 10 bucks for 5. The thing is, when I looked into it using google translate, it looks like it's a monthly subscription, not a one time donation. It's a support feature, not a "thank you".

paradadf

@hansolo77 I've been told that you can just make a one time donation, no need for a subscription.
Anyway, there is another way to get more threads... contributing to the database ;) I've done it so much already that I have 8 threads XD.

vbs

@hansolo77 said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:

But you don't get ANYTHING until you donate at least 5 bucks, and that's only 1 thread. 10 bucks for 5. The thing is, when I looked into it using google translate, it looks like it's a monthly subscription, not a one time donation. It's a support feature, not a "thank you".

Nah thats definitely not true: When you contribute to the site then you get more threads for free (without paying). After I assigned about 40 ROMs to games I had 4 threads. By now I assigned like ~300 ROMs and I have 5 threads. So I am quite sure you don't have to pay monthly to get more threads (that would be really uncool).
Assigning like 20 ROMs does not take more than 15 minutes I guess.

vbs

@hansolo77
I found the rules what you have to do for more threads:

http://screenscraper.fr/webapi.php?alpha=0&numpage=0

Combien puis-je gagner de « Threads » ?
Votre nombre de Thread dépend de votre participation :
(ssuser/uploadsysteme + ssuser/uploadinfos + ssuser/romasso + ssuser/uploadmedia)
-	1 participations validées : 2 Threads maximum ouvert simultanément
-	2 à 49 participations validées : 3 Threads maximum ouvert simultanément
-	50 à 199 participations validées : 4 Threads maximum ouvert simultanément
-	200 à 499 participations validées : 5 Threads maximum ouvert simultanément
-	500 à 749 participations validées : 6 Threads maximum ouvert simultanément
-	750 à 999 participations validées : 7 Threads maximum ouvert simultanément
-	+ de 999 participations validées : 8 Threads maximum ouvert simultanément

Et si vous faite une participation financière vous gagnez des « Threads Bonus »
(ssuser/contribution) : 2 = 1 Thread Supplémentaire / 3 et + = 5 Threads Supplémentaires
-	De 5 à 9 euros : 1 Thread bonus 
-	Au-delà de 10 euros : 5 Threads bonus

Well, it seems I lied: you need 50 contributions for 4 threads (but only 2 for 3 threads).

hansolo77

How do you assign roms though? I have a hard time using that website because of the French.

Also it looks like just using RetroPie (mix) is working. Only problem now, my 1 hour scrape on Atari ST is now going to take 10 Days, and it only on Rom #8. :(

hansolo77

I think I might have figured out how to assign ROMs to entries in their database. Sometimes it says something like "For instance, can you associate this ROM to it's game name in the above list" and shows a list of games to pick from. I've done that a bunch of times. Then it asks for something like a description and website where I found it. I provided that for a bunch of games too. Each time, the page refreshes saying my input has been saved or something, but I don't seem to be getting any credit for it. (and yes, I'm logged in). Am I doing something wrong? Do I need to be actually in a the system, and not just entering data from the homepage?

mattrixk

@screech: Cheers for the awesome scraper, the Standard (3img) makes my system look great.

Now a question. I want to make my own MIX Image Profile template (so I can create a rockin new theme for myself), but I'm having problems.
(Sorry, this got a bit long)

Here's my process:

Go to the Mix folder that is created when you first run Universal_XML_Scraper64.exe.
Copy Standard (3img).zip.
Paste Standard (3img).zip into a test folder on my desktop.
Unzip Standard (3img).zip within the test folder.
Duplicate the Standard (3img) folder and rename the duplicate Standard (test).
Open the config.xml within the Standard (test) folder.
Change <Profil Name="Standard (3img)"> to <Profil Name="Standard (test)">.
Save config.xml.
Zip the Standard (test) folder as Standard (test).zip.
Copy Standard (test).zip back into the Mix folder.
Run Universal XML Scraper.

Now, that's a few steps, but it's pretty straight forward. I've just renamed an existing Image Profile that I know works. I would expect it to show up in the profile list with it's own name, but with all the details of the (3img) profile.

3img
"Standard (3img)" looks like this. Note: "Standard (test)" is also in the list

Unfortunately, this is what I end up being shown:
test
"Standard (test)" with missing Author and Description data. Note: the example images shown are just a leftover from the last option selected. If I chose "Arcade (moon)" and then went back to "Standard (test)", it would continue to show the "Arcade (moon)" example images.

I also get a bunch of errors showing up in the log:
PasteBin link

If I run the Scraper with this Image Profile selected, it doesn't generate an image in the downloaded_images folder. It does generate the gamelist.xml, except it is missing the <image> tag.

I've tried using the tutorial in the template, but it doesn't seem to match up with the config.xml file in the MIX Image Profile templates. (eg: It's missing any info on the <compression>, <root> and <game> tags. The <element> tags seem to be vaguely similar, but still quite different and the tutorial refers to a config.ini file.)

Any ideas what I might be doing wrong?