[SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture

Furluge

I have been having some issues getting the software to pull down marquees when using the Retropie(Video) profile when scraping for final burn alpha. Nothing I seem to set for default picture gets me the marquees. Edit: It does not seem to want to pull down a7800 boxes either even though I see them on the site. :/ Does not seem to be pulling down a lot of a2600 boxes either. Still getting all the video though.

detron

on the General Configuration page of UXS, what does the option
"Add missing roms in XML" actually mean?

I have used UXS without it checked, and games that it did not have any artwork found still showed up.

though it seems straight forward, I would like to know what it really means before I re-scan some folders.

thanks

paradadf

@detron i believe that options just add an entry for the roms on the gamelist.xml, although without data, as no match was found.

Rion

@paradadf Yes that's right. I use it for rom translations and hacks. Then i manually add the information to that rom.

a7mag3ddon

I would like to add a suggestion if it hasn't been suggested already.

Can an option be added to skip the image if it already exists in the images folder.

Thanks

screech

normally it's already the case...

Aksen

@screech Can there be an option to scrape metadata but skip the image? I'd like to re-scrape at some point, because some of my scraped metadata was in french.

screech

@Aksen
Normally the basic option do that :
Recreate the XML, but if picture exist, don't touch it... Can you give it a try?

@mattrixk
Just release a new version : You can now put a size for every media you download (in the standard profil)
(Need to see for sys info/media now...)

Some new things added :
You can now add Genre Picto in the Mix
You can now put some text in Mix
You can now rotate picture in Mix
Lot's of correction on the fallback function (some pictures doesn't download exemple : 2DBox in standard profil now work nice ;) )
Added a Mirror server to avoid main server instability. (automatique fallback)

The Wiki is up to date with the new function for MIX : https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/wiki/[TUTO]-Making-a-MIX-Template

mattrixk

@screech You're a legend. Hopefully I'll get the chance to try this out today or tomorrow.

mattrixk

@screech I downloaded the latest version, but when I try to run it I get this error:

Universal XML Scraper fatal error !
(Time Out)

Edit: I'm on a Windows 7 PC.

Edit 2: I just realised it could be because screenscraper.fr is down. It would be nice if I could still play with the program, even if the website isn't working. I could keep working on Profiles without actually scraping, even if the website was down.

I just tried the previous version (v2.1.0.3) that worked yesterday. It doesn't work at the moment either.

fergyz

@mattrixk at least you confirmed I'm not crazy... i get the same error.

mattrixk

@fergyz yeah, I'm pretty sure it's because the website is down.

@screech, is it possible to open the program when the website isn't working? I had the idea of creating a MIX Profile to smush Local images into a single image.

Eg: Say I have 5 games in a system, and for each game I've already used a Scrape Profile to download Wheel, 3D Box Art and Screenshot, and suffixed each image with "-wheel", "-boxart" and "-screenshot" respectively.

Can I use UXS to format each of these images into a single image per game? Or maybe have a Scrape Profile that downloads the images separately, but also Mixes some of them together? That would be awesome.

EctoOne

Oh yeah, a tool to Mix offline images would be great. I spend so much time doing that manually for some rom hacks and when the Database was kinda empty in the beginning and I had to grab images from other sources.

BTW @mattrixk Can you upload the Mix Profil that you made to get all the single images or post the code somewhere?

mattrixk

@EctoOne
This is a WIP Scrape Profile for UXS.

http://pastebin.com/bZNYQ29h

It downloads screenshots, 3d box art (with 2d box art backup), fan art, game logo (wheel art), game cartridge (media) and game video. It saves them all to the 'downloaded_images' folder with suffixes of "-screenshot", "-boxart", "-fanart", "-logo", "-media" and "-video" respectively.

It also puts each of them into a gamelist.xml in <screenshot>, <boxart>, <fanart>, <logo>, <media> and <video> tags respectively.

It downloads each element as large as it can, up to 1024x1024 (videos will pretty much always be 640x480, as will screenshots, as that is the size of them within screenscraper.fr).

To use the Scrape Profile, save the xml as downloadeverything.xml (or whatever you want to call it), then put it in the "ProfilsFiles" folder within UXS. You can then choose it from the Scrape Profile menu.

Keep in mind that Scrape Profiles and MIX Profiles are 2 different things (something it took me a while to understand. I was confused as balls until I did, but once I did, it made things much easier).

Scrape Profiles are for scraping individual elements. MIX Profiles scrape the elements and combine them into a single image.

Once again, this is a work in progress, so I take no responsibility for your computer blowing up, or aliens abducting your loved ones.

screech

Sorry, it's impossible to launch UXS without a net connexion to the website... Too much problem after that...

But I have in my pretty big todolist : Make a MIX Generator soft ^^

Where you can scrape 1 game (with all the media/data it is possible) and create a MIX with them...
But it's a project nearly as huge as UXS >< so it's not for now ;)

If I can, I will try to make a "light" version of that ;)

mattrixk

No worries @screech. Love your work.

screech

Hi, the new 2.1.0.5 just come out.

A lot's of change :

Corrected :

° Better stability on downloaded images (Some never download :S and miss in the MIX)

Added :

° Dynamique Handle to API server, now I can force the server to ask (for the futur when we have a fallback server ;) or country based server )
° Lot's of new Picto media (Nb of player, Notes, Classification, Editor, Developer, Genre) -> Beta it can change soon.
° Added 3 news MIX Template for Bigshot Theme with these new picto (1 is directly on local, the 2 other are on the repository, you can download them directly from UXS)

WARNING Added things that can "broke" you existing MIX Template

° You need to put the full Xpath to the Source_Value node (exemple : Game need to start by 'Data/jeu/' and System by 'Data/systeme [id="%IDSYSTEM%"]')
° The CENTER/LEFT/RIGHT/UP/DOWN function in Target_TopLeftX, Target_TopLeftY (and other) now take the "origin" point of the picture (default are TopLeft corner of the Picture)
° You can now use Target_OriginPicX and Target_OriginPicY to fixe the origin point of the picture. So now to center a picture you need to put :

<Target_TopLeftX>CENTER</Target_TopLeftX>
<Target_OriginPicX>CENTER</Target_OriginPicX>
<Target_TopLeftY>CENTER</Target_TopLeftY>
<Target_OriginPicY>CENTER</Target_OriginPicY>

(The Origin point of the picture will be the center, and we center this origin point to the final picture)

This is exemple of the New MIX for BigShot Template :

text alternatif

mattrixk

This is huge. Great work @screech!

a7mag3ddon

Is it possible to use this scraper on roms folder on the PC hard drive instead of on the Pi itself? I have a backup of my pi roms folder on the PC so want to run it on that instead.

mattrixk

@a7mag3ddon you can download the 32 or 64 bit version of the scraper here and run it on your PC. It's what I use to build scrape and mix profiles. (I don't even know how to use it on the Pi itself shhh, don't tell anybody).