[SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture

SpaceSocks

@Lok i have the same problem

Lok

@SpaceSocks

Ok, I've read the thread you opened. If you sign up for free at http://www.screenscraper.fr it will work.

Thanks by the way.

Rambytes

Same problem... 3 days now and I cannot scrap my rom into Retropie with the Universal Scraper...

What the problem???

screech

Sorry.

We migrate both server and new website/db version at the same time...

And it wasn't a good Idea at the end of year (a lot's of new user)

So we had massive request on the new server aren't really ready for such a huge number of scraper..

We are working hard to gain back stability. And it seems it's going well today.

I can go back to UXS and correct some "new" probleme introduced with the new version..

But for information, the new DB allow more things and new data / media so you can see very soon the new improvement... (exemple : we already have some Brazilian media, and now UK isn't the same as US)

Thank you for your patience. We are doing what we can ;)

dirty_horror

Are you guys running the scraper from a virtual machine? I had this problem using the app on windows 10 on my mac with VMware fusion. I just now tried running it from my real PC and it's working perfect.

Nismo

@hansolo77 said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:

@vbs If that's the case, I'll just stick with my current batch of scrapes until Carbon is updated to allow videos. Because I have a bunch of systems installed on my RetroPie, I pretty much have to use Carbon. I don't like it, but it's the only theme that seems to work.

Carbon theme with video support:
http://www.mediafire.com/file/f32kjrkeopykjra/carbon_with_video.zip

Enjoy.

firewater

Is there any way so that the images of arcade are not antialiased? I tried forcing them to be downloaded in .png but they still are, and it looks awful since I only want to grab the thumbnail with no logo and the game doesn't look nearly as clear with all that "jpg" kind of compression on the borders... or are they this way on the database?

hansolo77

@Nismo said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:

@hansolo77 said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:

@vbs If that's the case, I'll just stick with my current batch of scrapes until Carbon is updated to allow videos. Because I have a bunch of systems installed on my RetroPie, I pretty much have to use Carbon. I don't like it, but it's the only theme that seems to work.

Carbon theme with video support:
http://www.mediafire.com/file/f32kjrkeopykjra/carbon_with_video.zip

Enjoy.

Awesome!! I'll give this a go on my next day off... Does this also include the added tags that are in the kids version of EmulationStation? If not, no biggie!

Nismo

@hansolo77 I suposse it has i only downloaded carbon from github and i added video support...

screech

@firewater what do you meen by antialiased ?

By default, MIX template use a png compression at the end of scrape. It may be that.

you can de-activate compression by editing the config.xml in the template you use by putting "false" on the <compression> node

firewater

@screech I've looked at the site and it seems that all images for arcade are like this from the database. So what I mean was that, this is how they look for arcade, and this is how it looks for a game on GBA. You can clearly see one has the pixels as presented on the game and the other has a bilinear filter on them (which looks awful, in my opinion).

MartinS

This post is deleted!

paradadf

@firewater you can always submit new images to the DB ;)
If you have better images, you can contact the people from SS to make a mass proposal for validation. This is easier if you are submitting hundreds of images. The only thing is that the names must (more or less) correspond.

screech

@firewater don't know where these Screenshot came from (I haven't submit it ^^) :S

Maybe it's a screenshot from an emulator (certainly) and there is some effect on it :S

For me, It didn't seem to have "so much" antialiasing...

And you compare a CPS1 game with a nintendo GB advance game ;)

But effectively, you are welcome if you have better Screenshot to submit some to the DB ;)

(there is a huge job to do on arcade system... The DB have just 1 year old ^^ lot's of work is comming ;) )

firewater

@paradadf @screech yes, be the change you want to see and such. Well to be fair I contribute on the film database of TMDB so I'm not that much of a leech of databases haha Not to mention I do appreciate that at least there are images, it's just I wanted to know if it was the program's problem or the database.

I downloaded a .zip that has 2000~ snapshots of MAME (don't remember where, it was yesterday, but it's probably failry common). Since the ROM name coincide, I think it's just a matter of appending a "-image" at the end to all files for ES to pick them up and problem solved... I hope.

On the other side, does anyone have a mix profile that has 2D box art + snapshot? This is I what I want (box at top, snapshot at bottom). I'll try making one myself if not.

screech

@firewater you can help the DB with your ZIP ;)

We can add directly your zip to the "Mass Import" system... Just need to associate them (with an auto association engine, if your image file are well named it's a question of "minutes" to update these 2000~ snapshot directly on the DB ;) )

For the MIX, sorry, I haven't one like that, but it's pretty easy to make the change ;)

just have a look here : https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/wiki/[TUTO]-Making-a-MIX-Template

If you have problem, you can contact me (or if I have some time I can do it ^^ )

If you managed to do it, don't hesitate to share it directly on the Github MIX Repository (https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/tree/master/MIX Repository ) So every one can dl it and use it directly from UXS ;)

(Or if you can't push it to github, send it to me ;) I'll add it ;) )

firewater

@screech I found the zip and it turns out it was a mirror of these. I think these are the standard snaps most use, so it's weird that there are other ones (the ones already uploaded, with bilinear filtering and so on). The whole pack is uploaded on Mega and is updated to the penultimate romset version so I think it's fair to assume it would be a good update from the ones that are included already... but I'll leave my involvement of this part at that, I'd rather get the the mix template (and the rest of my setup) working first.

So I'll get back to you soon... or in some hours depending on how difficult it is, with a MIX template result!

dankcushions

@firewater said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:

@screech I found the zip and it turns out it was a mirror of these. I think these are the standard snaps most use, so it's weird that there are other ones (the ones already uploaded, with bilinear filtering and so on).

right! this is a huge annoyance of mine with a lot of scraping databases. for any pre N64 era game, all screenshots should be lossless .PNGS at the exact resolution of the original game, and they should be about 30KB max. and other format will be loads worse looking and/or a larger filesize.

with stuff like mame, i've found some good snaps out there. see https://github.com/libretro/RetroArch/issues/4333#issuecomment-270173083

firewater

@dankcushions Yes! I agree completely. I also didn't know all (or most) databases did this, I've only ever dealt with boxarts most of my short frontend life so I assumed it was a specific problem on this one.

edit: well this is what I've got so far, it puts the box at the top and the screenshot at the bottom... more or less. Problems I've found is that 1. on NES, the boxart is not found 99% of the time (so it's empty) and 3. for arcade there's never a boxart at top (sure they don't have a "boxart" but it could put on a flyer, though I don't know how those are called).

0_1483561354455_Akumajou Special - Boku Dracula-kun (Japan)-image.png

<?xml version="1.0" encoding="UTF-8"?><Profil Name="Dual Vertical (2img)">
<Infos>
<Author>Anonymous</Author>
<Description>2 images assembled : 2D Box, and Screenshot</Description>
</Infos>
<General>
<Empty_Exemple>Empty_example.jpg</Empty_Exemple>
<Full_Exemple>Full_example.jpg</Full_Exemple>
<Target_Width>650</Target_Width>
<Target_Height>825</Target_Height>
</General>
<Compression>
<use>no</use>
<soft>pngquant.exe</soft>
<parameter>--force --verbose --ordered --speed=1 --quality=30-90 --ext .png</parameter>
</Compression>
<Root>
<Source_Value>Data</Source_Value>
<Target_Value>gameList</Target_Value>
</Root>
<Game>
<Source_Value>Data/jeu</Source_Value>
<Target_Value>gameList/game</Target_Value>
</Game>
<Country>
<Source_Value>Data/jeu/region</Source_Value>
</Country>
<Element>
<Name>box2dTEMP</Name>
<Source_Type>XML_Value</Source_Type>
<source_Origin>Game</source_Origin>
<Source_Value>Data/jeu/medias/media_boxs/media_boxs2d/media_box2d_%COUNTRY%</Source_Value>
<Target_Width>%0.5</Target_Width>
<Target_Height>%0.5</Target_Height>
<Target_TopLeftX>CENTER</Target_TopLeftX>
<Target_TopLeftY>UP</Target_TopLeftY>
<Target_Maximize>Yes</Target_Maximize>
</Element>
<Element>
<Name>screenTEMP</Name>
<Source_Type>XML_Value</Source_Type>
<source_Origin>Game</source_Origin>
<Source_Value>Data/jeu/medias/media_screenshot</Source_Value>
<Target_Width>%0.5</Target_Width>
<Target_Height>%0.5</Target_Height>
<Target_TopLeftX>CENTER</Target_TopLeftX>
<Target_TopLeftY>DOWN</Target_TopLeftY>
<Target_Maximize>Yes</Target_Maximize>
</Element>
</Profil>

mattrixk

@screech: I noticed you've updated the MIX tutorial, and that is awesome, thank you. However, I can't seem to get the Target_OriginX and Target_OriginY to work. I'm tyring to position the screenshot. If I don't use Target_Origin, the image shows up fine, with the Origin Point in the top-left of the image (basic default position).

If I use:

<Target_OriginX>%0.500</Target_OriginX>
<Target_OriginY>%0.500</Target_OriginY>

Then the screenshot image disappears completely.

If I use:

<Target_OriginX>CENTER</Target_OriginX>
<Target_OriginY>CENTER</Target_OriginY>

The screenshot shows up, but it ignores the Target_Origin, as though it isn't there.

Do I need to put it in a specific place in the element? Does it have to go before or after the Target_Width/Target_height? What about before/after the Target_TopLeftX/Y? Or does the position in the element not matter?