[SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture

[Dochartaigh]

I swore this was already gone over, but you can't search this entire thread (keeps on loading only sections at a time).... I'm having an issue where scraping Playstation adds a gamelist entry for every single file a certain PS1 game will have (some have just a .BIN and a .CUE - others have like 30 .BIN's and a single .CUE - so that one would have 31 files listed in the gamelist). How to fix?

[Jeck11]

@Dochartaigh add an exclusion for *.bin and you shouldn't get doubles.

Also, to search a specific thread on this site just click in the thread the hit ctrl-f. You should see something like thread:678533 show up in the search box.

[Dochartaigh]

@Jeck11 said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:

@Dochartaigh add an exclusion for *.bin and you shouldn't get doubles.

Also, to search a specific thread on this site just click in the thread the hit ctrl-f. You should see something like thread:678533 show up in the search box.

Thanks, I'll exclude .bin's!

And this forum only loads so many posts at once - so there's 380 posts in this topic, and instead of splitting it into multiple pages like the most common Phpbb and simple machines (and similar) forums do, it'll only load the last 10 or 20 posts so that's all you can search for with CTRL+F unfortunately (unless I'm missing something).

[screech]

Hello Every one.

Just pull out a fresh new release (v2.2.0.0)

What's new :

New Language Set
Now English Language are separated in 2 : English UK and English US. So now it handle correctly the media fallback (exemple : In UK fallback if a cover matching the rom is not found it will take the European one and the world one if not found. In US, it will take the US one and the world one... Same for the game naming)

A complete new Wizard.
I force it at start to make sure every one have a "good" install first.
I put a lot's of "verification" to make sure a standard install can scrape easy.
You can always tweek what you want after that ;)

A New "Options" Menu.
Now you will have a lot's of "easy" tweek" like :

• Mix or not.
• If not Mix : selecting Default picture, alt picture and for Retropie : Marquee and Videos
• Naming convention : Default is "Game Name + Options" so you can tweek the options.
• Name Tag options : Now you can put the country between [], and some tags between () like Clone, Beta, Demo, Proto, Trad, Hack, Unl, Alt, Best. (To come : Clone type and Hack type) -> So normally no more simple duplicate name (for mame for exemple). If still duplicate, it's because we haven't the info in DB. Don't hesitate to submit correction on that on www.screenscraper.fr
• Case for multiple metadata : Default/UPPERCASE/lowercase (depending on Theme, it may matter)

New Profil :
Profil change "a lot"... Not so much modification of existing things, but only 2 profils now : Recalbox/Retropie. With a lot's of new options (at the end).
It create dynamically the option menu.
We loose some "options" but I will put them back later (Like the "rom folder" profil or the "scummvm" profil)... I need to work on these.
Special dedicace to @mattrixk if you have any difficulty with modification on these profil, let me know.

New option in Gen config
It was asked, by default, UXS don't scrape "IN" zip file... So it may not found some game, but it's a way faster with romset like mame.
You can reactivate that in option.

Carbon Wheel on the UI
Now you can know what plateform you are scraping, it put a little carbon wheel on the UI.
This wheel depend on you language preset (so Genesis for US or Megadrive dor UK ;) )

Some debbuging

• Full Scrape works now ^^
• Full scrape have a good "Result" windows at the end
• Normally no more bug when scrape multiple system in the same session (you don't need to close and reopen UXS)
• Lot's of small debbuging.

To come :

• A new server (actually the small server we have suffer a lot. Specially at Week end) thanks to your donation we migrate soon to a new bigger and dedicated physical server.
• A new API (With the new server, little time after, we will have a new API with a lot's more new stuff to scrape and more easily ;) )
• A new Error handling (Actually error coming from the API aren't so easy to catch. So with the new API, I can have a better return and so a better information on what's going on when it doesn't work properly)
• lot's of new stuff in UXS ;)

Asking for Help
With this new version, some translation aren't up to date. So if you speak German, Spanish, Dutch, Portugaise, Japanaise or Italian, you are welcome to contribute.
(Juste look in the LanguageFiles Folder. Open the XML correspondinf. The stuff needing translation are at the end ;) )
If you think my English isn't so good (what is true) don't hesitate to correct the EnglishUK or EnglishUS XML ;)

@FlyingTomahawk : Just Open the Advanced Menu, Image configuration and select "Default" as extension. It will solve all your extension problem ;)

Thank you guys, and don't hesitate to tell me if something don't work well ;)

[Rion]

@screech
Thats a new hefty release list 😀
God to hear about the new server!

[ExarKunIv]

Having a problem getting this to launch

i got the latest version from Github v 2.2.0.0
I run Universal_XML_Scraper64.exe. i get the splash screen and then a error pops up

Line 38253 (file "c:\my stuff/ Universal-XML-Scraper-master\Universal_XML_Scraper64.exe"):

error subscript used on non-accessible variable

i looked in the logs folder and here is what i got

[11:03:09] - C:\My stuff\Universal-XML-Scraper-master\Universal_XML_Scraper64.exe (2.2.0.0)
[11:03:09] - WIN_8(X64) - 0409
[11:03:09] - Scrape in x64
[11:03:09] - > No updated files needed (Version : 2.2.0.0)
[11:03:09] - Starting files installation
[11:03:09] - > Ending files installation
[11:03:09] - /!\ Error Downloading File : C:\My stuff\Universal-XML-Scraper-master\Ressources\ServerList.txt
[11:03:09] - /!\ Bytes read: 0
[11:03:09] - /!\ Size: 0
[11:03:09] - /!\ Complete: True
[11:03:09] - /!\ successful: False
[11:03:09] - /!\ @error: 13
[11:03:09] - /!\ @extended: 12029
[11:03:10] - /!\ Error Downloading File : C:\My stuff\Universal-XML-Scraper-master\Ressources\ServerList.txt
[11:03:10] - /!\ Bytes read: 0
[11:03:10] - /!\ Size: 0
[11:03:10] - /!\ Complete: True
[11:03:10] - /!\ successful: False
[11:03:10] - /!\ @error: 13
[11:03:10] - /!\ @extended: 12029
[11:03:10] - /!\ Error Downloading File : C:\My stuff\Universal-XML-Scraper-master\Ressources\ServerList.txt
[11:03:10] - /!\ Bytes read: 0
[11:03:10] - /!\ Size: 0
[11:03:10] - /!\ Complete: True
[11:03:10] - /!\ successful: False
[11:03:10] - /!\ @error: 13
[11:03:10] - /!\ @extended: 12029
[11:03:10] - > -In 3 try and 0.86s

so i look in the Ressources folder for the ServerList.txt and i saw it was missing so i unziped Ressources zip and placed it in there. i saw that once i ran the exe it disappeared. and yes i did try in admin mode.

any ideas what im doing wrong here

[screech]

strange, serverList is here : https://raw.githubusercontent.com/Universal-Rom-Tools/Universal-XML-Scraper/master/Ressources/ServerList.txt

It's not a DB issue, it's like UXS can't download it... Have you open your firewall ?
(FYI it's the first file it try to download... So...)

[ExarKunIv]

yes i have the firewall turned off. still same thing.
Im currently at work. so i think it is something they have set for the firewall here. :(

ill give it a shot when i get home.
thx for the help

[ExarKunIv]

my work around is i just install my VPN here, and im useing that. lol

[DynaMight]

The update looks good but it isnt scraping anywhere near as many games? The previous update scraped 896 out of 900 roms (Megadrive) however the latest update only managed to scrape 565? Exactly the same roms. I re-ran it and it did exactly the same amount.

Any ideas?

[screech]

@ExarKunIv So it's ok ?
@DynaMight try to check the "Unzip zip" checkbox in the general config ;)

[DynaMight]

@screech No I didnt, I did see the option but didnt understand what it actually meant. All the roms are zipped. I'll try it tomorrow, but what does the option do/mean?

Thanks

[screech]

Before the 2.2 UXS do this :

look for the file hash in the DB
if not found, look for the rom file name in the DB
if not found, unzip the zip file (if zip) and look for the first file hash in the DB
if not found, look for the first file name in the DB
if not found, look for the second file hash in the DB
if not found, look for the second file name in the DB
And so on till it found a match or do all the file in the zip...
It work nice on some case (specialy console)

But it's horrible on other system (like Mame :S)

the unzip part is now an option you can activate or deactivate...
By default it's not activated because of that and to make a quick scrape..
it's faster but less precise because if not found in the 2 first case (Hash and filename of the zip)... it don't look further...

[mattrixk]

Looking good @screech. I'll hopefully have a chance to check it out in a few hours. (and you release this just after I completed my custom Scrape and MIX Profiles too :D)

[screech]

really sorry @mattrixk ^^
MIX profiles haven't change so normally it's ok ;)
For your Scrape profiles, send me the "old one" I'll adapt them to the "new" format if you want ;)

[mattrixk]

@screech No worries mate. I'll didn't get the chance to play with it today, but I'll have a look soon. I only have about 30-40 minutes of free time a day (during lunch at work), so I have lots of stuff to fit in there, and some times l like to chill and watch YouTube or something.

[screech]

WARNING : I confirm that the Kill emulationstation don't work :(

So you need to stop it manually till I correct that tonight...

[Squilenator]

@screech for some reason the new version is not scraping Wheel Art into the mix profiles for Arcade Zoom, and the Standard 2img sometimes scraps the wheel and sometimes not, it happens a lot now.

[SuperMagicom]

@Squilenator It's not just that mix profile either. The mix scraping is totally fubar in the latest version. I opened an issue ticket on github the other day so hopefully he fixes it soon. Until then I've gone back to the previous version and everything scrapes normally.

[SuperMagicom]

@screech Latest versions, including today's update, throwing "AutoIt error: Subscript used on non-accessible variable" and crashing after boot screen. 2.1.0.4 is the newest version that will run normally. :(