[SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture

chuzzwuzzer

@screech - Just trying this. Should it still be reading : 'Hashing please wait.... Time left 2d, 22h, 41 mins 516/17651 '

Seems a long time just to rewrite the gamelist.xml if its not redownloading the images or is it normal for it to take this long ?

screech

have you the last version (I don't think)...
17651 roms ? it's pretty huge...
How many thread have you ? just 1 ?
So maybe it's normal... But 2 day seems to me a little too much :S

hansolo77

It takes my setup 2 days and 20 hours to scan EVERY system automatically, using 3 threads. I have 25 systems (10,895 roms) and then an extra system MAME that has about 8,000 or so ROMs. Hitting the CD image systems takes the longest time, and I only have like 20 PSX games and 8 SegaCD ones.

screech

Hello Everyone.

Sorry for the problem you can have with UXS and Screenscraper.
For the new year we are migrating server to a new big one.
In the main time, we are migrating screenscraper database to V2. So lot's of new things are coming.
Thank you to be patient, it's a matter of day (hour ?)...

Bonjour tous le monde,

Désolé pour les problèmes que vous pouvez rencontrer actuellement avec UXS et Screenscraper.
Pour la nouvelle année nous migrons de serveur et passant sur un nouveau Dédié beaucoup plus performant.
En même temps, nous faisons évoluer le site et la BDD de Screenscraper en V2.
Donc plein de nouvelles choses qui arrivent.

Merci d’être patient, il ne s'agit que d'une question de jours (heures ?).

Again we want to thanks all of you who help us participating to the BDD and with some tipee or patreon donation. Have an happy christmas and an happy new year.

Encore une fois, nous remercions tous ceux qui nous aide en participant à la BDD et financièrement avec Tipee et Patreon. Joyeux noël et bonne année..

Rion

@screech Merry Christmas to you to! Looking forward to the new site.

Thak

hi !

First, Merry christmas :D, I hope everyone received some Rasp as a christmas gift to build some wonderfull homemade Arcade/console ^^

Now with the UXS...
@screech : I try to change the path in order to scrape my backup rom file directly on my computer, no more than 300 arcade rom (and all of them has a screen) but UXS find no rom, make no gamefile.xls and no pictures dowloaded...

Can we actually scrap a roml file from the same computer as the UXS?

Si c'est pas clair, je peux le refaire en francais :D

Thx!

KillerQ

Is anyone else having trouble with Universal XML scraper at the moment? Running on my windows PC for the roms on Retropie (connected on the network)

It's, by far, the best tool out there for scraping - it for the past two days it will say it successfully found the roms, but will only download empty images (0kb) or nothing at all.

I'm running this on roms that have worked 100% of the time with the same program in the past - that's how I know something isn't working correctly.

Thanks!!

paradadf

@KillerQ read a few posts before...
ScreenScraper is migrating to a better server and thus the current "issues".

KillerQ

@paradadf said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:

@KillerQ read a few posts before...
ScreenScraper is migrating to a better server and thus the current "issues".

Got it - thanks!!

hansolo77

I noticed in the release notes for the latest version (5 hours ago v2.1.0.0 git 78e5b95) that you have a new profile for video. :) How does this work? Do you need to use a specific theme to get videos to work, or will it work with the default theme? I think for best results it should use the default theme at least. Just curious! If it requires a special theme, it might not work in my case, and I can't test it for you.

hansolo77

Preliminary testing...

Scraped Atari 2600
Rebooted Pi
Entered Atari 2600
Looked through games
No videos
Noticed some static screenshot images without boxart (using MIX profile)

Is it possible that the system is using the IMAGE tag first, and the VIDEO tag second as a fallback? If so, it should be the other way around. So the video plays first, and if no video exists then show the image. In the case of the static screenshot without boxart, is there maybe a requirement for ES to have some kind of codec installed to allow playing the video rather than just showing the first frame? Or is it truly just a screenshot without boxart in the database? In either case, it needs more testing. From what I can tell, videos do NOT play while in the default Carbon theme.

vbs

@hansolo77
Quite sure that if you want videos to show you will have to use a theme that makes use of them.

giventofly

@hansolo77 well, for that exact same problem i talked to @screech (guess it was him) on the github to add the hash of those files on the pi directlty instead of "transfering" them first.

you have the option to do that adding the config in the .ini (see github for the code).

hansolo77

@vbs If that's the case, I'll just stick with my current batch of scrapes until Carbon is updated to allow videos. Because I have a bunch of systems installed on my RetroPie, I pretty much have to use Carbon. I don't like it, but it's the only theme that seems to work.

MartinS

@hansolo77 So, you have been able to get XML Scraper to work recently? I have tried the new version and old versions. When I select scrape, the program kind of hangs. I have to close it via task manager.

hansolo77

Their server is undergowing migration. They're moving everything to a larger/faster system. There have been hiccups, and delays. I tried probably a dozen times last night to just get UXS to open, it hangs, then crashes saying it timed out. The one time it DID open, I was able to get Atari 2600 to scrape. It looks like it worked. I just don't have a compatible theme to use the videos. It downloads both simultaneously (videos & images) and stores them side-by-side, and adds a <video>/path/</video> line to each game in the gamelist.xml. The only noticeable difference is the file extension.

KillerQ

Seems that there are still hiccups in the migration - or something.

I can scrape roms on my pi that contain less than about 100 ROMS - anything larger just sits there and does nothing. It will create an image folder and gamelist, but it never populates either.

I also noticed that some directors and paths under options have a period (.) before the paths... I wonder if that's a typo.

Just trying to help.

Thanks!

KillerQ

I figured something out. If your XMS just hangs and never starts try this... type the name of the system in your roms folder over again... that is, even though the rom folder may look to be named properly, and XMS sees it, before you scrape, just go into the pie and type the rom name folder over.

I renamed 'nes' to 'nes' again, and it worked perfectly!!

KillerQ

@screech said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:

Hello Everyone.

Sorry for the problem you can have with UXS and Screenscraper.
For the new year we are migrating server to a new big one.
In the main time, we are migrating screenscraper database to V2. So lot's of new things are coming.
Thank you to be patient, it's a matter of day (hour ?)...

Bonjour tous le monde,

Désolé pour les problèmes que vous pouvez rencontrer actuellement avec UXS et Screenscraper.
Pour la nouvelle année nous migrons de serveur et passant sur un nouveau Dédié beaucoup plus performant.
En même temps, nous faisons évoluer le site et la BDD de Screenscraper en V2.
Donc plein de nouvelles choses qui arrivent.

Merci d’être patient, il ne s'agit que d'une question de jours (heures ?).

Again we want to thanks all of you who help us participating to the BDD and with some tipee or patreon donation. Have an happy christmas and an happy new year.

Encore une fois, nous remercions tous ceux qui nous aide en participant à la BDD et financièrement avec Tipee et Patreon. Joyeux noël et bonne année..

Screech,

 Thanks for your hard work.  I figured something out that may help you out...

If your XMS just hangs and never starts try this... type the name of the system in your roms folder over again... that is, even though the rom folder may look to be named properly, and XMS sees it, before you scrape, just go into the pie and type the rom name folder over.

I renamed 'nes' to 'nes' again, and it worked perfectly!!

vbs

@KillerQ said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:

I renamed 'nes' to 'nes' again, and it worked perfectly!!

Sorry to say, but this cannot fix anything. Really not :)