[SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture

herb_fargus

@screech sselphs scraper is modular enough that you can specify the ROM path, image folder etc. Retropie just integrated it as a module with specific file paths. They can really be set to anything you want if you know what you're doing.

chuzzwuzzer

In each rom folder I have my gamelist.xml files plus a downloaded images folder in each rom folder. Only problem is each of my gamelist.xml files are empty which is why I'm not seeing the images. Any idea why they are empty ?

paradadf

@chuzzwuzzer you need to kill emulationstation before scraping. I believe the latest version of UXS has a bug within the ssh commands. So for the time being, do it through other way.

chuzzwuzzer

But I already have the scraped images in the downloaded images folder. Are you saying I need to rescrape the entire system again ?

paradadf

Yes.

daeks

Feature request:

(Optional) Scrape only roms with missing images.
(Optional) Cleanup image folders for missing roms

screech

@chuzzwuzzer If you rescrape and keep your image in your folder, UXS won't re-download them... And just create the XML.

@daeks you already have 2 scrape options :
New rom, who scrape only roms who aren't in your XML
Picture only, who only scrape picture without touching your XML.

And for the cleanup it's already in my todo list ;)
( https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/issues/28 )

chuzzwuzzer

@screech - Just trying this. Should it still be reading : 'Hashing please wait.... Time left 2d, 22h, 41 mins 516/17651 '

Seems a long time just to rewrite the gamelist.xml if its not redownloading the images or is it normal for it to take this long ?

screech

have you the last version (I don't think)...
17651 roms ? it's pretty huge...
How many thread have you ? just 1 ?
So maybe it's normal... But 2 day seems to me a little too much :S

hansolo77

It takes my setup 2 days and 20 hours to scan EVERY system automatically, using 3 threads. I have 25 systems (10,895 roms) and then an extra system MAME that has about 8,000 or so ROMs. Hitting the CD image systems takes the longest time, and I only have like 20 PSX games and 8 SegaCD ones.

screech

Hello Everyone.

Sorry for the problem you can have with UXS and Screenscraper.
For the new year we are migrating server to a new big one.
In the main time, we are migrating screenscraper database to V2. So lot's of new things are coming.
Thank you to be patient, it's a matter of day (hour ?)...

Bonjour tous le monde,

Désolé pour les problèmes que vous pouvez rencontrer actuellement avec UXS et Screenscraper.
Pour la nouvelle année nous migrons de serveur et passant sur un nouveau Dédié beaucoup plus performant.
En même temps, nous faisons évoluer le site et la BDD de Screenscraper en V2.
Donc plein de nouvelles choses qui arrivent.

Merci d’être patient, il ne s'agit que d'une question de jours (heures ?).

Again we want to thanks all of you who help us participating to the BDD and with some tipee or patreon donation. Have an happy christmas and an happy new year.

Encore une fois, nous remercions tous ceux qui nous aide en participant à la BDD et financièrement avec Tipee et Patreon. Joyeux noël et bonne année..

Rion

@screech Merry Christmas to you to! Looking forward to the new site.

Thak

hi !

First, Merry christmas :D, I hope everyone received some Rasp as a christmas gift to build some wonderfull homemade Arcade/console ^^

Now with the UXS...
@screech : I try to change the path in order to scrape my backup rom file directly on my computer, no more than 300 arcade rom (and all of them has a screen) but UXS find no rom, make no gamefile.xls and no pictures dowloaded...

Can we actually scrap a roml file from the same computer as the UXS?

Si c'est pas clair, je peux le refaire en francais :D

Thx!

KillerQ

Is anyone else having trouble with Universal XML scraper at the moment? Running on my windows PC for the roms on Retropie (connected on the network)

It's, by far, the best tool out there for scraping - it for the past two days it will say it successfully found the roms, but will only download empty images (0kb) or nothing at all.

I'm running this on roms that have worked 100% of the time with the same program in the past - that's how I know something isn't working correctly.

Thanks!!

paradadf

@KillerQ read a few posts before...
ScreenScraper is migrating to a better server and thus the current "issues".

KillerQ

@paradadf said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:

@KillerQ read a few posts before...
ScreenScraper is migrating to a better server and thus the current "issues".

Got it - thanks!!

hansolo77

I noticed in the release notes for the latest version (5 hours ago v2.1.0.0 git 78e5b95) that you have a new profile for video. :) How does this work? Do you need to use a specific theme to get videos to work, or will it work with the default theme? I think for best results it should use the default theme at least. Just curious! If it requires a special theme, it might not work in my case, and I can't test it for you.

hansolo77

Preliminary testing...

Scraped Atari 2600
Rebooted Pi
Entered Atari 2600
Looked through games
No videos
Noticed some static screenshot images without boxart (using MIX profile)

Is it possible that the system is using the IMAGE tag first, and the VIDEO tag second as a fallback? If so, it should be the other way around. So the video plays first, and if no video exists then show the image. In the case of the static screenshot without boxart, is there maybe a requirement for ES to have some kind of codec installed to allow playing the video rather than just showing the first frame? Or is it truly just a screenshot without boxart in the database? In either case, it needs more testing. From what I can tell, videos do NOT play while in the default Carbon theme.

vbs

@hansolo77
Quite sure that if you want videos to show you will have to use a theme that makes use of them.

giventofly

@hansolo77 well, for that exact same problem i talked to @screech (guess it was him) on the github to add the hash of those files on the pi directlty instead of "transfering" them first.

you have the option to do that adding the config in the .ini (see github for the code).