[SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture
-
@screech said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:
@hooperre You are right... Must add a save function to the original XML.
For the Donkey Kong File. I think there is a probleme with the ' in the name... Need to check that...I noticed that last night too. Tried to scrape Chip's Challenge and whatever i tried to name it, it couldn't find it. Also there was some problem with names that have multiple . in them. Like B.A.T. or P. P. Hammer.. I got those working by accidentally renaming them to NAME.ipf.zip
-
Normally, the "good" process is to identify your rom with CRC... Filename is just a "fallback" feature to scrape as much as it can ;) And so it's not very "reliable"...
-
@screech
Thanks for your reply.
Today it was faster, lot faster than yesterday. I have an account at screenscraper.fr but I haven't contributed ($$) anything yet.
Is it possible to run the scrape again but let it only scrape the roms from the missing list? -
Very nice ;)
FYI : $$ donation aren't necessary ;) just contribut to the DB is enough to grant you several Thread ;)
(We are missing lot's of sinopsys or media you can submit ;) )And of course you can just "add" the missing rom when you crape ;) in General menu, look at the scrape mode and select "update" ;)
-
@screech yeah i know, but I was scraping custom amiga uae files, so no hashes for me :)
-
@EctoOne try this :
https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/wiki/Rom-Missing-on-ScreenscraperPass your uae file in F-CRC and send them to me ;) if they are well named, they will be autoassociated to there legitimate game ;) so you can scrape directly with CRC ;)
-
@screech Thanks for this man. It's an amazing tool and you've enhanced my RetroPie experience greatly. So thank you.
Would creating a file directly from a URL to the screenscraper.fr database be difficult to incorporate for ROMs that don't pull easily? Sorry if this is a... novice question.
-
@hooperre sorry, my english must be too bad ^^ I don't understand your question :p
-
@screech I was wondering if you could add something that will make a MIX image simply by pasting a link to the ROM in the DB.
For example, for Donkey Kong 3, I would just paste "https://www.screenscraper.fr/gameinfos.php?plateforme=4&gameid=2181" into the XML Scraper and it would output the MIX image for that game.
-
unfortunatly, it's actually impossible...
To create a MIX, I need to catch the Media.
To Catch the Media, I need to have an API response from the DB.
To have an API response from the DB, I need to ask the API with a CRC32, MD5, SHA1, or Filename...I can't with the Game ID...
But I can ask the Screenscraper boss to add this to the API ;) but I think it's not for tomorrow ;) (he is pretty busy right now ^^)
-
@screech I don't think that would help in this case. UAE are just text config files and even with the automatic ways to create them i think they would be different from system to system. I mean it uses a template to create them and after that it already makes automatic changes to the controller name when you start them.. It was easier to batch rename them.
-
Arg.. ok ^^ CRC on this is useless ^^ (not very in Amiga ^^)
-
Speaking of complicated scraping. How exactly do i scrape ScummVM? I've found a rar containing all shortname.scumm files you uploaded once. When i put them in my rom folder (or their own subfolders), it says Roms Path can't be reached.
-
Hi Screech,
I am new tot he retropie community and love the way the universal xml 3 image art looks. My only problem is I have a MAC computer, and i can use the universal xml scraper. Do you have any suggestions on another way to quickly scrap this great image art to my retropie?
Thanks,
-
@screech said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:
@Dochartaigh 8+ hours for a NES romset with your config and 5 threads is pretty weird... normally a rom is scraped between 7 and 30s (depending on Server load and you config...)
So I gave up on NES, trying Genesis now. 1,707 ROMs from the No-Intro romset (totally standard). Turned CRC off and just going by filename to speed things up. AND it's reading directly off my hard drive. AND I joined the screenscraper site and logged in on the program. AND I contributed unique content (still only have 1 thread). AND the scraper site's server load is only 52% right now...
...AND it's still going to take me 14 hours to scrape Sega Genesis/Mega Drive....
Sorry, but if this entire program hinged on a slow flaky website (can't register from a Mac running Google Chrome either FYI...have to use Safari) to scape from it's completely flawed I'm sorry to say.
P.S. AND the program just crashed like 3 times in a row... lol
-
Many ANDs in there, lol
Just start it before going to bed and next day all is done. The time will change. Yesterday when I was scraping my NES roms (750+) the time constantly changed and got shorter and shorter. -
@Dochartaigh usung a filename isn't faster than the crc for small files.
I really believe your instabilities have something to do wth you running UXS on paralell. I would try it on a normal windows installation to compare.
It can also be that your firewall is causing problems, don't know. -
@Dochartaigh An other thing : don't look (too much) at the "estimated" time...
It is really "wrong" at start...Just to check, launch a scrape, scrape about 10 roms and cancel, look at the Time per Rom .
If it's between 5 and 30sec it's "ok"... you are at a normal speed (depending on your config/network/server load)
Try with more thread (just make some proposal on www.screenscraper.fr, wait for the Moderation validation, and put your ID in UXS) It will be really "fast" with just 2 or 3 threads...
(To explain thread : The Main EXE grab the API return and send it to X engine to manage them in parrallel. X is the thread number. The Engine will grab media, create the MIX and generate a "small" XML. When scrape is finished (or cancel) it will merge all the small XML in a final gamelist.xml )
-
Hi,
Recently came across this program but I have two issues I can't find answers for.
- I've noticed on the Screenscraper website that it doesn't list multiple disk names.
For example 'Another world' on the Amiga, it will scrape disk 1 but not disk 2.
Anyway around this?
- On the roms that don't get scraped, do I need to rename these as per the rom names on the website for the scrapper to work?
If not can they be manually added to the gamelist.xml file?
Thanks
-
@paradadf said in [SOFT] Universal XML Scraper V2 - Easy Scrape with High Quality picture:
@Dochartaigh usung a filename isn't faster than the crc for small files.
I really believe your instabilities have something to do wth you running UXS on paralell. I would try it on a normal windows installation to compare.
It can also be that your firewall is causing problems, don't know.I've tried it three ways: on Parallels inside OSX, on Windows Bootcamp (which is a native Windows partition on a Mac - NO different whatsoever than running Windows on a Windows PC), and on an old ASUS i3? dual-core laptop I had laying around. I can let it run for 30-60 minutes and it still always tells me something like 6 to 8 to 14+ hours to scrape a NES, SNES, or Sega Genesis romset. No firewall, heck, even turned off my VPN.
Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.
Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.