Please do not post a support request without first reading and following the advice in https://retropie.org.uk/forum/topic/3/read-this-first

Scrapes of full romsets available?



  • Is it possible to find complete packages of gamelist.xml files including images, e.g. for the complete romset for all Megadrive games, all SNES games, MAME 0.78 etc?

    I was thinking if the gamelists and the images-folder had everything in it, and you have maybe 40% of a romset installed, ES would only show the metadata and images for the games you actually had. If you acquire something new, then BAM images and metadata were already there.

    I'm having issues with scraping with both the built-in, the Universal XML scraper and the SSelph scraper, and right now it's nearing the weakest link of my setup, because I'm missing half my games, and the rest looks terrible.

    If we're all scraping for basically the same games, why is it not possible to share these images and metadata with each other?



  • @AndersHP

    Well, there's always the HyperSpin sets, with matching art sets from HyperSpin and EmuMovies. You could probably convert the HyperSpin XML so it can be used with ES or pick a different front-end. There's also art sets available for MAME for that purpose (although generally focused towards the latest version of MAME rather than the ones mostly used on the pi).



  • The difficulty with a "pre-made" XML is the link between the Rom File and the "Path" tag in the XML.
    Every one have a different romset, or different source for rom, with different naming convention. Some even rename manually each file...

    So it's "nearly" impossible to have a "universal" gamelist.xml... That's why "scrapers" exist ;) it use hash, or search function to match your own romfile with an existing game and generate the gamelist/media.

    Other advantage : it's easier to scrape your 200 Roms than downloading a gamelist.xml with media for 20000 games and clean it ;)



  • @screech

    Well, for the HyperSpin XML files, finding corresponding game sets is rather easy. :) What doesn't help is that ES requires you to clean up the files in case you don't have a full rom set; not all front-ends require that. Cleaning up can be scripted though, so it's not that hard. :)



  • @phulshof said in Scrapes of full romsets available?:

    finding corresponding game sets is rather easy. :)

    In your (or my) case certainly ;) because I think you already have a "clean" romset, or a well known romset like NO-INTRO without renamed file ;)

    but I can garanty it's not always so easy ;) I have associated manually about 4000 Roms to there own games (on 470000+ Screenscraper DB have) and sometime it's a real pain in the A. to match a file like "XmZ2.zip" to the good game ;) Even when you know on wich system it is ;)

    That's why I spoke about a "universal" file... It's just impossible it works on 100% case (even 80% will be a miracle ;) )

    I have an eye on what UXS Scrape in the world and what is found or not by the API on Screenscraper (API Error Result Page) and every day their is lot's of.... "silly thing ?"
    Gameboy game scraped as MegaCD game,
    File like "_1.zip",
    100Ko Iso file,
    And so much different case...

    An other thing, creating theses files mean creating a "new" local DB... because it will be very hard to have "complete" file.
    You always found someone who have the "special Hack Edition Unrealeased merged Turbo ' Alpha 2 zero Russian translation".
    That's why DB like Screenscraper, TheGameDb, and others were created ;)

    The only way it "may" works is to create these files only for Known Full romset. Like NO-INTRO, retrobution, Tosec,... But it mean 1 file by System and 1 file by romset creator... A huge work to do ^^

    In any way, feel free to take what you need on Screenscraper ;) it's CC ;)
    (And if your work is in the same "mind" with better info/media, we will certainly ask if it's possible to reverse the process to fill the DB ;) )



Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.