[Request] Update list of ROM names in EmulationStation for lr-mame2003

bleuge

Couldn't a rom be scanned against a cummulative DB of CRCs and show the coincident ones? This would be pretty easy.

Example: imagine a rom: 1942.zip
We don't care about the filename. Extract the CRCs (crc32, md5,etc) of the files inside (but crc32s are instant and enough, we are not doing crypto stuff here).

Check them against the DATs of the mame versions supporting this particular set of files. Show coincidences. Let user choose the mame version he wants to play this (or predefine one particular version)

Edit: in my point of view, checking against filenames (the same with bioses) is bad. CRCs is the way to go.

pjft

Part of me struggles to see why can't one just scrape the ROMS and have the right names for them?

Zigurana

@pjft I agree, this file-name matching should not be the front-end's job, but be done by whatever (scraping) mechanism the user employs to populate the game list metadata tables.
If anything, the current list should be removed from ES, not expanded.

Zigurana

@jlachniet, this is not to bash your suggestion, that must have come across very bluntly. Apologies.

Let me explain:
You are completely right: the current situation is a proper bug, and we should fix it in one way or the other.
I just think that patching the current design by adding more filename - gamename pairs will only lead to more maintenance work down the line. Therefore I propose another design: removing this particular functionality from ES, and let it be taken care of by whatever scraper. That way, it will not work directly out of the box, but at least it's clear where the responsibility lies.

pjft

@Zigurana I agree with your assessment in terms of avoiding long-term maintenance of something that's bound to be broken.

That being said, if it currently works for the majority of cases (I'm not sure it is effectively less than 10 that are missing, but it shouldn't be a significant percentage), I'm also not sure if actively removing it altogether is the best decision for the users.

I know, it's like the ugly stepchild living under the staircase, but at least right now it works for most* (YMMV) cases, and one can scrape the missing ones. I think it may be better to have this than to have nothing, for the average user.

I'm a big supporter of scraping it just to get it done right, especially if we're talking about just a handful of cases.

Ideally, one option would be to have ES "update" (i.e. retrieve, from a web service) the latest set of online MAME DB list for all prior versions and use that as an offline name-matching bulk-scraping dictionary, rather than rely on the current static one. But:
a) I don't think that such list exists in a consumable-friendly format;
b) We wouldn't own it, so it would eventually break as well (change URL, not be kept up to date, change format), which would mean increased maintenance efforts;

So yeah. Scraping would still be the solution, but in this case I was just focusing on the names.

In fact - and this is more a question for @sselph - when you do the hash and rom updates pre-scraping, does your scraper store an offline MAME database of sorts, with the hashes and such? Would it also have an offline list of the rom name and game names in it, that one could use for a quick, offline "name matching" rather than doing it one by one?

Would an offline "MAME name matching" option be possible in the current scraping infrastructure, with a slight tweak?

Just exploring options.

sselph

@pjft For MAME I use the file name minus the .zip to look up against an online database. I also pull from a local name to history mapping to get description information. I don't hash MAME files because I didn't want to go down that rabbit hole since I know very little about MAME.

ES's scraping code is written to allow offline scrapers as well as online scrapers. I don't know if that is the solution you want but it is possible. I'm familiar with writing scrapers in ES if there are specific questions about it.

pjft

@sselph thanks for the clarification and for the prompt reply. I suppose my question is, then: do you store currently an offline cache/lookup table of the entire MAME ROM name to game name?

Is that even such a thing, or are all lookups/scraping operations one by one at the moment?

I'm just thinking if we could, with minimal effort in the current scraper architecture, replace the current static ROM name to game name mapping.

It is not urgent nor important per se - we're just speculating on what one such scraper could be.

By the way, tangential comment and question: I love your scraper - it's been always my go to reference, on my Mac command line. Extremely fast and reliable! Thanks for putting it together. I know you've done a lot of work in it recently. Do you scrape videos for the ROMS these days?

Anyway, back on topic :)

sselph

@pjft I don't but that information is in these files http://www.progettosnaps.net/dats/ the MAME Dats ones.

As for the videos you can watch https://github.com/sselph/scraper/issues/146 and I will close it once I get it added.

FranceMSR

lr-mame2003 uses MAME set version 0.78.

You can extract the DAT file from MAME executable in this way:
https://retropie.org.uk/forum/topic/15233/mame-set-rebuilder/23

and convert ANY set you have in a simple way with this application:
https://retropie.org.uk/forum/topic/15233/mame-set-rebuilder

grant2258

ive already fixed this for myself for mame and mame2003+ i just made a gamelist from the dats of mame2003 and 2003+