Manually generate Mame2003 gamelist?

herb_fargus

still need to check out the php script, haven't gotten a free moment yet.

Though as an aside to parsing the xmls, as I haven't really got the greatest scripting skills, a quick hack you can download the sheet as a csv, upload that csv to here: http://www.convertcsv.com/csv-to-xml.htm and tweak a few settings and it pukes out a functional gamelist.xml

Once the spreadsheets are filled, should be simple enough to generate :)

UDb23

Very interesting web tool, will try it; thanks !

UDb23

@herb_fargus @caver01
Wrote some vba code in excel to parse History.dat (182) and extract the "official" mame .78 games descriptions.
Here's the resulting csv file, fields are: mame .78 rom name, latest mame name, game title and description. Total 4714 roms.

Should now be quite simple to integrate your Google sheet with this info (by vlookup ?) and make a "global" table to generate gamelist.

markwkidd

@herb_fargus if useful, there is now a 0.78 catver.ini in the mame2003 repository: https://github.com/libretro/mame2003-libretro/tree/master/metadata

One potentially nice thing about this is that corrections or updates can be made to the catver.ini in the libretro repository and the changes could percolate down to frontends and users of the core. I can see this RetroPie effort helping 'upstream' improvements.

Speaking of which, it may also be possible to add a history.dat to the mame2003 repository. history.dat is the only other of the major MAME metadata files that I haven't pursued yet in my quest to add them to the various mame repos.

@UDb23 that's some nice work! (I recently created a simple AHK script to regenerate catver.ini files from one version to another if you ever need it)

Two questions:

could your VBA be modified to produce a properly-formatted history.dat that we might try to add to the mame 2003 repository?
do you have a way to list the titles that are missing records in your csv so that the could be completed or at least tracked?

herb_fargus

@markwkidd Brilliant work. How did you handle the file renaming changes through the versions? or did you use the hashes from the dat?

Also the retropie wiki has 4705 games listed but your list is 4723 (wiki could be wrong I'll have to look over the dats again)

UDb23

@markwkidd Basically it starts from a list of roms and finds the corresponding descriptions in History.dat.
So if you need to extract additional or different roms it is very simple: just adding them to the initial list.
Note: History.dat contains current mame rom names (vs 78 names); therefore initial list must have 78. romname and current (182) romname (this last is the one used to search in History.dat's content).

What's the exact "properly formatted" structure you'd like History.dat to be "converted" ?

Please note that it contains huge amount of text. Descriptions already are very long.
I'm currently fixing a minor issue with CR/LF that is not correctly saved in my csv.

UDb23

@markwkidd To handle the name changes I wrote some other code some months ago based on renameset.dat by Antopisa.

markwkidd

@UDb23 Game names/zip file commonly names changed over time in the MAME sets. The game may have one filename and title in MAME 0.182 and be completely different in MAME 0.78.

The .dat formats also changed throughout the years. For example catver.ini and cheats.dat each go through at least three format changes between MAME 0.37b5 and now. Because I haven't been able to find an old history.dat for that era yet, I'm not sure if a modern history.dat can be read by the mame2003 core itself. I'll look into that though!

@herb_fargus you may be giving me credit for UDb23's work -- they are the one who created the awesome history.dat CSV ! : )

markwkidd

@UDb23 said in Manually generate Mame2003 gamelist?:

@markwkidd To handle the name changes I wrote some other code some months ago based on renameset.dat by Antopisa.

Nevermind my thoughts on name changes. Nice!

markwkidd

@herb_fargus said in Manually generate Mame2003 gamelist?:

@markwkidd Brilliant work. How did you handle the file renaming changes through the versions? or did you use the hashes from the dat?

Also the retropie wiki has 4705 games listed but your list is 4723 (wiki could be wrong I'll have to look over the dats again)

There should be a total of 4720 ZIP files in a MAME 0.78 set, of which 15 are BIOS. This is true of a set I have access to which scans as complete in ClrMamePro.

herb_fargus

@markwkidd ah thats right, when I listed it I removed the bios because I didn't consider them games. Ok. sure. um. I'll need to look them both over and compare against the name changes just to verify it all.

@UDb23 looks great so far.

UDb23

@herb_fargus Here's an improved version of the csv with the descriptions.
Basically I got rid of all those leading (and trailing) non readable characters (CR/LF etc) that were present in the description in History.dat.

btw: I wonder how they came up with that "unstructured" history.dat file format A little bit tricky to extract the descriptions without specific delimiters. ;-)

UDb23

Concerning number of ROMS:
my list includes 4714, so if ClrMamePro states 4720 it should be easy to find the 6 missing ones by comparing. Or maybe I just missed some of the BIOS files to be removed (actual game roms should be 4705 according to @markwkidd ).