How do you organise romsets? MAME and (in particular) consoles

AndersHP

I am aware that some win-based applications exist for sorting MAME roms, and have not tried these yet, but what about Master System, Genesis, NES and SNES romsets?

I have several romsets, with every game placed in a 7z file, and in each file there is 10-15 variants of the games. I really wanted a way to extract all the files to one folder (already here I have issues, since I'm running Mac) and sorting by [!] and maybe [U] for Complete, American versions of the games only.

How do you do this, especially on Mac? And do you have recommendations for sorting MAME / FBA romsets as well? In my test-run before my build is completed I have had luck with the recent FBA romset in combination with the MAME2003 0.78 romset. But I don't want to add a game at a time if I could just find a way to sort them and move them all to my Pi.

rbaker

@AndersHP

that will filter out the roms and shrink the collection to a manageable size. There are a selection of "lite" dat files out there for this purpose. So for example, just neogeo or just fba. Then you can transfer. It is important to start with a full correctly verified reference set first though.

AndersHP

OK, even though Romlister is for windows and not Mac, I will work this out.
But does it only work for MAME, or could I use it for FBA, and the console games also (NES, SNES etc.)?

rbaker

@AndersHP said in How do you organise romsets? MAME and (in particular) consoles:

and the console games also (NES, SNES etc.)?

Use romcenter for multiple emulators. But, the only way to get a clean set of nes roms in my experience is to acquire a "No Intro Collection". No hacks or bad dumps. No software I tried could clean up like you can with mame.

AndersHP

@rbaker Romlister needs a Mame list XML file... Can I use the compatibility sheet for FBA on Github?
Or where do I get this XML file for my FBA romset?

The other two files catvers and controls I'll have to try from here: https://www.tapatalk.com/topic/151580-19713

I really must say, I thought setting up ROMS so they'll work on RetroPi was going to be hard, but really, the hard part is getting these extense romsets cooked down to the essential games is the hard bit!

rbaker

@AndersHP I'm not sure as I have only really ever used MAME with clrmamepro over time. My "No Intro Collection" for the NES never needed cleaning.

dankcushions

@AndersHP said in How do you organise romsets? MAME and (in particular) consoles:

@rbaker Romlister needs a Mame list XML file... Can I use the compatibility sheet for FBA on Github?
Or where do I get this XML file for my FBA romset?

i think the .dat files we supply on the FBA/mame wiki pages are what you need. just rename from .dat to .xml. i don't know why they're often in .dat format when they're xml within. i guess it's historical.

Dochartaigh

@AndersHP said in How do you organise romsets? MAME and (in particular) consoles:
I really wanted a way to extract all the files to one folder (already here I have issues, since I'm running Mac) and sorting by [!] and maybe [U] for Complete, American versions of the games only.

How do you do this, especially on Mac? And do you have recommendations for sorting MAME / FBA romsets as well?

Mac actually excels at sorting files. Even unzipping like 3,000+ files from terminal will save you about 90% of time (vs. double-clicking all the roms at once and have whatever utility program you have installed unzip them all - that takes a TON more time).

Mac Finder (i.e. Mac's Windows Explorer) even has a built in renaming/find-replace function for multiple files.

Mac Terminal (a la linux-flavor) brings in so many other options it's not even funny. You can actually search for a certain keyword, make a .txt file from all those file names, edit that text file to your liking, then run a second easy command to either delete all those files at once, or do something like put them in a folder to sort them. It's REALLY powerful (can do similar/same thing in terminal on the Pi too).

For MAME, I suggest non-merged sets so each individual version of a single game has ALL the files needed to run it in one file. Certainly saves me tons of time since I usually only play 1 version (maybe 2) of a certain rom.

@rbaker said in How do you organise romsets? MAME and (in particular) consoles:

My "No Intro Collection" for the NES never needed cleaning.

I disagree. No Intro collection still has like 5-10ish versions of a single game, like for 1000+ games PER system. Looking at their NES right now (from the bottom), and there's 5x Yoshi Cookies, 6x Zelda II's, 3x Zoids, 2x and 3x of nearly everything else...just from from Y-Z.

If you play every version of a game then that's great (and speak like 10 different languages ;) . But for all the rest of us there's a LOT to cleanup if you want ONLY 1 good version (from your language/region hopefully) of every game. I've still not found a good way to automate this search and delete process to filter out the ones I don't like - except by doing it mostly manually.

rbaker

@Dochartaigh I see, it appears then that I do not have the said "no intro" set. I have 655 roms and they are all (U) versions with no duplicates.

markwkidd

@rbaker said in How do you organise romsets? MAME and (in particular) consoles:

@Dochartaigh I see, it appears then that I do not have the said "no intro" set. I have 655 roms and they are all (U) versions with no duplicates.

It sounds like your set may have been processed by a "1 Game 1 Rom" or 1G1R DAT file. They are available at the No-Intro site and maybe what the OP is looking for.

markwkidd

@AndersHP when it comes to using the catver.ini file, please be aware that you need to use a catver.ini with the same version as your ROMs and your MAME emulator.

For example you can find a MAME 2003/MAME 0.78 catver here: https://github.com/libretro/mame2003-libretro/tree/master/metadata

You might have luck running the MAME category sorting (and clone removing) tool that I have been developing, but as with the others it's a Windows app. Not sure if it will work in OS X via WINE, as no one has tried: https://retropie.org.uk/forum/topic/7606/simple-mame-category-sorter-get-rid-of-mahjong-rom-tool

AndersHP

@markwkidd said in How do you organise romsets? MAME and (in particular) consoles:

@rbaker said in How do you organise romsets? MAME and (in particular) consoles:

@Dochartaigh I see, it appears then that I do not have the said "no intro" set. I have 655 roms and they are all (U) versions with no duplicates.

It sounds like your set may have been processed by a "1 Game 1 Rom" or 1G1R DAT file. They are available at the No-Intro site and maybe what the OP is looking for.

YES! This is what I'm looking for, just had my hopes up that the No Intro Romsets were split in regional subfolders... But no.

I'm no programmer, I cannot find a "1g1r dat file", and I'm not sure how to process it if I had one.

I'm now using finder on Mac to first find Europe versions, copying these into a different folder, and then World and doing the same, I guess....

geneworld

Just write a python program to unzip all the zips files to a single directory, then with the help of a variable [say keep = (U)] have it move all the files you want with the '(U)' in the file name to a different directory and you can see what your left with. After that you can just 'sudo rm -r' the old directory with the remaining roms and folders . It's simple actually ;)

markwkidd

@AndersHP my only experience is on Windows with ClrMamePro

With the correct DAT file ClrMamePro made it easy for me to set a preferred region of 'USA' and only add ROMs from other regions if there was not a USA version. (I know your intended region may be different but the same process should apply)

For example, depending on the specific system and what regions was available in, I could set ClrMamePro to filter my set based on a preference of USA ROMs > then EUR > then JPN.

The resulting set of ROMs will feature a USA version of a game whenever possible, then look for a EUR region ROM for that title, and finally use the JPN only if no USA or EUR ROMs are in the folder. All regions with releases in a given system are supported (in other words you could set a preference of Spain > Europe > USA > Italy > Japan > etc.)

Rion

@AndersHP I use hyperspin datfiles for my rom collections. One game one region.

The problem is that the output_file don't have the file extensions intact. So you have to add that after you run your roms thru the No-Intro collection that Hyperspin is using as it's source.

AndersHP

I just tried testing ClrMamePro, but I'm puzzled as to how it works. Tried making a test of rebuilding a "romset" consisting of dkong.zip, ddragon.zip, ddragon2.zip and ddragon3.zip.

I was using the DAT file "lite" from the Github page for MAME2003 (0.78) and thought the result was Donkey Kong and Double Dragon 1-3. But no, the result was chinagat, darktowr, ddragon1-3 (check!) and ddungeon. The only games that worked was the Double Dragon games and Donkey Kong wasn't even there.

I manually copied the DKong.zip onto my RPi, and the game functioned.

I don't get why the other "games" got copied into my romset, since neither dkong nor the ddragon has any parents...?

AndersHP

@markwkidd said in How do you organise romsets? MAME and (in particular) consoles:

@AndersHP my only experience is on Windows with ClrMamePro

With the correct DAT file ClrMamePro made it easy for me to set a preferred region of 'USA' and only add ROMs from other regions if there was not a USA version. (I know your intended region may be different but the same process should apply)

For example, depending on the specific system and what regions was available in, I could set ClrMamePro to filter my set based on a preference of USA ROMs > then EUR > then JPN.

The resulting set of ROMs will feature a USA version of a game whenever possible, then look for a EUR region ROM for that title, and finally use the JPN only if no USA or EUR ROMs are in the folder. All regions with releases in a given system are supported (in other words you could set a preference of Spain > Europe > USA > Italy > Japan > etc.)

@markwkidd can you be more specific as to how you did this, or perhaps link to a guide?

markwkidd

@AndersHP No-Intro creates and distributes the DAT files for 1G1R processing, and it turns out they have published a brief guide for how to remove the duplicated region ROMs. You might start with that guide: http://datomatic.no-intro.org/stuff/xml_details.txt

AndersHP

@markwkidd WOW, this is awesome, truly grateful for the link!
This totally did the trick on my SNES and MegaDrive roms. I cannot get the NES one to work, though, something about a missing XML file in the ClrMamePro header...?!

Also, what about MAME roms, specifically Finalburnalpha (arcade) and Finalburnalpha (neogeo)?

I cannot find DAT files on the DAT-o-MATIC site for these..?

Dochartaigh

@AndersHP said in How do you organise romsets? MAME and (in particular) consoles:

k on my SNES and MegaDrive roms. I cannot get the NES one to work, though, something about a missing XML file

I was getting all sorts of errors too. I honestly spent a couple hours on it then gave up. Tried following like 4 different tutorials - all sorts of weird errors (which when I google that exact error there's 0 search results found on the entire internet!). I just broke down and manually edited my NES, SNES, and Sega Genesis ROMs.