Which system emlator should I use? I only want to play "Space Invaders"

dankcushions

@Zigurana said in Which system emlator should I use? I only want to play "Space Invaders":

@dankcushions said in Which system emlator should I use? I only want to play "Space Invaders":

...you mostly need the whole romset on your system to ensure everything will work, as they have dependencies between each other. invaders.zip, however, does not...

Yeah, about that, how do you know what files are needed? Is there some sort of overview of all the inter-dependencies available?
I am guessing this is somehow used in clrmamepro when you want to repack roms (or is it done in romlister?) I've been trying to get a greatly reduced set working correctly, but without much success.

it's summarised in the wiki: https://github.com/RetroPie/RetroPie-Setup/wiki/Managing-ROMs#step-5--rebuild-a-rom-set (parent/clone section)

or you could look in the compatibility lists. anything with something in the parent or bios column requires those zip files present in the directory to run.

ps, we already have a variety of dat files available from which you can build reduced sets, on the wiki linked above.

Riverstorm

I don't know if it helps but sound samples are used in conjunction with games that either don't have emulated sound or used external sound samples via another device like a tape. An example of that would be Journey-Separate Ways (yeah the game/band/song). They are named the same as the ROM. If they are present when the ROM is launched the emulator will use them.

ROMs are a little tricky. They rip the code from the ROM chips (black centipede looking chips) located on the actual arcade games PCB (motherboard sort to speak). So when you see a ROM file named something like 3cb_c1.rom it is code from one of the chips located inside the arcade machine. They basically rip the code from all the chips in a arcade machine to make a ROM set. That ROM set is what makes up a game.

Each game may have multiple versions such as, clones, bootlegs, etc. So every version needs the code ripped from each and every ROM chip. Then comes the issue of organization.

When it comes to organizing ROMs they are setup 3 ways. This is a limitation of the emulator or probably better said it's just how the ROMs need to be organized/archived/presented to the emulator in order to run games.

Merged Set - All the ROMs for a specific game are put in one single archive file. What this basically means is they place all the ROM files for ALL versions and variations of an arcade game into one big zip archive. This allows you to play ANY version of a game. Master, clone, bootleg, etc. You have no dependencies a master archive being present.

This is the best use of space and the fewest files but has a big drawback. When you load an emulator like MAME and want to play a game. It might contain a game like Defender and it will be listed 5 times. So you're looking at this list going why is Defender listed 5 times. Well it might be the master game, 2 clones and 2 bootlegs in that one ROM archive. Hence the name merged set. Most people probably don't care about every variation of a game. It's a bit messy because with multiple variations of potentially 100's of games your game list tends to get long. Do you want every variation possible?

Non-Merged Set - Every version of a game whether it's the master, clone, bootleg, etc. has it's own set of ROM files stored in it's own archive file. So a game like Defender you might see multiple archives named defender.zip, defendg.zip, defendw.zip. This would be 3 different versions of Defender and each runs on its own with no dependency on other ROM archives. You could copy the master (defender.zip) or a clone (defendg.zip) or a bootleg (defendw.zip) to your ROMs folder and run the game. Every ROM file needed for the set is contained in each individual zip archive. Basically each game archive set is a self-contained game. This is nice because you only have the exact version of the game you want and only the ROM files you need in the archive with no extra ROM files and no extra archive files in the folder. Very clean.

But also this makes a HUGE number of zip archive files, is the worst use of space and takes a while to parse with a ROM manager. The reason being is let's say the defender.zip contains a ROM file named defend.1 and this file is needed in 3 variations of the game. It makes a copy of that file and places it in each game archive set. Which wastes lot's of space when building game ROM archives. Hence the name non-merged set. On a side note you may delete the unwanted files after your set is built. I usually use this method, copy the games I want and delete the rest.

The big benefit here is you can copy the exact version of a game you want to your ROMs folder with no extra files. A lot of people might use ClrMamePro to make a non-merged set and only copy the master games.

Split Set - This is the most common way you will find ROM archive sets on websites but harder to understand. It designates one version of a game to be the master. For example defender.zip. Now defender.zip contains all the ROMs needed to run that game.

Now let's say they find a variation of the same game and rip the code from all the chips in that arcade game and call it defendw.zip. They compare the two archives and notice that both archives contain defend.1, defend.2 and defend.3 ROM files. So what they do is remove those duplicate files from the clones, bootlegs, etc. and tell the emulator to look to the master ROM archive because it will have the additional files needed to run the game. In a split-set you have a dependency of the master being in the same folder. The master ROM in a split-set basically contains the all master ROM files but also any common ROM files used among the clones, bootlegs, etc. A big space saver.

This is also a great use of space because no files are duplicated but it's also the hardest to work with. For example let's say you want to play only the defendw.zip version of the game. You might copy over defendw.zip archive and try to play the game but it fails to run complaining about missing ROM files. Why? Well the rest of the ROM files are contained in the master archive. So you copy over defender.zip to your ROMs folder and ding it works! So what's the drawback of this method?

If you only wanted defendw.zip and not the master game you don't get a choice. It needs both archives (master and clone) to work as some of the ROM files needed to make the game run are stored in one archive the rest in the other archive. So when you look at your game list in the emulator you have two versions of Defender even if you only wanted one. The master and a clone.

Well what if you don't want two versions but a nice small list of only the games and variations you remember playing in the arcade. Make a non-merged set and copy only the game versions you want.

If you placed every file from a full merged set OR a full non-merged OR a full split-set into your ROMs folder your list will be the exact same. The only difference is how the zip archive files are organized in the folder. If you have a complete set audited in any of the 3 ways listed above you can re-audit them to any other way listed above because it's the same files just organized in 3 different ways. Once you get this part down it makes managing game ROM archives and setting them up how you want much easier.

So why doesn't everyone build non-merged sets and just grab the games they want? It's time consuming creating a non-merge set and then copying the individual games you want but that granularity is why it's so useful. Where just dumping a whole version set is quick and easy but you have a huge messy list. Unless you're a completionist and want the whole set of games. It's all a matter of preference as all methods work.

I think their are several ROM management programs but ClrMamePro is one of the more popular ones. If you open the DAT file for a specific version of MAME it will list all the ROMs needed for each game and variation. It's a HUGE list. That DAT files tells the ROM management software exactly what ROM files are needed for each game archive. There's no room for error nor variations. Each ROM file in every ROM archive has an exact CRC hash associated with it.

I believe there's programs out there to parse the DAT files and make a list of the ROM files needed for each game archive. I find this useful when I am building one or two games that I might have missed and I am not sure exactly which ROMs are needed to run properly.

You might get lucky and the same ROM file works across several versions of MAME but that's not a guarantee. Every few months a new version of MAME comes out and the files may change due to "no dump" or "bad dump" ROM files. So every few months you have to re-audit your ROM set to keep it current. Each and every version of MAME that is released is very exact on what ROMs are needed. This is important to remember.

As with RetroPie it uses specific versions of MAME such as for example 0.36. Which means every game and every ROM file contained in that game archive has a specific CRC with no room for substitutions or variations.

Also some games used BIOS files for say Neo-Geo, CPS-1, CPS-2, etc. If you're not running any Neo-Geo games for example then you don't need the BIOS ROM files needed for that "family" of games.

I don't know much about Romlister but I thought that's more for searching/organizing like if you wanted only spinner games or trackball games or 4-way or 8-way joystick games. I am sure there's a lot more to it but I usually just use ClrMamePro for ROM building and auditing.

Here's a pic of how the ROM files are stored in archives depending on whether it's merged, non-merged or split-set:

https://www.flickr.com/photos/chadwilson/14197659441/in/dateposted-public

Zigurana

@Riverstorm Wow!
That is a great explanation of the ROM issue! I realize that much of this information was already on the wiki, but your explanation finally made it 'click' for me. (And I've been reading here for quite a while now).

Thanks so much for taking the time to write that all down!

(We should really see how to add this info to the wiki, as it currently focusses mainly on the workflow for clrmamepro, and not on understanding the issue. )

Riverstorm

@Zigurana said in Which system emlator should I use? I only want to play "Space Invaders":

@Riverstorm Wow!
That is a great explanation of the ROM issue! I realize that much of this information was already on the wiki, but your explanation finally made it 'click' for me. (And I've been reading here for quite a while now).

Thanks so much for taking the time to write that all down!

(We should really see how to add this info to the wiki, as it currently focusses mainly on the workflow for clrmamepro, and not on understanding the issue. )

You're welcome. That makes it worth the time. The Wiki is spot on but it reads more like a technical manual. I always understand better when it can be applied like a real scenario/example.

dankcushions

@Riverstorm said in Which system emlator should I use? I only want to play "Space Invaders":

Merged Set - All the ROMs for a specific game are put in one big single archive file. What this basically means is they place all the ROMs for ALL versions and variations of an arcade game into one zip archive. This allows you to play ANY version of a game. Master, clone, bootleg, etc.

This probably the best use of space but has a drawback. When you load an emulator like MAME and want to play a game. It might contain a game like Defender and it will be listed 5 times. So you're looking at this list going why is Defender listed 5 times. Well it might be the master game, 2 clones and 2 bootlegs. Most people probably don't care about the variations but some do. It's a bit messy because with multiple variations of potentially 100's of games your game list tends to get long. Do you want every variation possible?

your game list where? if you use a merged set, your game list in emulationstation should only have one entry for each game (as there is only one zip), so it's much shorter. it's up to the emulator itself to parse that zip for all the various clones/bootlegs/etc and present you with a menu once the emulator is loaded. the problem is, i'm not sure if any arcade emulators shipped with retropie do this (?) so will just load the parent, or maybe nothing at all?

Split Set - This is the most common way you will find ROM archive sets on websites. It designates one version of a game to be the master. For example defender.zip. Now defender.zip contains all the ROMs needed to run the game.

Now let's say they find a variation of the same game and rip the code from all the chips in that arcade game and call it defendw.zip. They compare the two archives and notice that both archives contain defend.1, defend.2 and defend.3. So what they do is remove those duplicate files from the clone, bootleg, etc. and tell the emulator to look to the master ROM archive because it will have the additional files needed to run the game.

This is the best use of space because no files are duplicated

actually merged sets are the smallest by some margin, although i'm not sure why as in my head they're just a different arrangement of the same files in split sets. weird!

Riverstorm

@dankcushions said in Which system emlator should I use? I only want to play "Space Invaders":

It's up to the emulator itself to parse that zip for all the various clones/bootlegs/etc and present you with a menu once the emulator is loaded. the problem is, i'm not sure if any arcade emulators shipped with retropie do this (?) so will just load the parent, or maybe nothing at all?

Hey Dank, it should work that way but I am not sure with RetroPie. I will have to merge a game set and see if it presents a list. Yeah it's the emulator/front ends job.

actually merged sets are the smallest by some margin, although i'm not sure why as in my head they're just a different arrangement of the same files in split sets. weird!

You're right as a merged set contains way fewer files. So it depends on your hard disk sector size and where a file ends can potentially waste a lot more space with more files. If you have a 512k sector and the file uses only 3k of the "next" sector the rest of that sector is wasted space. I believe each file has to start on a sector boundary. The more files you have the more wasted space. That might be the marginal difference you're talking about.

A merged set vs a split set contains way fewer files.

A non-merged set vs a split-set contain the same number of files but the non-merged set file sizes are larger as they are complete games. A split-set moves all common ROM files from the clones, bootlegs, etc. to the master. The split-set is the only type with a dependency on the master ROM archive.

Riverstorm

@dankcushions Ok I did a quick test and it's based off the name. I used Donkey Kong 3 for my test and I merged two versions of the game. Donkey Kong 3 (US) and Donkey Kong 3 (Japan). If it was named dkong3.zip it showed up as Donkey Kong 3 (US) and launched that game fine. If I changed the name to dkong3j.zip it showed up as Donkey Kong 3 (Japan) and launched the game fine too. I think the front-end of the emulator has to be "aware" of that you're using merged sets.

I think the Wiki states it will launch the master/parent in a merge set. Which it will but only because the merge set borrows the master ROM name. Technically you could name a merge set any valid name from the set and it will launch.

herb_fargus

@Riverstorm yeah perhaps we should add a note about rom names with merged sets for the wiki just for those that want to play clones or in the odd case the clone works but the parent doesnt

Riverstorm

@herb_fargus said in Which system emlator should I use? I only want to play "Space Invaders":

@Riverstorm yeah perhaps we should add a note about rom names with merged sets for the wiki just for those that want to play clones or in the odd case the clone works but the parent doesnt

That's a good point I never thought of that. The clone or bootleg on other hardware might work fine but the parent is broke in which case it would require a rename to a working/preferred version. Especially on these older versions of MAME.

dankcushions

@Riverstorm said in Which system emlator should I use? I only want to play "Space Invaders":

@dankcushions Ok I did a quick test and it's based off the name. I used Donkey Kong 3 for my test and I merged two versions of the game. Donkey Kong 3 (US) and Donkey Kong 3 (Japan). If it was named dkong3.zip it showed up as Donkey Kong 3 (US) and launched that game fine. If I changed the name to dkong3j.zip it showed up as Donkey Kong 3 (Japan) and launched the game fine too. I think the front-end of the emulator has to be "aware" of that you're using merged sets.

thanks for the test! but what you mean by "front end"? emulation station figures out the names by the file name and a hard-coded cross reference of 'full' names. it doesn't know anything about the contents or type of file. you could rename outrun.zip to dkong3.zip and get the same result i think :) i'm more curious as to what is actually being loaded by the emulators in this scenario.

Riverstorm

@dankcushions said in Which system emlator should I use? I only want to play "Space Invaders":

@Riverstorm said in Which system emlator should I use? I only want to play "Space Invaders":

@dankcushions Ok I did a quick test and it's based off the name. I used Donkey Kong 3 for my test and I merged two versions of the game. Donkey Kong 3 (US) and Donkey Kong 3 (Japan). If it was named dkong3.zip it showed up as Donkey Kong 3 (US) and launched that game fine. If I changed the name to dkong3j.zip it showed up as Donkey Kong 3 (Japan) and launched the game fine too. I think the front-end of the emulator has to be "aware" of that you're using merged sets.

thanks for the test! but what you mean by "front end"? emulation station figures out the names by the file name and a hard-coded cross reference of 'full' names. it doesn't know anything about the contents or type of file. you could rename outrun.zip to dkong3.zip and get the same result i think :) i'm more curious as to what is actually being loaded by the emulators in this scenario.

Hey Dank, I've been on holiday and I usually don't do much with technology during that time, well, mostly. ;)

I would call ES a front-end. Like Attract Mode, Hyperspin, Maximus Arcade, etc. there's a lot of front-ends for MAME. They just make running games a bit more aesthetically pleasing and usually easier by passing command line parameters and setting different options.

I agree it probably does scan the folder for archives, cross references another list to add the full name in place of the shortened archive name or something similar. Unless it's an unknown archive name then it would probably leave the name as is.

You could rename outrun.zip to dkong3.zip but it will fail immediately complaining of missing ROMs like dk3c.5h and other files from the archive.

The only point I was trying to make above is that you can use a merged archive which could contain all the ROM files needed to run a parent, clone, bootleg, etc. Then name it accordingly and it will run successfully. The front-end/emulator doesn't care about "extra" files it just needs to contain the correct files the run the game as it's named.

So when I dumped both the US and Japan versions of Donkey Kong 3 into one archive it could run either game completely dependent upon the name as you were saying. Once the front-end passes control to the emulator it is looking for specific ROMs within the archive needed to run that game and in this case the archive contains all the files needed to run multiple versions from the parent down to a bootleg.

When I said front-end aware basically it would need to scan the master/parent archive looking for complete ROM sets. If it's missing a single file then it would not list the game as working. I don't know what criteria they use (one file, multiple files, complete set) to generate a game list from a merged set but I suppose it would be something like that.

dankcushions

@Riverstorm the reason i asked what you meant by 'frontend' is that some people mean different things (retroarch, mame tab menu, etc) by it, but luckily my assumption you meant emulation station was correct :)

The only point I was trying to make above is that you can use a merged archive which could contain all the ROM files needed to run a parent, clone, bootleg, etc. Then name it accordingly and it will run successfully. The front-end/emulator doesn't care about "extra" files it just needs to contain the correct files the run the game as it's named.
So when I dumped both the US and Japan versions of Donkey Kong 3 into one archive it could run either game completely dependent upon the name as you were saying. Once the front-end passes control to the emulator it is looking for specific ROMs within the archive needed to run that game and in this case the archive contains all the files needed to run multiple versions from the parent down to a bootleg.

ahh, i understand what you mean :) emulation station is 'dumb' in this regard because it just looks at the .zip name and then cross references it with a list of full games, and displays the title. BUT the underlying mame emulator, when launched, will take the .zip name as the game to load, and attempt to locate the relevant rom files of that game within that .zip, and either succeeds or fails. in this manner it can run all variants of a game in a merged .zip.

so contrary to what i said before, there is no need for underlying mame emulator to present the user with a menu to choose what variant from the merged .zip you are loading - it just loads whatever one the name is for. if you want to load something other than the parent/normal, you just rename :)

Riverstorm

@dankcushions said in Which system emlator should I use? I only want to play "Space Invaders":

so contrary to what i said before, there is no need for underlying mame emulator to present the user with a menu to choose what variant from the merged .zip you are loading - it just loads whatever one the name is for. if you want to load something other than the parent/normal, you just rename :)

Exactly! The one thing I am not quite sure on is who (front-end or emulator) contains the parent/clone relationship information to use a merged set.

I know the current versions of MAME have approximately twice as many clones/variants as it does parents so it could be useful when using a merged set especially for different regions that want to play the game as remembered.

dankcushions

@Riverstorm said in Which system emlator should I use? I only want to play "Space Invaders":

@dankcushions said in Which system emlator should I use? I only want to play "Space Invaders":

so contrary to what i said before, there is no need for underlying mame emulator to present the user with a menu to choose what variant from the merged .zip you are loading - it just loads whatever one the name is for. if you want to load something other than the parent/normal, you just rename :)

Exactly! The one thing I am not quite sure on is who (front-end or emulator) contains the parent/clone relationship information to use a merged set.

emulator i guess. all emulationstation does is pass run a command line which is basically (for a libretro emulator):
/some/dir/retroarch -L /some/dir/libretro-core.so /roms/mame/gamename.zip

so the emulator works out what it is trying to run from the game name, and then looks for the individual files within that. so if you had this game:

game1.zip:
game1.rom
game2.rom
commonfile.rom

if you call it game1.zip, mame would look for game1.rom, and commonfile.rom (and find them), and if you rename it to game2.zip, it would look for game2.rom, and commonfile.rom (and find them). that is basically my understanding of merged sets :)

Riverstorm

@dankcushions said in Which system emlator should I use? I only want to play "Space Invaders":

if you call it game1.zip, mame would look for game1.rom, and commonfile.rom (and find them), and if you rename it to game2.zip, it would look for game2.rom, and commonfile.rom (and find them). that is basically my understanding of merged sets :)

I agree completely but by renaming the archive you're basically using a non-merged or split set parent with extra files.

We know in non-merged and split sets that all archive names exist as an actual folder file entry so it's pretty straight forward.

With merged it's not quite the same. The DAT files are created from the mame.exe file in each version. That means the DAT has all the information on interdependence between the parent and clones for a merged set.

Is MAME aware of merged sets. When I look at the command line I don't see any way to run say dkong3j.zip from dkong3.zip. Dkong3.zip has all the dkong3j.zip ROM files in it's archive but I don't see command line parameters. A hypothetical example: mame.exe dkong3.zip dkong3j.zip where you might specify the parent and then the clone to run. MAME is aware of the children but I don't see the options to work with them.

That's why I was asking is the front-end delving into the archives to verify all the clones and creating lists and archives for clones on the fly which doesn't seem likely. A front-end is more like an extension of the emulator passing the command line parameters by checking boxes here and there in a GUI and sprucing up the looks. The other possibility is the emulator is handling it.

I mean to cross reference a list by hand and rename an archive to the correct clone almost seems silly as they are such short archaic names from the 8.3 filename era and difficult to remember. That's why I keep thinking there must be some way to run any game from a merged set with the correct command line parameters without renaming it every time. Some clones like Gauntlet you might have a dozen versions.

If not then it's just a nice way to save a bit of space. With a big drawback of dropping out to a command prompt to rename your archive when you want to try an alternate version. If that's the case a split set would be more appropriate.

dankcushions

@Riverstorm

Is MAME aware of merged sets. When I look at the command line I don't see any way to run say dkong3j.zip from dkong3.zip. Dkong3.zip has all the dkong3j.zip ROM files in it's archive but I don't see command line parameters. A hypothetical example: mame.exe dkong3.zip dkong3j.zip where you might specify the parent and then the clone to run. MAME is aware of the children but I don't see the options to work with them.
That's why I was asking is the front-end delving into the archives to verify all the clones which doesn't seem likely. A front-end is more like an extension of the emulator passing the command line parameters by checking boxes here and there in a GUI and sprucing up the looks of it. The other possibility is the emulator is handling it.

Currently, no, emulationstation has absolutely no idea what it's running. All the fancy mame names are derived from a simple cross reference, I believe, purely based on the zip name. this is why we get lots of users complaining that ES shows the mame games in its gamelist, but they still don't work. well, es is happy as long as the filename is right.. the actual contents of the files is mame's problem :)

we're dealing in hypotheticals here, really. mame might have ways of handling loading sets via command line (i can't find any), but you also need our forks of mame (advmame, lr-mame*) to pass the same commands, and the frontend/runcommand to provide an interface to let the user choose. right now you're stuck with renaming.

chavatar

@dankcushions

All the fancy mame names are derived from a simple cross reference
Can I ask how this works a little more? Genuinely trying to get my head around it!

Riverstorm

@dankcushions said in Which system emlator should I use? I only want to play "Space Invaders":

we're dealing in hypotheticals here, really. mame might have ways of handling loading sets via command line (i can't find any), but you also need our forks of mame (advmame, lr-mame*) to pass the same commands, and the frontend/runcommand to provide an interface to let the user choose. right now you're stuck with renaming.

Good point but most MAME forks have the same core functionality plus whatever "extra" features they want to add. I would imagine if it works in MAME then all the forks would port the code over too.

I was more curious rather than a wanted feature. I've been using MAME since the first year of it's release and I've never used merged sets except for quick and dirty testing. I basically stick to non-merge or split. They just come with the territory when doing ROM management.

After typing it all out it's really a straight forward question that maybe I can post on a MAME forum to get a better understanding of their purpose and how to use them. Usually they have good creative reasons for what they had in mind or where trying to accomplish, usually! ;)

chavatar

@chavatar PS Ignore the formatting, and don't dumb it down ;)

caver01

I don't know if this helps or hurts the discussion, but for those of us who would rather manage a subset of ROMs on our systems (saves space, cleaner--no clones), I have always just generated non-merged archives as needed.

For example, I typically play Joust (Solid, Red Label). This is, of course, a specific clone. From a split set, you can't play joustr.zip unless you also have joust.zip (parent). But I don't want both of these in my menu. Solution: simply create a non-merged archive by unzipping both and copying the files from joust.zip into joustr.zip. Re-zip. Delete the parent. Done. Now, ES sees only Joust (Solid Red Label) and when launched, MAME finds the parent ROMs inside too.

If you have a merged archive (joust.zip with ALL clone components) You don't have to stop and rename it when you want to play a clone--just make a copy and give each copy the name of the clone/parent that you want to play and they will both be in the ES menu.