Script Request: Sort Roms by Folder After Using Datfile

[Used2BeRX]

Hey everybody,

When I wrap everything up for the NES, I'm going to be offering the full Xtras collection somewhere-undecided with datfiles (no roms).

I was hoping somebody could help me with a script that would make this process easier. Well... two scripts actually. I was hoping for one for Windows and another for people who prefer Linux. I think with what I can provide that this would be too hard to do for somebody with some basic coding knowledge.

As it stands now, you can only create a single folder for a datfile. This is a problem for the Xtras that are divided in many different folders based off of category. Not only would it require that I create around 25-30 datfiles for the NES collection, but even worse it would require the end user to actually have to run 30 datfiles against their rom collection and make sure their roms were exactly where they are intended to be, which would be a nightmare.

I'm hoping we could have a script that would separate roms into their intended folders after they've been run in either Romcenter or clrmamepro. This would make creating a perfect Romset that works with all of the included media a two-step process instead of a 60 step process.

For the NES collection I've been working on what seems like forever, here's the folders we're working with:

(1) Licensed
(1) Licensed\(1_1) US Licensed
(1) Licensed\(1_2) US Unlicensed ------ [Tengen Games, which are technically unlicensed, as well as Color Dreams, etc.]
(1) Licensed\(1_3_1) NES Zapper Games
(1) Licensed\(1_3_2) NES Power Pad Games
(1) Licensed\(1_4) Europe
(1) Licensed\(1_5) Japan
(1) Licensed\(1_6) Australia
(1) Licensed\(1_7) VS Games
(2) Translated
(3) Unlicensed - Homebrew
(3) Unlicensed - Homebrew\(3_01) Unlicensed -----[Foreign unlicensed games like Sachen games]
(3) Unlicensed - Homebrew\(3_02) Prototypes
(3) Unlicensed - Homebrew\(3_03) Pirate Carts
(3) Unlicensed - Homebrew\(3_04) Pirate - Fighting Games
(3) Unlicensed - Homebrew\(3_05) Original Homebrew
(3) Unlicensed - Homebrew\(3_06) Castlevania Updates
(3) Unlicensed - Homebrew\(3_07) Final Fantasy Updates
(3) Unlicensed - Homebrew\(3_08) Mega Man Updates
(3) Unlicensed - Homebrew\(3_09) Metroid Updates
(3) Unlicensed - Homebrew\(3_10) Super Mario Bros Updates
(3) Unlicensed - Homebrew\(3_11) Zelda Updates
(3) Unlicensed - Homebrew\(3_12) Other Updates
(4) Famicom Disk System
(4) Famicom Disk System\(4_1) Translated
(4) Famicom Disk System\(4_2) Untranslated
(4) Famicom Disk System\(4_3) Update Hacks
(5) Adult Games
(6) Non-Working
(7) Needs Alternate Controller

NOTE: Rom location isn't Critical when it comes to the XBox. It's nice to have the roms separated by category instead of having over 2,000 of them of every flavor in the same folder, but they would work with all of the Xtras just fine that way.

The problem comes with the nature of the RetroPie, since it relies on a pre-made "gamelist.xml" file that loads up when the pi is turned on and tells the system exactly where all the media for a given rom is located. If a single rom is not located in the exact spot on the Pi that the gamelist points to than none of the Xtras media will work for it. You can't even have the case wrong with linux. It must be perfect. (Meleu's script for handling a lot of the gamelist creation is amazing and literally saves me hundreds to thousands of hours).

I'm slowly working on creating lists for all of the roms that are to be in specific folders as work progresses.

Does anybody here think that this could be done with this information? Is there anything else that I would need to provide?

Thanks

EDIT: Currently I'm updating everything to work with GoodSETS as much as possible. There will be some games that can't be found that way, such as newer translations and hacks and such, but all of the official games, and most of the prototypes, pirates, unlicensed and such should all be hits by the time I create the datfiles.

Some of the translations and hacks that aren't hits are actually in GoodSETs, but I re-patched every game by hand to the exact specs the authors noted and sometimes this resulted in a different CRC in the end. The Good versions should probably work, but I wanted to make sure I was working with the exact roms that the authors used.

A spreadsheet noting all of the work being done is forthcoming. I still have some more stuff I want to finish before I make it public though.

[Used2BeRX]

So I am nearly done with the NES GoodSET integration and re-testing (on the XBox). I still have to test everything on the Pi, but I'm assuming anything that works on a 10+ year old emulator won't have any problems on the Pi.

I've been running the sets now against the latest No-Intro dat and so far all of the official releases are a 100% match there with the GoodNES 3.23b set, which is great news. Anybody who wanted to get a huge head start on hacks, translations and some more obscure games would want to get the GoodNES set, but anybody who was only interested in the offical releases would be fine with just the No-Intro stuff.

All of this has been documented on the spreadsheet. It's pretty cool seeing all 3 sets together to compare the different versions. As far as I know there's never been anything out there like this. I've read quite a bit about how GoodSETS are "outdated", but I can say for a fact right now that this is not true, at least for the NES. It might be extremely bloated at around 3GB, but the roms verified as [!] are truly "Good", and are a 100% match to the very latest No-Intro datfile however many years later.

The only real difference is that the No-Intro team completely stripped out the iNES headers from all of the games, so only newer emulators that can play a game without them will work with that set. In order to verify my set against no-intro, I had to download a "P/Clone XML" version of the NES database that has something coded into it to ignore the header when checking the validity of your roms.

More to come. Hopefully somebody will see this spreadsheet when I release it and have an idea about how to create these scripts.