Share your collections

cyperghost

@thelostsoul said in Share your collections:

I have a lot of collections. Can't post them now, as I am not at home.

@cloudlink
Thanks for this huge collection ;)
That's amazing!

Just for an example your Warner Bros Collection was translated by my script to this. So even you naming convention isn't "correct" I was able to get half of your settings on first run. I think that's okay....

But we have to dig deeper into somehow... Maybe checksums ;) But now we could merge your collection and mine in one file ;) The script needs a bit improvment.

It need a first loop with a check if files can in the whole collection are available and put it in an array with array+=("$romfile" "$available") so if file is available the $available will be setted to 1, else to 0 ;)

/home/pi/RetroPie/roms/gb/Tiny Toon Adventures - Wacky Sports (U) [!].gb
/home/pi/RetroPie/roms/gb/Taz-Mania 2 (U) [!].gb
/home/pi/RetroPie/roms/nes/Bugs Bunny Birthday Blowout, The (E) [!].nes
/home/pi/RetroPie/roms/gbc/Tiny Toon Adventures - Dizzy's Candy Quest (Europe) (En,Fr,De,Es,It,Nl).zip
/home/pi/RetroPie/roms/atari2600/Bugs Bunny (Atari) (Prototype) [!].a26
/home/pi/RetroPie/roms/snes/Speedy Gonzales - Los Gatos Bandidos (U) (V1.1) [!].smc
/home/pi/RetroPie/roms/gbc/Looney Tunes - Twouble! (USA) (En,Fr,Es) (GB Compatible).zip
/home/pi/RetroPie/roms/gbc/Looney Tunes - Carrot Crazy (USA) (En,Fr,Es) (GB Compatible).zip
/home/pi/RetroPie/roms/gbc/Speedy Gonzales - Aztec Adventure (USA, Europe) (GB Compatible).zip
/home/pi/RetroPie/roms/snes/Porky Pig's Haunted Holiday (U) [!].smc
/home/pi/RetroPie/roms/megadrive/Taz-Mania (U) [!].bin
/home/pi/RetroPie/roms/snes/Tiny Toon Adventures - Buster Busts Loose! (E) [!].smc
/home/pi/RetroPie/roms/snes/Daffy Duck - The Marvin Missions (U) [!].smc
/home/pi/RetroPie/roms/snes/Animaniacs (U) [!].smc
/home/pi/RetroPie/roms/gbc/Marvin Strikes Back! (USA) (En,Fr,Es).zip
/home/pi/RetroPie/roms/megadrive/Animaniacs (E).bin
/home/pi/RetroPie/roms/megadrive/Sylvester and Tweety in Cagey Capers (UEJ) [!].bin
/home/pi/RetroPie/roms/snes/ACME Animation Factory (E) [!].smc
/home/pi/RetroPie/roms/gb/Looney Tunes (U) [!].gb
/home/pi/RetroPie/roms/gbc/Daffy Duck - Fowl Play (USA, Europe) (GB Compatible).zip
/home/pi/RetroPie/roms/gb/Animaniacs (U) [S][!].gb
/home/pi/RetroPie/roms/snes/Road Runner's Death Valley Rally (U) [!].smc
/home/pi/RetroPie/roms/snes/Bugs Bunny - Rabbit Rampage (U) [!].smc
/home/pi/RetroPie/roms/megadrive/Tiny Toon Adventures - Buster's Hidden Treasure (E) [!].bin
/home/pi/RetroPie/roms/gb/Daffy Duck - The Marvin Missions (UE) [!].gb
/home/pi/RetroPie/roms/megadrive/Daffy Duck in Hollywood (JUE) [!].bin
/home/pi/RetroPie/roms/snes/Taz-Mania (U) [!].smc
/home/pi/RetroPie/roms/megadrive/Bugs Bunny in Double Trouble (4) [!].bin
/home/pi/RetroPie/roms/gbc/Looney Tunes Collector - Alert! (USA) (En,Fr,Es).zip
/home/pi/RetroPie/roms/gb/Tiny Toon Adventures - Babs' Big Break (E) [!].gb
/home/pi/RetroPie/roms/gb/Speedy Gonzales (U) [!].gb

thelostsoul

@cyperghost This is great!
I didn't test it out yet, but now we are talking. :D First, I want thank you for doing/starting this. I was thinking about this "problem" a while back and didn't know how to do it properly. I was more thinking of a Python script, but Bash is fine too. ;-) Currently I am doing other stuff. I'll test this later and will have a look into the script. From a different script I have, I use this to strip down the clutter from filename:

sed -i s/"(.*$"//g "$file"

which deletes everything after (including) the first brace. I use this to get a clean name of the filename for a quick list. But thats not gonna help for your script, as you use those different parts. What do you think about doing it like this: Strip down on both sides (custom collection and files) everything after brace and compare. Found identical entry, use original filename from drive and add to new collection. So, you don't deal with the clutter. Not sure about this approach, if it would work better. It just sounds more simple. What do you think about it?

Custom Collections with max. of 13 elements per collection:
The entries are like showcases for a specific genre or what they are for. I try to get different type of games for different systems. The systems in use are following: atari2600, mame2003, fba, neogeo, gamegear, gb, gba, gbc, megadrive, sega32x, mastersystem, nes, snes, pcengine, psx, n64. The SNES have two extensions, sfc and smc. PC Engine have two extensions, pce and cue.

Genre:

Other:

Insider Tips:

Custom Collections From Theme, for main front page without a limit in elements per collection:

Just a note here. I did a pre selection of games. That means, I often have only one or two best versions of games, with some exception. In example:

I have Earthworm Jim for SNES, because I grew up with it. Found out, the Mega Drive/Genesis version is better, but excluded it for not having duplicates, even if there are differences. Finally added the Mega CD/Sega CD version of the game, as definitive version.
I have Street Fighter 2 versions (Super, Turbo) for SNES and Arcade. Because I play Console versions with gamepad and Arcade versions with arcade stick. And there are duplicates for some personal reason.
I often have only one version of a game, in example the Arcade versions. In example of Bubble Bobble, later I added the NES version to be able to build the correct NES Mini collection.

cyperghost

@thelostsoul said in Share your collections:

which deletes everything after (including) the first brace. I use this to get a clean name of the filename for a quick list. But thats not gonna help for your script, as you use those different parts. What do you think about doing it like this: Strip down on both sides (custom collection and files) everything after brace and compare. Found identical entry, use original filename from drive and add to new collection. So, you don't deal with the clutter. Not sure about this approach, if it would work better. It just sounds more simple. What do you think about it?

The collection needs to fit exactly with file pathes. So the way how to strip down bash internal, sed, python isn't important. Important is, how the files can be retrieved. The bash script here works in 3 levels:

Just compare file name, is 1 is returned everything is fine if 0 (=no file found) then go to level 2
Compare filename with stripped brackets and original file extension. If one file os found (returns 1) then add the file to collection. If no file is found(0 returned) go to level 3
Compare filename with stripped bracktes and ignore file extension. If one file is found then add to collection, if 0 is returned then make nothing else.

You see fail by design. If 2 entries are found then the script also fails ;) As I said: I can live with that becasue I have no savegames or SRM-states in my ROM folder.

Annother (maybe better) approach would be following.

We use a tool like md5tree and all roms will be scanned for md5 checksum. So we have filename and it's checksum stored in one huge file. We call it md5_romcollection.dat

But this won't save us if someone uses zipped files. But let us be independent of file pathes. I must think about it a bit. I can work with the script I coded ... it covers 75% of my usecase but it's not coded in a good manner and uses unnecessary uses cases and selections.

I would be glad if someone else would step in...

thelostsoul

@cyperghost My idea was using wildcards to get full filename. So strip the clutter path and find file with wildcard. In example "find Bik*" gives me "Biker Mice From Mars (U) [!].smc". Ok, the save files have to be filtered out. Maybe this can be used? This could be used as fallback, if exact filename of different variations aren't found. The md5checksum approach would be good too, but custom collections don't have a md5checksum. And what do you think about stripping down the "-" between Batman and its subtitle and all spaces, plus making everything lower case while comparing?

cyperghost

@thelostsoul That's exactly how it's working
Look: match=$(find "${line%% (*}"*) cuts all charcters till the braket appears and set a wildcard * - that's the searchstring for level 3. So it fits exact your example.

In level 2: match=$(find "${line%% (*}"*."${line##*.}")this cuts all charcater till the braket and set a wildcard * but with extension retrieved from the custom collection.

Your example would be proccesed in my script like following
level 1 - find Biker Mice From Mars (U) [!].smc
level 2 - find Biker Mice From Mars*.smc
level 3 - find Biker Mice From Mars*

I think to filter out all possibles filenames will be a good choice. And the more people share the collections the more the chance to get a full working collection. But as I said I'm satisfied how the script is working. But there is space for lot's of improvements.

See... level 3 is very critical

Because it will also find
Biker Mice From Mars II (U) [!].smc or Biker Mice From Mars (U) [!].smr or Biker Mice From Mars (U) [b].smc or Biker Mice From Mars (J) [!].smc or even worse Biker Mice From Mars (U) [!].autostate

But now I've an idea...... ;)
Why not ask the user for right choice?
So if more then one entry appears open a dialog and ask the user for input.

Annother thing that has to be solved is.... if we process the custom collection.
Should we just add the new found entry? Or should we overwrite the existing one? Because if we just add a new entry we have to check the whole custom collection before we write anything because we will append data on every run. So the overwriting should be more solid! And that's a real easy sed command ;)

thelostsoul

@cyperghost OH man, sorry I was just blind. I only did a brief look into the script. And I forgot about all the other files, so a pure find Bik* would be really a bit greedy. I think asking the user wouldn't be good, its the job of the script to do this. After while with different collections, it could get annoying or the user chose wrong decision, say a noob did it.

Snes roms have mostly 2 different types, .sfc and .smc. Also .zip files are supported too. Maybe we could do it with ls and grep. I did this with a script of mine and it worked.

output="$(ls "$1" | grep -E ".$2")"

Variable $1 is the base directory where all files are. The final output of the command is saved in variable $output. I used this in another script as a function. Sorry for the German comments, it never was intended to release. But now, it helps explaining how I used it.

#!/bin/bash
# Important, this script was intended to be used under Linux / Ubuntu. Not on Raspberry Pi!
base="smb://retropie/roms/"
temp="./games.txt"

cd $base
rm "$temp" 2> /dev/null

# Listet und filtert Verzeichnisse nach Dateityp.
# Argumente: 
#   1 = Verzeichnis
#   2 = Dateityp
function lsfiles ()
{    
    # Führt ls Kommando aus und filtert jede Zeile durch grep.
    # "$()" Konstrukt ermöglicht es als Kommando auszuführen.
    # -E ist nötig, um mehrere Dateitypen zu ermöglichen.
    output="$(ls "$1" | grep -E ".$2")"
    # Speichert die Ausgabe der Variablen in der Datei.
    echo -e "$1\n$output\n" >> "$temp"
}

lsfiles "atari2600" "bin"
lsfiles "mame-libretro" "zip"
lsfiles "fba" "zip"
lsfiles "neogeo" "zip"
lsfiles "gamegear" "gg"
lsfiles "gb" "gb"
lsfiles "gba" "gba"
lsfiles "gbc" "gbc"
lsfiles "megadrive" "md"
lsfiles "sega32x" "32x"
lsfiles "mastersystem" "sms"
lsfiles "nes" "nes"
lsfiles "snes" "sfc|smc"
lsfiles "pcengine" "pce|cue"
lsfiles "psx" "cue"
lsfiles "n64" "z64"

# Pro Zeile, alles nach erstem Klammer löschen.
# -i ist nötig, damit es in der Datei gespeichert wird.
sed -i s/"(.*$"//g "$temp"

Try it manually with:

ls "./" | grep -E ".sfc|smc"

And then we have the filtered list.

cyperghost

@thelostsoul I understand the comments ;) And this will create a list of ROMs without brackets and extension.

But I would stick to find this supports RegEx so wie can include/exclude filetypes ;) and more...

I would also use the level model
I would use sed to directly modify custom collection file rather to add a new entry
Then there are special folder like neogeo/cps/fab/aracade/mame ;) These should be investigated step by step ;)

thelostsoul

Ok, back to plan A. :D

cyperghost

@thelostsoul Definitly use find it outmatches ls easily. I think I can provide a working script for all usecases. But thank you for posting your script. It gives me inspiration.

cyperghost

@thelostsoul I trying to write a script ...
Followup is here Pathfinder for custom collections

cyperghost

Updated main topic ... We can now use filefinder script to use collections from other users with the personal rom-set
Thanks to @thelostsoul for testing and @pjft for developing the custom collections feature for ES ;)

thelostsoul

New stuff for you geeks:

Sega Mega Drive Collection based on newest iteration of Mega Drive/Genesis game collection for upcoming PS4 (and others release).
Source: denofgeek.com
Neo Geo Mini based on the announced Neo Geo Mini, something like the NES and SNES Mini hardware.
Source: wccftech.com

I hope, the lists are complete. But be careful, this is a personal list and so not everything is correct for you. For the Mega Drive collection in example I used Story of Thor, The (Germany) version and the US version have a different title. So the script will not find your game, it should be impossible. I think the US version is named Beyond Oasis. And about Neo Geo games, I don't know how the script will work for you. The games I have are all under neogeo folder, but you could have them in fba or Arcade too. I know some of you use Mame to play Neo Geo games, so it could be under Mame folders too!

cyperghost

@thelostsoul Thx ;)
As small info... I missed 7 only 2 roms out of your MegaDrive Collection with the filefind.sh script ;) (since version 1.10_060918) It was forced to change every file extension from .md to .bin

DEcapAttack = Decap Attack, so no chance as it seems to be a typing error in your ROMslist
Sonic 3D Blast ~ Sonic 3D Flickies' Island (USA, Europe).md ;), I don't have this game, so no chance for the script.

So the result is 49 of 51 ROMs detected
One ROM was added manually (Decap Attack) and so the result is 50 out of 51
The missing ROM isn't in my list.... but I left the entry into the list.

Thx for sharing

thelostsoul

No problem. Your script does well and helps to minimize the manual work. I did a lot of changing regional roms and the only thing I needed to do was using your script. ;-) But one thing, shouldn't the script do the change of Roman numericals automatically?

Did you try the Neo Geo Mini list?

cyperghost

@thelostsoul said in Share your collections:

No problem. Your script does well and helps to minimize the manual work. I did a lot of changing regional roms and the only thing I needed to do was using your script. ;-) But one thing, shouldn't the script do the change of Roman numericals automatically?

No... but I think I will try annother search level that just cuts the last space. Then you might get a big list of selections. But this would help to get all ROMs very quick

Did you try the Neo Geo Mini list?

Yes! 100% hit

jamesnj

I love this thread, thanks.

I've started making some collections, my favorite right now are pinball games.

If you are interested, my pinball and other lists can be gotten from my retropie github repo.

I have about 20 games listed there ... if anyone has some other hidden pinball gems please send them my way.

Thanks!

thelostsoul

@jamesnj Ouh thats cool, as I thought about sharing some pinball games too. Found some interesting ones. I will check your list later and post mine, when I build a new collection.

thelostsoul

@jamesnj Here is the fresh list:

Pinball

Some games not found in your list are: There are multiple titles from the Crush series (first three items MD, SNES and PCE), so you may don't want integrate all, as they get similar.

Dragon's Revenge (MD)
Jaki Crush (SNES)
Alien Crush (PCE)
Time Cruise (PCE)
Pinball - Revenge of the Gator (GB)
Super Pinball Action, rom:spbactn.zip (MAME2003)

jamesnj

Thanks, some good ones there. 1 or two I can't find so I'll have to dig around online. One challenge is seems like these roms have some slight naming differences, but I was able to sort out.

Full disclosure, I added the Super Pinball Action to my list but it is of a mild adult theme .... though the games seem sufficiently difficult enough to make "the prize" near impossible to attain. Or maybe I'm just terrible at video games ;)

cyperghost

@jamesnj I can recommend this tool: Pathfinder for Custom Collections.

One challenge is seems like these roms have some slight naming differences, but I was able to sort out.

No problem! ;)

Thanks for your collection - used your Arcade Classic and this is really a nice one.