Share your collections

cyperghost

@thelostsoul said in Share your collections:

which deletes everything after (including) the first brace. I use this to get a clean name of the filename for a quick list. But thats not gonna help for your script, as you use those different parts. What do you think about doing it like this: Strip down on both sides (custom collection and files) everything after brace and compare. Found identical entry, use original filename from drive and add to new collection. So, you don't deal with the clutter. Not sure about this approach, if it would work better. It just sounds more simple. What do you think about it?

The collection needs to fit exactly with file pathes. So the way how to strip down bash internal, sed, python isn't important. Important is, how the files can be retrieved. The bash script here works in 3 levels:

Just compare file name, is 1 is returned everything is fine if 0 (=no file found) then go to level 2
Compare filename with stripped brackets and original file extension. If one file os found (returns 1) then add the file to collection. If no file is found(0 returned) go to level 3
Compare filename with stripped bracktes and ignore file extension. If one file is found then add to collection, if 0 is returned then make nothing else.

You see fail by design. If 2 entries are found then the script also fails ;) As I said: I can live with that becasue I have no savegames or SRM-states in my ROM folder.

Annother (maybe better) approach would be following.

We use a tool like md5tree and all roms will be scanned for md5 checksum. So we have filename and it's checksum stored in one huge file. We call it md5_romcollection.dat

But this won't save us if someone uses zipped files. But let us be independent of file pathes. I must think about it a bit. I can work with the script I coded ... it covers 75% of my usecase but it's not coded in a good manner and uses unnecessary uses cases and selections.

I would be glad if someone else would step in...

thelostsoul

@cyperghost My idea was using wildcards to get full filename. So strip the clutter path and find file with wildcard. In example "find Bik*" gives me "Biker Mice From Mars (U) [!].smc". Ok, the save files have to be filtered out. Maybe this can be used? This could be used as fallback, if exact filename of different variations aren't found. The md5checksum approach would be good too, but custom collections don't have a md5checksum. And what do you think about stripping down the "-" between Batman and its subtitle and all spaces, plus making everything lower case while comparing?

cyperghost

@thelostsoul That's exactly how it's working
Look: match=$(find "${line%% (*}"*) cuts all charcters till the braket appears and set a wildcard * - that's the searchstring for level 3. So it fits exact your example.

In level 2: match=$(find "${line%% (*}"*."${line##*.}")this cuts all charcater till the braket and set a wildcard * but with extension retrieved from the custom collection.

Your example would be proccesed in my script like following
level 1 - find Biker Mice From Mars (U) [!].smc
level 2 - find Biker Mice From Mars*.smc
level 3 - find Biker Mice From Mars*

I think to filter out all possibles filenames will be a good choice. And the more people share the collections the more the chance to get a full working collection. But as I said I'm satisfied how the script is working. But there is space for lot's of improvements.

See... level 3 is very critical

Because it will also find
Biker Mice From Mars II (U) [!].smc or Biker Mice From Mars (U) [!].smr or Biker Mice From Mars (U) [b].smc or Biker Mice From Mars (J) [!].smc or even worse Biker Mice From Mars (U) [!].autostate

But now I've an idea...... ;)
Why not ask the user for right choice?
So if more then one entry appears open a dialog and ask the user for input.

Annother thing that has to be solved is.... if we process the custom collection.
Should we just add the new found entry? Or should we overwrite the existing one? Because if we just add a new entry we have to check the whole custom collection before we write anything because we will append data on every run. So the overwriting should be more solid! And that's a real easy sed command ;)

thelostsoul

@cyperghost OH man, sorry I was just blind. I only did a brief look into the script. And I forgot about all the other files, so a pure find Bik* would be really a bit greedy. I think asking the user wouldn't be good, its the job of the script to do this. After while with different collections, it could get annoying or the user chose wrong decision, say a noob did it.

Snes roms have mostly 2 different types, .sfc and .smc. Also .zip files are supported too. Maybe we could do it with ls and grep. I did this with a script of mine and it worked.

output="$(ls "$1" | grep -E ".$2")"

Variable $1 is the base directory where all files are. The final output of the command is saved in variable $output. I used this in another script as a function. Sorry for the German comments, it never was intended to release. But now, it helps explaining how I used it.

#!/bin/bash
# Important, this script was intended to be used under Linux / Ubuntu. Not on Raspberry Pi!
base="smb://retropie/roms/"
temp="./games.txt"

cd $base
rm "$temp" 2> /dev/null

# Listet und filtert Verzeichnisse nach Dateityp.
# Argumente: 
#   1 = Verzeichnis
#   2 = Dateityp
function lsfiles ()
{    
    # Führt ls Kommando aus und filtert jede Zeile durch grep.
    # "$()" Konstrukt ermöglicht es als Kommando auszuführen.
    # -E ist nötig, um mehrere Dateitypen zu ermöglichen.
    output="$(ls "$1" | grep -E ".$2")"
    # Speichert die Ausgabe der Variablen in der Datei.
    echo -e "$1\n$output\n" >> "$temp"
}

lsfiles "atari2600" "bin"
lsfiles "mame-libretro" "zip"
lsfiles "fba" "zip"
lsfiles "neogeo" "zip"
lsfiles "gamegear" "gg"
lsfiles "gb" "gb"
lsfiles "gba" "gba"
lsfiles "gbc" "gbc"
lsfiles "megadrive" "md"
lsfiles "sega32x" "32x"
lsfiles "mastersystem" "sms"
lsfiles "nes" "nes"
lsfiles "snes" "sfc|smc"
lsfiles "pcengine" "pce|cue"
lsfiles "psx" "cue"
lsfiles "n64" "z64"

# Pro Zeile, alles nach erstem Klammer löschen.
# -i ist nötig, damit es in der Datei gespeichert wird.
sed -i s/"(.*$"//g "$temp"

Try it manually with:

ls "./" | grep -E ".sfc|smc"

And then we have the filtered list.

cyperghost

@thelostsoul I understand the comments ;) And this will create a list of ROMs without brackets and extension.

But I would stick to find this supports RegEx so wie can include/exclude filetypes ;) and more...

I would also use the level model
I would use sed to directly modify custom collection file rather to add a new entry
Then there are special folder like neogeo/cps/fab/aracade/mame ;) These should be investigated step by step ;)

thelostsoul

Ok, back to plan A. :D

cyperghost

@thelostsoul Definitly use find it outmatches ls easily. I think I can provide a working script for all usecases. But thank you for posting your script. It gives me inspiration.

cyperghost

@thelostsoul I trying to write a script ...
Followup is here Pathfinder for custom collections

cyperghost

Updated main topic ... We can now use filefinder script to use collections from other users with the personal rom-set
Thanks to @thelostsoul for testing and @pjft for developing the custom collections feature for ES ;)

thelostsoul

New stuff for you geeks:

Sega Mega Drive Collection based on newest iteration of Mega Drive/Genesis game collection for upcoming PS4 (and others release).
Source: denofgeek.com
Neo Geo Mini based on the announced Neo Geo Mini, something like the NES and SNES Mini hardware.
Source: wccftech.com

I hope, the lists are complete. But be careful, this is a personal list and so not everything is correct for you. For the Mega Drive collection in example I used Story of Thor, The (Germany) version and the US version have a different title. So the script will not find your game, it should be impossible. I think the US version is named Beyond Oasis. And about Neo Geo games, I don't know how the script will work for you. The games I have are all under neogeo folder, but you could have them in fba or Arcade too. I know some of you use Mame to play Neo Geo games, so it could be under Mame folders too!

cyperghost

@thelostsoul Thx ;)
As small info... I missed 7 only 2 roms out of your MegaDrive Collection with the filefind.sh script ;) (since version 1.10_060918) It was forced to change every file extension from .md to .bin

DEcapAttack = Decap Attack, so no chance as it seems to be a typing error in your ROMslist
Sonic 3D Blast ~ Sonic 3D Flickies' Island (USA, Europe).md ;), I don't have this game, so no chance for the script.

So the result is 49 of 51 ROMs detected
One ROM was added manually (Decap Attack) and so the result is 50 out of 51
The missing ROM isn't in my list.... but I left the entry into the list.

Thx for sharing

thelostsoul

No problem. Your script does well and helps to minimize the manual work. I did a lot of changing regional roms and the only thing I needed to do was using your script. ;-) But one thing, shouldn't the script do the change of Roman numericals automatically?

Did you try the Neo Geo Mini list?

cyperghost

@thelostsoul said in Share your collections:

No problem. Your script does well and helps to minimize the manual work. I did a lot of changing regional roms and the only thing I needed to do was using your script. ;-) But one thing, shouldn't the script do the change of Roman numericals automatically?

No... but I think I will try annother search level that just cuts the last space. Then you might get a big list of selections. But this would help to get all ROMs very quick

Did you try the Neo Geo Mini list?

Yes! 100% hit

jamesnj

I love this thread, thanks.

I've started making some collections, my favorite right now are pinball games.

If you are interested, my pinball and other lists can be gotten from my retropie github repo.

I have about 20 games listed there ... if anyone has some other hidden pinball gems please send them my way.

Thanks!

thelostsoul

@jamesnj Ouh thats cool, as I thought about sharing some pinball games too. Found some interesting ones. I will check your list later and post mine, when I build a new collection.

thelostsoul

@jamesnj Here is the fresh list:

Pinball

Some games not found in your list are: There are multiple titles from the Crush series (first three items MD, SNES and PCE), so you may don't want integrate all, as they get similar.

Dragon's Revenge (MD)
Jaki Crush (SNES)
Alien Crush (PCE)
Time Cruise (PCE)
Pinball - Revenge of the Gator (GB)
Super Pinball Action, rom:spbactn.zip (MAME2003)

jamesnj

Thanks, some good ones there. 1 or two I can't find so I'll have to dig around online. One challenge is seems like these roms have some slight naming differences, but I was able to sort out.

Full disclosure, I added the Super Pinball Action to my list but it is of a mild adult theme .... though the games seem sufficiently difficult enough to make "the prize" near impossible to attain. Or maybe I'm just terrible at video games ;)

cyperghost

@jamesnj I can recommend this tool: Pathfinder for Custom Collections.

One challenge is seems like these roms have some slight naming differences, but I was able to sort out.

No problem! ;)

Thanks for your collection - used your Arcade Classic and this is really a nice one.

MrSkyle

Not really here to share a collection, but rather pose a question:

Is it possible to add entire systems or romsets to a collection instead of adding single games?

I'm trying to create an "Arcade" collection where I would put every MAME, FBA and Neo-Geo game in it. I know there is an "arcade" rom folder but that doesn't allow me to have different copies of the same game (MAME romset version, FBA romset version) in the same folder nor do I want to have to change emulator each time I launch a game to see where it runs better. My idea is simply to have all arcade roms in the same collection, where I know which system it originates from (as they appear listed at the end of the title) and allows me to better compare between versions.

However, doing this adding a single game each time is too daunting. Is there a way to add entire system libraries to a collection?

cyperghost

@mrskyle Of course that is possible. Go to your ROMs dir and just type
ls -d1 $PWD/* > "/opt/retropie/configs/all/emulationstation/collections/custom-YOURCOLLECTION.cfg"

Use > to give the file a one shot entry, doing this again will overwrite the whole file
Use >> to append other entries, with a second ls-command

YOURCOLLECTION presents the name of this. After this restart EmulationStation to make the collection appear and then set it active in Main-Menu > Game Collection Settings