Create a custom ES system able to launch games for many systems

meleu · 19 Jun 2017, 09:09

@TMNTturtlguy about the issue #1, those fancy scripts are clearly lacking the permission to execute. A simple chmod +x scriptname should solve it.

dmmarti · 20 Jun 2017, 17:05

@meleu Here is a 7zip download containing all of the scripts and configuration files for adding 50 new collections into Emulation Station. I've also included a readme file where I tried to document how it all works. Users keep their roms and matching media in the normal folders, everything is based upon symbolic links (for both the roms/media themselves and the specialized gamelist.xml files).

This method adds new emulator folders in /opt, new systems into es_systems.cfg, new /roms folders, new scripts, and new specialized gamelist.xml files for them all.

The only prerequisite is that the scripts expect the rom filenames to be named according to the Hyperspin/EmuMovies naming convention.

But...hopefully you can gleam some usefulness out of it and be able to update/adapt it to make it more generic and global for use. Hope my included readme text file explains it well enough as well as you looking over the shell scripts themselves to make sense. Also, of course, if you add these, you'll need to update whatever theme you use as well so they'll show properly in ES.

I copied them over and zipped them on Windows ... so you might have to chmod 755 on the scripts to get them back to executable on your Pi (not sure if they kept those permissions or not).

https://mega.nz/#!5Zt2ga6b!mgcwbDPW1o18hS1DT_QwL7nHiqEDk7Eg7ZA3RVeOnwk

pjft · 20 Jun 2017, 17:14

@dmmarti thanks for sharing this - great to see the momentum and engagement here.

I plan on making all of these obsolete, but we're likely looking at an August timeframe given that we're at the end of June and I'm still working on the all, last played and favorites auto systems.

Still, great to see the use cases in the interim :)

meleu · 20 Jun 2017, 17:35

@pjft said in Create a custom ES system able to launch games for many systems:

I plan on making all of these obsolete

Do you really mean ALL. Including the "custom systems" like konami, cps2, megaman, shoot'em ups, etc?

TMNTturtlguy · 20 Jun 2017, 17:39

@meleu said in Create a custom ES system able to launch games for many systems:

Do you really mean ALL. Including the "custom systems" like konami, cps2, megaman, shoot'em ups, etc?

You forgot the reason this all started....TMNT! Cowabunga!

pjft · 20 Jun 2017, 17:42

@meleu I don't mean making the systems obsolete, but the way we create them. I envision this all being done within EmulationStation, but it's still a ways off.

UDb23 · 20 Jun 2017, 21:54

Shoot'em Up SVG logo ready !

Available here.

TMNTturtlguy · 17 Jul 2017, 14:21

@meleu Is there anyway that we can point the custom folders with the symlinks to use a custom launching image? right now the symlinks in the script point the linked roms to use the launching image associated with the /opt/retropie/configs/folder for the system the rom originated from. So if mega man was from the nes folder, it would use the nes launching screen found in /opt/retropie/configs/nes

Is there a way to set it so that all of the sym links in the megaman folder use a launching image specific for just megaman? can the custom runcomand script in the es_systems.cfg for the custom systems have a line to point to the symlink folder for the launching image? Thanks

meleu · 17 Jul 2017, 14:42

@tmntturtlguy said in Create a custom ES system able to launch games for many systems:

@meleu Is there anyway that we can point the custom folders with the symlinks to use a custom launching image?

We can think in a workaround for it, but currently not.

Maybe it's better to use the game specific launching image feature.

More info in the docs: https://retropie.org.uk/docs/Runcommand/#adding-custom-launching-images

TMNTturtlguy · 17 Jul 2017, 16:05

@meleu Thanks Man,

I thought about the game specific but then it would show the mega man launch image if I launch it from the NES system....I guess it isn't the end of the world, but I would like to get it so that mega man.zip shows the mega man lunch when launched from the mega man system and mega man.zip has the nes launch when launched from the nes system.

This is not a bid deal at all, so no worries. Thanks

namyzark · 9 Aug 2017, 22:22

@TMNTturtlguy & @meleu

I love this. I've tried following along with this due to the design of what my wife wants, but it isn't working.

Let me describe the end goal: the wife says there are too many game systems. CPS1, CPS2, CPS3, Capcom, FBA, Galaxian, GameGear, GBC, GBA, NES, SNES, Genesis, etc etc etc

She wants a 'couple' main system directories. Like Nintendo, Arcade, & Sega. In each of these 'main system' groups there will be sub-systems. Nintendo would have GBC, GBA, NES, & SNES.

Thus far I've managed to create a new main system. The sub folders are just that, sub folders. They don't launch correctly (incorrect emulator is my best guess). Anything either of you can do to help and get me on the correct path to fix the issue I am having?

Thank you in advance.

meleu · 13 Aug 2017, 01:20

@namyzark I suggest you to use the @TMNTturtlguy 's trick here: https://retropie.org.uk/forum/topic/11499/change-es_systems-cfg-script

meleu · 13 Aug 2017, 01:37

@hansolo77 and all RetroAchievements users, if you to create a custom system for RetroAchievements, the hascheevos tool can help you in this task. Here are the steps:

Create an achievements custom system as described in the OP.
Go to the hascheevos repo and follow the instructions to install it.
Use the trick to create a list of all ROMs that have cheevos in a directory. Here is an example for Mega Drive games:

./hascheevos.sh -u USER -p PASSWORD ~/RetroPie/roms/megadrive/* > ~/megadrive-roms-with-cheevos.txt

Get the add-game-to-custom-system.sh script (I talked about it in a post above):

wget https://raw.githubusercontent.com/meleu/share/master/add-game-to-custom-system.sh
chmod a+x add-game-to-custom-system.sh

Give the created list's content as arguments for the add-game-to-custom-system.sh:

cat ~/megadrive-cheevos.txt | xargs -d '\n' ./add-game-to-custom-system.sh -d ~/RetroPie/roms/achievements

[optional] Repeat the steps 3 and 4 for other supported systems.
Done!

Meanwhile we keep trying to make it less laborious (thanks to the big contributions from @pjft ).

hansolo77 · 13 Aug 2017, 02:09

EEEEEE!!!!!! Something to test! I'm now officially on vacation (from work, but staying home), so I have all week to play around with this.

hansolo77 · 13 Aug 2017, 23:32

Ugh migraines. Trying to figure out the PowerBlock problem I'm having and then to tackle this is just too much. I'm on vacation for crying out loud. Why'd I have to go and have a migraine? So if I understand correctly, I have to first create the "folder" achievements, then run the scrape tool to get a text list of my games. then use ANOTHER tool to convert those lists into symbolic links, and then go back and run yet another tool to add in the missing metadata? Seems like a lot of manual labor. Can't wait for some automation. :) Might have to try this tomorrow hen my head feels better.

namyzark · 14 Aug 2017, 00:23

@meleu said in Create a custom ES system able to launch games for many systems:

https://retropie.org.uk/forum/topic/11499/change-es_systems-cfg-script

This was a great starting point @meleu. Thank you so much. Now to create custom folders for each user with only their games!!!

mattrixk · 15 Aug 2017, 00:04

@hansolo77 said in Create a custom ES system able to launch games for many systems:

I'm now officially on vacation (from work, but staying home)

The best kind of vacation.

hansolo77 · 15 Aug 2017, 00:24

@meleu - I'm getting hung up at step 8 in the OP:

create the /home/pi/bin/runcustom.sh script (the code is pretty small):

The problem comes when I try to save the file because the /bin part of the path doesn't exist. Does this need to be in that path, or can I put it somewhere else and just make sure the es_systems knows where it is? It looks like it should reside with the runcommand.sh file, or maybe with the other ones (runcommand-onstart.sh and runcommand-onend.sh) located at /opt/retropie/configs/all.

Also, are you sure that ~/.emulationstation/es_systems.cfg is the correct location for that file? When I edited, that file didn't exit. On my system, that file is located at /etc/emulationstation/es_systems.cfg. Are you sure ES will pick it up?

EDIT - Decided to write the script in /home/pi/RetroPi and changed the es_systems.cfg to match. I then installed the hascheevos script. It's scraping MegaDrive right now. Looks like it's gonna take some time, but that's ok. The text file it creates is viewable through SFTP, so I checked it and it is grabbing full path names like it should! I wonder if a future version of this could maybe include some intelligence.. Have it scrape the entire /home/pi/RetroPie/roms path recursively and be like:

Skipping Atari 2600, no cheevos exist
Skipping Atari 5200, no cheevos exist
...etc

Then when it gets to a system it knows to have cheevos, it can go through and scan that folder. And then in the end, have it write out the text file including all systems rather that a separate file for each system.

EDIT - Another feature request.. maybe have a way to have it log the WARNING and ERROR games as well. If I have a game that has cheevos, but the script doesn't match the hash, I assume it's not going to work in RetroArch either. This is a bummer to me. I'd like to know what games need replaced with ones that DO work.

Also, perhaps another feature, or maybe part of the same. Have it list MISSING games. I say this because I know there are a lot of games that are HACKS that you simply don't get in a full ROM pack. To know there is a game available you don't have would be nice to see so you can go out and get it. It's related to the other request, so maybe just combining the two would help?

---> I think this belongs to a different thread that doesn't exit. Perhaps something like "HasCheevos Support" in the Ideas/Development?

EDIT - After the system was scraped, I installed the script to make symlinks to all the files in the /achievements directory. It was successful. Then I tried to do step #9 in the OP, and got error:

pi@retropie:~ $ ./custom-system-autoscraper.sh ~/RetroPie/roms/achievements/
ERROR: you must give a directory as argument.
Usage:
./custom-system-autoscraper.sh /path/to/custom/system/directory

I got this with /home/pi/RetroPie/roms/achievements/ as well... So I decided, meh, no metadata for now. Lets test it. Restarted ES.. and ES craps out. Says my systems file is missing or corrupt, and I'll have to manually recreate it. Then ES crashes back to terminal displaying:

lvl0:     es_systems.cfg is missing the <systemList> tag!
lvl0:     Error while parsing systems configuration file!

Wonderful. :( I suspect it's because of the custom system I added to that random folder somewhere... gonna test moving it into the actual existing es_systems.cfg file that works.

EDIT - More success, more set backs. I erased the file ~/.emulationstation/es_systems.cfg and added the contents of that to /etc/emulationstation/es_systems.cfg. That worked. When I rebooted, there is now an "achievements" system on my home screen. However, there is no working artwork for it (yet) so the screen is all messed up. No biggie for now. I tried to load up a game, and all it does is zoom in, go to black, and zoom back out. My SSH terminal shows this:

sh: 1: /home/pi/runcustom.sh: Permission denied
INFO: Failed to set keyboard mode

~~Do I need to set ownership or chmod or something to that file?~~ Bingo! Did a quick and dirty chmod a+x runcustom.sh (saw you did that in another step) and it worked! I was able to get the game to load up just fine. :) Now I just need to get a proper background and system svg from @TMNTturtlguy and then figure out why your script for metadata doesn't work. I also need to scrap the other systems too. BTW.. I tried that whole export Path blah blah stuff and it didn't do anything.

meleu · 15 Aug 2017, 06:25

@hansolo77 hey dude, you are being a bit vindictive, aren't you? :-) I confuse you with all those scripts and commands and you confuse me with all those questions and edits in a same post. :D

Let me try:

when I say "create the /home/pi/bin/runcustom.sh script", the creation of the bin directory if it doesn't exist is implicit in that sentence.
"Also, are you sure that ~/.emulationstation/es_systems.cfg is the correct location for that file?"
Yes, I am. If you edit /etc/emulationstation/es_systems.cfg any change you made will be lost after an EmulationStation update.
the feature requests for the hascheevos tool can be made on this thread: https://retropie.org.uk/forum/topic/11859/what-about-adding-a-cheevos-flag-in-gamelist-xml
The error you're having ERROR: you must give a directory as argument. is because you are using the wrong script (sorry, I need to make it more clear in the OP). I've created a better script, which takes the path for a file and automatically creates the symbolic links and copy the original metadata. I talked about it in this post
I didn't understand this part of your post "I tried that whole export Path blah blah stuff and it didn't do anything".

hansolo77 · 15 Aug 2017, 06:43

LOL I'm going to bed now before my head explodes.

1 - Does it NEED to be in that folder at all though? Is it safe where I have it in /home/pi/?

2 - Does ES look in the ~/.emulationstation/es_systems.cfgfirst? If so, I'll just make a copy of the file to that location and edit from there. The error I got about that was because it was missing all the other systems, and thus the extra parts at the beginning.

3 - When I get up in the morning, I'll transfer over my requests for you there. :)

4 - I didn't really read through your thread here. Only to the point where you tagged my name and then the OP. So I will check back and see if you have a clearer update to the correct script.

5 - https://github.com/meleu/hascheevos#create-a-list-of-all-roms-that-have-cheevos-in-a-directory (step #4)

# adapt the path below to your setup!
export PATH="$PATH:/path/to/hascheevos/bin"

I tried to map it to /home/pi/hascheevos/bin, /home/pi/hascheevos/bin/hascheevos.sh, ~/hascheevos/bin/ etc.. none of them work. I read up on the export command. It's for making that path appear as though I'm always in that path, so I can run the command without having to type out the entire path to it. Well.. I can verify with the env command that it did add those paths, but they don't work. When I try to run the script locally without being in that folder, it just tells me it's not a valid file or something.

Anyway, yeah.. it's almost 3am here. I haven't been up this late in years lol. I need to rest my noggin. :)