Create a custom ES system able to launch games for many systems

mattrixk

@hansolo77 said in Create a custom ES system able to launch games for many systems:

I'm now officially on vacation (from work, but staying home)

The best kind of vacation.

hansolo77

@meleu - I'm getting hung up at step 8 in the OP:

create the /home/pi/bin/runcustom.sh script (the code is pretty small):

The problem comes when I try to save the file because the /bin part of the path doesn't exist. Does this need to be in that path, or can I put it somewhere else and just make sure the es_systems knows where it is? It looks like it should reside with the runcommand.sh file, or maybe with the other ones (runcommand-onstart.sh and runcommand-onend.sh) located at /opt/retropie/configs/all.

Also, are you sure that ~/.emulationstation/es_systems.cfg is the correct location for that file? When I edited, that file didn't exit. On my system, that file is located at /etc/emulationstation/es_systems.cfg. Are you sure ES will pick it up?

EDIT - Decided to write the script in /home/pi/RetroPi and changed the es_systems.cfg to match. I then installed the hascheevos script. It's scraping MegaDrive right now. Looks like it's gonna take some time, but that's ok. The text file it creates is viewable through SFTP, so I checked it and it is grabbing full path names like it should! I wonder if a future version of this could maybe include some intelligence.. Have it scrape the entire /home/pi/RetroPie/roms path recursively and be like:

Skipping Atari 2600, no cheevos exist
Skipping Atari 5200, no cheevos exist
...etc

Then when it gets to a system it knows to have cheevos, it can go through and scan that folder. And then in the end, have it write out the text file including all systems rather that a separate file for each system.

EDIT - Another feature request.. maybe have a way to have it log the WARNING and ERROR games as well. If I have a game that has cheevos, but the script doesn't match the hash, I assume it's not going to work in RetroArch either. This is a bummer to me. I'd like to know what games need replaced with ones that DO work.

Also, perhaps another feature, or maybe part of the same. Have it list MISSING games. I say this because I know there are a lot of games that are HACKS that you simply don't get in a full ROM pack. To know there is a game available you don't have would be nice to see so you can go out and get it. It's related to the other request, so maybe just combining the two would help?

---> I think this belongs to a different thread that doesn't exit. Perhaps something like "HasCheevos Support" in the Ideas/Development?

EDIT - After the system was scraped, I installed the script to make symlinks to all the files in the /achievements directory. It was successful. Then I tried to do step #9 in the OP, and got error:

pi@retropie:~ $ ./custom-system-autoscraper.sh ~/RetroPie/roms/achievements/
ERROR: you must give a directory as argument.
Usage:
./custom-system-autoscraper.sh /path/to/custom/system/directory

I got this with /home/pi/RetroPie/roms/achievements/ as well... So I decided, meh, no metadata for now. Lets test it. Restarted ES.. and ES craps out. Says my systems file is missing or corrupt, and I'll have to manually recreate it. Then ES crashes back to terminal displaying:

lvl0:     es_systems.cfg is missing the <systemList> tag!
lvl0:     Error while parsing systems configuration file!

Wonderful. :( I suspect it's because of the custom system I added to that random folder somewhere... gonna test moving it into the actual existing es_systems.cfg file that works.

EDIT - More success, more set backs. I erased the file ~/.emulationstation/es_systems.cfg and added the contents of that to /etc/emulationstation/es_systems.cfg. That worked. When I rebooted, there is now an "achievements" system on my home screen. However, there is no working artwork for it (yet) so the screen is all messed up. No biggie for now. I tried to load up a game, and all it does is zoom in, go to black, and zoom back out. My SSH terminal shows this:

sh: 1: /home/pi/runcustom.sh: Permission denied
INFO: Failed to set keyboard mode

~~Do I need to set ownership or chmod or something to that file?~~ Bingo! Did a quick and dirty chmod a+x runcustom.sh (saw you did that in another step) and it worked! I was able to get the game to load up just fine. :) Now I just need to get a proper background and system svg from @TMNTturtlguy and then figure out why your script for metadata doesn't work. I also need to scrap the other systems too. BTW.. I tried that whole export Path blah blah stuff and it didn't do anything.

meleu

@hansolo77 hey dude, you are being a bit vindictive, aren't you? :-) I confuse you with all those scripts and commands and you confuse me with all those questions and edits in a same post. :D

Let me try:

when I say "create the /home/pi/bin/runcustom.sh script", the creation of the bin directory if it doesn't exist is implicit in that sentence.
"Also, are you sure that ~/.emulationstation/es_systems.cfg is the correct location for that file?"
Yes, I am. If you edit /etc/emulationstation/es_systems.cfg any change you made will be lost after an EmulationStation update.
the feature requests for the hascheevos tool can be made on this thread: https://retropie.org.uk/forum/topic/11859/what-about-adding-a-cheevos-flag-in-gamelist-xml
The error you're having ERROR: you must give a directory as argument. is because you are using the wrong script (sorry, I need to make it more clear in the OP). I've created a better script, which takes the path for a file and automatically creates the symbolic links and copy the original metadata. I talked about it in this post
I didn't understand this part of your post "I tried that whole export Path blah blah stuff and it didn't do anything".

hansolo77

LOL I'm going to bed now before my head explodes.

1 - Does it NEED to be in that folder at all though? Is it safe where I have it in /home/pi/?

2 - Does ES look in the ~/.emulationstation/es_systems.cfgfirst? If so, I'll just make a copy of the file to that location and edit from there. The error I got about that was because it was missing all the other systems, and thus the extra parts at the beginning.

3 - When I get up in the morning, I'll transfer over my requests for you there. :)

4 - I didn't really read through your thread here. Only to the point where you tagged my name and then the OP. So I will check back and see if you have a clearer update to the correct script.

5 - https://github.com/meleu/hascheevos#create-a-list-of-all-roms-that-have-cheevos-in-a-directory (step #4)

# adapt the path below to your setup!
export PATH="$PATH:/path/to/hascheevos/bin"

I tried to map it to /home/pi/hascheevos/bin, /home/pi/hascheevos/bin/hascheevos.sh, ~/hascheevos/bin/ etc.. none of them work. I read up on the export command. It's for making that path appear as though I'm always in that path, so I can run the command without having to type out the entire path to it. Well.. I can verify with the env command that it did add those paths, but they don't work. When I try to run the script locally without being in that folder, it just tells me it's not a valid file or something.

Anyway, yeah.. it's almost 3am here. I haven't been up this late in years lol. I need to rest my noggin. :)

meleu

@hansolo77 said in Create a custom ES system able to launch games for many systems:

1 - Does it NEED to be in that folder at all though? Is it safe where I have it in /home/pi/?

No. Yes, just change es_systems.cfg accordingly.

2 - Does ES look in the ~/.emulationstation/es_systems.cfgfirst?

Yes.

5 - https://github.com/meleu/hascheevos#create-a-list-of-all-roms-that-have-cheevos-in-a-directory (step #4)
# adapt the path below to your setup!
export PATH="$PATH:/path/to/hascheevos/bin"
I tried to map it to /home/pi/hascheevos/bin, /home/pi/hascheevos/bin/hascheevos.sh, ~/hascheevos/bin/ etc.. none of them work.

Oh, I got it. I would need to know what exactly you typed to help with this. But this is subject for that other thread. Feel free to ask there, but please give more details of what you typed and error you got.

Have a good night Han!

cyperghost

@meleu
Just logged in for small comment. You are awesome! I am still in holiday and enjoying it.

meleu

@cyperghost thanks for the kind words bro. Enjoy your time!

hansolo77

Yesterday I realized I had a lot of my modded roms (that I got for cheevos) only in my "Achievements" sub folders. So I moved those back into their root system folders with the rest of their rom brothers. That then prompted me to have to re-scrape all my roms so that when I migrate them over to the new custom system, they'll all still have proper metadata. Took me all day yesterday, then I realized I didn't have the naming scheme set right (the last scrape I did was for MAME so I had changed it). So today I'm re-scraping again, but this time I made sure to have the name scheme set right! Oh well, I'll get there eventually.

meleu

@hansolo77 what do you mean by "naming scheme set right"? What would be the right naming scheme?

hansolo77

@meleu I was using USX and had it configured to provide just the Game Name in the gamelist.xml. Since I'm using a no-intro romset, the resulting gamelist had a bunch of duplicate game names. This is a bother to me, because if there are multiple versions of games, I want to know which one I'm loading up. So now I'm scraping using the scheme "filename", so it will provide me with the extra bits of info at the end (like USA, Japan, v1.0, v1.2, etc.

TMNTturtlguy

@hansolo77 why did you have to rescrape? Just because you moved the roms to a different folder? You could just use notepad++ and do a replace all to all the files in the folder. Takes about 10 seconds.

hansolo77

There's more to it than that though. All my achievement artwork was saved with the folder name as part of the filename. So they all looked like _ACHIEVEMENTS_Sonic 1 (etc).mp4. It's all good. I'm in the process of making the symlinks now. Fingers crossed!

I also create a system logo for Achievements. I was going to upload it the ComicBook theme page, but everything says the file is corrupted or something. I even tried to copy it back over from the Pi and still no good. If it's corrupted, it shouldn't work with the Pi either right? I dunno. It works, that's all that matters.

hansolo77

I think I got it working! Took forever adding the last couple of systems, probably due to memory or something. All told, I now have a custom system named "Achievements", and there are currently 1426 games to choose from! :) I need to do some further checking and testing, but so far all the games look like they have proper metadata installed.

Next on my list - figure out how to get the ComicBook loading screens to install.

Thanks @meleu !!!

meleu

@hansolo77 said in Create a custom ES system able to launch games for many systems:

figure out how to get the ComicBook loading screens to install.

Didn't rpie-art work for you?

hansolo77

I never progressed any further with it because I can't get it to load up in the Retropie menu. I'm tackling this today.

namyzark

@meleu said in Create a custom ES system able to launch games for many systems:

https://retropie.org.uk/forum/topic/11499/change-es_systems-cfg-script

Wonderful! I have combined @TMNTturtlguy 's change ES system script with @meleu 's 'Custom ES System able to launch games for many systems. I have a very happy family right now.

Next: to see if I can combine symbolic links with actual games files so that multiple people in my house can play the same game without overwriting each others save games/save states.

pjft

Hi all,

The ability to create custom collections (effectively, custom systems) from within ES is now available.

Best.

TMNTturtlguy

@meleu said in Create a custom ES system able to launch games for many systems:

@tmntturtlguy said in Create a custom ES system able to launch games for many systems:

@meleu Is there anyway that we can point the custom folders with the symlinks to use a custom launching image?

We can think in a workaround for it, but currently not.

Maybe it's better to use the game specific launching image feature.

More info in the docs: https://retropie.org.uk/docs/Runcommand/#adding-custom-launching-images

So I know most users are now using the neat and easy custom collections in ES, but for anyone still using this symlink method, i have figured out how to do what I had originally asked in an early post, quoted above. Now when I launch a rom from my TMNT collection my TMNT launch screen appears based on the theme. When I launch a Mega Man game from my Mega Man collection, a Mega Man launch screen appears. If i launch the same game from the NES system, the NES launch screen appears! Here is how to do it, if anyone is interested:

Go to /opt/retropie/configs/all/emulationstation
- Create a folder called “Launching”
  - Place folders for each system within this folder, with the launching image inside that folder
  - Example, place TMNT folder here with “Launching.png”
Create a folder called “runcommand”
- Place a custom runcommand.sh file for each custom system here named with the custom system at the end:
  - Example: runcommonandtmnt.sh
- Edit runcommand starting at line 915

# look for custom launching images
 if [[ "$IS_SYS" -eq 1 ]]; then
images+=(
"$HOME/.emulationstation/Launching/TMNT/launching"
)
Fi

Go to /home/pi/bin/
- Add custom runcommand for each system
  - Example: runcustomtmnt.sh
- Edit the last line to launch the custom runcommand from above
  - Example:
    - /opt/retropie/configs/all/emulationstation/runcommand/runcommandtmnt.sh 0 _SYS_ "$system" "$rom"
In putty, go to both new runcommand files and type the following:
- chmod +x nameofruncommand.sh
Go to your es_systems.cfg
- Modify your runcommand for each system to the new runcustom.sh you created. Example: runcustomtmnt.sh

So anyone crazy enough to have read through this without just leaving the post might want to know why I am still running the custom systems through the script and not using the newest updates to ES. There are a few reasons:

I already went through all the work to set these up about 2 months prior to this being available in ES.
Using this symlink script I am able to have a gamelist for each custom system. I can then modify my images and marquees to be different than in the standard system list.
I now have the ability to have the custom launching images for each custom system.
Bottom line, it is a lot more work for the same results, but I have a lot more flexibility to set them up exactly how I want them.

cyperghost

@tmntturtlguy That's very complicate! And we have the ES collections as @pjft mentioned. But of course that scripting provides much more flexibility ... You are so a kind of person... here we go!

You forget... there are other "entrypoints" as only the runcommand.sh as it's the "brain" of our RetroPie I would leave it untouched.

I would do following:
You have already a bash-file that provides $rom (that's the runcustom.sh)
So you can identify the system by filepath...

Best method is to compare es_system.cfg rom filepathes with extraced one = 100% working methof
Set the filepathes on your own script so /home/pi/roms/nes could be NES system

Next step...

As an example...
You identified it as SNES system
Then you extract the original launch-image for SNES system and temporarily move it somewhere and copy launch image of a custom system...

To genereate a generel script you can use command line parametes like -megaman or -tmnt ....

So for your MEGANMAN "system" in es_systems.cfg you set

<command>/home/pi/bin/runcustom.sh %ROM% -megaman</command>

and then create "usecases" for the parameters "-megaman" "-tmnt" "-mario".... within @meleu runcustom.sh script!

Last line reverts all back to normal!

TMNTturtlguy

@cyperghost Thanks as always for your kind and quick response to help me out!

I have a few questions for you as I am not fully understanding your instructions:

In regards to the runcomand.sh file - Yes i understand it is the brain of running the roms. I have left the runcommand.sh in its original location and it is untouched. It still runs all the roms for the normal systems. The modified runcommands are moved to the .emulationstation folder, thus they will not be orverwritten, and I can simply delete them change my es_systems.cfg back to the original path for my custom systems. In theory, if anything does not function properly, I can simply revert back with no harm done. Do you see any issues with this thinking?
I have my full system setup, and don't plan to change how anything is executed outside of the retroarch files themselves. I have only modified the location in which the runcommand pulls the launch image. What if any are the downsides of doing this? All other runcommand functions are untouched and as it is the "brain" it should not see or execute any differently than the standard runcommand.

Now onto your solution, I think this might be more elegant than my attempt, however I am unclear on how it works.

Best method is to compare es_system.cfg rom filepathes with extraced one = 100% working methof
Set the filepathes on your own script so /home/pi/roms/nes could be NES system

what are you stating here? I don' t follow. are you moving the roms location? if i repath the roms folder, then i would no longer be launching the same rom from the original location, which defeats the purpose of @meleu's script. Please explain more if you have the time.

Last line reverts all back to normal!

what do you mean here? Currently @meleu's runcustom.sh script tells to launch using the original runcommand for the rom in its original system folder. That is the end of the runcustom.sh and it turns everything over to the runcommand.sh file. How would I create a last line to set all back to normal in the runcustom.sh script?

Again, thank you for all of your help!