Create a custom ES system able to launch games for many systems

meleu

UPDATE: The method described here became outdated after @pjft implemented custom collections to EmulationStation.

I'll keep the post for historical reasons, but I recommend you to use the ES custom collection feature instead.

[original post]

Oh, hello there! Great to see you again.

I've been reading a @TMNTturtlguy 's post about creating a new custom system view for a given theme and I participated on that thread too. And at some point I remembered that @UDb23 created some cool custom SVG logos and shared with us here in the forum.

All those posts have given me a great idea! Let's try something!

We are going to try to create a KONAMI EmulationStation system and use it to place some classic console games from this company.

In the end it looks like this:
0_1496824655136_Captura de tela de 2017-06-07 04:47:44.png

As you can see below, it doesn't have metadata. If you want it, you have to scrape it again (UPDATE: or use the script I'm talking about on step #9 below.

0_1496824683046_Captura de tela de 2017-06-07 05:03:15.png

Results after using the ES builtin scraper (UPDATE: you can also use the script I'm talking about on step #9):

0_1496850312208_Captura de tela de 2017-06-07 12:41:08.png

Now let's make it!

[credits: what we will make here is strongly based on @TMNTturtlguy 's post, and I think @lilbud deserves a credit for the idea too.]

[bonus: the first person to say which is the reference of the text used in the disclaimer gets an upvote.]

create a rom folder for konami:

mkdir -p ~/RetroPie/roms/konami

create symbolic links to the games you want to see in the KONAMI system (note: the link target must have the full path):

# going to the konami directory is useful to avoid much typing
cd ~/RetroPie/roms/konami
# This is how the roms are organized in my system! YOU MUST ADAPT IT ON YOUR END!
ln -s ~/RetroPie/roms/nes/Teenage\ Mutant\ Ninja\ Turtles\ \(USA\).nes TMNT-\(NES\).nes
# ... added more TMNT games ...
ln -s ~/RetroPie/roms/snes/Sunset\ Riders\ \(USA\).zip Sunset\ Riders\ \(SNES\).zip
ln -s ~/RetroPie/roms/nes/Castlevania\ \(USA\).nes Castlevania\ \(NES\).nes
# ... added more Castlevania games for SNES, Mega Drive and Gameboy Advance...
# don't forget to add games from the Contra series!!

UPDATE: there's a way to simplify the symbolic links creation, see the FAQ below.

get the @UDb23 's konami logo SVG file. Click this link, click on the more folder and then download a konami logo (I prefer the konami_old.svg)
copy the carbon theme folder to the user-specific ES files:

mkdir -p ~/.emulationstation/themes
cp -R /etc/emulationstation/themes/carbon ~/.emulationstation/themes/
mv ~/.emulationstation/themes/carbon ~/.emulationstation/themes/myCarbon

It prevents you to lose your changes after a Carbon theme update.

copy a system theme as a template (I used nes):

cd ~/.emulationstation/themes/myCarbon
cp -R nes konami

name the SVG logo you downloaded as myCarbon/konami/art/system.svg:

# removing unnecessary files
rm ~/.emulationstation/themes/myCarbon/konami/art/*
# copying the logo
cp /PATH/TO/FILE/konami_old.svg ~/.emulationstation/themes/myCarbon/konami/art/system.svg

add a konami entry in ~/.emulationstation/es_systems.cfg (create the file if it doesn't exist):

  <system>
    <name>konami</name>
    <fullname>Konami</fullname>
    <path>/home/pi/RetroPie/roms/konami</path>
    <extension>.zip .ZIP .smd .SMD .smc .SMC .sfc .SFC .nes .NES .bin .BIN .pbp .PBP</extension>
    <command>/home/pi/bin/runcustom.sh %ROM%</command>
    <platform>all</platform>
    <theme>konami</theme>
  </system>

create the /home/pi/bin/runcustom.sh script (the code is pretty small):

#!/usr/bin/env bash

# testing if it's a symbolic link
if [[ ! -L "$1" ]]; then
    dialog --msgbox "ERROR: \"$1\" is not a symbolic link." 0 0
    exit 1
fi

# getting the name of the file that the symbolic link points to
rom="$(readlink "$1")"

# getting the system based on the folder the rom is stored
system="$(echo "$rom" | sed 's|\(.*/RetroPie/roms/[^/]*\).*|\1|' | xargs basename)"

# now launch runcommand normally
/opt/retropie/supplementary/runcommand/runcommand.sh 0 _SYS_ "$system" "$rom"

Done! But I have a BONUS!!

I wrote a script to let you use the already existent metadata. So, there's no need to scrape the games in konami folder.

The script can be found here: https://github.com/meleu/share/blob/master/custom-system-autoscraper.sh

And you can download it with this command:

wget https://raw.githubusercontent.com/meleu/share/master/custom-system-autoscraper.sh

Launch it with --help to see the usage:

$ ./custom-system-autoscraper.sh -h
This script gets all the symbolic links in a custom ES system directory and
tries to create a gamelist.xml based on already existent metadata, boxart,
marquee, and video.

More info here: https://retropie.org.uk/forum/post/84125

Usage:
./custom-system-autoscraper.sh /path/to/custom/system/directory

FAQ

1. Can I adapt it to make a "character" or "beat'em ups" ES system?

Sure. Get a nice SVG logo and do it!

2. If I launch Castlevania - Aria of Sorrow (GBA) from Konami menu, and after I launch the same game from Gameboy Advance menu, will I be able to load my progress?

Yes. The .srm files are stored in the same folder as the real ROM file. Same thing for save states.

3. Is it possible to create symbolic links on a Windows machine and move them to my RetroPie later?

No. Symbolic links can't be created on Windows and can't be placed on a Windows partition. But if your ROMs are stored on a Windows partition, you can use the method described here. You just have to place the symbolic links themselves on a Linux partition (target files can be in any valid path). Click here to see the conversation about it.

4. Is there a way to automate the symbolic links creation?

UPDATE: Yes, I wrote a script for this and talked about it here in this post.

Yes, but you'll need shell scripting knowledge, at least the basic of repetition structures.

5. What about metadata and boxart?

UPDATE: you can use the script I'm talking about on step #9.

You'll have to scrape the Konami "system" again. My suggestion is to use sselph's scraper, as it uses the the ROM's hash to get metadata. The ES builtin scraper uses filenames to get metadata and the scraping process will be very tedious with the user having to deal with conflicts.

If you have a small game list, you can create a gamelist.xml by hand and use full path to reference the files (boxart, video, etc). It would avoid files duplication.

6. What launching image will be displayed when I launch a game from Konami menu?

The real system's launching image. Example: If you launch Contra - Hard Corps, you'll see the Mega Drive launching image; if you launch Castlevania - Aria of Sorrow, you'll see the Gameboy Advance one, etc...

spruce_m00se

YES!
I would love a theme that allows me to organise my games by either "character" or "developer" ie have all sonic games from all systems in one menu, that would be great,

meleu

@spruce_m00se I would love to have a Mega Man category, I just need a cool SVG logo for it...

mattrixk

@meleu Is it possible to create symbolic links on a windows machine and move them over, or does it have to be done on the Pi itself. I see them in my mind basically as shortcuts to the real file, but I'm guessing it's a bit more complicated than that.

I have the idea to have a new System for "Classics" and then in the have folders and subfolders within that system eg:

Classics
  - Super Mario
    - NES
      - Mario 1.nes
      - Mario 2.nes
    - SNES
      - Mario 3.sfc
  - Tetris
    - Gameboy
      - Tetris.gb
    - NES
      - Tetris 2.nes
    - SNES
      - Super Tetris.sfc

Etc.

Do you think that would be possible?

spruce_m00se

@meleu maybe if im free tommorow at work I will knock something up in photoshop.

meleu

@mattrixk perfectly possible (but I wouldn't use so many directories, too many button presses until launch a game).

Unfortunately symbolic links can only be created in a Linux partition.

EctoOne

Is there a way to batch create the links? I really want to combine gb, gbc, nes, snes etc under Nintendo and Gamegear, Sega 32x etc under Sega. And even i don't have that many Roms it seems like a lot of work.
Also is it possible to remove the unused systems afterwards from the es_system.cfg without breaking this method?

meleu

@EctoOne assuming you're comfortable with the basics of a Linux shell, these commands can help you (you need to adapt to your needs):

# in this example I'm creating links for every file in gameboy directory
original_system="/home/pi/RetroPie/roms/gb"
destination_system="/home/pi/RetroPie/roms/nintendo"
while read -r file; do
  ln -s "$original_system/$file" "$destination_system/$file"
done < <(ls "$original_system")

You guys need to keep in mind that scraped data don't work using this method.

Also is it possible to remove the unused systems afterwards from the es_system.cfg without breaking this method?

I didn't test, but I can't see a problem.

EctoOne

@meleu said in Create a custom ES system able to launch games for many systems:

You guys need to keep in mind that scraped data don't work using this method.

Well in that case i won't use it. Thanks for getting my hopes up! :P (Just kidding)

TMNTturtlguy

@EctoOne @meleu are you able to create a new gamelist within the kanomi folder that will recognize the linked games?

@EctoOne if you see the steps in my original post that inspired @meleu's method, you would be able to create a new gamelist for the "kanomi" system and have metadata etc. in the coming days I will have an updated process for my method. It works just like the Arcade folder.

meleu

@meleu said in Create a custom ES system able to launch games for many systems:

You guys need to keep in mind that scraped data don't work using this method.

Actually I meant "scraped data don't work automatically". You'll need to edit gamelist.xml.

edit: I updated the OP with this info.

TMNTturtlguy

@meleu perfect! And my recommendation is to copy paste the gamelist info for each rom from the original gamelist and just update the path for the ~~draped~~ data to point to the folder location.

Edit: Scraped - stupid phone!

EctoOne

@meleu Ahh good to know, will try it on the weekend.

lilbud

Can the games be scraped in a custom system?

meleu

@lilbud I think scrapers that uses hashes cand do it. Sselph can, but I think you'll need to invoke it from command line (not the retropie_setup frontend).

UPDATE: I tested Sselph's scaper from RetroPie-Setup menu and it works pretty well. There's no need to do any tweak, just select konami in Scrape chosen system.

meleu

@lilbud but it would duplicate several files...

When I have a free time I'll try to automate this thing...

Edit: Duh! With an automated way to create a gamelist.xml there's no need to use symbolic links... Why can't I stop writing these damn scripts?!?! Am I a bash addicted? Should I look for a professional treatment?

meleu

@lilbud I've just used the ES builtin scraper, and when I chose only KONAMI to scrape, I got this warning:

0_1496850171125_Captura de tela de 2017-06-07 12:36:42.png

And indeed the results aren't very accurate, but it works reasonably well:

0_1496850312208_Captura de tela de 2017-06-07 12:41:08.png

Let me see what sselph's scraper can do...

UPDATE: sselph scraper in RetroPie-Setup runs pretty well and (as expected) much faster than the ES scraper.

TMNTturtlguy

@meleu any update on how save states work? Where do the save files get saved? If I play TMNT in the konami folder and save progress. Will I be able to open my saved game from the Konami folder? I am guessing that the script runs the same as we discussed in the other thread? Games played in konami will save in Konami, and i open the same game in NES folder, it will not have that saved game info.

Basically, the 2 folders are independent. I can't play in one folder and then access the saved info from the other folder. Correct?

Edit: also launching images - I assume that the launching image will match whatever the current launch image is within the NES folder? So if all my NES roms launch an NES splash, when i launch from konami i will still get the NES splash?

backstander

@TMNTturtlguy

Where do the save files get saved?

The Save States will be saved next to the rom in the same folder unless you manually override this in /opt/retropie/configs/konami/retroarch.cfg. You might be able to fix this if you use Symbolic Links for the Save States in the Konami as well.

also launching images

It will look for the "Konami" system launching image at /opt/retropie/configs/konami/launching.jpg and each games's launching image /opt/retropie/configs/konami/RomName-launching.png

Symbolic Links are your friend!

This is why @lilbud said "Don't try this at home, kids!"

TMNTturtlguy

@backstander Hmmmmmm...Maybe @meleu can expand on this, but i think the point of @meleu script is that we are not creating /opt/retropie/configs/konami/ therefore none of the locations you mention in your post exist. In my original build of how this works i created the configs folder and the retroarch.cfg, thus launching.jpg and save files would work. The problem was you had to place your rom in a new roms folder of the same name, thus having 2 instances of 1 rom as well as needing to set the roms emulator. My understanding of the sybmolic link is that it is linking to the rom in its original folder or example nes and then the script says, use the ./configs/nes/retroarch.cfg
This would mean that it isn't using the konami.cfg and there is no location to launch an image from either.