Symbolic Links and External Storage Potential

Used2BeRX

The Symbolic Link in Linux is such a powerful tool. Think a Windows Shortcut but on steroids.

So.... What is a symbolic link then????

I wanted to test out all of the Sega CD games on a Pi Zero a while back. My thought was that I was going to have to transfer only a few CDs at a time via FTP onto my very small 16GB SD card and that this was going to take about a week to do it this way. Then I looked into using a USB stick which was 64GB and benefited from a MUCH faster transfer rate, and also wouldn't burden the Pi Zero.

There are a few ways to set up the USB stick to work with the RetroPie, but hands down the Symbolic Link is the best way to do it. Not only is it relatively easy to do and does not require you to alter any system files, but as it turns out, if you have properly set up your gamelist.xml files beforehand, your RetroPie will benefit from all of the images or videos you have for that system even though the media isn't actually on the SD card where the XML file is pointing to!

For this example, I will use the Sega CD roms folder:

/home/pi/RetroPie/roms/segacd/

We actually delete the above folder, and then by entering a command in the Linux console we replace that folder with the symbolic link pointing to the proper folder on your USB drive. (I figured out an even easier way to do this in WinSCP, but I don't remember it right now. I will update this guide at a later date with this alternate method).

Put all of your Sega CD games and directories into a directory called /segacd/ in the root directory on your thumb drive, and plug it into your Pi.
Go to the Linux console on your Pi (by hitting F4 from EmulationStation).
type ls /media/usb0 and it should show the folders in the root of your USB stick. If it doesn't, replace usb0 with usb1 all the way up to usb7 until you get the one showing the media. Remember the #.
Go to the system folder for Sega CD on your pi (/home/pi/RetroPie/roms/segacd/ in our example)
type sudo ln -s /media/usb#/segacd segacd (where # is the # you got from step 3).

This will create a "virtual folder" in /home/pi/RetroPie/roms/ that points to the /segacd/ folder on your USB drive that will in every way behave as if the media were exactly in that spot on your SD card!!!!

This means that if you don't have the USB stick in your Pi when you fire it up, Sega CD won't even show up as a selection in the main menu at all, but when you have the stick with your CD games in it and boot up the Pi the Sega CD will show up and if your gamelist.xml was set up properly, the romlist will show all of the artwork, videos and synopsis information as you scroll thru the roms!!!!!!

Also, this means that any other roms for any other system that you have on the SD card that are not in any folders you've changed in this way will still be usable as normal.

It is also possible to point your entire roms folder and have all of them on a USB stick and none of them on your SD Card. You can do this with as many folders as you want to.

DISCLAIMER: This is a work in progress guide that I will polish up once I get back to testing out CD based games and really putting my Pi together. Currently I'm neck deep in an NES project so it will be a while, but @waimea had said that he wanted RetroPie to support being able to launch games from the SD Card as well as a USB drive at the same time and with Symbolic Links that can already be done.

I thought I had written a good guide to it somewhere else, but it was just some basic notes. I'm going to put them up here for now so waimea doesn't have to wait forever for me to finally get around to it. I hope it helps anybody else who wondered if this was possible too. If anybody has any questions, please don't hesitate to ask. I will do my best to update this and go deeper into the possibilities of the Symbolic Link in the future. It's been a while since I've done it though, so I don't want to talk about things that I'm not exactly sure are possible here until I'm actively doing them again.

edmaul69

@used2berx you could save the trouble and edit your es_systems.cfg to just look for your sega cd roms in /media/usb0/segacd/ instead of /home/pi/RetroPie/roms/segacd/

edmaul69

@used2berx i read roms from both my hard drive and sd card at the same time just fine by editing my es_systems.cfg

Used2BeRX

True, but I think this is something important for people who want to tinker with things to learn. I've done a few other really cool things with it that I'll talk about when I get back to that work.

Also, I don't know if your media would show up after editing es_systems.cfg with a new destination for your media with your gamelist.xml as is. I suspect that you'd have to re-scrape things, or do whatever you did in the first place to regenerate a new gamelist.xml. At the very least I'd imagine you'd have to do a big find and replace session for the new spot on all of the effected xmls.

Maybe I'm wrong about that. I haven't tested it myself.

All I know is that with the symbolic link, the remote media is treated as if it were exactly where that folder is on your SD card.

herb_fargus

@used2berx said in Symbolic Links and External Storage Potential:

There are a few ways to set up the USB stick to work with the RetroPie, but hands down the Symbolic Link is the best way to do it

I disagree. It depends on what the user's needs are and they need to understand the ramifications of modifying configs with symlinks as opposed to the simpler built in method of mounting the USB directly on the Roms folder.

https://retropie.org.uk/docs/Running-ROMs-from-a-USB-drive/

cyperghost

@used2berx said in Symbolic Links and External Storage Potential:

All I know is that with the symbolic link, the remote media is treated as if it were exactly where that folder is on your SD card.

You don't know all about it - be carefull with such statements!
What do you think happens if you use this on a device with FAT32? As FAT32 does not support user permissions and all file flags by default. Therefore the wiki tells to use fstab or the automount - these are the general methods for a proper mounting.

You can read more about here

In most cases the symlink works ... yes :) But it's not the 100% working method for all usecases. But it's rather better than editing es_systems.cfg(imho)

EDIT: @herb_fargus was quicker ... Good morning/evening ;)

meleu

@edmaul69 said in Symbolic Links and External Storage Potential:

@used2berx you could save the trouble and edit your es_systems.cfg to just look for your sega cd roms in /media/usb0/segacd/ instead of /home/pi/RetroPie/roms/segacd/

I think this editing will be lost after an ES update.

edmaul69

@meleu not if you put your es_systems.cfg in /opt/retropie/configs/all/emulationstation/ which is where i keep it anyways because i put my systems in order by manufacturer and age. And the retropie menu is always last for me so i can find it easily.

edmaul69

@cyperghost i edit the es_systems.cfg for everything. You can place the systems in any order you want, you can find roms from any location, and link to any theme systems you want. If you put your es_systems.cfg in /opt/retropie/configs/all/emulationstation/ then you can have all your edits done in one file and not be overwritten. And i have never had a single issue doing it this way. Everything works exactly as it should and as i want. If anyone uses advmess or lr-mess, you have to edit the es_systems.cfg anyways as retropie doesnt make any system configs for them other than a few for systems for advmess that run better on other emulators already.

BuZz

@edmaul69 that method is not recommended. It may work for you, but means users need to manage their own systems XML.

The recommended method is the one herb linked and ensures the usb is mounted with the correct options.

I wouldn't recommend users replace their ROM folders with symlinks either.

cyperghost

@edmaul69 There is a difference between what you do on your mashine at home and what you recommend to other users.
It is a question of maintaining RetroPie and help other users. If you tell them to use the procedure as descriped in the wiki there is no failure. But if you do symlink, editing XMLs then other issues will occour - think about filesystems, permissions flags, typos, editing with root instead of user pi....

As you are a veteran in this forum you know what issues may occur if a new member get in touch with RetroPie ;)

Used2BeRX

@herb_fargus

https://retropie.org.uk/docs/Running-ROMs-from-a-USB-drive/

Would I be able to edit things in folders that are currently denied because of permissions using this method?

I'm still not very happy that I can't edit the NstDatabase.xml file for Nestopia or all of the hoops you have to jump through to change control configurations. Would be nice if this method was a workaround for that.