• Recent
  • Tags
  • Popular
  • Home
  • Docs
  • Register
  • Login
RetroPie forum home
  • Recent
  • Tags
  • Popular
  • Home
  • Docs
  • Register
  • Login

Symbolic Links and External Storage Potential

Scheduled Pinned Locked Moved Ideas and Development
symbolic linksexternal drivestorage
12 Posts 6 Posters 7.2k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • U
    Used2BeRX
    last edited by 7 Oct 2017, 01:24

    The Symbolic Link in Linux is such a powerful tool. Think a Windows Shortcut but on steroids.

    So.... What is a symbolic link then????

    I wanted to test out all of the Sega CD games on a Pi Zero a while back. My thought was that I was going to have to transfer only a few CDs at a time via FTP onto my very small 16GB SD card and that this was going to take about a week to do it this way. Then I looked into using a USB stick which was 64GB and benefited from a MUCH faster transfer rate, and also wouldn't burden the Pi Zero.

    There are a few ways to set up the USB stick to work with the RetroPie, but hands down the Symbolic Link is the best way to do it. Not only is it relatively easy to do and does not require you to alter any system files, but as it turns out, if you have properly set up your gamelist.xml files beforehand, your RetroPie will benefit from all of the images or videos you have for that system even though the media isn't actually on the SD card where the XML file is pointing to!

    For this example, I will use the Sega CD roms folder:

    /home/pi/RetroPie/roms/segacd/

    We actually delete the above folder, and then by entering a command in the Linux console we replace that folder with the symbolic link pointing to the proper folder on your USB drive. (I figured out an even easier way to do this in WinSCP, but I don't remember it right now. I will update this guide at a later date with this alternate method).

    1. Put all of your Sega CD games and directories into a directory called /segacd/ in the root directory on your thumb drive, and plug it into your Pi.

    2. Go to the Linux console on your Pi (by hitting F4 from EmulationStation).

    3. type ls /media/usb0 and it should show the folders in the root of your USB stick. If it doesn't, replace usb0 with usb1 all the way up to usb7 until you get the one showing the media. Remember the #.

    4. Go to the system folder for Sega CD on your pi (/home/pi/RetroPie/roms/segacd/ in our example)

    5. type sudo ln -s /media/usb#/segacd segacd (where # is the # you got from step 3).

    This will create a "virtual folder" in /home/pi/RetroPie/roms/ that points to the /segacd/ folder on your USB drive that will in every way behave as if the media were exactly in that spot on your SD card!!!!

    This means that if you don't have the USB stick in your Pi when you fire it up, Sega CD won't even show up as a selection in the main menu at all, but when you have the stick with your CD games in it and boot up the Pi the Sega CD will show up and if your gamelist.xml was set up properly, the romlist will show all of the artwork, videos and synopsis information as you scroll thru the roms!!!!!!

    Also, this means that any other roms for any other system that you have on the SD card that are not in any folders you've changed in this way will still be usable as normal.

    It is also possible to point your entire roms folder and have all of them on a USB stick and none of them on your SD Card. You can do this with as many folders as you want to.

    DISCLAIMER: This is a work in progress guide that I will polish up once I get back to testing out CD based games and really putting my Pi together. Currently I'm neck deep in an NES project so it will be a while, but @waimea had said that he wanted RetroPie to support being able to launch games from the SD Card as well as a USB drive at the same time and with Symbolic Links that can already be done.

    I thought I had written a good guide to it somewhere else, but it was just some basic notes. I'm going to put them up here for now so waimea doesn't have to wait forever for me to finally get around to it. I hope it helps anybody else who wondered if this was possible too. If anybody has any questions, please don't hesitate to ask. I will do my best to update this and go deeper into the possibilities of the Symbolic Link in the future. It's been a while since I've done it though, so I don't want to talk about things that I'm not exactly sure are possible here until I'm actively doing them again.

    E H 3 Replies Last reply 7 Oct 2017, 01:33 Reply Quote 1
    • E
      edmaul69 @Used2BeRX
      last edited by edmaul69 10 Jul 2017, 02:36 7 Oct 2017, 01:33

      @used2berx you could save the trouble and edit your es_systems.cfg to just look for your sega cd roms in /media/usb0/segacd/ instead of /home/pi/RetroPie/roms/segacd/

      M 1 Reply Last reply 7 Oct 2017, 11:00 Reply Quote 0
      • E
        edmaul69 @Used2BeRX
        last edited by edmaul69 10 Jul 2017, 02:39 7 Oct 2017, 01:38

        @used2berx i read roms from both my hard drive and sd card at the same time just fine by editing my es_systems.cfg

        1 Reply Last reply Reply Quote 0
        • U
          Used2BeRX
          last edited by Used2BeRX 10 Jul 2017, 02:44 7 Oct 2017, 01:38

          True, but I think this is something important for people who want to tinker with things to learn. I've done a few other really cool things with it that I'll talk about when I get back to that work.

          Also, I don't know if your media would show up after editing es_systems.cfg with a new destination for your media with your gamelist.xml as is. I suspect that you'd have to re-scrape things, or do whatever you did in the first place to regenerate a new gamelist.xml. At the very least I'd imagine you'd have to do a big find and replace session for the new spot on all of the effected xmls.

          Maybe I'm wrong about that. I haven't tested it myself.

          All I know is that with the symbolic link, the remote media is treated as if it were exactly where that folder is on your SD card.

          C 1 Reply Last reply 7 Oct 2017, 04:39 Reply Quote 0
          • H
            herb_fargus administrators @Used2BeRX
            last edited by 7 Oct 2017, 04:30

            @used2berx said in Symbolic Links and External Storage Potential:

            There are a few ways to set up the USB stick to work with the RetroPie, but hands down the Symbolic Link is the best way to do it

            I disagree. It depends on what the user's needs are and they need to understand the ramifications of modifying configs with symlinks as opposed to the simpler built in method of mounting the USB directly on the Roms folder.

            https://retropie.org.uk/docs/Running-ROMs-from-a-USB-drive/

            If you read the documentation it will answer 99% of your questions: https://retropie.org.uk/docs/

            Also if you want a solution to your problems read this first: https://retropie.org.uk/forum/topic/3/read-this-first

            U 1 Reply Last reply 7 Oct 2017, 23:37 Reply Quote 1
            • C
              cyperghost @Used2BeRX
              last edited by cyperghost 10 Jul 2017, 05:48 7 Oct 2017, 04:39

              @used2berx said in Symbolic Links and External Storage Potential:

              All I know is that with the symbolic link, the remote media is treated as if it were exactly where that folder is on your SD card.

              You don't know all about it - be carefull with such statements!
              What do you think happens if you use this on a device with FAT32? As FAT32 does not support user permissions and all file flags by default. Therefore the wiki tells to use fstab or the automount - these are the general methods for a proper mounting.

              You can read more about here

              In most cases the symlink works ... yes :) But it's not the 100% working method for all usecases. But it's rather better than editing es_systems.cfg(imho)

              EDIT: @herb_fargus was quicker ... Good morning/evening ;)

              E 1 Reply Last reply 7 Oct 2017, 18:49 Reply Quote 0
              • M
                meleu @edmaul69
                last edited by 7 Oct 2017, 11:00

                @edmaul69 said in Symbolic Links and External Storage Potential:

                @used2berx you could save the trouble and edit your es_systems.cfg to just look for your sega cd roms in /media/usb0/segacd/ instead of /home/pi/RetroPie/roms/segacd/

                I think this editing will be lost after an ES update.

                • Useful topics
                • joystick-selection tool
                • rpie-art tool
                • achievements I made
                E 1 Reply Last reply 7 Oct 2017, 18:39 Reply Quote 0
                • E
                  edmaul69 @meleu
                  last edited by edmaul69 10 Jul 2017, 19:40 7 Oct 2017, 18:39

                  @meleu not if you put your es_systems.cfg in /opt/retropie/configs/all/emulationstation/ which is where i keep it anyways because i put my systems in order by manufacturer and age. And the retropie menu is always last for me so i can find it easily.

                  1 Reply Last reply Reply Quote 0
                  • E
                    edmaul69 @cyperghost
                    last edited by 7 Oct 2017, 18:49

                    @cyperghost i edit the es_systems.cfg for everything. You can place the systems in any order you want, you can find roms from any location, and link to any theme systems you want. If you put your es_systems.cfg in /opt/retropie/configs/all/emulationstation/ then you can have all your edits done in one file and not be overwritten. And i have never had a single issue doing it this way. Everything works exactly as it should and as i want. If anyone uses advmess or lr-mess, you have to edit the es_systems.cfg anyways as retropie doesnt make any system configs for them other than a few for systems for advmess that run better on other emulators already.

                    B C 2 Replies Last reply 7 Oct 2017, 18:59 Reply Quote 0
                    • B
                      BuZz administrators @edmaul69
                      last edited by 7 Oct 2017, 18:59

                      @edmaul69 that method is not recommended. It may work for you, but means users need to manage their own systems XML.

                      The recommended method is the one herb linked and ensures the usb is mounted with the correct options.

                      I wouldn't recommend users replace their ROM folders with symlinks either.

                      To help us help you - please make sure you read the sticky topics before posting - https://retropie.org.uk/forum/topic/3/read-this-first

                      1 Reply Last reply Reply Quote 0
                      • C
                        cyperghost @edmaul69
                        last edited by 7 Oct 2017, 19:56

                        @edmaul69 There is a difference between what you do on your mashine at home and what you recommend to other users.
                        It is a question of maintaining RetroPie and help other users. If you tell them to use the procedure as descriped in the wiki there is no failure. But if you do symlink, editing XMLs then other issues will occour - think about filesystems, permissions flags, typos, editing with root instead of user pi....

                        As you are a veteran in this forum you know what issues may occur if a new member get in touch with RetroPie ;)

                        1 Reply Last reply Reply Quote 0
                        • U
                          Used2BeRX @herb_fargus
                          last edited by 7 Oct 2017, 23:37

                          @herb_fargus

                          https://retropie.org.uk/docs/Running-ROMs-from-a-USB-drive/

                          Would I be able to edit things in folders that are currently denied because of permissions using this method?

                          I'm still not very happy that I can't edit the NstDatabase.xml file for Nestopia or all of the hoops you have to jump through to change control configurations. Would be nice if this method was a workaround for that.

                          1 Reply Last reply Reply Quote 0
                          12 out of 12
                          • First post
                            12/12
                            Last post

                          Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                          Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.

                            This community forum collects and processes your personal information.
                            consent.not_received