Preferred long-term storage for the roms.
-
@Neuromante Since i have been working through all of these issue with @pjft and @Sano and just moved to a 128gb usb drive I can tell you that it is pretty slick once it works. The speed is a noticeable improvement, and depending on what type of external drive you get, you might have slower spin rates on the disks....unless you go solid state, which will be much more expensive. I am not sure what currency you are using, but I can tell you that you have 2 options with 128gb usb sticks, you can get USB 2.0 for under 20 and USB 3.0 for under 30 where a 1TB is about 50.
As for favorite roms to avoid browsing. You can create "Favorites Games" systems . You can modify the es_systems.cfg. Lets say you want to have your favorite NES games in one system so you can quickly access them. Copy the NES system in the .cfg and paste it. Rename it whatever you would like. lets say nesFAV.
Edit <path>/home/pi/RetroPie/roms/nes</path> to read <path>/home/pi/RetroPie/roms/nesFAV</path>
Now go into your Roms folder and create a folder named 'nesFAV' and put the roms you want into that folder. Now when you reboot you will see a nes system and right next to it an nes favorites system. You can leave your art in the original path, just tell the gamelist to look to that location. To take this to the next level, you can modify the theme by adding a different logo for the selection screen. If you are interested in this let me know. I can give more information on how to do this.
Alternatively, you could create multiple es_systems.cfg files. Just add a number to the end of the ones you are not using and they won't be recognized. Downside is you have to be able to access the files to change the names when you want to switch what systems you see. Upside, you can keep all your roms loaded at all times and only see the ones you want.
Last, and easiest, buy a couple of smaller USB sticks for each setup you want. If you want to only play your favorites, insert the USB with your favorites on it, Want to access a bigger library, throw in a bigger drive.
-
@Neuromante yeah, USB stick prices are still higher than external HDDs. A few things to consider as well are that you may need to get external power for the HDD, depending on its consumption. I think I've heard people say they use a USB hub that takes separate power.
Also, I believe that the RPi 3 is still only USB 2.0 compatible, meaning that you won't be able to fully benefit from USB 3.0. So you may end up opting for a cheaper drive, if cost is a key factor.
The NFS share is certainly another good option for storing an archive and moving specific games on and off as needed. :)
-
Is corruption unavoidable when using the SD card as a global storage device for the system, the saves and the roms?
I don't mind making backups now and then, but I'm worried the corruption problems are as frequent as they seem in this forums. Any way to avoid SD card corruption? What are the causes?
-
@Serj-Targarien I haven't had my SD card become corrupted, but I run my ROMs on a USB. That being said, plenty of people seem to have had no significant problems either, so I suppose your mileage may vary but they don't seem to be that common.
The most common causes are usually shutting the system off/losing power/removing a card while the card is being written to. If you always shut off the system via the menu, and don't turn it off from the actual power before the card stops having any activity (i.e. yellow light stops blinking), you should be on the safe side for the most part.
-
@pjft thanks for the answer. I always shut down my pi from the emulationstation menu. Mistakes can happen, but that what I have backups for. I hope to keep using retropie this way and will report back with results, even if positive, in a few months, to calm down people who are overworried about the matter.
-
@Serj-Targarien said in Preferred long-term storage for the roms.:
@pjft thanks for the answer. I always shut down my pi from the emulationstation menu. Mistakes can happen, but that what I have backups for. I hope to keep using retropie this way and will report back with results, even if positive, in a few months, to calm down people who are overworried about the matter.
Threads like this one were surprising to me, since it makes the microSD corruption seem like a rampant problem. To further complicate matters, I've seen people report that they are corrupting their cards while trying to save a backup image, either with the PC/Mac imager or through the Linux command line. That's not good if the very thing you're trying to safeguard against is the thing that ends up causing the disaster!
But it's also not at all clear to me that this is a pervasive issue, it has yet to happen to me after a few weeks of setting up the Pi and doing several backups, and my hope is that it's usually caused by a bad power supply, user error (cutting power too early), or something else preventable. Since I'm not versed in Linux, there have been a couple of times where I got stuck and basically had to cut power without doing a proper shutdown (with no ill result). One time, I was experimenting with setting up a bluetooth keyboard - this caused a crash to something that looked like a command prompt, except it wouldn't respond to "emulationstation" or any of the restart/reboot sudo commands I looked up online.
The speed improvement would be a bonus, I suppose, except that I'm generally amazed how swiftly everything runs "as is", and I don't try to emulate N64 or any of the really new stuff that would really push the system, anyway. There are also drawbacks to having external storage, including increased power draw from pulling data through the USB port, and having a thumbdrive protruding from my awesome retro NES case all the time, which I don't want. Relatively minor drawbacks, yes, but I'm sticking with the microSD + periodic backup route for the time being.
Given the impressive functionality of RetroPie (and impressive connectivity of the Pi 3), I'm a little surprised there's no option for cloud saves or uploading save files for safekeeping. Maybe that's more complicated that it seems, however, since a "save file" is really more of a programming snapshot of your ROM frozen in time...
Finally, I've also observed that there's a lot of comments from experienced Linux users on the forums that are basically "power users" and "tinkerers" obsessed with finding the most efficient/appropriate ways to do everything. I don't mean that as a negative at all - I think that's awesome, and those people have been a very friendly resource to me anytime I've asked a silly novice question. But for a user like me that is looking to obtain just enough knowledge to operate and maintain a retro game system, there's probably such a thing as being over-worried about things that are out of my control and above my head, when I should just be enjoying the system and doing the occasional backup to get me a "save point" to return to :)
-
@StormJH1 great synopsis of this thread and several others! To your point, you shouldn't be overly worried about corruption as it is not all that common, however it does happen so we need to be aware of it. For the casual user, corruption will probably only be an issue if you loose power while retropie is running, or you do not properly shut down the system. Even doing these things incorrectly does not mean you will corrupt your card, they just increase the chances it will happen.
You will probably find the corruption happens more often than not to those who do a lot of "tinkering' and those who try to overclock. I am guilty of having 2 cards corrupt on me, however I also have several builds going at the same time, all running slightly differently for different reasons. I have also identified the source of my corruption, my computer is linked to a company dropbox as i work remotely, if dropbox is syncing when I insert by sd card, dropbox wants to open the card and import its contents. First time it happened i just canceled the dropbox request and thought nothing of it. After running the backup, the card was "locked". Thought it was a bad card. Second time it happened, same results. Mind you I have made about 40 backups over the past few months as i test out different things and create setups for several different controller types. In all of those backups, I corrupted twice, the only 2 times that dropbox was syncing and asked to access my card. I have now disabled dropbox from seeing my sd card devices so this will not be an issue moving forward.
So back to your point. You should know there are issues out there, and find a way to backup or limit your exposure to them. It is your choice how you backup and run your system. I would not loose sleep over the potential issues out there. Just use the knowledge we all share and hopefully you can avoid any issues!
-
@TMNTturtlguy Thanks! I'm totally saving your post about the favorite thing for future reference. I'm pondering right now if having "everything" in the USB is possible (I'm working with small USB's right now, mor on this later).
Also, for what I understand of your post, if I have a /whatever-system/ folder, retropie will read whatever subfolders are in it?
The several drives idea seems to be a good one, having one for Nintendo, other for SEGA and such, as it seems is the best alternative ATM.
@pjft My other pi has an external HDD and yeah, I need to use a specific charger to provide power, as with a "random one" (The one I use to charge the phone, for instance), there wasn't enough juice to power it.
Will using USB hubs (To connect more USB's, or just to connect more gamepads, so I can play with someone else) drive the energy consumption up, right? I was thinking, maybe, on 1 USB, Keyboard, Mouse and 2 Pads, expandable to 4.
And yeah, the Pi3 uses USB 2.0. "Fortunately", as it greatly lowers the price of the drive, if I end up buying one. (I did a search again, and yeah, anything greater than 64 GB is out of the question. I'm in Spain, so the almighty Euro is my currency, and prices start to ramp up as soon as you leave the 32GB field).
@StormJH1 Regarding your last point, my main concern towards storage does not has anything to do with "being obsessed with the best way of doing something", but with the system's long term reliability. One of the main points (for me) is being a cheap device (Unless you go full modding, but well, the sky is the limit there), and understanding differences between diffeerent storage options allows me to better assess risks of using this or that option. I had a strong feeling that USB sticks are prone to fails, long term wise, but if those are the preferred option (Taken into account that SD cards are slower -something I did not know- and hard drives are a bit overkill, price wise), I can move on "prevent possible breakdowns".
Which brings me to that part of your post:
" Given the impressive functionality of RetroPie (and impressive connectivity of the Pi 3), I'm a little surprised there's no option for cloud saves or uploading save files for safekeeping. Maybe that's more complicated that it seems, however, since a "save file" is really more of a programming snapshot of your ROM frozen in time..."
Right now, I'm considering two options:
- Installing an owncloud client on the retropie (Which would bring compatibility and performance questions to the table). Given what you are writing in your post, this seems to be a bit overkill (As it needs you to install an owncloud server, and is a bit tricky process).
- Using the rsync command to syncronize and create the backups. (Also a bit overkill, but it seems to be the best/easiest option, starting from scratch).
For what I know, this could comprise reconfiguring save routes and allowing ssh connections. Of course, everything aimed so the process is as simple and automated as possible.
But well, that's for the future. First I need to assess the total size of the ROMs I will have installed, so I can think on keeping my 8GB drive or getting a 32 one, as it seems an USB stick is the way to go (And I have to confess that is way easy, as I can just unplug it from the pi, plug it on the PC, and update the roms).
Cheers!
-
@Neuromante My comment on USB hubs was that you could use a USB hub that has external power (the link is an example, I don't have that or any similar device). :)
-
@TMNTturtlguy - Thanks for your comments! I feel a little better now. Many of the people here who did suffer corruptions were able to diagnose it to a specific cause (losing power; bad power supply; overclocking; etc).
@Neuromante - No offense intended by the "obsessed with finding the best way of doing something" comment; I actually meant that as a compliment. As in, users like you are playing a higher stakes game of poker and pushing the envelope of what is possible with this inexpensive little computer, and then relaying that information back to more novice users like me. So, thanks!
Interesting project idea with the game saves/cloud thing, also!
For purposes of simply backing up the ROM files, I do plan on keeping an uncompressed list of ROMS on a HDD or flash drive that mirrors the one in my ROMs folder on the Pi. Obviously, this would be contained within a full microSD card image, also, but I'd like to have the collection ready to go in case I need to start over (like with a new operating system or different device).
Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.
Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.