Backup Retropie to external USB Drive

PARALAX

Hi!

I am currently planning to purchase a Raspberry Pi 4B for Retropie and have been dealing with a few questions over the last few days, to which I could not find an answer for all of them. One of them is making a backup of the entire system, similar to what I know from "True Image" on the PC. The website mainly refers to the backup to another SD card using the PC tool "Image Writer" or via "Rpi-clone", which can make a complete backup of the card, but this also has some disadvantages, e.g. the immense size of the image and writing it back to a different manufacturer's card of the same size. However, the solution via SFTP would be less suitable for me, since I do not intend to use the Internet connection permanently.

I would therefore prefer to back up the necessary files manually using an external exFAT formatted USB stick on the Raspberry.

Hence my question as to whether it is possible to simply copy the necessary folders and their configuration (emulator cores, settings, save states, themes, images, etc.) from the internal Ext4 SD Card to the USB stick with a file manager and then use it again on the PC, to pack the corresponding folders and files with 7zip to save space.

Is it possible to simply copy the entire system including all files 1:1 to the USB stick, even if Retropie or the Linux system is running in the background, or are there problems like Windows 10, which is not all due to file access restrictions Files can be read and copied? And if that's not possible: Which folders would have to be copied to at least save the Retropie & Retroarch environment including all emulators and their settings, so that you can restore all settings when reinstalling an image? Is it even possible the way I planned it?

And what file manager can I use? Is something like "Total Commander" or "Midnight Commander" already built into the Retropie system and if so, where can I find it?

I would be very grateful for detailed instructions and assistance.

windg

I have successfully backup my system with dd
command .

I follow the instructions from this site : [https://www.tomshardware.com/how-to/back-up-raspberry-pi-as-disk-image] The only thing i didn't try was to shrink the image.
My suggestion is to install RetroPie on a spare sd card, experiment here first and when you are sure, try to backup your main system .

PARALAX

@windg Does it also work with exFAT formatted drives? Because I ordered an 128 GB Micro SD Card with Micro SD Card Reader for USB: https://www.amazon.de/Samsung-Speicherkarte-USB-Kartenleser-MB-MD512KB-WW/dp/B09D3MNRGH/

I want to use this SD Card as external drive on Raspberry for loading ROM's and storing save states & configurations. I read that this should be possible, but there are no informations about using the exFAT Format.

https://retropie.org.uk/docs/Running-ROMs-from-a-USB-drive/

If it's possible, I would use this card for backups as well for automatic mount by creating the "retropie-mount" folder on PC.

windg

@PARALAX
Yes, RetroPie support exFat out of the box.

mitu

Hence my question as to whether it is possible to simply copy the necessary folders and their configuration (emulator cores, settings, save states, themes, images, etc.) from the internal Ext4 SD Card to the USB stick with a file manager and then use it again on the PC, to pack the corresponding folders and files with 7zip to save space.

You could, but that's not recommended since some folders may use symlinks, which are not guaranteed to work on other (non-Linux) filesystems. You could archive the folders on your backup destination though, the archive will keep the files' attributes.

Is it possible to simply copy the entire system including all files 1:1 to the USB stick, even if Retropie or the Linux system is running in the background, or are there problems like Windows 10, which is not all due to file access restrictions Files can be read and copied?

You can, but it's not recommended. Plus, you'll need a compatible filesystems - see my previous note.

And if that's not possible: Which folders would have to be copied to at least save the Retropie & Retroarch environment including all emulators and their settings, so that you can restore all settings when reinstalling an image? Is it even possible the way I planned it?

RetroPie is installed in /opt/retropie and most of the configurations are created in /opt/retropie/configs (and symlinked in $HOME where the emulators expect them).

And what file manager can I use? Is something like "Total Commander" or "Midnight Commander" already built into the Retropie system and if so, where can I find it?

Midnight Commander is already included, you can start it with the File Manager entry in the RetroPie system. RetroPie is using RasPiOS Lite underneath (which in-turn is based on Debian), so whatever programs you need you can install via apt.

If you have another PC, I think the easiest option is to keep your system on a smaller SDcard and your ROMs on an external drive (as mentioned in the docs). Your ROMs are already on your USB drive and backing-up a smaller SDcard would be easier.

Another option is to use something like this, which can back-up your system to a minimal .img file on the USB drive and you can exclude the $HOME/RetroPie folder (that should be mounted from the USB drive). To restore, just write the .img file back and connect the USB with your ROMs.

PARALAX

@mitu said in Backup Retropie to external USB Drive:

You could, but that's not recommended since some folders may use symlinks, which are not guaranteed to work on other (non-Linux) filesystems. You could archive the folders on your backup destination though, the archive will keep the files' attributes.

So that means, I should better use the dd command like discribed here?
https://www.tomshardware.com/how-to/back-up-raspberry-pi-as-disk-image#xenforo-comments-3636441

And does it also work with exFAT formatted drives, so that the created image will keep all symlinks and file attributes? Instead of using the "sudo pishrink.sh -z myimg.img" command, I simply could compress the img-file directly with 7zip on my PC to another SSD and could spare the USB drive (in my case the 128 GB Micro SD connected via Micro SD Card Adapter) from extensive write cycles. Then this would be my favourite method, as 7zip is the most effective packer I know. The question is, wether the Raspberry Pi Imager can handle with 7zip files as well. If not, I'm forced to follow the instructions from the website or I have to compress it in Gzip-format from PC with another packer. 7zip also allows compressing in "tar"-format but I don't know wether Raspberry Pi Imager will also accept this when writing back to SD card.

If you have another PC, I think the easiest option is to keep your system on a smaller SDcard and your ROMs on an external drive (as mentioned in the docs). Your ROMs are already on your USB drive and backing-up a smaller SDcard would be easier.

That's exactly what I intend to do, although the Retropie and Retroarch interface, as well as the emulator cores, could also be included in the backup in a pinch. In the worst case, you can undo everything if one of the updates is bugged or one of the emulators deteriorates as a result. However, it would be more elegant if you could save the Retropie & Retroarch environment and the actual emulators directly as a whole folder or copy them to the external USB drive - like I do with my emulators on PC which are mostly portable versions.

Another option is to use something like this, which can back-up your system to a minimal .img file on the USB drive and you can exclude the $HOME/RetroPie folder (that should be mounted from the USB drive). To restore, just write the .img file back and connect the USB with your ROMs.

This goes exactly the way I intend. However, I don't intend to store the entire retropie archive including all settings on the USB drive, just the ROM's, the save states and the saves (battery powered SRAM like "srm" files from the SNES). The rest should remain on the internal 32 GB SD card, which is intended for write-intensive operations. In this case, I opted for a Samsung Pro Endurance, which is intended for precisely such purposes.

https://www.amazon.de/Samsung-MB-MJ32GA-EU-microSDXC-Endurance/dp/B07CY3QSST

The external USB drive, on the other hand, should only be used for reading ROMs and saving game-related data, as well as for the occasional system backups that are to be made before major Retropie or core updates.

Sorry for the many questions, but when it comes to Linux and Debian I'm an absolute layman, even though I've had to deal with it before because of our webserver. Fortunately, you can read most of it in many places like here today.

mitu

@PARALAX said in Backup Retropie to external USB Drive:

So that means, I should better use the dd command like described here?

You can, though cloning a running system with dd is not recommended (as said before).

And does it also work with exFAT formatted drives, so that the created image will keep all symlinks and file attributes ?

The image can be copied to any formatted drive, it's a single file containing a block-by-block copy of your installation.

The question is, wether the Raspberry Pi Imager can handle with 7zip files as well.

Not yet, but 7-zip can compress to xz or gz, which the RPI imager understands.

However, I don't intend to store the entire retropie archive including all settings on the USB drive, just the ROM's, the save states and the saves (battery powered SRAM like "srm" files from the SNES)

This is exactly what Running Roms from a USB drive does.

PARALAX

@mitu So you mean backupping like this would be the better option?

"image-backup $HOME/RetroPie"

This will also keep all symlinks and file attributes?

How do I open the command line or is there an option in Retropie for this? And how exactly do I specify to save the content to my USB drive excluding the mentioned directorys (or more folder)? Sorry if I ask, but I'm not very familiar with Linux commands.

mitu

@PARALAX said in Backup Retropie to external USB Drive:

@mitu So you mean backupping like this would be the better option?

As I said, an image backup isn't impacted by the filesystem where it is copied/created. It will preserve any symlinks inside.

And how do I exactly specify to save the content to my USB drive excluding the mentioned directorys (or more folder)? Sorry if I ask, but I'm not very familiar with Linux commands.

Look up the topic I mentioned and experiment with the examples given by the tool's author.

PARALAX

@mitu I think, I will prefer the "image-backup" tool which you suggest. But how can I use the download in the thread you mentioned, if I don't have a desktop or browser in Retropie? Or should I download and unzip the tool on my Windows PC and copy it to an external USB drive and then copy it via Midnight Commander to the Ext4 Partition of the internal Raspberry SD card to start it in a shell windows? Sorry if I ask, but it's still new territory for me and I want to be well prepared.

Does the RAW backup then also occupy as much space as there is data on the SD card or is it then a 1:1 image of the entire SD card and its capacity? If necessary, I would also like to copy the image to other SD cards that may have the same capacity and that can differ depending on the manufacturer. This is also the main reason why I don't want to use the "W32 Disk Imager", as it always creates a backup of the entire data medium, including the unused blocks which could cause problems with other SD cards of the same capacity due to manufacturer tolerances. I would then like to pack the image created by "image backup" tool and, if necessary, be able to unpack it and write it back with the "Raspberry Pi Imager".

mitu

You can install the desktop on the RetroPie image and other packages (including a browser) so you can operate directly from the Pi. See here.

Does the RAW backup then also occupy as much space as there is data on the SD card or is it then a 1:1 image of the entire SD card and its capacity?

If you look in the topic for the utility I pointed, there's an explanation on how much space does it take - it will initially create a full block copy of your sdcard, then shrink it to the actual size of used blocks. So you need to have enough space on your USB disc for a full sdcard copy.