Backup SD Card by copying out files?

mediamogul

but do i just save the script code you posted in a .xml and then save that file to /home/pi/RetroPie/retropiemenu? That seems to easy.......Thanks

It's just about that easy. Save it with the extension .sh and then run:

chmod +x /home/pi/RetroPie/retropiemenu/name-of-script.sh

After that it should be good to go.

TMNTturtlguy

@mediamogul Awesome, you are the best and made my day!

pjft

@TMNTturtlguy you'd need a second card, but not the same size (unless your ROMS are there). But yeah, the other folders are the key ones.

TMNTturtlguy

@pjft unfortunately my roms are on my SD card. I started my builds before i joined this community and read all the documents and made my first build on a 128GB and it worked great, so I made an image and started make base copies for each of my additional builds (stand up arcade machines as well as a plug and play TV setup). All of my roms are set up with the correct emulators and have game cores set and they are all scraped to the location on the SD Card. I would love to simply move the roms off the card so if and when the SD card corrupts I don't have to replace it with a 128gb, but I believe that would take a lot of effort? Right now i have all my builds with the same base so I hope this will allow me to create a backup of the mods i make for each setup. Reinstall the base image and overwrite the folders in the base with the USB backup. Downside is i have to replace a 128 SD card instead of an 8 gb card.

mediamogul

@TMNTturtlguy

I would love to simply move the roms off the card so if and when the SD card corrupts I don't have to replace it with a 128gb, but I believe that would take a lot of effort?

There's a very easy way to move the entire RetroPie folder externally that was introduced a few month's back.

TMNTturtlguy

@mediamogul would i need to repath all of my gamelists?

mediamogul

@TMNTturtlguy

No, the beauty part is that because the new location is mounted over the old, the entire system never knows there was a change.

TMNTturtlguy

@mediamogul WOW - that is awesome....ok, so now a few more question as I am trying to compute all of this info - Thanks again as i am learning a lot today!

I am currently running 128gb cards, lets say i get 128 gb usb stick and copy over the roms. Now i am left with a 128gb card running with no roms on it. Could I use your script to copy over the important files, move the roms over, get a new 8gb card and throw a clean retropie image onto it, copy the folders i pulled off onto the clean image on the 8gb card and continue to run as if nothing changed? I guess to simplify, can i copy the roms and folders and build an 8gb card that will run identical to my current 128 SD card build?
The instructions have you turn on the USB transfer service, they don't say to turn it off after the transfer? Do you just leave it on at all times?
after i transfer the roms and it is mounted, can i then take the USB out and plug it back into my windows computer to add roms, or is it best to still add roms over WinSCP?

pjft

@TMNTturtlguy I can't answer for others, but you can certainly use rpi-clone to, after copying the ROMS to the SD card and removing them from the SD card, create an exact copy on a smaller SD card and run it from there. I can tell you how to do that as well.

And yes to number 3, that's how I sometimes do it as well. Works like a charm. Obviously I still backup the ROMS as well every now and then :)

I don't know about number 2 as I don't use the service myself but rather mount it using the old method.

TMNTturtlguy

@pjft Thanks for the response. I like the plan of copying off the roms and then deleting them and transferring the data onto a smaller SD card with the rpi-clone tool. Does the clone tool make the smaller sd card bootable as well? I did find a version on github, not sure if it is the same tool you are speaking of. If you have time, any directions or link would be appreciated. If not, I will be resourceful and dig into some reading! I really appreciate all the help and responses, i think this will really improve my setups.

I am going to run to the local computer store and buy a few SD and USB sticks. Do you think an 8gb sd is the right size? All of the games will be on the SD card, i suppose scraped art that is in home/pi/.emulationstation will remain on the SD Card? Edit: looks like a 16gb class 10 card is only $1.00 more for me, so i might as well just get those.

As for #2 - after reading the manual instructions, the manual instructions say not to use the automatic transfer, so i assume that once the transfer is complete I will want to turn that service off so it doesn't try to transfer every time.

mediamogul

@TMNTturtlguy

I must have missed your last post.

(1) Yes, all of that would work.

(2) Yes, you would leave it on. I believe it triggers an action that would set everything back when deactivated.

(3) Yes again. At that point you could just plug the external drive into your computer to transfer ROMs.

Edit:

so i assume that once the transfer is complete I will want to turn that service off so it doesn't try to transfer every time.

It doesn't transfer every time.

TMNTturtlguy

@mediamogul Thanks! Off to pick up some parts and give this try!

mediamogul

@TMNTturtlguy

Good luck. It sounds like either method will get you where you want to be.

TMNTturtlguy

@mediamogul @pjft this could take awhile! I have about 90 gigs to transfer, it has been running for 10 minutes and we are up to 1gb transferred! I am doing this for 2 builds, do you know if i can simply copy the files from one usb stick to another over my PC? or does the script do something to format the usb stick when it is first inserted? I noted that when looking at the retropie manager that the mount is named /media/usb0 - that is making me wonder if I have to do this process twice instead of a simply copy?

Also, I noticed that the link for formatting USB drives for Windows does not provide proper information for formatting USB drives larger than 32GB. Windows will not allow you to format drives larger than 32GB as fat32. Luckily the drives I purchased where already formatted fat32, but the docs might want to get updated? Thanks again

mediamogul

@TMNTturtlguy

do you know if i can simply copy the files from one usb stick to another over my PC? or does the script do something to format the usb stick when it is first inserted?

That should work. Just make sure you rename the copied 'RetroPie' folder to 'retropie-mount' in the root of the external drive and it should see that it's already been copied and act accordingly.

TMNTturtlguy

@mediamogul @pfjt - Still running! I am at 76 gb transferred. Once the roms/folders are transferred to the USB stick and i want to delete them from the SD card, what is the best process to do so? Can I just go into WinSCP and select the rom folder and delete all the folders within? or does that file structure need to remain on the SD card and I then go into each system folder and delete the contents from within each one? Also, can retropie run without the USB plugged in now that the entire folder has been moved, or does the USB always need to be plugged in when running? I want to make sure i don't delete any ROM files since it has taken so long to transfer them!

pjft

@TMNTturtlguy Ok, so a few things:

My recommendation is to leave part of the folder structure in the SD card, but with no ROMs there. Perhaps just one of those folders with placeholder shortcuts (like Amiga's "Launch UAE" or something) would work. Or a single ROM in a single folder. The reason is that, if/when you need to boot RetroPie without the USB there, you still want it to be able to launch ES properly, and I don't know what would happen if you had no systems/ROMs there.
As I was saying, as long as the base folder structure is there, it should boot fine. I imagine you'll be mounting the USB on top of the RetroPie folder, so both BIOS and roms folders would benefit form existing in the SD card, even if empty.

In my case, that's what I do when I want to run rpi-clone. I boot it without the USB mounted, and then run the script. It'll copy whatever's needed to the new SD card.

Depending on the script, though, if you do have the USB plugged in, and if it's mounted in ~/RetroPie, it may start copying over the ROMs to the backup (as it doesn't know that that folder is actually now residing on a USB stick). Just a note, from personal experience :)

Other than that, you should hopefully be good to go.

ibiza7

@herb_fargus
Bonjour je. NE sais pas qui contacté je voudrais me désinscrire désabonner du forum le plutot possible je recois plus de 100 mail de retropie par jour. Cordialement

pjft

@ibiza7

Click your picture on the top right > Settings >

Uncheck:
- Email: Send an email if a new chat message arrives and I am not online
- Email: Send an email when replies are made to topics I am subscribed to
- Watch: Watch topics you create
- Watch: Watch topics that you reply to
Subscribe to Digest: Off

Hope this works.

StormJH1

I didn't know corrupting SD cards during the backup process was a thing. Yikes, one more thing to worry about.

This is an interesting alternative way to do backups - one thing it might do is provide another workaround for the problem where SD card images grow to the size of the card being cloned, and won't necessarily fit on other cards of the same "size" (particularly different brands).

If this works as advertised, then I should be able to just do a fresh RetroPie install on any new SD card (even a smaller one, theoretically), and then transfer the date over using SFTP. So long as the data itself doesn't exceed the size of the new SD card, it should work.

A few questions though:

(1) What is the rough size of all those files? Is it just the sum of all its parts (ROMs, cover art, config information), or do the files themselves also "grow" to the size of the available storage?

(2) Does this retain ALL the setup information? Will my controllers just automatically work like they did before?

(3) In @mediamogul 's list of folders to copy, he used an external thumb drive for ROMS. If I have my RetroPie folder just on the microSD card (like I assume the majority of us do), can I backup and re-transfer that just the same? IIRC, the "RetroPie" folder contains roms, gamelist, boxart files, and metadata, so shouldn't that all still work in the new transfer?

I kinda want to try this just to see if I can make a functional backup. I have a 32GB off-brand microSD card. I couldn't do straight image copy from my SanDisk 32GB card because the SanDisk had slightly larger capacity. The "fresh RetroPie install followed by copying backed up folders" method might work!