Backup images failed. File locations?

jca2112

So BOTH backup images of my 8GB sdcard retropie install fail when trying to clone them onto new 8GB cards.

One backup made via ApplePi-Baker fails, the other backup made via command line (on a Mac) also fails. I tried Etcher as well and it will not burn either image because of a size mismatch.

Both backup versions of the card fail with kernel panics and "VFS: can't mount root fs (179,2)" errors. I've tried all the suggestions listed here and on other sites -- I'm ready to give up on the backups being usable to burn a new clone of my original install.

Jumping through several hoops on my Mac, I can mount the ext partition from one of the backup image files and see all the existing folders/files outside the boot partition. My question is:

What files/folders do I need to copy over to a fresh install of Retropie to retain all my settings, controllers, tweaks, images, media, overlays, changes, etc.?

All of my roms are stored on an external usb drive via the retropie-mount folder, so I'm covered on the actual roms, etc. It's just everything else spread out through all those other folders on the sdcard.

Any help on recreating all this from scratch would be appreciated.

mediamogul

@jca2112

What files/folders do I need to copy over to a fresh install of Retropie to retain all my settings, controllers, tweaks, images, media, overlays, changes, etc.?

It really depends on what all you've altered. However, the following should cover everything.

  /boot/config.txt
  /etc/fstab
  /etc/udev/rules.d
  /etc/rc.local
  /etc/samba/smb.conf
  /opt/retropie/configs
  /home/pi/.config
  /home/pi/.local

The contents of /boot/config.txt, /etc/fstab, /etc/udev/rules.d, /etc/rc.local, /etc/samba/smb.conf should be added to the existing files of a new install manually to avoid possible permissions problems.

If you happen to run ioQuake3 and/or OpenMSX, you should also backup:

  /home/pi/.openMSX
  /home/pi/.q3a

jca2112

@mediamogul thanks for the info -- this helps out a lot. I don't know how often threads get a sticky, but this info may be valuable to someone else.

Also, if anyone has any recommendations on how to successfully copy/clone then burn a Retropie image back to an sdcard, please let me know. (I can say to avoid ApplePi-Baker in my experience. The command line version has never worked for me as well.)

Thanks.

pjft

@jca2112 I have used ApplePi-Baker with success in the past, though the most recent attempt did fail. Did you per chance choose to backup to a zip file? If so, the way I got it to restore was to unzip the image, rename the file extension (it'll be named "-", so I named it "-.img") and then restore from the unzipped image.

That worked for me at least. See if it helps.

pjft

@jca2112 Alternatively, I can only speak wonders of rpi-clone, though you're backing up to another card, and need a USB-to-SD-card adapter to run. You also should remove the ROMs USB drive before attempting it.

mediamogul

@jca2112 said in Backup images failed. File locations?:

I don't know how often threads get a sticky, but this info may be valuable to someone else.

It's probably best that we only officially reference disk image backups for simplicity sake. However, I personally use a script to backup the directories listed above and have restored them on numerous occasions without issue. Two major benefits are easy access to the backed up files and of course the space that's saved. All the changes I've made in the past two years, as well as all my game saves and scraped artwork for over 10,000 games only takes up around 500MB. I have the script set to backup up automatically once a week, but it can also be activated manually from Emulation Station and it's configurable to only keep so many previous backups. Right now, I have it set to four, which is admittedly overkill, but you never can be too careful ;).

jca2112

@mediamogul said in Backup images failed. File locations?:

However, I personally use a script to backup the directories listed above and have restored them on numerous occasions without issue.

Care to share that script? :)

mediamogul

@jca2112 said in Backup images failed. File locations?:

Care to share that script? :)

Sure. You'll notice there's also a function that creates a text file in the backup folder that lists everything installed at the time of backup. This comes in handy for reference on new installs, as you can just go down the list and add anything not already installed by default without having to remember what all you had before.

#!/bin/bash

##Set backup directory name based on timestamp

DIRNAME=$(date +%Y%m%d%H%M%S)

##Create backup directory structure

mkdir -p /media/usb0/Backups/RetroPie/"$DIRNAME"/backup

##Backup files and folders

sudo cp --parents -rfv \
  /boot/config.txt \
  /etc/fstab \
  /etc/udev/rules.d \
  /etc/rc.local \
  /etc/samba/smb.conf \
  /opt/retropie/configs \
  /home/pi/.config \
  /home/pi/.local \
  /home/pi/.openMSX \
  /home/pi/.q3a \
  /home/pi/RetroPie/roms/ports/*.sh \
  /media/usb0/Backups/RetroPie/"$DIRNAME"/backup

##Backup specified file types recursively from specified locations

sudo rsync -Ravm --include='*.srm*' -f 'hide,! */' /home/pi/RetroPie/roms/ /media/usb0/Backups/RetroPie/"$DIRNAME"/backup/
sudo rsync -Ravm --include='*.cfg*' -f 'hide,! */' /home/pi/RetroPie/roms/ /media/usb0/Backups/RetroPie/"$DIRNAME"/backup/
sudo rsync -Ravm --include='*.state*' -f 'hide,! */' /home/pi/RetroPie/roms/ /media/usb0/Backups/RetroPie/"$DIRNAME"/backup/
sudo rsync -Ravm --include='*.png*' -f 'hide,! */' /home/pi/RetroPie/roms/*/overlays/ /media/usb0/Backups/RetroPie/"$DIRNAME"/backup/
sudo rsync -Ravm --include='*.bat*' -f 'hide,! */' /home/pi/RetroPie/roms/pc/ /media/usb0/Backups/RetroPie/"$DIRNAME"/backup/

##Create text file listing installed components

cd /media/usb0/Backups/RetroPie/"$DIRNAME"/ || exit

INSTALLED="./installed.txt"

touch $INSTALLED

echo >> $INSTALLED "emulators:"
echo >> $INSTALLED
ls >> $INSTALLED /opt/retropie/emulators/
echo >> $INSTALLED

echo >> $INSTALLED "libretrocores:"
echo >> $INSTALLED
ls >> $INSTALLED /opt/retropie/libretrocores/
echo >> $INSTALLED

echo >> $INSTALLED "ports:"
echo >> $INSTALLED
ls >> $INSTALLED /opt/retropie/ports/
echo >> $INSTALLED

echo >> $INSTALLED "supplementary:"
echo >> $INSTALLED
ls >> $INSTALLED /opt/retropie/supplementary/
echo >> $INSTALLED

##Keep only specified number of backups. Change numeral in "head -n -*" to however many backups are to be kept.

cd /media/usb0/Backups/RetroPie/  || exit

ls -1tr | head -n -4 | xargs -d '\n' rm -rfv --

##Clean up onscreen text

clear

jca2112

Copying things over manually isn't working, I'm sorry to say. I'm mounting an .img of the original retropie backup (that won't work when burned to an sd card) using ext4fuse on the Mac. (To see the ext4 partition of the img.)

I'm then trying to use SFTP to copy over files/folders from that mounted img over to the network to the pi. Unfortunately a lot of folders/files, such as many folders in /opt/retropie/configs/all are locked and have no read/write permissions on the Mac. (I assume because they are owned by the pi user?)

Not sure how to proceed, but it's becoming frustrating dealing with the Pi via a Mac.

mediamogul

@jca2112 said in Backup images failed. File locations?:

it's becoming frustrating dealing with the Pi via a Mac.

I primarily use a Mac myself, but I find it best to eliminate it from the equation as much as possible. I always backup to the same external drive I keep my ROMs on and only use the Mac for SSH access to issue commands remotely. Also, with all the garbage files the Mac Finder creates, I've gotten to the point where I use the terminal for local transfers to thumb drives to avoid the headaches. Overall, Macs just don't play well with other systems in my experience.

weirdocollector

@mediamogul said in Backup images failed. File locations?:

@jca2112

What files/folders do I need to copy over to a fresh install of Retropie to retain all my settings, controllers, tweaks, images, media, overlays, changes, etc.?

It really depends on what all you've altered. However, the following should cover everything.
  /boot/config.txt
  /etc/fstab
  /etc/udev/rules.d
  /etc/rc.local
  /etc/samba/smb.conf
  /opt/retropie/configs
  /home/pi/.config
  /home/pi/.local
The contents of /boot/config.txt, /etc/fstab, /etc/udev/rules.d, /etc/rc.local, /etc/samba/smb.conf should be added to the existing files of a new install manually to avoid possible permissions problems.

If you happen to run ioQuake3 and/or OpenMSX, you should also backup:
  /home/pi/.openMSX
  /home/pi/.q3a

Sorry to exhume this old thread with what is probably a stupid question.

How can I 'manually' add the content of my old files to the new ones indicated above on the fresh install?

Those files seems to be owned by 'root' and do not have the permission to change them as 'pi' user.

The same problem presents itself when I try to change the files directly on the SD card on a Linux system.

Can someone help me?

Thanks!

mediamogul

@weirdocollector said in Backup images failed. File locations?:

How can I 'manually' add the content of my old files to the new ones indicated above on the fresh install?

I always install the emulators I was using from the RetroPie-Setup first. Then, assuming my backup is recent, I copy over my previous configs all in one go with:

cp -rfv /path/to/backup/configs /opt/retropie/

Everything else, I like to compare file differences and add the content manually to the new files using either vi or nano.

weirdocollector

Thanks!!