Copying files from SD card to USB key?

jca2112

I have a 128gb microSD card full of roms, files, etc. I'm using the Automount method described here:

to automatically copy the RetroPie folder and all of its contents from the SD to a 128GB USB key. I inserted the USB with the added retropie-mount folder and rebooted the Pi to start the copy script.

How do you see the status of that automatic copying of files to USB to find out when it is finished?

I used the htop function on the command line, and I see the rsync -rtu /home/pi/RetroPie/ /media/usb0 process is running -- I assume that is the file copy process? The Pi has been running for over a day with that process still going. Is there a better way to determine when the copy process is done so I can remove the files from the internal microSD card?

Appreciate any help/info.

-jca2112

Clyde

@jca2112 I don't know of a way to see the ongoing progress, but you can compare the occupancy of the source and destination folders with the du command (disk usage) to see how much has been copied yet.

du -s /home/pi/Retropie
du -s /media/usb0

The option -s stands for summarize. You can add -h for a more human readable output (-s -h or -sh).

jca2112

@clyde Thanks that is helpful. (The du for the usb took quite a while). I'm wondering if there is a method to check rsync status after the fact -- i see lots of methods (piping output, etc.) when first running rsync, but not afterwards as it's running.

jca2112

It looks like watch can show what a current process is doing:

sudo watch ls -l /proc/###/fd/1

where ### is the process id number. So I put in the pid of the rsync -- not sure if the copy script is running more than one active rsync process?

jca2112

Also, once this USB copy script has finished, is it safe to delete the entire /home/pi/RetroPie/ folder? (Or do I need to leave the /RetroPie folder structure, and just delete out the rom files, etc, leaving empty folders?)

mitu

@jca2112 Delete the ROM files, but leave the ROM folder structure in place. Technically, you only need the /home/pi/RetroPie/roms folder to exist, but I'd leave the systems' folders there also, just in case you'll need it.

Clyde

Another way to somewhat show rsync's progress after invocation: iotop

It's like htop, but shows disk IO. You can limit the output to either only active processes by -o or to only one specific processs by -p <process number>. The -a option will show the acumulated data instead of bandwith. So, if rsync is process 1234, you should be able track its copied amount since you started iotop this way:

sudo iotop -ap 1234

iotop isn't installed by default on Retropie. You can install it from the official repos:

sudo apt install iotop

Just for your and others' information.

timekills

Thread necro, but an easy way to watch any already in progress action, including cp, mv, rsync, etc. is using the command line tool called "progress" (appropriately.)

I put the simple instructions on the unix/linux stackexchange page below. I used it for exactly this situation, when transferring 100+GB of ROMS/videos/etc. takes a long time and you have no idea if it's still going or what the status is.

https://unix.stackexchange.com/a/581462/407699

Clyde

@timekills Thanks for that! Even after 13 years of Linux, I didn't know of this program. For others who might be interested, here's a more elaborate guide about progress:

https://www.tecmint.com/progress-monitor-check-progress-of-linux-commands/