[SCRIPT] Setup script to sync saves to cloud services

Newellj79

Very cool!

jandalf81

Just a heads up, I moved the files from DropBox to GitHub:
https://github.com/Jandalf81/rclone_script

I've also written a quite extensive documentation there...

HumanMoose

I want to use this, but every time I install it and reboot my Pi becomes super slow to respond... almost completely unresponsive at boot. I've luckily been able to finally get in the last 2 times and struggle to get it to uninstall with the script and reboot, then things are back to normal. Before I've had it totally bring down my system and had to re-image the card.
I'm using a Pi 3B, not overclocking, and just using the stock 4.4 image that I update just using the retropie-setup update menu... There any log files I can attempt to look at?

jandalf81

@HumanMoose
Yes, the scripts creates log files in it's own directory /home/pi/scripts/rclone_script/rclone_script.log
Unfortunately, I have no idea what could happen here. There's nothing this script installs which starts at boot...?

I'll try to reproduce this. Did you update to 4.4.2 before or after applying this script? Your Pi is a 3B, not 3B+, right?
Can you send the logfiles rclone_script-install.log and rclone_script.log so I can take a look at them?

HumanMoose

@jandalf81
Sorry it's taken me a bit to get back with you, I've been busy...
Yes, my Pi is a 3B not a 3B+, and I updated before running the script.
I just tried reinstalling the script and once I told it to go ahead and reboot, it tanked my system like before. It took a lot longer than before but I was finally able to SSH and SCP into the system, I could only find the rclone_script-install.log in the rclone_script folder, I looked around for the rclone_script.log but couldn't find one anywhere. Looked around the /var/log folder and just got the syslog and messages from there.
What is the best way to send you the log that I do have?

jandalf81

@HumanMoose
Please open a new issue and add the files here: https://github.com/Jandalf81/rclone_script/issues

It seems right that you only found one log file, rclone_script.log is only created when you actually sync the saves upon starting or stopping a ROM. If you did that, there should be a log file in the directory.

I'll try to reproduce it with my Pi now:

fresh install of RetroPie 4.4
update to 4.4.2 via RetroPie Setup
install rclone_script
define a remote for rclone
reboot

Can you please tell me what kind of remote you have configured? Which cloud service do you use? Did you install anything else?

EDIT:
I just played Batman after doing all the steps I mentioned... I don't see any slowdown on my side. Could you check what the htop command gives you? This is kind of like Window's Task manager, it should show you the most expensive processes...

And just to be clear, we are talking about heavy slowdowns right when RetroPie starts, right? We are not talking about starting any game?

HumanMoose

@jandalf81
Well, it looks like I was able to fix it myself somehow....?
Previously when I had been running through the installer, when it gets to the point of asking for the remote base directory on my Dropbox, I was going with retropie. It would create it fine, but I noticed last time that after that and I told it to use the internet as the sync network it seemed to start slowing down my system then.
And the slowdown has been heavy from bootup, it would start loading processes and then when it gets to the splash screens it would throw up error messages at the bottom of the screen that it couldn't load them then after a couple minutes it would be able to start mpeg123 and play my background music, then a few minutes later would give me the Emulation Station load screen and I would just let it sit there and it maybe would finish booting but the menu would still be unresponsive. This is at least 10 minutes into boot. I might be able to SSH into it and if so commands were slow to run, just getting a password prompt takes about 5 seconds instead of being instant.

So this time I uninstalled, rebooted and the system was normal and fine. I started the install process again and this time when it got to asking for the remote base folder on the Dropbox, I left it at RetroArch and it finished up, I didn't notice any slowdown when selecting the internet and rebooted. Since then it has been running fine, I've rebooted several times without issue and was even able to install your other script for the auto save state selector. For whatever reason it seems to not have liked giving the remote folders the retropie name? I'll let you know if I have any other issues, but I seem to be OK now....

jandalf81

@HumanMoose
Thank you very much for this update! It still is strange, though... There is nothing in the script that starts during the installation process or right after boot. I'll try to reproduce that with a fresh install myself.

EDIT:
I just tried the following:

fresh install of RetroPie 4.4.0
updated to RetroPie 4.4.2
installed rclone_script
created a remote to DropBox named "retropie"
reboot
play a game
look at htop

There is no slowdown there. I'm really sorry but I don't see a problem here. Everything on my side looks the way it should.

Carcayu

Sounds pretty interesting. But according to this:

starting a game will trigger RCLONE to download save files for the game from the remote

stopping a game will trigger RCLONE to upload save files for the game to the remote

What happens if I play and save a state out of the network? The next time I start a rom with the rasp connected, It will download and overwrite with the older state?

Also, if I play and save a state out of the network and I don't play it again that rom with the rasp connected, then It will never upload the state? Is there any way to manually update and upload all the states in the raspberry?

jandalf81

Yeah, you got most things right.

Starting or stopping a game will trigger the sync process, but only newer files will be transfered.
If you played a game without internet connection before, your local save is newer than the remote one, so no download should happen.

There's a menu item to trigger a systemwide sync of all save files in both directions. Of course, this also only transfers newer files to the other side. In theory, both sides should contain the newest version of each save afterwards.

Please let me emphasize that I'm a hobby programmer. I make errors. I'll try to help but I cannot guarantee that there'll be no failures ;-)

Carcayu

@jandalf81 thanks a lot! Then I'll download it

robotronman

@jandalf81 Love the script and thank you. My log keeps complaining the gateway cannot be PINGed, even though my pi has access and I can ping my default gateway fine. DNS resolution works fine too.

Looking forward to using this script, as I just lost a major game save on my psx due to my tinkering :)

Thanks agian,
Robo

robotronman

@jandalf81 Just a quick update: Manual full syncs to/from Dropbox work fine, and saw the current network check issues documented on your github page. I added the "return 0" workaround to the script as you mentioned and it now works like a charm! Love the script, very helpful!

Addison

I get the following error in the installation menu and then it kicks me to the command prompt.

1. RCLONE                                                                              
x       1a. Test for RCLONE binary                      [ NOT FOUND   ]                
x       1b. Download RCLONE binary                      [ FAILED      ]

Is this setup script too outdated now?

I finally figured out cloud sharing on my Windows computer and also Android devices today through RetroArch. :)

Would very much love to get that rolling as well in Retropie.

For reference, I did follow the initial instructions found here like suggested:
https://github.com/Jandalf81/rclone_script