RetroArch's version is pinned in the install script, installing from source will still get you the pinned version. If you update your RetroPie-Setup script, the latest version available will be 1.9.7.
If you wish to build the latest master or and another version, you need to modify the retroarch package script and change the branch used here. Note that RetroPie applies some patches to the source (here), if they don't apply correctly to the version you want to build, you'll need to comment the patches before installing from source.
@tripplies we don't support third party installs, but to get the original games off you would need to transfer them off. see: https://retropie.org.uk/docs/Transferring-Roms/#transferring-roms. essentially you're doing the same process, but in the other direction. samba, SFTP or the manual copy would work. after that, install your new system on your new SD, and transfer them back in the same manner.
i would be reticent to transfer configs and the emulationstation setup as those settings are likely incompatible with current retropie and could cause issues/break it.
@GreenHawk84 I'd definitely back things up every now and then (I do it more or less once a month, but it's more because of my lack of availability than anything else), and keep at least two historical backups (in case you backup something that's already broken, you can always revert to a previous safe backup).
I don't update packages nor emulators unless I need to - if it's working and there's nothing new that I really need there, then I don't risk breaking things. The latest thing I've wanted to update for (but still didn't) was several months back when RetroArch launched the lookahead features to reduce input lag. That's one of the key things I'd be looking for. Still, then you read that the latest RA and lr-fbalpha can cause slight frame drops on CPS3 games in certain setups and you struggle to revert those changes if needed.
So, my preference: unless there's a meaningful new feature for you or a critical security patch, I'd stay with what you have that's working for you.
Also, I am a big supporter of keeping the ROMs in a USB drive and the OS in the SD Card. Easier to backup, transfer roms, and also to set up fresh in a worst case scenario.
To keep files off the SD card I changed the .emulationstation to link else where. I did the update all option and saw it changed the symbolic link too. I guess the reasoning is to recreate a clean slate for those who have broken their existing setup.
It's probably me but to me it looks like @Clyde and @cyperghost are saying two different things?
Not really, we more of complemented each other. @cyperghost's comment about open ports in the router's firewall are valid, though they should be closed in the router's initial factory setup. It doesn't hurt to check a router's setup for open ports, though, especially if multiple people have admin access to it, or the device was bought pre-owned.
It also doesn't hurt to read the tech news about vulnerabilites in common routers, or an occational web search for one's own router model in that regard.
Finally, the advice to set strong passwords also applies to one's router, of course, lest it may be manipulated like @cyperghost said. ]:}
... So, in the end, I guess, the main problem with the OP was that the Pi with unchanged credentials was connected to a compromised internet connection? The way I read it as first was that the unchanged credentials was exactly what made the connection an unsafe one but it seems now that there must've been a step or two before that that went awry.
Supposedly yes, but without more details from @RikFlorence, we can only suspect that much.
EDIT Looks like it got past the stall by exiting ES and running the setup manually. Might be something to add to the setup script when updating ES to say something like "This must be ran outside of ES."