ensuring ES gracefully finish and save metadata in every system shutdown

meleu

UPDATE: many guys are reporting that this trick isn't working anymore. Maybe something has changed after an OS update, but I'm not sure. I don't use GPIO buttons to shutdown my RetroPie myself.

I'm currently very busy on other projects and unable to provide support here. Sorry mates...

It's getting common some users coming here to report issues with some ES metadata not being saved, notably Favorites and Last Played. Specially users who use GPIO pins connected to a button to shutdown the whole system (Mausberry, Powerblock, etc).

In order to provide a generic solution to encompass as many users as possible I did some research on how systemd works and I think I've found a nice approach (at least it's working fine for me).

Let's get it started!

goal

Gracefully finish EmulationStation, letting it save all metadata, in every system shutdown. No matter how this shutdown is invoked (it can be by EmulationStation quit menu, command line via SSH, Mausberry button triggering a script, etc).

step 1 - `killes.sh` script

Here is the script responsible to gracefully finish EmulationStation:

#!/bin/bash

# Check if EmulationStation is running. Finish the script if doesn't.
espid="$(pgrep -f "/opt/retropie/supplementary/.*/emulationstation([^.]|$)")" || exit 0

# the "sed" command below isn't a crypted message :), it's just a trick to
# make $emucall regex-safe to use in the "pgrep -f" below.
emucall="$(sed '4!d; s/\([\\"]\|[[:alnum:]_]\+=[^ ]* \)//g; s/[][(){}^$*.|+? ]/\\&/g' /dev/shm/runcommand.info)"

# If there's an emulator running, we need to kill it and go back to ES
if [[ -n "$emucall" ]]; then
    emupid="$(pgrep -f "$emucall" | tr '\n' ' ')"
    pkill -P "$(echo $emupid | tr ' ' ',')"
    kill "$emupid"
    wait "$emupid"
    sleep 5 # maybe it can be lesser
fi

kill "$espid"
wait "$espid"
sleep 5 # maybe it can be lesser

Copy this script and save it where you prefer. I used /etc/killes.sh.

Make sure to set the script as executable:

chmod a+x /etc/killes.sh

step 2 - `killes.service` systemd unit

In order to make that script be executed right before a shutdown, you must create the file /etc/systemd/system/killes.service with this content (Note: this step must be done with super user privileges):

[Unit]
Description=Kill EmulationStation
After=autologin@tty1.service

[Service]
Type=oneshot
RemainAfterExit=true
ExecStop=/etc/killes.sh

[Install]
WantedBy=multi-user.target

Adapt the ExecStop= line with the path to your killes.sh script!

After creating this file you have to perform this command

sudo systemctl enable killes

step 3 - shutdown the system

Add/remove some games to you Favorites, and then shutdown/reboot the system with your fancy power button (or even perform a sudo shutdown -r now via SSH), and after a reboot see if your metadata was saved.

Repeat the same test with an emulator running and see if your metadata was saved.

bonus

How to perform a software shutdown with the Mausberry and the diode/transistor hack

acknowledgements

I've reached this solution thanks to all brainstorm here in the forums. Here are some guys who somehow contributed until I got this solution: @cyperghost @pjft @TMNTturtlguy @hansolo77 @lostless @caver01 @madmodder123 and others.

meleu

@cyperghost

Well, systemd is tricky. IMHO it doesn't follow the KISS principle as most of UNIX tools historically do. Now I understand why it's so criticized. There is even a wiki dedicated to criticize systemd.

Maybe this gif can summarize a little:
systemd_is_hungry

As it's the init system chosen by Debian/Raspbian projects, who am I to criticize it? :P

Oh... Let's get back to your question (as it's not a shell script and we are not messing others people's thread, we can discuss in this thread here :) ).

@cyperghost said in shell scripting topic:

Is tty is setting the trigger to start the service? Don't get me wrong I thought tty is just the terminal like ssh ;) Can you please explain?

I'll try to summarize the explanation first and then will try to explain the lines in the killes.service file.

With systemd the shutdown is the reverse order of startup. Then what I did was to discover what service was launching emulationstation (autologin@tty1.service) and then making the killes.service run right after it. But killes.service does nothing until it receive instruction to stop, and that's when the killes.sh script is launched.

Now, I've added some comments in the .service file to explain what each line does:

[Unit]
Description=Kill EmulationStation
# err... I think Description is quite descriptive. :)

After=autologin@tty1.service
# autologin@tty1.service is the service that at some point will launch
# emulationstation. Putting it in the "After=" directive makes killes.service
# be started after autologin@tty1.service, and consequently makes killes.service
# be stopped before autologin@tty1.service when the system is shutting down.
# This is the effect we want.
# Pretty tricky, huh? I agree. It should be simpler, but it's the systemd world...

[Service]
Type=oneshot
# Indicates that the process will be short-lived and that systemd should wait
# for the process to exit before continuing on with other units. If it's not
# used this way, the killes.sh script would be executed in parallel with other
# stuff that should kill EmulationStation abruptly (not letting it save metadata).

# The ExecStart directive is used to launch something when starting this service,
# but we don't want to launch anything. Therefore I omited the ExecStart directive.

RemainAfterExit=true
# The killes.service should remain active after the process launched by ExecStart
# finishes. We didn't launch anything in ExecStart, remember? ;)

ExecStop=/home/pi/bin/killes.sh
# This is what should be launched when stopping this service. This is what
# happens when the system is shutting down.

[Install]
WantedBy=multi-user.target
# In a not very precise way, we can say that in systemd "targets" are like
# runlevels in System V. The multi-user.target is something like runlevel 3.
# If you don't know what it means, we can simplify it as: in a normal boot
# this service will be started.

If you want to go further, here is a good reading:
https://www.digitalocean.com/community/tutorials/understanding-systemd-units-and-unit-files

The Arch wiki entry for systemd also has some good info: https://wiki.archlinux.org/index.php/Systemd

But I have to be honest: once I accomplished what I wanted, I stopped reading about systemd. :/ I think I agree with many of those critics...

crazydude2

For some reason this didn't seem to work for me over ssh or using my atxraspi circuit. Any idea what i could have done wrong. I don't have much experience using linux. I didn't get any errors.

pjft

@crazydude2 what do you experience? Do you have "save metadata on exit" set to true?

cyperghost

@meleu Thanks for stepping into this I think there also annother approach ;)

https://raw.githubusercontent.com/LowPowerLab/ATX-Raspi/master/shutdownchecksetup.sh
This install the watch GPIO script (Python and BASH - it's your choice!)

and then here the shutdown script... That's very interesting!
https://lowpowerlab.com/guide/atxraspi/full-pi-poweroff-from-software/

It's even using the NPN-hack for software powerdown ;)

So we can restart ES, reboot the system and shutdown ;)
I think the ATX Rasp works same as the mausberry ;)

I don't want to mess this thread ... If there is a reason to step in we open a new thread ;)

crazydude2

@pjft Hi Thanks for getting back to me. I press the button to shutdown and when i reboot it has not saved the meta data. It is set to save. Doing from emulation station works.

cyperghost

@crazydude2 Let us talk here

meleu

@crazydude2 said in ensuring ES gracefully finish and save metadata in every system shutdown:

For some reason this didn't seem to work for me over ssh or using my atxraspi circuit. Any idea what i could have done wrong.

Hello mate. Sorry for the late replay, I was busy with Real Life stuff. I made intensive tests before posting that trick, then I would like to make it work for you.

I think it was my fault, because I didn't explicitly said to the user create the /home/pi/bin directory. It's implicit, but not very clear for inexperienced Linux users.

Can you please paste here the output of these commands:

cat /home/pi/bin/killes.sh

... and paste the output here (don't forget to format your post, like @cyperghost showed you in your thread).

cat /etc/systemd/system/killes.service

... and paste the output here.

Have a nice day! :-)

meleu

Changelog: I've changed the script's path from /home/pi/bin/ to /etc/. Edited OP accordingly.

crazydude2

@meleu Hi Thanks for getting back to me. I have found another solution which works but when i get more time i will set up a new image and give it a try. I am a bit worried about breaking it at the moment.

Yahmez

@meleu
Thanks for all the work you have put into this, I was excited when I was pointed to this thread, sadly this did not work for me though. I am using the Pololu mosfet board in a Retroflag/NESPi case. I followed your steps but it was very odd because when I powered down via the switch the power LED on the case remained lit (faintly) after the pi shut down. It also did not save the meta data when shutting down from ES or from an emulator. Does the script HAVE to be located in /etc/ ?

meleu

@yahmez no. You can put it wherever you want (and adapt your killes.service file accordingly). Make sure to make the script executable (I forgot to mention this in the OP. Will update it soon.).

chmod a+x /path/to/killes.sh

cyperghost

@yahmez Can you please provide your shutdown script? Maybe this gives the kick ;)
@meleu I'm hunting down that rabbit - it's name is BASH

Yahmez

@cyperghost
I thought this would work for any shutdown script? 😜
Here it is: https://pastebin.com/25GKw9xv

Yahmez

@meleu said in ensuring ES gracefully finish and save metadata in every system shutdown:

chmod a+x /path/to/killes.sh

Thank you, this got the script working. Only problem is the case LED remains faintly lit. I cannot figure this out... I mean everything shuts down, the fan turns off, but the LED is still getting a little bit of power. Somehow by using the killes script the mosfet switch stays on. The only thing that can keep it on after the power button is is turned off is a pi GPIO staying high. But then again it's not fully on because the pi and fan are off... it's puzzling. I added a GPIO.cleanup() line to the end of my script but it did not seem to matter.

Yahmez

After 5 minutes the LED fully extinguishes, so I guess this is a non issue even though it perplexes me. Thank you @meleu for working coming up with this graceful exit to ES. I really appreciate it.

meleu

@yahmez said in ensuring ES gracefully finish and save metadata in every system shutdown:

@cyperghost
I thought this would work for any shutdown script? 😜

Yes. This is the intention of my approach here. ;)

I'm glad to know it's now working for you!

Have a nice weekend.

cyperghost

@yahmez Yes the solution from @meleu is working in general (thanks to him!)

... but I just want to take a look how the mass of these scripts are written. I'm currently working on annother way to shutdown and to give the user more control about the Pie.

One disadvantage will be that the shutdown scripts (bash, python or any other language) needs to be modified a bit. On the other hand, there will be no need to add something manually to systemd.

I don't know which method is better and I don't want to judge about. In the end the user decides on his own. The way how this script was born can be read on the first posting. And we can thank @meleu for his long breath and brillance in improve such scripts.

For me: It is always better to have two working ways ;)
Thanks for your help...

meleu

@BuZz do you think it would be valuable to add the trick described in the OP as an option in emulationstation.sh script module?

Summing up what it is: a way to ensure that ES will cleanly exit saving all metadata right before any shutdown/reboot.

If yes I can submit a PR for your evaluation.

EDIT: I'm thinking about an option like "Cleanly exit ES at any shutdown/reboot."

BuZz

@meleu It's an issue with 3rd party reboot scripts? Looking briefly at op, I don't see why this is needed in RetroPie (it's not a RetroPie problem).