RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login
    Please do not post a support request without first reading and following the advice in https://retropie.org.uk/forum/topic/3/read-this-first

    ensuring ES gracefully finish and save metadata in every system shutdown

    Scheduled Pinned Locked Moved Help and Support
    shutdown scriptemulationstatiofavoriteslast playedmetadata issues
    96 Posts 26 Posters 37.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • YahmezY
      Yahmez @meleu
      last edited by

      @meleu God, I miss that guy.

      1 Reply Last reply Reply Quote 0
      • dankcushionsD
        dankcushions Global Moderator @stoo
        last edited by

        @stoo said in ensuring ES gracefully finish and save metadata in every system shutdown:

        Does ES only save metadata on shutdown?

        Wouldn't it make sense to just flush metadata to storage every X minutes if there are changes? At least then you only lose very recent changes if ES goes down "unclean".

        most stuff in retropie only saves on exit - game saves will only save to disk on exit of a game. retroarch settings only save on exit of retroarch, etc.

        when you're dealing with SD card storage, you generally don't want to do multiple writes; it will kill your card quicker, especially with something as intense as an ES xml write.

        meleuM stooS 2 Replies Last reply Reply Quote 0
        • S
          stet
          last edited by

          I just wanted to put in my thanks for putting this together. I use a RemotePi board to power down my RetroPie setup and this solved the problem of ES not saving the favorites list. :)

          1 Reply Last reply Reply Quote 0
          • meleuM
            meleu @dankcushions
            last edited by

            @dankcushions You just gave me an idea for a feature request: an option on the ES menu to Save Current Metadata (something like the "Save Current Configuration" on RetroArch) Then a careful user can do it to prevent loss of data.

            My suggestion for such option: in the OPTIONS menu (the one that appears when the users press Select).

            @Zigurana @Hex @pjft @jdrassa and other ES hackers, do you guys think it's feasible?

            • Useful topics
            • joystick-selection tool
            • rpie-art tool
            • achievements I made
            dankcushionsD 1 Reply Last reply Reply Quote 0
            • stooS
              stoo @dankcushions
              last edited by stoo

              @dankcushions Hey Dank, thanks for your input!

              Before I begin, I'm not 100% sure about the behaviour of ES (and many other things!). I don't actually know if scrapers write the XML files directly at runtime or if they let ES handle it. The following assumes that ES does it:

              A change like I suggested wouldn't make much difference in terms of number of writes. If it were up to me, ES would write out metadata on a - let's say 10 mins for argument - schedule unless a scraper is running (no point writing mid-scrape) and only if there are changes to write.

              I don't think ES records things like duration spent in a game or number of game launches, so the only data written would be:

              1. Scraped data (which is going to be written on shutdown anyway)
              2. Data the user has explicitly changed e.g. added favourites (which is going to written on shutdown anyway)

              In this situation, writing before shutdown and writing at shutdown are the same, the only difference is that we mitigate the case where user metadata is lost when ES is killed before it gets a chance to write it. From a quick glance at the source it appears to me (an almost total non-coder) that ES only writes if there are changes anyway.

              (Incidentally, I wrote a tiny change for ScummVM to help scrapers ID ScummVM titles by hash. The PR I submitted had .svm files write only if they didn't exist or were empty. The PR was changed to write them all out every time ScummVM launches. Why? No idea.)

              @meleu I had a similar thought :)

              1 Reply Last reply Reply Quote 0
              • HexH
                Hex
                last edited by

                @meleu Cant the MD be saved while exiting from any game? That is the only time MD needs to be updated. The other part would be when MD is manually editted.

                Delaying the MD saving can help with minimal writes to the SD card but this problem happens with GPIO shutdown. Would it be more useful to have the script that handles safe shutdown signal ES that the System is going down and it should perform cleanup. Another option would be to have a service that handles the MD saving only and let it handle the clean shutdown.

                My recommendation is to update the safe shutdown script to first send sig_term to ES and thus enabling safe storage of MD.

                Sent from 20,000 leagues under the sea.

                Powersaver Emulation station : https://github.com/hex007/EmulationStation
                ES dev script : https://github.com/hex007/es-dev/blob/master/es-tests.sh

                meleuM 1 Reply Last reply Reply Quote 0
                • meleuM
                  meleu @Hex
                  last edited by

                  @hex said in ensuring ES gracefully finish and save metadata in every system shutdown:

                  Would it be more useful to have the script that handles safe shutdown signal ES that the System is going down and it should perform cleanup. Another option would be to have a service that handles the MD saving only and let it handle the clean shutdown.

                  eh... pretty much what this whole thread is all about. :D

                  • Useful topics
                  • joystick-selection tool
                  • rpie-art tool
                  • achievements I made
                  1 Reply Last reply Reply Quote 0
                  • G
                    G30FF
                    last edited by

                    I have some new information for you.

                    After doing a bit of research, I tried to test out the killes service. If I SSH into a terminal session and run "sudo systemctl stop killes" while the killes service is running, EmulationStation will quit. The next time I start up EmulationStation, the metadata was saved correctly.

                    So the service does work, and IS invoking the shell script to kill EmulationStation when the service stops. However, it seems that the service is stopping AFTER EmulationStation is already killed. For reference, I updated my RetroPie install along with the kernel tonight to see if it would help.

                    1 Reply Last reply Reply Quote 1
                    • dankcushionsD
                      dankcushions Global Moderator @meleu
                      last edited by

                      @meleu said in ensuring ES gracefully finish and save metadata in every system shutdown:

                      @dankcushions You just gave me an idea for a feature request: an option on the ES menu to Save Current Metadata (something like the "Save Current Configuration" on RetroArch) Then a careful user can do it to prevent loss of data.

                      My suggestion for such option: in the OPTIONS menu (the one that appears when the users press Select).

                      @Zigurana @Hex @pjft @jdrassa and other ES hackers, do you guys think it's feasible?

                      doesn't restarting emulationstation accomplish the same goal? seems like a redundant option to me.

                      cyperghostC 1 Reply Last reply Reply Quote 0
                      • cyperghostC
                        cyperghost @dankcushions
                        last edited by

                        @dankcushions
                        I totally agree, most of these "errors" seems to be somehow user made. The service should work and if somehow the power is cut then it's a hardware error and not software.

                        @meleu
                        The only thing that can be improved imho is the emulator detection routine. The detection of ES PID is done 100% and I think there is no more improvement to do in ES itself. The ES restart via options menu is exactly the feature you mentioned.

                        mediamogulM 1 Reply Last reply Reply Quote 0
                        • mediamogulM
                          mediamogul Global Moderator @cyperghost
                          last edited by

                          @cyperghost said in ensuring ES gracefully finish and save metadata in every system shutdown:

                          The detection of ES PID is done 100%

                          This is as good a time as any to thank you guys for putting this together. I dual boot between RetroPie and OSMC and a while back I used part of this to create a script that launches from the ES Kodi system menu that allows ES to shut down and save all MD properly before rebooting into OSMC. This was a very welcome addition after the update to 'favorites', 'last played', etc. Very keen and thanks again.

                          RetroPie v4.5 • RPi3 Model B • 5.1V 2.5A PSU • 16GB SanDisk microSD • 512GB External Drive

                          caver01C 1 Reply Last reply Reply Quote 1
                          • caver01C
                            caver01 @mediamogul
                            last edited by caver01

                            @mediamogul quick side note here if I may: what were the broad steps you followed to build your OSMC dual boot? I have Kodi setup, but I really do prefer the OSMC flavor. Always have.

                            My 4-player cocktail style cabinet built as a custom "roadcase"

                            mediamogulM 1 Reply Last reply Reply Quote 0
                            • mediamogulM
                              mediamogul Global Moderator @caver01
                              last edited by mediamogul

                              @caver01

                              NOOBS was used for the setup I'm currently using. Since both sides will need to be redone for Stretch, I plan to use PINN to replace it in the months to come, as it offers RetroPie as one of it's default install options. As you may be aware, using NOOBS for this involves converting a RetroPie SD image to be used. After that, it's just a matter of booting it up and selecting RetroPie to install alongside the already available OSMC option.

                              When it's all said and done, you either have the option of using a dual boot selection screen every time the Pi starts, or you can take a page out of the Multi Boot Pi book like I do and use a custom addon for Kodi that launches RetroPie, along with a custom script in RetroPie to return to OSMC. All that said, besides mentioning PINN as an option, I'm probably burying the lead in that you could just get an install image from the Multiboot Pi page with OSMC and RetroPie already configured for this. While it will install older versions of both, they can of course be updated independently as they would normally.

                              RetroPie v4.5 • RPi3 Model B • 5.1V 2.5A PSU • 16GB SanDisk microSD • 512GB External Drive

                              1 Reply Last reply Reply Quote 1
                              • F
                                Fitz2380
                                last edited by

                                @meleu I am still not able to get the metadata to save from my process. Is there something I could look at to help troubleshoot the issue.

                                I have verified that the killes.service is enabled and therefore running.

                                I can execute the killes.sh script from the command line and emulation station processes do terminate. After I power down and backup, the metadata has been saved.

                                My issue is when I execute the 'shutdown' command from within emulation station as a menu item. The command performed is the 'shutdown now' command, but the meta data is never saved when I execute it this way.

                                Any thoughts that would help me debug would be great.

                                1 Reply Last reply Reply Quote 0
                                • S
                                  Semper 5
                                  last edited by

                                  Any updates on the MD not saving?

                                  I've made no progress. MD only saves when performing shutdowns or restarts within ES menu....not button :/

                                  1 Reply Last reply Reply Quote 0
                                  • F
                                    Fitz2380
                                    last edited by

                                    I just wanted to update my previous post as I did not include the proper information before. I am running Retropie v4.3, RPi3 Model B, 5v @2.5A PSU, 32GB SandDisk Microsd.

                                    I have installed the killes service and installed the killes.sh script. I can successfully save metadata by running the script from the command line. I cannot save metadata if I run a script that does the 'shutdown now' command which is executed from the main retropie menu as was posted here: https://retropie.org.uk/forum/topic/10736/power-on-off-option-in-main-menu/3

                                    Additionally how do you put your system information into a footer in your post

                                    1 Reply Last reply Reply Quote 0
                                    • B
                                      Bananu @Yahmez
                                      last edited by

                                      @yahmez

                                      Hi! I´m on your previous situation: Modded Nespi Case soft shutdown, following PDF (working GREAT) and after some iso backups, I installed the killes.sh, but is not working... adding some favs and pressing power down button, and after power it up, favs didn´t save.

                                      I can see killes.service enabled from :~ $ systemctl status killes
                                      ● killes.service - Kill EmulationStation
                                      Loaded: loaded (/etc/systemd/system/killes.service; enabled)
                                      Active: active (exited) since Sat 2018-03-24 10:05:18 UTC; 3min 46s ago

                                      Is the fresh install your only way to go? For me, it would be a nightmare re-install 30gb of roms and ports, gamepad config, overclock... I´m making backups every now and then just to not repeat what it´s done (like 6 months of work here)

                                      So... how can I emulate the "fresh install" on my scripts? is something I should wipe from systemctl or something?

                                      Thank you!

                                      YahmezY 1 Reply Last reply Reply Quote 1
                                      • YahmezY
                                        Yahmez @Bananu
                                        last edited by

                                        @bananu
                                        Sorry man, I really don't know what is causing this issue for people. Especially because the killes script worked for me both on a system I had already set up completely, and with a fresh install. Wish I could be more of a help here...

                                        1 Reply Last reply Reply Quote 0
                                        • J
                                          julenvitoria Banned
                                          last edited by

                                          @meleu HI! I'm trying to make the gracefully finish to work but I can't and I don't know why... I follow all the steps even create the script in the same path than you (/etc) but it does not work. I'm using a raspberry pi 3 with the last version of retropie and shutdown script with a button connected to GPIO 3. Could you guide me about this?
                                          Thanks

                                          1 Reply Last reply Reply Quote 0
                                          • QuackwalksQ
                                            Quackwalks
                                            last edited by

                                            When using this with the Kintaro 9000 case, the power switch doesn't respond but the reset switch does. Using the reset switch won't save metadata. If anyone has experience with this, I'd love some help.

                                            Pi Model or other hardware: Pi 3b
                                            Power Supply used: 5.1v 2.5a
                                            RetroPie Version Used: 4.3.17
                                            Built From: Pre made SD Image on RetroPie website, updated to 4.3.17
                                            USB Devices connected: dongle for wireless keyboard
                                            Controller used: Official PS3 Controller
                                            How to replicate the problem: Followed install instructions from Kintaro 9000, then followed instructions in this post

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.