RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login

    a gamelist.xml cleaner tool

    Scheduled Pinned Locked Moved Ideas and Development
    gamelist.xmlscript
    56 Posts 21 Posters 26.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • meleuM
      meleu @TMNTturtlguy
      last edited by

      @TMNTturtlguy the script does exactly what it is saying it does: check if the rom exists (the file in <path> field) and generate a cleaner gamelist only with <game> entries with valid <path>s. (By valid I mean existing files)

      The script wouldn't change any of the examples you gave.

      Give the script a try. It doesn't change your gamelist, but creates a cleaner one. ;-)

      • Useful topics
      • joystick-selection tool
      • rpie-art tool
      • achievements I made
      1 Reply Last reply Reply Quote 1
      • meleuM
        meleu
        last edited by meleu

        NOTE: the script doesn't work if the gamelist.xml has invalid xml characters. Such as & in Track & Field. I will try to take a look at it soon.

        Fortunately I implemented the --update feature, then the users can always try to run the script with that argument to get the latest version :-)

        EDIT: HA! Looks like EmulationStation auto fix the gamelist.xml files with invalid xml characters as soon as it needs to update a gamelist.xml file. Then I'm not worried about this issue for a while! :D

        • Useful topics
        • joystick-selection tool
        • rpie-art tool
        • achievements I made
        theWinterDojerT hansolo77H 2 Replies Last reply Reply Quote 1
        • theWinterDojerT
          theWinterDojer @meleu
          last edited by theWinterDojer

          @meleu Awesome! I was looking for this and didn't know it existed. I'm going to give it a try tonight, thank you.

          How possible do you think it would be to do the same thing but for downloaded_images? I have duplicates of images from multiple scrapes taking up space but there is no way I can go through them all. I'd be happy to test it :D

          meleuM 1 Reply Last reply Reply Quote 1
          • meleuM
            meleu @theWinterDojer
            last edited by

            @thewinterdojer it's a fair request. Will implement it as time permits. :-)

            • Useful topics
            • joystick-selection tool
            • rpie-art tool
            • achievements I made
            1 Reply Last reply Reply Quote 1
            • SanoS
              Sano
              last edited by Sano

              A (very) quick and (extremely) dirty script I just made to clean up images after your script, @meleu :

              for sys in .emulationstation/downloaded_images/*
                      do
                      sys=`echo $sys|awk -F "/" '{print $NF}'`
                      for file in .emulationstation/downloaded_images/$sys/*
                              do
                              cfile=`echo $file | sed s/\&/\&amp\;/g`
                              if ! grep -q "$cfile" .emulationstation/gamelists/$sys/gamelist.xml
                                      then
                                      echo deleting $file
                                      # rm "$file"
                              fi
                      done
              done
              
              theWinterDojerT 1 Reply Last reply Reply Quote 3
              • theWinterDojerT
                theWinterDojer @Sano
                last edited by

                @sano Awesome, so what exactly does this do? Will it just delete the images from the downloaded_images folder, or create a new clean folder like Meleu's script?

                SanoS 1 Reply Last reply Reply Quote 0
                • SanoS
                  Sano @theWinterDojer
                  last edited by Sano

                  @thewinterdojer It's really quick and dirty, so if you don't understand it, you should wait for @meleu to add this feature in his script in a cleaner manner ;)
                  For this reason I didn't put the bash header, and commented the rm line.
                  It was just a proof of concept to help @meleu (but I think he will come with a cleverer solution anyway).

                  Basically, after @meleu script, all games listed in gamelist.xml files are existing games.

                  This script just browse the downloaded_images folder for each system, and verify that the images are used in the corresponding gamelist.xml file.
                  If an image is not used, it prints "deleting image" (and delete it only if the rm line is uncommented).

                  theWinterDojerT meleuM 2 Replies Last reply Reply Quote 3
                  • theWinterDojerT
                    theWinterDojer @Sano
                    last edited by

                    @sano Gotcha, I understand the concept of how it works, but I don't understand how to tell Linux that. Good stuff.

                    1 Reply Last reply Reply Quote 1
                    • OmnijaO
                      Omnija
                      last edited by Omnija

                      Awww sweet, I've always hated the ES logs telling me how games are missing... and NOW "THEY SHALL BE GONE". :D

                      Considering im always looking for means of saving space, that other scripts to clean up images sounds great.

                      Systems: Raspberry Pi 2
                      Overclocked: 1050 | 525 | 350 | 425 | 256
                      My Themes: Metro Theme | Simpler Turtlepi | Future Mini

                      1 Reply Last reply Reply Quote 1
                      • meleuM
                        meleu @Sano
                        last edited by meleu

                        @sano thanks for the inspiration. It really helps! ;-)
                        I'll implement that when I have a chance.

                        • Useful topics
                        • joystick-selection tool
                        • rpie-art tool
                        • achievements I made
                        SanoS 1 Reply Last reply Reply Quote 4
                        • SanoS
                          Sano @meleu
                          last edited by

                          @meleu said in a gamelist.xml cleaner tool:

                          @sano thanks for the inspiration. It really helps! ;-)

                          According to you bash skills, I don't think it's the truth ;)
                          Nevertheless, I had fun forgetting work for 10min, and doing a bash script for pleasure.

                          1 Reply Last reply Reply Quote 1
                          • AndersHPA
                            AndersHP
                            last edited by

                            This is really cool! Good work, guys!

                            My "Bubble Bobble" Themed Bartop Arcade
                            My Gameboy

                            1 Reply Last reply Reply Quote 0
                            • KaltinrilK
                              Kaltinril
                              last edited by Kaltinril

                              Suggestion:

                              Script should clean up all related items to the file (As @Sano has suggested with his images deletion).

                              Change default behavior to:

                              1. Copy gamelist.xml to gameslist<DATE>.xml
                              2. Modify the gamelist.xml file directly
                              3. Move all images/video's into a sub-folder "Unused"

                              New Option: -r (remove)
                              Removes any files in the Unused folder (If supplied during the initial scan, does not move files to Unused, just deletes them)

                              The default behavior would mean if you have 15 systems, you have no work to do, but you still have the "backup" file.
                              The new option means if you are confident or don't care, after running the cleanup script a user would have nothing to do.

                              Thoughts:

                              1. Do people cross link images/movies for 1 system to the same game in another system or folder? If so, the script would need to check for ANY/ALL occurrences or references to the image/movie.
                              2. If a user provides this new -r option, should a backup of gameslist.xml be created at all?
                              3. Perhaps create multiple other shell scripts for the separate functions, and a master script that calls them.
                                A) gamelist-cleanup-master.sh - orchestrates the cleanup process by calling all sub-scripts and passing in parameters
                                B) gamelist-cleaner.sh - Does what is has always done.
                                C) downloaded-images-cleaner.sh - Would remove un-linked images and movies if no reference is found in any of the XML files
                              meleuM 1 Reply Last reply Reply Quote 2
                              • meleuM
                                meleu @Kaltinril
                                last edited by

                                @kaltinril oh, I forgot this tool on my TODO list. :-)

                                Will try to improve it next week. I'm currently pretty busy.

                                • Useful topics
                                • joystick-selection tool
                                • rpie-art tool
                                • achievements I made
                                theWinterDojerT 1 Reply Last reply Reply Quote 1
                                • theWinterDojerT
                                  theWinterDojer @meleu
                                  last edited by

                                  @meleu I ended up manually going through my SNES folder and deleting the unused images. Took a few days, I'm gonna go ahead and patiently wait for you to do the other ones haha. It's not a space issue or anything, I'm just a perfectionist and don't want any unnecessary files on my machine.

                                  Thanks for all the other tools, and really no rush. Real life takes precedent to our Pi's :)

                                  1 Reply Last reply Reply Quote 1
                                  • KaltinrilK
                                    Kaltinril
                                    last edited by

                                    Going to clone and edit it and then send a pull request.

                                    You can accept whatever you want, or reject it all :)

                                    1 Reply Last reply Reply Quote 1
                                    • KaltinrilK
                                      Kaltinril
                                      last edited by

                                      I've got the -r working. I also added -a for "all" so that I can just type this:
                                      ./gamelist-cleaner.sh -a -r

                                      and it cleaned up all folders and replaces the gamelist.xml instead of creating the -clean version.

                                      https://github.com/kaltinril/share

                                      I want to do some more testing before I submit a pull request.

                                      meleuM 1 Reply Last reply Reply Quote 1
                                      • meleuM
                                        meleu @Kaltinril
                                        last edited by

                                        @kaltinril you can submit a PR and then we can discuss about the code on github. ;-)

                                        • Useful topics
                                        • joystick-selection tool
                                        • rpie-art tool
                                        • achievements I made
                                        KaltinrilK 1 Reply Last reply Reply Quote 2
                                        • KaltinrilK
                                          Kaltinril @meleu
                                          last edited by

                                          @meleu Submitted.

                                          theWinterDojerT 1 Reply Last reply Reply Quote 2
                                          • theWinterDojerT
                                            theWinterDojer @Kaltinril
                                            last edited by

                                            @kaltinril

                                            Is this ready to be used?

                                            meleuM 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.