RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login

    How do you edit the gamelist.xml manually?

    Scheduled Pinned Locked Moved General Discussion and Gaming
    gamelist.xmlscraper
    20 Posts 6 Posters 6.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • thelostsoulT
      thelostsoul
      last edited by

      For starters: "what is the gamelist.xml?": gamelist.xml is the "database" file with all the information for the scraped games. Usually you don't need to edit it manually. Using a scraper tool will do all the needed work to add new games or update existing entries. However, sometimes games in special cases are not known to the scraper tool or database it uses. These cases are in example adding new romhacks. Then I have to edit the gamelist.xml and add all files manually. So, don't edit the file, if you don't know how to work with it.

      My actual question is, which tools do you use, if any to edit the gamelist.xml properly? All the time until now I did it with a plain text editor and simple copy and paste. And everytime I use a scraper and let the file generate, all changes are gone. So I end up having a second file with all the <game> entries and put them back after the scraping process. Are there any common tools?

      πŸ“œ RE/SET: 100 SNES Games for your RetroPie, 🎁 Share your hidden gems and insider tips

      WeirdHW 1 Reply Last reply Reply Quote 1
      • Impman66I
        Impman66
        last edited by

        I use programmers notepad. Which is free to download and use, and it allows you to open and edit a number of other stuff too.

        Have a look

        link to Programmers Notepad

        Pi Model or other hardware: Raspberry Pi 3 Model B
        Power Supply used: Generic 5v 2.5A Panasonic
        RetroPie Version Used : 4.5.1
        Built From: Retropie website
        USB Devices connected: 2 x Dragonrise N64 Encoders

        thelostsoulT 1 Reply Last reply Reply Quote 0
        • thelostsoulT
          thelostsoul @Impman66
          last edited by

          @Impman66 Oh, I wasn't asking for a notepad replacement. My question was more about if there are specialized tools to deal with the gamelist.xml format. So the answer I get from you is that you use an editor like me. Btw I am on Ubuntu and have multiple editors, so that is not the issue.

          Actually I wrote a script and gui application (in Python, also good exercise to learn it) the last several days to help me with dealing directly with that specific XML format. My hope is, that there are others who made/use a more comprehensive tool.

          πŸ“œ RE/SET: 100 SNES Games for your RetroPie, 🎁 Share your hidden gems and insider tips

          1 Reply Last reply Reply Quote 0
          • Impman66I
            Impman66
            last edited by

            Apologies, I am just a beginner in all this, so wouldn't have a clue on what to recommend. I only said about the Programmers Notepad, as this has been invaluable to learning, by looking at others codes for a theme. Then using that to develop my own.

            Good luck in your search

            Pi Model or other hardware: Raspberry Pi 3 Model B
            Power Supply used: Generic 5v 2.5A Panasonic
            RetroPie Version Used : 4.5.1
            Built From: Retropie website
            USB Devices connected: 2 x Dragonrise N64 Encoders

            1 Reply Last reply Reply Quote 1
            • mituM
              mitu Global Moderator
              last edited by

              I think ARRM can help you edit the gamelist.xml file without resorting to direct edits.

              thelostsoulT 1 Reply Last reply Reply Quote 0
              • thelostsoulT
                thelostsoul @mitu
                last edited by

                ARRM
                @mitu Interesting app. I wasn't aware of it and it is quite impressive. Unfortunately I also can't test it, because its Windows only and I am using Linux. It could run under WINE, but don't know how well it works and I don't want install it for now (also quite big).

                πŸ“œ RE/SET: 100 SNES Games for your RetroPie, 🎁 Share your hidden gems and insider tips

                1 Reply Last reply Reply Quote 0
                • WeirdHW
                  WeirdH @thelostsoul
                  last edited by

                  @thelostsoul said in How do you edit the gamelist.xml manually?:

                  All the time until now I did it with a plain text editor and simple copy and paste. And everytime I use a scraper and let the file generate, all changes are gone.

                  I don't know if this works the same for all scrapers, but I use the Steven Selph one: in the scraper menu I selected 'append' instead of 'overwrite' (or whatever the other option is called). This only scans games that aren't in the list already and leaves all other info alone.

                  1 Reply Last reply Reply Quote 0
                  • mituM
                    mitu Global Moderator
                    last edited by mitu

                    I'm not sure if an editor is going to help much, but I see a few workarounds for this:

                    • when editing your gamelist.xml to add hacks info, add a custom attribute to the game entry
                      <game mine=1>
                      <path>ROM_Path</path>
                      <name>Blah</name>
                      </game>
                      
                      Before scraping, backup these entries to an additional file with xmlstarlet:
                       xmlstarlet sel -t -m '//game[@mine=1]' -c . ~/roms/ngpc/gamelist.xml > ~/roms/ngpc/gamelist_mine.xml
                      

                    When using skyscraper, instruct it to ignore any non-scraped entries and to no add them to the gamelist, with the skipped flag. After scraping is done, add the entries from your backup to your main gamelist.

                    • (best option) add the necessary ROM metadata to Skyscraper's cache - either using the import module (if you need image/video) or via the cache editing options.

                    EDIT: forgot that skyscraper has an ES gamelist scraping module. As an addendum to the 2nd option presented above, you can scrape your games using it. The info you added to the games in the gamelist.xml files should be added to the cache, so it will help when re-generating the gamelist.xml again.

                    1 Reply Last reply Reply Quote 0
                    • B
                      Bluescreen
                      last edited by

                      Hi
                      I am trying to learn programming. I have created a small program. it works for me but it has a few bugs.
                      manager.jpg

                      thelostsoulT 1 Reply Last reply Reply Quote 0
                      • thelostsoulT
                        thelostsoul @Bluescreen
                        last edited by

                        @Bluescreen This looks great and is what I was looking for. Is this more of a private helper application that you developed for yourself?
                        @WeirdH When I learned and experimented with RetroPie and the scrapers (years back now) I experienced a few problems with append operation. Don't ask me what problems these was. From that point I decided to go with overwrite and two files, just in case.
                        @mitu Actually, the introduction of a custom attribute is a good idea to quickly find any custom or edited entry. For the way you suggest: if I have to copy the custom entries and save them to a custom xml and reinstert them after scraping... that is actually more work for me than what I do now. I could just cut out the first work of getting them from the main list. But I guess this is just a workflow thing and could be automated anyway, so thanks for the idea.
                        I never used Skyscraper. It looks promising and maybe I should change my default scraper, especially if it have the functionality I need.

                        So for now I need to evaluate what to do next and this can take ages. In the meantime if someone else have ideas or tools which helps you, you can post them here too.

                        πŸ“œ RE/SET: 100 SNES Games for your RetroPie, 🎁 Share your hidden gems and insider tips

                        1 Reply Last reply Reply Quote 0
                        • B
                          Bluescreen
                          last edited by

                          @thelostsoul Yes its a private helper application. i learn coding, and this is my learning projekt.

                          ItΒ΄s works for the most. The program is slow from time to time and has a few small errors.
                          works very well for a learning project. I can send you the program if you want

                          -Features
                          -XML read write
                          -check whether files are available.
                          -ID and Info: opens the Screenscraper page with the ID of the respective game (cover, marquee, video etc)
                          -Bugs ;)

                          sorry my english is not the best.

                          Impman66I thelostsoulT 2 Replies Last reply Reply Quote 0
                          • Impman66I
                            Impman66 @Bluescreen
                            last edited by

                            @Bluescreen said in How do you edit the gamelist.xml manually?:

                            @thelostsoul Yes its a private helper application. i learn coding, and this is my learning projekt.

                            ItΒ΄s works for the most. The program is slow from time to time and has a few small errors.
                            works very well for a learning project. I can send you the program if you want

                            -Features
                            -XML read write
                            -check whether files are available.
                            -ID and Info: opens the Screenscraper page with the ID of the respective game (cover, marquee, video etc)
                            -Bugs ;)

                            sorry my english is not the best.

                            Your English is very good actually.
                            Looking at this I think there may be a few more people interested in your little "learning coding" project.
                            May be the mods can direct as to a way of getting it out there - with your permission of course.

                            Pi Model or other hardware: Raspberry Pi 3 Model B
                            Power Supply used: Generic 5v 2.5A Panasonic
                            RetroPie Version Used : 4.5.1
                            Built From: Retropie website
                            USB Devices connected: 2 x Dragonrise N64 Encoders

                            B 1 Reply Last reply Reply Quote 0
                            • B
                              Bluescreen @Impman66
                              last edited by

                              @Impman66 said in How do you edit the gamelist.xml manually?:

                              Your English is very good actually.
                              Looking at this I think there may be a few more people interested in your little "learning coding" project.
                              May be the mods can direct as to a way of getting it out there - with your permission of course.

                              I do not have a problem with that. the code only looks dirty;)
                              but i think most only want the program and not the code

                              G 1 Reply Last reply Reply Quote 0
                              • G
                                Gamesman @Bluescreen
                                last edited by

                                @Bluescreen I would be interest in the code. The program looks nice and promising.

                                1 Reply Last reply Reply Quote 0
                                • thelostsoulT
                                  thelostsoul @Bluescreen
                                  last edited by

                                  @Bluescreen I guess it is a Windows only app, as you are learning right now. Does it run in Linux? I would want to have a little look at the application and if possible in the code, just curious.

                                  πŸ“œ RE/SET: 100 SNES Games for your RetroPie, 🎁 Share your hidden gems and insider tips

                                  1 Reply Last reply Reply Quote 0
                                  • B
                                    Bluescreen
                                    last edited by Bluescreen

                                    @thelostsoul
                                    ItΒ΄s a Windows only app. i don't know much around linux.

                                    GamelistManager Download

                                    password is: retropie

                                    only the app with 4 test entries for the snes. I can also upload the 20gb (without video and roms) or the full 80gb (without roms)

                                    @Gamesman i can upload the code or the complete projectfile

                                    1 Reply Last reply Reply Quote 1
                                    • thelostsoulT
                                      thelostsoul
                                      last edited by thelostsoul

                                      I wasn't happy, so I made my own tool: https://retropie.org.uk/forum/topic/27466/gamelist-addon-an-assist-tool-to-manually-add-new-game-entries. Unlike the other tools, this is really just to add new games and not a general scraper or editing tool.

                                      @Bluescreen, I had a quick look at the folder structure to see whats inside. Because its interesting. :-) About the 20gb, I am not really interested into the images itself (I suppose thats what the 20gb are).
                                      What test entries do you have and what are you testing? I am absolutely not sure here what to test.

                                      πŸ“œ RE/SET: 100 SNES Games for your RetroPie, 🎁 Share your hidden gems and insider tips

                                      B 1 Reply Last reply Reply Quote 0
                                      • B
                                        Bluescreen @thelostsoul
                                        last edited by Bluescreen

                                        @thelostsoul open the snes list via my program. the first 4 entries are with game information, cover video and marquee.
                                        to see what it looks like

                                        edit

                                        my tool does exactly the same as yours. the only difference is video marquee and video display. It can't scrape. just edit the gamelist.xml.

                                        I'm glad you programmed one yourself.

                                        thelostsoulT 1 Reply Last reply Reply Quote 0
                                        • thelostsoulT
                                          thelostsoul @Bluescreen
                                          last edited by

                                          @Bluescreen Unfortunately I can't test it, because as said before I am on Linux. I tried it on a very old Win10 virtual machine, but it does not work. Probably because of too old libraries.
                                          The tool I have created do not edit in the sense of deleting stuff and it does not show a table. Its really just adding new games only, as this is exactly the only thing I need.

                                          πŸ“œ RE/SET: 100 SNES Games for your RetroPie, 🎁 Share your hidden gems and insider tips

                                          1 Reply Last reply Reply Quote 0
                                          • thelostsoulT
                                            thelostsoul
                                            last edited by

                                            Just saw this app related to this topic. Its not exactly what I am looking for, just want to link it here: RetroPi Media - gamelist.xml generator

                                            πŸ“œ RE/SET: 100 SNES Games for your RetroPie, 🎁 Share your hidden gems and insider tips

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.