RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login

    Making premade gamelist.xml with xtra media collections

    Scheduled Pinned Locked Moved Ideas and Development
    gamelist.xmlpremade scrapeeasy config
    448 Posts 16 Posters 151.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Used2BeRXU
      Used2BeRX @meleu
      last edited by Used2BeRX

      @meleu Hey Meleu,

      Just in case you miss the thread I made asking the question, I thought you might have an idea about this. I was wondering if you could "hijack" a .CUE file and just launch a rom in Sega CD with it. Pier Solar is different than any other CD image and just consists of a .BIN and a .ISO.

      Because I filter out everything but .CUE files in the segacd romlist, the .BIN file needed to launch the game will not show up in the romlist. Because of the way we are handling the segacd in our script, it will also not find this game either. Being able to create a .CUE file that just launches the game as if you selected the .BIN file in the romlist and launched it with genesis-plus-gx would avoid both of these issues.

      I figure if anybody I know knows how to do this it's you. :)

      1 Reply Last reply Reply Quote 0
      • Used2BeRXU
        Used2BeRX @meleu
        last edited by

        @meleu Everything is re-cased. I re-ran the script and only had 6 errors across all of the systems. Nothing to worry about. Just things I neglected to sync between my PC and the Pi when I fixed them last time around, so it's kind of a good thing I've had to re-start this stuff from scratch a few times to really iron out all the hiccups.

        At this point I'm ready to start trying things to test out that the other pieces of artwork are all going to work fine, but I won't be able to test videos until I get a better Pi.

        I should have some time this week to start looking into github and seeing what to do about that.

        1 Reply Last reply Reply Quote 1
        • Used2BeRXU
          Used2BeRX @meleu
          last edited by

          @meleu Thanks for replying to my e-mail. I don't know if the site was actually down all day. I had a browser issue and I had to remove firefox and re-install it and when I did this site was back up and not giving me an "under maintenance".

          I'm answering your email here since I didn't want to split up our conversation on this over email.

          About the script, the '--no-desc' option perfectly describes what it means: "do not generate <desc> entries". Only this, nothing more.

          You can use '--full' and '--no-desc' in the same command line and it will generate that bloated xml but with empty <desc> fields.

          Maybe you're asking for a '--minimal' option where the xml output would have only the <path> field filled?

          Oh... lol. I didn't realize that you could use '--no-desc' and '--full' together. I thought --no-desc was giving you the bare minimum. Well.. that's awesome. :) When I'm ready to call it a night I'm going to re-run the script and see if it finds all the other pics then when I run --full and --no-desc together.

          So, using --no-desc, it would have added marquees and videos if I had them on the Pi, since EmulationStation/RetroPie takes advantage of these file types now?

          Now the idea would be to find a way to make quick edits to the xml files to change which file type is the <image> without having to wait 12 hours for the pi to re-create all the gamelist.xml files. That way I can verify that all of the other media types are all working without any issues.

          My next project on my end is going to figure out where Emulationstation finds all of the gamelist.xml files and edit that. I want to be able to simply run your program and when it's done I can just reboot the Pi and all of gamelist.xml files that your script makes will load up from the same folder with the names you gave them.

          That will end up saving me about 15 minutes of moving and re-naming files every time I re-run all the xmls.

          meleuM 1 Reply Last reply Reply Quote 0
          • meleuM
            meleu @Used2BeRX
            last edited by

            @Used2BeRX

            So, using --no-desc, it would have added marquees and videos if I had them on the Pi, since EmulationStation/RetroPie takes advantage of these file types now?

            I tested here and the script generate valid <video> entries.

            The script doesn't fill the <marquee> field. I didn't find any image that looks like a "marquee" in the Xtras pack I downloaded. Do you have such kind of image on your system? What would be the path to them? Media/Artwork/...?

            Now the idea would be to find a way to make quick edits to the xml files to change which file type is the <image> without having to wait 12 hours for the pi to re-create all the gamelist.xml files. That way I can verify that all of the other media types are all working without any issues.

            Didn't get what you mean here. When searching for image files, the script looks only for files ending with .png or .jpg (case insensitive).

            My next project on my end is going to figure out where Emulationstation finds all of the gamelist.xml files and edit that.

            You can find the answer here: https://github.com/retropie/EmulationStation/blob/master/GAMELISTS.md#gamelists

            I want to be able to simply run your program and when it's done I can just reboot the Pi and all of gamelist.xml files that your script makes will load up from the same folder with the names you gave them.

            According to the info in the link above, you can't load an arbitrary gamelist.xml. The file must be one of those. But I think that making the script generate files directly in the right places is a fair request... :-)

            • Useful topics
            • joystick-selection tool
            • rpie-art tool
            • achievements I made
            Used2BeRXU 5 Replies Last reply Reply Quote 0
            • Used2BeRXU
              Used2BeRX @meleu
              last edited by Used2BeRX

              @meleu said in Making premade gamelist.xml with xtra media collections:

              @Used2BeRX

              So, using --no-desc, it would have added marquees and videos if I had them on the Pi, since EmulationStation/RetroPie takes advantage of these file types now?

              I tested here and the script generate valid <video> entries.

              The script doesn't fill the <marquee> field. I didn't find any image that looks like a "marquee" in the Xtras pack I downloaded. Do you have such kind of image on your system? What would be the path to them? Media/Artwork/...?

              It would be /Media/{system name}/Artwork/Marquee/

              I don't have any images of this type in my sets. My friends who are working on their own Pi skin have them though. It is a feature that was added in the last few months around the same time that the video previews were added. Their skin makes use of the <marquee> field in the .XML files.

              Now the idea would be to find a way to make quick edits to the xml files to change which file type is the <image> without having to wait 12 hours for the pi to re-create all the gamelist.xml files. That way I can verify that all of the other media types are all working without any issues.

              Didn't get what you mean here. When searching for image files, the script looks only for files ending with .png or .jpg (case insensitive).

              .png and .jpg should be just fine. That's not what I meant, sorry.

              Here's my problem. It takes about 12 hours for me to create --no-desc gamelist.xml files for all of the systems, so testing all of the image types can take me about 4-5 days by re-running this for each image type.

              First, I will run one --full and --no-desc command to generate all of the xml files with all of the fields. (You say that this is possible, and I will run it overnight while I'm sleeping and verify it when it's complete).

              Then I would like a 2nd script that would change which file type had the <image> tag from these complete gamelist.xml files, which should only take a few seconds per file instead of 12 hours to re-run everything.

              If I had the standard xmls with the Box Front as image and I wanted instead to have the Title Shot as the image that shows in the romlist, first the <image> tags would need to be changed to <boxfront>, then the <title> tags would need to be changed to <image>.

              The script doing this, however, would need to leave any <folder> entries alone and only make these changes to the images inside all of the <game> entries.

              I hope I'm explaining this right. Please let me know if I'm not. I think this is a very important feature for the script that could potentially save hundreds of hours of re-scanning everything any time you wanted to change which images were displayed in RetroPie/EmulationStation.

              My next project on my end is going to figure out where Emulationstation finds all of the gamelist.xml files and edit that.

              You can find the answer here: https://github.com/retropie/EmulationStation/blob/master/GAMELISTS.md#gamelists

              I want to be able to simply run your program and when it's done I can just reboot the Pi and all of gamelist.xml files that your script makes will load up from the same folder with the names you gave them.

              According to the info in the link above, you can't load an arbitrary gamelist.xml. The file must be one of those. But I think that making the script generate files directly in the right places is a fair request... :-)

              I already solved this issue :)

              I now have your script in /home/pi/RetroPie/Gamelists/.

              I run the script and just leave the files right in there. If I need to re-run anything I can back them up on my PC first and delete them or append to them.

              Then I went into all the /opt/pi/retropie/configs/all/emulationstation/gamelists/{system_name}/ and made symbolic links to every one of the gamelist.xml files that your script creates.

              Now all I have to do when your script finishes running is type in emulationstation in the Linux console and when it boots it automatically uses the new gamelist.xml files :)

              I actually like this solution better than making your script put the files in the correct places. This way all of the true files remain in the same folder.

              1 Reply Last reply Reply Quote 0
              • Used2BeRXU
                Used2BeRX @meleu
                last edited by Used2BeRX

                @meleu

                Hey bud. What was the download link for this script again? I've been just updating it for a while now and somebody else who's got our media wants to try it out.

                I'm writing up instructions and put this link to download it: https://raw.githubusercontent.com/meleu/share/master/Used2BeTXT.sh

                But looking at that it seems like it is an older version that doesn't have things like '--no-desc` and the image options???

                Thanks :)

                Nevermind... Figured it out. :)

                Let me know if I need to re-explain what I said in the previous post or if you need any clarification.

                1 Reply Last reply Reply Quote 0
                • Used2BeRXU
                  Used2BeRX @meleu
                  last edited by Used2BeRX

                  @meleu I ran --full on the segacd and it looks like the only extra media it found was <threedbox>.

                  The 3D Boxes are .PNG files. Everything else on my Pi Zero setup was .JPG, so I was wondering if possibly the code for --full is only looking for .PNG files in the Artwork folders?

                  I haven't tried it for <marquee> or '<video>` yet. I will have to add some of these and re-run that to test them out as well.

                  EDIT: Could you change the code also so it doesn't change the name of the gamelist if you use options like nes_FULL_gamelist.xml? Now that I have the Symbolic Links set up, this is working great, but if I use an option like that it saves differently and won't work with the symbolic link unless I manually edit the files when it's done.

                  1 Reply Last reply Reply Quote 0
                  • Used2BeRXU
                    Used2BeRX @meleu
                    last edited by Used2BeRX

                    @meleu Well.... github makes absolutely no sense to me.

                    I think if I share the synopsis files it's better to just share them from my MegaNZ account.

                    If you have any compelling reasons why github would be better let me know. At this point I still don't know what anybody would get out of just having the synopsis files without any of the other media.

                    1 Reply Last reply Reply Quote 0
                    • Used2BeRXU
                      Used2BeRX @meleu
                      last edited by

                      @meleu Hey man. I seen you on around here but not replying, so sorry if I've been asking too much without giving enough back or something.

                      I'm going to make sure one last time that the Synopsis files are ready to go and then just upload them to a file sharing service. If somebody wants to do something and set them up on github so they're of more use to people, feel free to do it.

                      You haven't told me what use it would be on github, and since I have no idea what you're thinking or even a clue how to use github, I feel lost. I've spent 2 hours trying to figure github out and I haven't gotten anywhere.

                      meleuM 1 Reply Last reply Reply Quote 0
                      • meleuM
                        meleu @Used2BeRX
                        last edited by

                        @Used2BeRX I talked about it above. But never mind. Forget it and let's play some game! :-)

                        • Useful topics
                        • joystick-selection tool
                        • rpie-art tool
                        • achievements I made
                        Used2BeRXU 2 Replies Last reply Reply Quote 0
                        • Used2BeRXU
                          Used2BeRX @meleu
                          last edited by

                          @meleu Too much work for me to still do man. Hope you get a chance to look at the issues I'm still having with the script at some point.

                          Right now I'm working on converting my old romsets as much as possible to no-intro. Yesterday I finished up adding the new games I was going to add (about 30-40 of them on 4 different systems), and I can't wait to play Pier Solar at some point.

                          Then I have to figure out a good way to distribute this stuff without roms that is easier than github. That site makes no sense to me. I'm thinking that is partially because I'm a Linux dummy or something.

                          UDb23U 1 Reply Last reply Reply Quote 0
                          • UDb23U
                            UDb23 @Used2BeRX
                            last edited by

                            @Used2BeRX said in Making premade gamelist.xml with xtra media collections:

                            github

                            Initially it may be intimidating but it's not complicated at all.
                            I upload files just using the web browser both under windows and Mac.
                            The only thing that's unusual is that there is no direct "make folder" command but, instead, you create folders by adding a new file (even a dummy txt file) and putting the directory path were you want the file to be located.

                            Used2BeRXU 1 Reply Last reply Reply Quote 0
                            • Used2BeRXU
                              Used2BeRX @UDb23
                              last edited by

                              @UDb23 Thanks. lol... I don't even see any options for adding any new files though. All I see is <> Code, Issues, Pull Requests, Projects, Wiki, Settings, and Insights.

                              I've got a few Insights.... but I'll keep them to myself for now :)

                              meleuM 1 Reply Last reply Reply Quote 0
                              • Used2BeRXU
                                Used2BeRX @meleu
                                last edited by

                                @meleu So am I supposed to be uploading just zip files, or the individual synopsis files?

                                I finally opened a folder to upload to and when I drag and drop it says "YOWZA! That's a lot of files. Try it again with 100 or less."

                                No. I don't want to try it again with 100 or less. You're the computer. You do what I tell you to do.

                                1 Reply Last reply Reply Quote 0
                                • meleuM
                                  meleu @Used2BeRX
                                  last edited by

                                  @Used2BeRX don't worry about github. It's a tool for developers and other people can take advantage of it. But if it is confusing for you and you don't see a reason to learn, just move on. ;-)

                                  I was thinking on other guys with the same background as you: people coming from the XBox emulation scene. But I'm not from this scene and don't even know if they want this kind of thing. Let's forget it and move on! We have other stuff in our To Do list, right? :-)

                                  • Useful topics
                                  • joystick-selection tool
                                  • rpie-art tool
                                  • achievements I made
                                  1 Reply Last reply Reply Quote 0
                                  • meleuM
                                    meleu
                                    last edited by

                                    By the way, I'll improve the script to let you change the art on <image> field. No need for another tool to achieve that.

                                    • Useful topics
                                    • joystick-selection tool
                                    • rpie-art tool
                                    • achievements I made
                                    Used2BeRXU 2 Replies Last reply Reply Quote 1
                                    • Used2BeRXU
                                      Used2BeRX @meleu
                                      last edited by

                                      @meleu If you're sure. I just don't want to come off like I'm not contributing. I have a lot of plans for this stuff, but I have to figure out the best way to do it all. I would be more receptive to github if I saw the benefit of it, but since I don't even understand how to use it other than dumbly typing in commands from Console that other people tell me to type I can't envision the value of putting anything up there.

                                      I said it before. I'm just a big dumb donkey. I just kind of brute force my way around everything and hope that other people a lot smarter than me see value in my work and help ease the work load.

                                      Here's a to-do list on the script so you don't have to go back and look at stuff:

                                      1. Change it so that when you use --full or another option that it doesn't alter the gamelist.xml name. (Eg: no _FULL_ added to nes_gamelist.xml.

                                      With symbolic links I have everything set up to run perfect now after running your script so all you need to do is reboot when it's done. When the names are changed like this it adds the step of having to change names before rebooting.

                                      I actually wrote a guide at the other place that I'm still working on. You can see at the bottom what I did with your script and the symbolic links in depth. I'm going to do a similar write up here as soon as we get this working completely. :)

                                      1. Make sure that script is checking for both .JPG as well as .PNG for the extra fields on a --full command. --full only found .PNG files that were located in the 3D Boxart folders. It did not find any of the .JPG files in the other folders such as Titles and Action.

                                      2. Add <marquee> search and entry to standard script run without needing --full. <marquee> is a usable tag in EmulationStation now, even though my packs don't currently contain any artwork with them. A few of my friends are making skins and have made their own marquees by cutting out the title in the title screen of every game. :)

                                      3. The ability to do a quick switch of which type of artwork has the usable <image> tag after the --full command has been run.

                                      By just changing which type of artwork has this tag after all of the gamelist.xml files have been made, it could literally save about 12 hours by not having to completely re-scan everything on the Pi Zero.

                                      After we can do this stuff, I can go back in and test that all of the other artwork is working properly. I suspect that it all should be, but I do want to verify with my own eyeballs to make sure. Takes quite a while to move through 8,000-10,000 games on the romlist to verify every one has artwork, but it's worth the effort. :)

                                      1 Reply Last reply Reply Quote 0
                                      • Used2BeRXU
                                        Used2BeRX @meleu
                                        last edited by

                                        @meleu Hey meleu. I was thinking that maybe you could upload a small synopsis pack to github the way that you think it would be of most benefit. Maybe Atari 7800 or Atari LYNX since they're only 80 or so files.

                                        I believe you could share or even change ownership of a project.

                                        If you did that, I might be able to figure out how to do the rest of it on my own. I'm pretty sure you have old synopsis data for all the systems now, but that doesn't matter. If I can just get an idea of how you would like to see it presented for the end user I might be able to figure out how to do it.

                                        Been adding a lot more files on my end. Currently in the middle of a project adding hundreds of translated games to the NES and SNES. :)

                                        meleuM 1 Reply Last reply Reply Quote 1
                                        • meleuM
                                          meleu @Used2BeRX
                                          last edited by

                                          @Used2BeRX I'm a little busy currently. Maybe I'll do it when I have a chance. :-)

                                          • Useful topics
                                          • joystick-selection tool
                                          • rpie-art tool
                                          • achievements I made
                                          Used2BeRXU 3 Replies Last reply Reply Quote 0
                                          • Used2BeRXU
                                            Used2BeRX @meleu
                                            last edited by

                                            @meleu No problem...

                                            I was just thinking today while I was putting more of these synopsis files together that it would be great if they were online and people could have access to make altered versions of them.

                                            I was thinking that if they were all out there, somebody might want to go through and make things like the <genre> uniform so any sort functions would work even better.

                                            Stuff I'd like to do some day, but I'm just way too busy doing bigger things right now.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.