Making premade gamelist.xml with xtra media collections

meleu

@Used2BeRX said in Making premade gamelist.xml with xtra media collections:

@meleu How would they be useful to the community by themselves?

I thought that those files (that Xtras stuff) from the XBox community were popular, then I thought it would be useful for those guys who are willing to migrate to RetroPie. But TBH I have no idea how popular it is.

why that would make you recover the will to work on the script :)

Because it would benefit the community. :)

EDIT: Knowing what you have in mind would also help me plan a way to present the information.

Well, to be honest I don't have a clear idea of what to do next. I have some ideas, but not a clear one.

Anyway, having those files easily available on github (and not those obscure mega/sendspace links) would increase the chance to involve people to do something with that info.

What I found most valuable in your files was the synopses for hacks and homebrews, maybe those descriptions should be submitted to some of the popular databases.

Used2BeRX

@meleu I literally replied to you about 3 seconds before you replied. I did talk about what you just talked about, as well as had a question regarding the script. :)

meleu

@Used2BeRX said in Making premade gamelist.xml with xtra media collections:

I was wondering if I could do a quick alteration to include both CD or Compact Disc until you get a chance to put it in your official script so when I make an update in the future it's still in there.

I like to follow a principle that says:
The software should adapt to humans, not vice versa.

That's why I am constantly asking to not change your files. :)

I'll make the script recognize Compact Disc as CD, so no need to change your files.

EDIT: no rush to post your files to github, but consider posting when you think it's ready! :-)

Used2BeRX

@meleu Of course :)

I was asking if there was a way I could temporarily alter your script so I could re-run it on my end right now since you were away from it yourself.

Just to be sure though, keep the CD in there as well as Compact Disc. The CD works perfect for TurboGrafx CD.

BTW... if you want to share any of your ideas that you mentioned you might have for the synopsis and what not, feel free to let me know what they are.

I would like to share this with people, but as you can imagine it's a daunting task. If sharing roms wasn't a problem I could just make some huge SD card images and be done with it. Since that's not an option there's about 100 times more work on my end to try to figure out a way to make this work for other people.

meleu

@Used2BeRX

I was asking if there was a way I could temporarily alter your script so I could re-run it on my end right now

Not exactly in the script, but you can replace every Compact Disc by CD in the synopsis with sed. Do the following:

change the directory to where the synopsis are placed:

cd /path/to/segacd/synopsis

sed can do the magic:

sed -i 's/Compact Disc/CD/' *.txt

(that .txt is case sensitive, so it can be .TXT on your end)

meleu

@Used2BeRX don't mind about the sed stuff above. I already changed the script. Please --update and test it.

Used2BeRX

@meleu Wow, that's a cool trick. I literally changed over 100 entries for the `Famicom Disk System~ by hand the other night. It took me about 20 minutes.

I'm just a big dumb donkey. I like to surround myself with smart people. :)

I'm not going to change anything. I've got some other stuff to do and I'm just going to turn in early after I can get some file transfers cued up to my thumb drives so I can start testing the Sega CD and Turbo CD games tomorrow hopefully. I'll just wait until you can change the script.

meleu

@Used2BeRX said in Making premade gamelist.xml with xtra media collections:

I'll just wait until you can change the script.

I already changed the script. We are posting almost at the same time! :-)

Used2BeRX

@meleu Heh.. yeah, I saw that, but I had already moved my keyboard to the Pi to update and run it.

100% success rate with boxart on Sega CD cue files. :)

Now I have to figure out how to use a USB stick on the Pi for CD testing. I had all the images for the Sega and Turbo CD in ISO/MP3 format and it just just a little over 50 GB. Now that they're all in BIN/CUE and ISO/CUE they're over 150GB!!!!

I'll be doing that and in between re-casing everything like I said I would. AFter that I should be able to look into github.

Thanks again man.

meleu

@Used2BeRX said in Making premade gamelist.xml with xtra media collections:

Wow, that's a cool trick. I literally changed over 100 entries for the `Famicom Disk System~ by hand the other night. It took me about 20 minutes.

It's just a little bit of an example of the power of Linux shell tools. That's why I said that it would be easier to get what you wanted to do directly on Linux. :)

Maybe one day your old computer will become useless to run Windows and you'll try Linux. When it happen you'll say "Wow! This system is fast even on this old computer! Look how fast it boots up!"

Used2BeRX

@meleu

Maybe one day your old computer will become useless to run Windows and you'll try Linux. When it happen you'll say "Wow! This system is fast even on this old computer! Look how fast it boots up!"

That day might be soon. Windows 10 is terrible. It does have some cool features that earlier windows didn't have, but the search function is crippled to uselessness and other useful things have been stripped out or changed.

For example, last night before being able to get CD images on my USB to test on the Pi, I had to delete a lot of stuff off of a 64GB stick. There were tons of files since it was a backup I was using for Xtras before I got my 2TB drive. It took almost an hour to delete the files on the stick!!!!!! That is unacceptable, and it never would have happened on Windows XP or earlier.

They're trying to cater to the smartphone/tablet crowd now, and old dogs like me are being pushed out. If it keeps going this way I'll probably stop using Windows out of necessity.

meleu

@Used2BeRX said in Making premade gamelist.xml with xtra media collections:

They're trying to cater to the smartphone/tablet crowd now, and old dogs like me are being pushed out

"old dogs" with a Windows XP background are exactly the Linux Mint target audience. You should give it a try when you have a chance. ;-)

Used2BeRX

@meleu I originally started out with DOS and 3.1. I remember being resistant to upgrading to XP from Windows 98 at the time, but XP really was a great system.

I think it's been all downhill for M$oft ever since Vista. Every new OS seems to be worse than the previous.

Once I get an actual job I'm going to get a new computer and I'm going to turn this rig into a full fledged Linux PC.

I just figured out how to direct a symbolic link to the USB drive and I was amazed that when done properly it will benefit from all of the Xtras in the gamelist.xml. Linux is pretty kick ass.

Used2BeRX

@meleu Damn near 100% of Sega CD and TurboGrafx CD games work on a Pi Zero! So awesome using Symbolic Links to a USB drive after making the gamelist.xml files too.

I CaMeL CaSeD all of the media and roms, so I'm in the process of deleting all the old stuff and re-uploading all of the re-cased stuff. Next step will be to re-make all of the gamelist.xml files and check that everything worked. That would be the first time checking that everything works perfect in one run rather than making edits and appending stuff to the xml files.

Hopefully I can start looking into github this weekend or sometime next week to see about at least getting the synopsis files up there.

Used2BeRX

@meleu Hey Meleu,

Just in case you miss the thread I made asking the question, I thought you might have an idea about this. I was wondering if you could "hijack" a .CUE file and just launch a rom in Sega CD with it. Pier Solar is different than any other CD image and just consists of a .BIN and a .ISO.

Because I filter out everything but .CUE files in the segacd romlist, the .BIN file needed to launch the game will not show up in the romlist. Because of the way we are handling the segacd in our script, it will also not find this game either. Being able to create a .CUE file that just launches the game as if you selected the .BIN file in the romlist and launched it with genesis-plus-gx would avoid both of these issues.

I figure if anybody I know knows how to do this it's you. :)

Used2BeRX

@meleu Everything is re-cased. I re-ran the script and only had 6 errors across all of the systems. Nothing to worry about. Just things I neglected to sync between my PC and the Pi when I fixed them last time around, so it's kind of a good thing I've had to re-start this stuff from scratch a few times to really iron out all the hiccups.

At this point I'm ready to start trying things to test out that the other pieces of artwork are all going to work fine, but I won't be able to test videos until I get a better Pi.

I should have some time this week to start looking into github and seeing what to do about that.

Used2BeRX

@meleu Thanks for replying to my e-mail. I don't know if the site was actually down all day. I had a browser issue and I had to remove firefox and re-install it and when I did this site was back up and not giving me an "under maintenance".

I'm answering your email here since I didn't want to split up our conversation on this over email.

About the script, the '--no-desc' option perfectly describes what it means: "do not generate <desc> entries". Only this, nothing more.

You can use '--full' and '--no-desc' in the same command line and it will generate that bloated xml but with empty <desc> fields.

Maybe you're asking for a '--minimal' option where the xml output would have only the <path> field filled?

Oh... lol. I didn't realize that you could use '--no-desc' and '--full' together. I thought --no-desc was giving you the bare minimum. Well.. that's awesome. :) When I'm ready to call it a night I'm going to re-run the script and see if it finds all the other pics then when I run --full and --no-desc together.

So, using --no-desc, it would have added marquees and videos if I had them on the Pi, since EmulationStation/RetroPie takes advantage of these file types now?

Now the idea would be to find a way to make quick edits to the xml files to change which file type is the <image> without having to wait 12 hours for the pi to re-create all the gamelist.xml files. That way I can verify that all of the other media types are all working without any issues.

My next project on my end is going to figure out where Emulationstation finds all of the gamelist.xml files and edit that. I want to be able to simply run your program and when it's done I can just reboot the Pi and all of gamelist.xml files that your script makes will load up from the same folder with the names you gave them.

That will end up saving me about 15 minutes of moving and re-naming files every time I re-run all the xmls.

meleu

@Used2BeRX

So, using --no-desc, it would have added marquees and videos if I had them on the Pi, since EmulationStation/RetroPie takes advantage of these file types now?

I tested here and the script generate valid <video> entries.

The script doesn't fill the <marquee> field. I didn't find any image that looks like a "marquee" in the Xtras pack I downloaded. Do you have such kind of image on your system? What would be the path to them? Media/Artwork/...?

Now the idea would be to find a way to make quick edits to the xml files to change which file type is the <image> without having to wait 12 hours for the pi to re-create all the gamelist.xml files. That way I can verify that all of the other media types are all working without any issues.

Didn't get what you mean here. When searching for image files, the script looks only for files ending with .png or .jpg (case insensitive).

My next project on my end is going to figure out where Emulationstation finds all of the gamelist.xml files and edit that.

You can find the answer here: https://github.com/retropie/EmulationStation/blob/master/GAMELISTS.md#gamelists

I want to be able to simply run your program and when it's done I can just reboot the Pi and all of gamelist.xml files that your script makes will load up from the same folder with the names you gave them.

According to the info in the link above, you can't load an arbitrary gamelist.xml. The file must be one of those. But I think that making the script generate files directly in the right places is a fair request... :-)

Used2BeRX

@meleu said in Making premade gamelist.xml with xtra media collections:

@Used2BeRX

So, using --no-desc, it would have added marquees and videos if I had them on the Pi, since EmulationStation/RetroPie takes advantage of these file types now?

I tested here and the script generate valid <video> entries.

The script doesn't fill the <marquee> field. I didn't find any image that looks like a "marquee" in the Xtras pack I downloaded. Do you have such kind of image on your system? What would be the path to them? Media/Artwork/...?

It would be /Media/{system name}/Artwork/Marquee/

I don't have any images of this type in my sets. My friends who are working on their own Pi skin have them though. It is a feature that was added in the last few months around the same time that the video previews were added. Their skin makes use of the <marquee> field in the .XML files.

Now the idea would be to find a way to make quick edits to the xml files to change which file type is the <image> without having to wait 12 hours for the pi to re-create all the gamelist.xml files. That way I can verify that all of the other media types are all working without any issues.

Didn't get what you mean here. When searching for image files, the script looks only for files ending with .png or .jpg (case insensitive).

.png and .jpg should be just fine. That's not what I meant, sorry.

Here's my problem. It takes about 12 hours for me to create --no-desc gamelist.xml files for all of the systems, so testing all of the image types can take me about 4-5 days by re-running this for each image type.

First, I will run one --full and --no-desc command to generate all of the xml files with all of the fields. (You say that this is possible, and I will run it overnight while I'm sleeping and verify it when it's complete).

Then I would like a 2nd script that would change which file type had the <image> tag from these complete gamelist.xml files, which should only take a few seconds per file instead of 12 hours to re-run everything.

If I had the standard xmls with the Box Front as image and I wanted instead to have the Title Shot as the image that shows in the romlist, first the <image> tags would need to be changed to <boxfront>, then the <title> tags would need to be changed to <image>.

The script doing this, however, would need to leave any <folder> entries alone and only make these changes to the images inside all of the <game> entries.

I hope I'm explaining this right. Please let me know if I'm not. I think this is a very important feature for the script that could potentially save hundreds of hours of re-scanning everything any time you wanted to change which images were displayed in RetroPie/EmulationStation.

My next project on my end is going to figure out where Emulationstation finds all of the gamelist.xml files and edit that.

You can find the answer here: https://github.com/retropie/EmulationStation/blob/master/GAMELISTS.md#gamelists

I want to be able to simply run your program and when it's done I can just reboot the Pi and all of gamelist.xml files that your script makes will load up from the same folder with the names you gave them.

According to the info in the link above, you can't load an arbitrary gamelist.xml. The file must be one of those. But I think that making the script generate files directly in the right places is a fair request... :-)

I already solved this issue :)

I now have your script in /home/pi/RetroPie/Gamelists/.

I run the script and just leave the files right in there. If I need to re-run anything I can back them up on my PC first and delete them or append to them.

Then I went into all the /opt/pi/retropie/configs/all/emulationstation/gamelists/{system_name}/ and made symbolic links to every one of the gamelist.xml files that your script creates.

Now all I have to do when your script finishes running is type in emulationstation in the Linux console and when it boots it automatically uses the new gamelist.xml files :)

I actually like this solution better than making your script put the files in the correct places. This way all of the true files remain in the same folder.

Used2BeRX

@meleu

~~Hey bud. What was the download link for this script again? I've been just updating it for a while now and somebody else who's got our media wants to try it out.~~

~~I'm writing up instructions and put this link to download it: https://raw.githubusercontent.com/meleu/share/master/Used2BeTXT.sh~~

~~But looking at that it seems like it is an older version that doesn't have things like '--no-desc` and the image options???~~

~~Thanks :)~~

Nevermind... Figured it out. :)

Let me know if I need to re-explain what I said in the previous post or if you need any clarification.