Making premade gamelist.xml with xtra media collections

Used2BeRX

@meleu Tons of new stuff done for the NES in the last week. I've pretty much re-done about 70% of the synopsis files... everything that wasn't US Licensed. Those were all really done right. I also made sure all of the games have the "Players" field. That information wasn't online for most non-US/EU games. Now every Japan, hack, translation, prototype, pirate, unlicensed, etc have that information. I verified every game by hand.

I'm working on the Famicom Disk System now. Once that's done, I'm going to start adding the 180 or so translations, and then look into adding just some really good hacks and homebrew.

If anybody knows where to find a REALLY good Famicom Disk Image, I would appreciate it. It doesn't matter what game it is for as long as it's clean. I was able to find good images for pretty much every FDS game, but it's just the label on the Disk. I need to have a large and clean image of the disk for a template to put these labels on.

Much appreciated. :)

Used2BeRX

@meleu So... my problem now with FDS synopsis is that the Platform: field for all of them says Nintendo Entertainment System instead of Famicom Disk System.

I remember I had to do that because otherwise the script would separate these games into an entirely different folder (I believe it was fds, but I don't have the pi available at the moment and I don't use that folder).

At first, I thought this wouldn't be a problem and I could just cut all the info from the gamelist_fds.xml, but the problem is that it doesn't fill in any of the media fields or even the roms because it is looking for them in the fds folder.

Is there a way we could have the NES look for the FDS games instead of separating them? Now that I'm doing all of this work re-doing all of the synopsis files I really don't like having the wrong information for the console type on all of the FDS games just to get the script to work right.

meleu

@used2berx said in Making premade gamelist.xml with xtra media collections:

@meleu So... my problem now with FDS synopsis is that the Platform: field for all of them says Nintendo Entertainment System instead of Famicom Disk System.

I remember I had to do that because otherwise the script would separate these games into an entirely different folder (I believe it was fds ...)

Yes, it is fds, as this system is named in RetroPie.

You shouldn't rename Platform on your synopsis, man, but asked me to make Famicom Disk System synopsis content to be added to the same gamelist as the Nintendo Entertainment System.

@used2berx said in Making premade gamelist.xml with xtra media collections:

Is there a way we could have the NES look for the FDS games instead of separating them?

This would be a workaround in your system, not in the script.

Do you want me to make the script spit metadata from synopsis where Platform: Famicom Disk System AND Nintendo Entertainment System be the same? I can change 1 or 3 lines of code for that...

Used2BeRX

@meleu said in Making premade gamelist.xml with xtra media collections:

@used2berx said in Making premade gamelist.xml with xtra media collections:

@meleu So... my problem now with FDS synopsis is that the Platform: field for all of them says Nintendo Entertainment System instead of Famicom Disk System.

I remember I had to do that because otherwise the script would separate these games into an entirely different folder (I believe it was fds ...)

Yes, it is fds, as this system is named in RetroPie.

You shouldn't rename Platform on your synopsis, man, but asked me to make Famicom Disk System synopsis content to be added to the same gamelist as the Nintendo Entertainment System.

Yeah. I know. It was a stupid thing to do. Didn't you say there was a command that you could run that could easily change that line in all of the FDS synopsis files in Linux? I mean, changing Platform: Nintendo Entertainment System back to Platform: Famicom Disk System?

There are only about 100 entries, but it still took quite a while to change every one by hand. I seem to remember you saying something that there is a single command I could run that would change all of them in a few seconds. :)

@used2berx said in Making premade gamelist.xml with xtra media collections:

Is there a way we could have the NES look for the FDS games instead of separating them?

This would be a workaround in your system, not in the script.

Do you want me to make the script spit metadata from synopsis where Platform: Famicom Disk System AND Nintendo Entertainment System be the same? I can change 1 or 3 lines of code for that...

I'm not exactly sure what you're asking, but what I would like to see is any synopsis with either Platform: Famicom Disk System OR Platform: Nintendo Entertainment Systemthat is found in the Media/nes/Synopsis/ folder would only look in the /nes/ directories for the roms and media instead of /fds/.

Now that I think about it, that would probably be the best way to go. Somebody else might actually like splitting the two systems up and if they had synopsis files for the Famicom Disk System in Media/fds/Synopsis/ then it would still make a separate gamelist for the FDS.

Did I explain that good enough? I hope.... :)

Used2BeRX

@meleu BTW man... I sent you a message at the other place. Firefox keeps messing up my gmail account so I can't send any mail unless I uninstall and re-install it and I don't have time for that right now.

Used2BeRX

So I found this image:

alt text

It was 1000x1080 pixels, but didn't need all that blank space around the border so I minimized that wasted space while also keeping the removed background transparent.

Here's an example of the "labels" that I manged to snag. Not all of them include part of the cart like this... it was a really mixed bag:

alt text

Here's what the transparent label looks like when the background junk is removed:

alt text

And finally... we have our finished image. An actual FDS Disk image with the proper label for the game. :)

alt text

Can't even tell, can ya? ;)

959x1154 pixels pretty much ensures that this should still look good on a 4k TV down the road. Currently, according to one of my friends, a Pi 3 on a 1080 TV doesn't even handle images that are larger than 700 pixels on the large dimension, and these are essentially a square, so they'll probably even need to be shrunk down smaller than that to prevent lag on the Pi 3.

Used2BeRX

I decided I didn't like the original color of the image and that it didn't reflect the actual color of the FDS disks. I also noticed that some of the edges were rough, so I cleaned them up a bit and re-made the image for the first game restoration.

Here's the final look of the template, and what all the FDS disk images will look like in the collection.

alt text

meleu

@used2berx said in Making premade gamelist.xml with xtra media collections:

Didn't you say there was a command that you could run that could easily change that line in all of the FDS synopsis files in Linux? I mean, changing Platform: Nintendo Entertainment System back to Platform: Famicom Disk System?

I think you're talking abou this:

cd /path/to/FDS/synopsis/
sed -i 's/Platform: Nintendo Entertainment System/Platform: Famicom Disk System/' *.txt

Please, make a backup before running this command! I think it's perfectly OK, this warning is just to make you not blame me if something goes wrong. :-)

UDb23

@used2berx With Carbon theme (most commonly used) @1080p the expected scrape image width is around 500 pixels (bigger images are automatically resized).
Therefore your images will look and fit nicely.

phulshof

Out of curiosity: would it make sense to simply convert the HyperSpin XML files (where matching rom sets exist), and combine those with the matching art sets from HyperSpin and EmuMovies? That would save a lot of scraping, and provide you with complete sets of high quality art.

Used2BeRX

@meleu Yes. I believe that's the command. No worries.... I'll back it up first. :)

@UDb23, thanks for the info. I only use SD since I'm rocking the Pi Zero and I don't have an HDTV, so it's good to hear from somebody who knows. I'm trying to build all of this with the highest quality and future expansion in mind.

@phulshof Nah, that won't work for me. No offense to HS and EM, but their "high quality" art is hit or miss, as is everbody else's. I've grabbed the highest quality NES artwork from multiple sources, as well as google image search and at this point I've touched up 500-600 images, I've made a few hundred myself and I've easily done some minor cropping to another 1,000 images to get rid of rough edges.

In total right now, including the FDS games, there is nearly 1,950 games in this collection, and the Box and Cart artwork in JPG format is 2.79GB. There is a box and a cart for every game in the collection.

Nobody has an NES artwork set that looks anything close to this good. ;)

phulshof

@used2berx said in Making premade gamelist.xml with xtra media collections:

@phulshof Nah, that won't work for me. No offense to HS and EM, but their "high quality" art is hit or miss, as is everbody else's. I've grabbed the highest quality NES artwork from multiple sources, as well as google image search and at this point I've touched up 500-600 images, I've made a few hundred myself and I've easily done some minor cropping to another 1,000 images to get rid of rough edges.

In total right now, including the FDS games, there is nearly 1,950 games in this collection, and the Box and Cart artwork in JPG format is 2.79GB. There is a box and a cart for every game in the collection.

Nobody has an NES artwork set that looks anything close to this good. ;)

Fair enough. :) Might it be a good idea to hook up with the truRIP group then? They're currently working on creating complete sets (with planned PD distribution), and have access to high quality scans (since they own (almost) all of the games). They could provide good source material, while they could use the help in sorting out material.

Used2BeRX

@phulshof Yeah. I keep meaning to do that. Sorry I haven't gone over there yet. Would like to peek at their NES art just to see what they've got.

Going to be away from this for a few weeks now, but I'll join up after that.

Used2BeRX

@meleu Hey bud.

Going to be away from this work for a little while. Just wanted to update the script "wishlist" in case you get around to tinkering with it while I'm away.

Create the backup folder if it doesn't exist, and backup any gamelist.xml file before creating a new one if one of the append options isn't used on an exisiting gamelist.
Make option to change which artwork tag is set as <image> so you can switch between Box Front, Cart, Titles, Action and 3D Boxarts types after the gamelists are already made.
Platform: Nintendo Entertainment System and Platform: Famicom Disk System entries in the synopsis.txt files located in /Media/nes/Synopsis/ folder will treat these entries as if they were NES and not FDS games and look for roms and all media in the /nes/ folders instead.
Add <marquee> tag to both regular and FULL operations, looking for the media in /Media/[System]/Marquee/ folders. (Search for both .JPG and .PNG).

I believe that was it at this point.

Thanks again man. This script is awesome.

meleu

@used2berx OK bro, will do it when I have a chance.

meleu

@used2berx said in Making premade gamelist.xml with xtra media collections:

Platform: Nintendo Entertainment System and Platform: Famicom Disk System entries in the synopsis.txt files located in /Media/nes/Synopsis/ folder will treat these entries as if they were NES and not FDS games and look for roms and all media in the /nes/ folders instead.

Done! Please test it.

Add <marquee> tag to both regular and FULL operations, looking for the media in /Media/[System]/Marquee/ folders. (Search for both .JPG and .PNG).

Do you really mean that dir? I'm asking because looking at the directory structure of the Xtras files I have, I would bet in a folder like Media/[SYSTEM]/Artwork/Marquee.

EDIT: I've made the script look for marquees in Media/[SYSTEM]/Artwork/Marquee, tell me if it needs to change.

EDIT 2:
@used2berx said in Making premade gamelist.xml with xtra media collections:

Create the backup folder if it doesn't exist, and backup any gamelist.xml file before creating a new one if one of the append options isn't used on an exisiting gamelist.

I've just added the backup feature. But currently it doesn't work as you requested, the script creates backup at the start of every execution. The backups will be placed at used2betxt-backups/ folder. The file will be gzipped and the name will be the same as the original with a trailing time stamp. Example: megadrive_gamelist_2017-08-04_100510.xml.gz.

Used2BeRX

@meleu Hey bud,

Thanks for the update. I'm literally walking out the door right now so I probably won't be able to test this stuff for another 10 days to 2 weeks. :(

You were right about the Marquee folder. Thanks for catching that mistake.

I'm sure the backup feature the way you made it is just fine. I'll let you know if I have any suggestions on it when I get back and test it out.

All that is left at this point I believe is the ability to change what artwork is displayed on the fly. I was thinking that maybe the option to turn any of the folders into a <marquee> tag at some point might be beneficial as well since those are now two separate image types supported for any skins that allow for two.

Just because the tag says <marquee>, that doesn't mean that you have to use Marquee images. You could, for instance, have /Artwork/Box Front/ set as <image> and /Artwork/Cart/ set for Marquee, and then a skin that uses both images would actually display the Box Front and the Cart image for every game while you scrolled through the list.

Just an idea for the future!

Anyways.... I'll talk to you when I get back and let you know how the script tests out. Thanks again for all of your help meleu.

Stay out of trouble. ;)

Used2BeRX

@meleu said in Making premade gamelist.xml with xtra media collections:

@used2berx said in Making premade gamelist.xml with xtra media collections:

Platform: Nintendo Entertainment System and Platform: Famicom Disk System entries in the synopsis.txt files located in /Media/nes/Synopsis/ folder will treat these entries as if they were NES and not FDS games and look for roms and all media in the /nes/ folders instead.

Done! Please test it.

I'm back, but in limited capacity for a while. :)

I basically re-wrote all of the synopsis files, and one of the "fixes" was to change all of these back to have the Famicom Disk System instead of the NES. I will update your script and re-run everything and let you know how it goes in the next few days hopefully.

Add <marquee> tag to both regular and FULL operations, looking for the media in /Media/[System]/Marquee/ folders. (Search for both .JPG and .PNG).

Do you really mean that dir? I'm asking because looking at the directory structure of the Xtras files I have, I would bet in a folder like Media/[SYSTEM]/Artwork/Marquee.

EDIT: I've made the script look for marquees in Media/[SYSTEM]/Artwork/Marquee, tell me if it needs to change.

I'll throw some random images in a Marquee folder just to see if it's working. I don't actually have any marquee images, but like I said in the previous post, we could still actually make use of that tag in other ways, like displaying two images at the same time in skins that are being made to use that new tag.

EDIT 2:
@used2berx said in Making premade gamelist.xml with xtra media collections:

Create the backup folder if it doesn't exist, and backup any gamelist.xml file before creating a new one if one of the append options isn't used on an exisiting gamelist.

I've just added the backup feature. But currently it doesn't work as you requested, the script creates backup at the start of every execution. The backups will be placed at used2betxt-backups/ folder. The file will be gzipped and the name will be the same as the original with a trailing time stamp. Example: megadrive_gamelist_2017-08-04_100510.xml.gz.

I'll be sure to make backups on my PC of everything and just let it run and see what happens. I'll make any suggestions for tweaks after I see what it does. :)

Thanks again. I'll let you know when I've got a new gamelist.xml and tested it out.

Used2BeRX

@meleu Hey man. Sorry I haven't gotten back to you with a re-test yet. Things have been really busy for a while in RL. I've had a little time this week to get some major things done and I'm doing a lot of updates to the synopsis files for the NES.

I was wondering if I could ask for another request before I get around to a re-test. I was hoping that we could add an option to leave out the website information in the gamelist.xml entries. You probably know that every synopsis entry contains the link where the info came from. (Even better would probably be to leave the website out by default and only add it if you use an option to add them).

I don't believe that there is any way that this information could be beneficial for the end user. Maybe if there was some sort of brower functionality added to EmulationStation one day it would be cool to be able to open the links, but until then I think it would be a better presentation to leave this information out.

I like having the info in all of the files, so I was hoping that we could add some code that would eliminate the links. Maybe the best way to do it would be to finish copying the body of each file once a link has been encountered?

Thanks again man. I can't wait to try this out again once I get all of these files up to date and resize the artwork and give it a shot.

meleu

@used2berx can you provide an example?