Making premade gamelist.xml with xtra media collections

phulshof · 3 Aug 2017, 09:34

@used2berx said in Making premade gamelist.xml with xtra media collections:

@phulshof Nah, that won't work for me. No offense to HS and EM, but their "high quality" art is hit or miss, as is everbody else's. I've grabbed the highest quality NES artwork from multiple sources, as well as google image search and at this point I've touched up 500-600 images, I've made a few hundred myself and I've easily done some minor cropping to another 1,000 images to get rid of rough edges.

In total right now, including the FDS games, there is nearly 1,950 games in this collection, and the Box and Cart artwork in JPG format is 2.79GB. There is a box and a cart for every game in the collection.

Nobody has an NES artwork set that looks anything close to this good. ;)

Fair enough. :) Might it be a good idea to hook up with the truRIP group then? They're currently working on creating complete sets (with planned PD distribution), and have access to high quality scans (since they own (almost) all of the games). They could provide good source material, while they could use the help in sorting out material.

Used2BeRX · 3 Aug 2017, 18:46

@phulshof Yeah. I keep meaning to do that. Sorry I haven't gone over there yet. Would like to peek at their NES art just to see what they've got.

Going to be away from this for a few weeks now, but I'll join up after that.

Used2BeRX · 3 Aug 2017, 21:48

@meleu Hey bud.

Going to be away from this work for a little while. Just wanted to update the script "wishlist" in case you get around to tinkering with it while I'm away.

Create the backup folder if it doesn't exist, and backup any gamelist.xml file before creating a new one if one of the append options isn't used on an exisiting gamelist.
Make option to change which artwork tag is set as <image> so you can switch between Box Front, Cart, Titles, Action and 3D Boxarts types after the gamelists are already made.
Platform: Nintendo Entertainment System and Platform: Famicom Disk System entries in the synopsis.txt files located in /Media/nes/Synopsis/ folder will treat these entries as if they were NES and not FDS games and look for roms and all media in the /nes/ folders instead.
Add <marquee> tag to both regular and FULL operations, looking for the media in /Media/[System]/Marquee/ folders. (Search for both .JPG and .PNG).

I believe that was it at this point.

Thanks again man. This script is awesome.

meleu · 4 Aug 2017, 00:16

@used2berx OK bro, will do it when I have a chance.

meleu · 8 Apr 2017, 15:12

@used2berx said in Making premade gamelist.xml with xtra media collections:

Platform: Nintendo Entertainment System and Platform: Famicom Disk System entries in the synopsis.txt files located in /Media/nes/Synopsis/ folder will treat these entries as if they were NES and not FDS games and look for roms and all media in the /nes/ folders instead.

Done! Please test it.

Add <marquee> tag to both regular and FULL operations, looking for the media in /Media/[System]/Marquee/ folders. (Search for both .JPG and .PNG).

Do you really mean that dir? I'm asking because looking at the directory structure of the Xtras files I have, I would bet in a folder like Media/[SYSTEM]/Artwork/Marquee.

EDIT: I've made the script look for marquees in Media/[SYSTEM]/Artwork/Marquee, tell me if it needs to change.

EDIT 2:
@used2berx said in Making premade gamelist.xml with xtra media collections:

Create the backup folder if it doesn't exist, and backup any gamelist.xml file before creating a new one if one of the append options isn't used on an exisiting gamelist.

I've just added the backup feature. But currently it doesn't work as you requested, the script creates backup at the start of every execution. The backups will be placed at used2betxt-backups/ folder. The file will be gzipped and the name will be the same as the original with a trailing time stamp. Example: megadrive_gamelist_2017-08-04_100510.xml.gz.

Used2BeRX · 4 Aug 2017, 18:03

@meleu Hey bud,

Thanks for the update. I'm literally walking out the door right now so I probably won't be able to test this stuff for another 10 days to 2 weeks. :(

You were right about the Marquee folder. Thanks for catching that mistake.

I'm sure the backup feature the way you made it is just fine. I'll let you know if I have any suggestions on it when I get back and test it out.

All that is left at this point I believe is the ability to change what artwork is displayed on the fly. I was thinking that maybe the option to turn any of the folders into a <marquee> tag at some point might be beneficial as well since those are now two separate image types supported for any skins that allow for two.

Just because the tag says <marquee>, that doesn't mean that you have to use Marquee images. You could, for instance, have /Artwork/Box Front/ set as <image> and /Artwork/Cart/ set for Marquee, and then a skin that uses both images would actually display the Box Front and the Cart image for every game while you scrolled through the list.

Just an idea for the future!

Anyways.... I'll talk to you when I get back and let you know how the script tests out. Thanks again for all of your help meleu.

Stay out of trouble. ;)

Used2BeRX · 21 Aug 2017, 17:13

@meleu said in Making premade gamelist.xml with xtra media collections:

@used2berx said in Making premade gamelist.xml with xtra media collections:

Platform: Nintendo Entertainment System and Platform: Famicom Disk System entries in the synopsis.txt files located in /Media/nes/Synopsis/ folder will treat these entries as if they were NES and not FDS games and look for roms and all media in the /nes/ folders instead.

Done! Please test it.

I'm back, but in limited capacity for a while. :)

I basically re-wrote all of the synopsis files, and one of the "fixes" was to change all of these back to have the Famicom Disk System instead of the NES. I will update your script and re-run everything and let you know how it goes in the next few days hopefully.

Add <marquee> tag to both regular and FULL operations, looking for the media in /Media/[System]/Marquee/ folders. (Search for both .JPG and .PNG).

Do you really mean that dir? I'm asking because looking at the directory structure of the Xtras files I have, I would bet in a folder like Media/[SYSTEM]/Artwork/Marquee.

EDIT: I've made the script look for marquees in Media/[SYSTEM]/Artwork/Marquee, tell me if it needs to change.

I'll throw some random images in a Marquee folder just to see if it's working. I don't actually have any marquee images, but like I said in the previous post, we could still actually make use of that tag in other ways, like displaying two images at the same time in skins that are being made to use that new tag.

EDIT 2:
@used2berx said in Making premade gamelist.xml with xtra media collections:

Create the backup folder if it doesn't exist, and backup any gamelist.xml file before creating a new one if one of the append options isn't used on an exisiting gamelist.

I've just added the backup feature. But currently it doesn't work as you requested, the script creates backup at the start of every execution. The backups will be placed at used2betxt-backups/ folder. The file will be gzipped and the name will be the same as the original with a trailing time stamp. Example: megadrive_gamelist_2017-08-04_100510.xml.gz.

I'll be sure to make backups on my PC of everything and just let it run and see what happens. I'll make any suggestions for tweaks after I see what it does. :)

Thanks again. I'll let you know when I've got a new gamelist.xml and tested it out.

Used2BeRX · 5 Oct 2017, 23:46

@meleu Hey man. Sorry I haven't gotten back to you with a re-test yet. Things have been really busy for a while in RL. I've had a little time this week to get some major things done and I'm doing a lot of updates to the synopsis files for the NES.

I was wondering if I could ask for another request before I get around to a re-test. I was hoping that we could add an option to leave out the website information in the gamelist.xml entries. You probably know that every synopsis entry contains the link where the info came from. (Even better would probably be to leave the website out by default and only add it if you use an option to add them).

I don't believe that there is any way that this information could be beneficial for the end user. Maybe if there was some sort of brower functionality added to EmulationStation one day it would be cool to be able to open the links, but until then I think it would be a better presentation to leave this information out.

I like having the info in all of the files, so I was hoping that we could add some code that would eliminate the links. Maybe the best way to do it would be to finish copying the body of each file once a link has been encountered?

Thanks again man. I can't wait to try this out again once I get all of these files up to date and resize the artwork and give it a shot.

meleu · 6 Oct 2017, 00:34

@used2berx can you provide an example?

Used2BeRX · 6 Oct 2017, 02:23

@meleu Sure thing. :)

Murder on the Mississippi.txt

Murder on the Mississippi
Original Title: Mississippi Satsujin Jiken
Platform: Nintendo Entertainment System
Region: Translated (Japan)
Media: Cartridge
Controller: NES Gamepad
Genre: Text Adventure
Gametype: Translated
Release Year: 2017
Translated By: GAFF Translations
Version: 1.00
Players: 1
_________________________

The famous detective Sir Charles Foxworth is taking a holiday on the Mississippi with his servant Regis on the Delta Princess. He's just taking a stroll around the upper deck of the ship when he discovers an open cabin door. Behind it: a dead man. As it turns out, the body of Raleigh Cartwright III. Sir Charles has three days to find the murderer, because then, the ship will reach New Orleans, giving the culprit a safe chance to escape.

The joystick controls the player's sprite to move into the respective direction, and pressing the button opens a menu with appropriate commands for the current situation. Walk around, Inspect an item, or examine evidence.

You will meet some characters who also happen to be suspects. Use these encounters as a fact-finding session. Take notes on what he or she has said. Share these notes with other passengers and crew members. But your first job is to find the body. Until you do, any information you gather will be of little value. Once you find the body, the other passengers will prove to be much more talkative.

Collect as many notes and evidence as you can to solve the mystery. Once you think that you have it solved, make an accusation by going to the character that you want to accuse and by using the joystick, point to Talk to [character's name], and press the joystick button. Then point to Accuse [character's name] and press the joystick button. Depending on the amount of evidence collected, you will know if your hunch was right or wrong.

Murder on the Mississippi has four possible endings with many variations on each one. Don't let the Delta Princess arrive to New Orleans before you solve the mystery, or the murderer will get away.

http://www.romhacking.net/translations/3133/
http://www.mobygames.com/game/nes/murder-on-the-mississippi

meleu · 6 Oct 2017, 11:58

@used2berx removing any line with only an URL is OK for you?

Used2BeRX · 6 Oct 2017, 19:48

@meleu That would be perfect.

At one point my synopsis files would contain links on occasion inside of the body. This was generally for hacks and translations that would have addendum patches to them. I'm in the process of updating all of these files and that type of information won't be in the synopsis anymore. If somebody wants to read any of that, that's what the links are for ;)

The new synopsis files I'm working on will be short, sweet and to the point. They will talk about the game and not about patch information.

That being said, there will probably be weird issues on some of the files for the non-NES systems that haven't been updated, but I'll get to everything eventually. :)

Used2BeRX · 25 Oct 2017, 02:09

@meleu Hey buddy.

I was wondering if there was any way to revert the gamelist.xml back into the individual synopsis.txt files?

I was thinking that it would be cool to be able to take the gamelist.xml file and put it all into a spreadsheet so I could do some serious editing of individual cells, so I could have uniform names for all of the different game genres, for instance.

I haven't figured out any good way to convert my synopsis.txt files into a spreadsheet, but I'm thinking that I should have much better luck doing that with the XML file. I thought it would be a lot easier for me to do that and edit them that way rather than individually going through about 2,000 games. I was also thinking that this would make my work MUCH easier when I finally start on other systems to have a way to easily edit and convert these things to the 3 different formats. (synopsis.txt, gamelist.xml and a spreadsheet).

I'd, of course, have to figure out how to convert a spreadsheet back to a gamelist.xml file before converting them back to synopsis.txt as well.

I'm just kind of spit-balling ideas here since I'm not really good at this type of stuff at all. I've spent a ridiculous amount of hours doing most of this work by hand individually for all the games, but a system like this could eventually save me thousands of hours once I get around to all of the other systems. I'm kind of in the process of learning "what not to do" in the future.

I'm open to any ideas you have, or anybody else who has a good idea.

BTW... I've also added a few more categories for the synopsis files. For instance, the Translated Games now have a Release Year: (Original Game Release Year) and a Translation Release Year:. I now have synopsis information on the Translations that reflects both the translation itself as well as the original game information. They also show the Developer: and Publisher: of the original game as well as the Translated By: field.

I'm in the process of "upgrading" around 300-400 hacks to do the same. They'll have a Hack Release Year: on them.

I've also added Players: information for every Translation and Hack in the set by verifying the players on each game individually as I was testing them.

Lot's of work being done behind the scenes man. I still don't want to make the spreadsheet public, but if you have the link for it still you should check it out. Lots of big updates reflected there, and it should be a lot more self-explanatory than it was the last time you looked.

Take it easy. ;)

meleu · 25 Oct 2017, 11:26

@used2berx gamelist.xml to synopsis.txt can be done. But converting to/from a spreadsheet is not that easy. But if you are able to edit xml files on Excel or other spreadsheet program, then probably you're also able to save your modifications on a xml file. Aren't you?

When I find some time I'll try to make that gamelist.xml to synopsis.txt feature.

Cheers!

Used2BeRX · 25 Oct 2017, 16:44

@meleu Well, the reason I wanted to do it in a spreadsheet is because then for any different field I could alphabetaize them and go through and make any fixes for any typos and stuff. I keep using the Genre as an example because it really needs the work. Sometimes it's RPG and other times it's Role Playing Game. Sometimes it's Sports - Basketball, other times it's Sports / Basketball.

If I was somehow able to get them all into a spreadsheet to edit, and then somehow revert them, it would be much easier to do this editing.

Looking at them in the romlist on the Pi will be able to help a little, but the problem there is that about 1/2 of the fields in my synopsis files aren't yet implemented at all in EmulationStation so I can't check them in there.

meleu · 25 Oct 2017, 17:36

@used2berx sometimes I feel like if you were my neighboor we would solve all this editing stuff that takes hundreds of hours of your time in an afternoon. :-)

Basic regular expression and sed knowledge would help you a lot!

Used2BeRX · 25 Oct 2017, 17:48

@meleu You're probably right.

I was never very good at studying. I tried to get into computer programming but it was such a drag.

Funny thing is I can literally spend hundreds or even thousands of hours doing stuff by hand rather than learn how to code things. It soothes my OCD itch. There's not many things I'd less rather do than try to learn from a book. Maybe if I had a good teacher it would be a different story.

I've got to do something about this problem here though. There's only 2,000 games in the NES compared to all of them in the other systems. I'm getting to the age where I don't know if I have enough time left on earth to get it all done if I don't start doing a few things differently.

Used2BeRX · 31 Oct 2017, 22:14

@meleu Hey Meleu...

You don't know of any program or an easy way I could input the gamelist.xmlfile and be able to sort the values inside like a spreadsheet or like in MS Access, do you?

Maybe I'm making this more complicated than it needs to be.

EmulationStation itself already does this for any of the fields that are displayed in the romlist/skins. Maybe there's another simple program out there that can just display all of the info if I tell it what fields to look for without actually launching any roms or anything?

meleu · 31 Oct 2017, 23:51

@used2berx sorry, I didn't get what you want exactly. Could you give an example?

Used2BeRX · 31 Oct 2017, 23:59

@meleu I don't have the Pi available right now, but I recall that somewhere in the romlists you can choose to sort by different fields, such as Genre or # of players.

I'd be looking for some sort of way to be able to do this for all of the fields... even the ones not currently supported by ES.

It would be ideal if I could actually edit the XML from wherever I was seeing this sorted data because I'd be able to convert it back to synopsis.txt files if you made that script. But even if I can't still edit this info from here I'd at least know which synopsis.txt files I needed to go in and edit.