Making premade gamelist.xml with xtra media collections
-
@mediamogul I didn't ban him. If I did, it's because I'm on mobile and it was accidental with random button presses. I don't know if there are logs on who did what, @BuZz or @dankcushions ?
If you read the documentation it will answer 99% of your questions: https://retropie.org.uk/docs/
Also if you want a solution to your problems read this first: https://retropie.org.uk/forum/topic/3/read-this-first
-
@herb_fargus @mediamogul Thanks for looking into it guys. Sorry about creating a new account, but I couldn't figure out any other way to contact anybody here about the ban.
-
it wasn't me, officer! :)
-
Almost everyone is accounted for except for that suspicious new moderator, @ColonelMustard.
-
I'm going to take the liberty of unbanning your original account, as it has all the markings of an accident. Since every ban is added to your record, I also made a note that it seems to have been unintentional in some way.
-
@mediamogul Thanks to you and everyone else for looking into it.
If you do find out why somebody did it and that it wasn't an accident, please let me know what I did and I will not do it again in the future.
-
@UDb23 I was in the middle of giving you a long reply about the program. It looks great. I tried 5-10 of the Atari 7800 txt files and there weren't any problems. I tried 5 NES files and 2 of them gave me errors. I had them all copied to send to you, but my laptop with all my pi work died on me. I think the motherboard or at least the GPU is fried. I don't know how long it's going to take me to get set up again. I don't have any of the work on this old PC I'm typing from and I can't afford a new laptop.
I might have time over the next few days to try to at least round up the synopsis files from other systems to give you a list, but I'm kind of depressed now. I just lost a few months of work.
-
@Used2BeRX uh, that's bad! Really sorry about it.
You should still be able to access the hd of your laptop. There are quite cheap adaptors to connect laptop hd's to a PC. Hope you can fix your laptop and/or recover your files. -
@UDb23 said in Making premade gamelist.xml with xtra media collections:
@Used2BeRX uh, that's bad! Really sorry about it.
You should still be able to access the hd of your laptop. There are quite cheap adaptors to connect laptop hd's to a PC. Hope you can fix your laptop and/or recover your files.LOL... I don't think I'm going to be able to fix the laptop. It's now sitting in my garbage in about a thousand pieces :)
I'm sure I can get the data off of it. The failure was either the motherboard or GPU. I'm just seriously limited on funds right now and need to get a job. I have to wait on my brother to get me a 2TB external drive to get back into any serious work on this stuff and to get the data off of that drive. He loaned it out to somebody else a while back but they're only using it for Wii games.
This work is too important though and I didn't want to lose your interest by stalling so I pulled all the synopsis files for all of the systems so I could test this out again on another computer and give you initial feedback. I'm really impressed with it so far and I can't wait to see where this goes.
Brace yourself.... I've got a LOT of feedback for you, and I'm going to have more info for you hopefully later today. Here we go........
I started with the 7800 even though you made this with the 2600 in mind. It was a much smaller list of only 90 games so I was able to test every single file in about 10 minutes.
7800 TEST RUN (EVERY GAME TXT FILE TESTED):
I got an error and couldn't create the file for the following games:
Beef Drop - 10 new levels.txt
Beef Drop - More Beef Drop.txt
bonQ.txt
Centipede.txt
FailSafe.txt
Fatal Run.txt
Meltdown.txt
Ms Pac-Man.txt
Pac-Man Collection.txt
Robotron 2084.txt
Snowday Jr Pac-Man.txt
Tomcat F 14 Simulator.txt
Tower Toppler.txtHere's the Error:
Unhandled exception has occurred in your application. If you click Continue, the application will ignore this error and attempt to continue. If you click Quit, the application will close immediately. The given path's format is not supported. DETAILS: See the end of this message for details on invoking just-in-time (JIT) debugging instead of this dialog box. ************** Exception Text ************** System.NotSupportedException: The given path's format is not supported. at System.Security.Permissions.FileIOPermission.EmulateFileIOPermissionChecks(String fullPath) at System.Security.Permissions.FileIOPermission.QuickDemand(FileIOPermissionAccess access, String fullPath, Boolean checkForDuplicates, Boolean needFullPath) at Microsoft.VisualBasic.FileIO.FileSystem.NormalizePath(String Path) at Microsoft.VisualBasic.FileIO.FileSystem.CombinePath(String baseDirectory, String relativePath) at Microsoft.VisualBasic.MyServices.FileSystemProxy.CombinePath(String baseDirectory, String relativePath) at MetaX.Form1.ConvertButton_Click(Object sender, EventArgs e) at System.Windows.Forms.Control.OnClick(EventArgs e) at System.Windows.Forms.Button.OnClick(EventArgs e) at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent) at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks) at System.Windows.Forms.Control.WndProc(Message& m) at System.Windows.Forms.ButtonBase.WndProc(Message& m) at System.Windows.Forms.Button.WndProc(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m) at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam) ************** Loaded Assemblies ************** mscorlib Assembly Version: 4.0.0.0 Win32 Version: 4.6.1648.0 built by: NETFXREL3STAGE CodeBase: file:///C:/Windows/Microsoft.NET/Framework/v4.0.30319/mscorlib.dll ---------------------------------------- MetaX Assembly Version: 1.0.0.0 Win32 Version: 1.0.0.0 CodeBase: file:///C:/Users/The%20Dude/AppData/Local/Apps/2.0/Q5PQQAWH.RQH/9XWRQPVQ.KZM/meta..tion_9bae49a2533cd5b2_0001.0000_1813f733e3f252fe/MetaX.exe ---------------------------------------- Microsoft.VisualBasic Assembly Version: 10.0.0.0 Win32 Version: 14.6.1586.0 built by: NETFXREL2 CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/Microsoft.VisualBasic/v4.0_10.0.0.0__b03f5f7f11d50a3a/Microsoft.VisualBasic.dll ---------------------------------------- System.Windows.Forms Assembly Version: 4.0.0.0 Win32 Version: 4.6.1586.0 built by: NETFXREL2 CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Windows.Forms/v4.0_4.0.0.0__b77a5c561934e089/System.Windows.Forms.dll ---------------------------------------- System Assembly Version: 4.0.0.0 Win32 Version: 4.6.1647.0 built by: NETFXREL3STAGE CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System/v4.0_4.0.0.0__b77a5c561934e089/System.dll ---------------------------------------- System.Drawing Assembly Version: 4.0.0.0 Win32 Version: 4.6.1586.0 built by: NETFXREL2 CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Drawing/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll ---------------------------------------- System.Configuration Assembly Version: 4.0.0.0 Win32 Version: 4.6.1586.0 built by: NETFXREL2 CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Configuration/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll ---------------------------------------- System.Core Assembly Version: 4.0.0.0 Win32 Version: 4.6.1647.0 built by: NETFXREL3STAGE CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Core/v4.0_4.0.0.0__b77a5c561934e089/System.Core.dll ---------------------------------------- System.Xml Assembly Version: 4.0.0.0 Win32 Version: 4.6.1586.0 built by: NETFXREL2 CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Xml/v4.0_4.0.0.0__b77a5c561934e089/System.Xml.dll ---------------------------------------- System.Runtime.Remoting Assembly Version: 4.0.0.0 Win32 Version: 4.6.1586.0 built by: NETFXREL2 CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Runtime.Remoting/v4.0_4.0.0.0__b77a5c561934e089/System.Runtime.Remoting.dll ---------------------------------------- Accessibility Assembly Version: 4.0.0.0 Win32 Version: 4.6.1586.0 built by: NETFXREL2 CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/Accessibility/v4.0_4.0.0.0__b03f5f7f11d50a3a/Accessibility.dll ---------------------------------------- ************** JIT Debugging ************** To enable just-in-time (JIT) debugging, the .config file for this application or computer (machine.config) must have the jitDebugging value set in the system.windows.forms section. The application must also be compiled with debugging enabled. For example: <configuration> <system.windows.forms jitDebugging="true" /> </configuration> When JIT debugging is enabled, any unhandled exception will be sent to the JIT debugger registered on the computer rather than be handled by this dialog box.I took a look at the failed files, and I think I spotted the potential problems with them.
Beef Drop - 10 New Levels.txt, Beef Drop - More Beef Drop.txt, Robotron - 2084.txt, Snowday - Jr Pac-Man.txt, Tomcat F 14 Simulator.txt
These have a ":" in the title. There are a LOT of games over all the systems that will have this character as I thought it looks a lot better in the romlist than using the " - " between the title and sub-title. For example "Beef Drop: 10 New Levels".
(NOTE: Snowday Jr Pac-Man.txt also has a "." in the title, so probably would have failed for that as well)
bonQ.txt
The title has an asterisk in it. It is actually b**nQ. Would only be a problem with games like Q*bert and such.Centipede.txt
A ";" in the Players line. Players: 1 or 2 VS; CO-OP
I believe there are a lot of instances of this over all the systems.Ms Pac-Man.txt
Maybe the "." in "Ms. Pac-Man"???Pac-Man Collection.txt
Maybe the "!" in "Pac-Man Collection!" or the "(" and ")" in "Region: Homebrew (USA)"???Tower Toppler.txt
Maybe the ":" in the description? There's a section toward the bottom that has hints and says "Hints: "I'm thinking that the program should avoid any colons while it's scraping text since a lot of the Game Titles will have them and they will probably creep up in other places like this example.
This would also explain the failures for Meltdown.txt, Fatal Run.txt and FailSafe.txt
I hope I'm right about all of these and saved you some time trying to figure out what went wrong.
I also tested the first 25 text files for the NES and had 10 failures. I didn't check why they failed yet. Here's the list of games:
10-Yard Fight.txt
1942.txt
1943.txt
3-D Battles of WorldRunner - Part II
720.txt
75 Bingo.txt
8 Eyes.txt
A Boy and His Blob.txt
A Nightmare On Elm Street.txt
Abadox.txtHere's a full upload of the NES and 7800 synopsis files for you to check out:
https://www.sendspace.com/file/ueaeig
I will try to look at files from every system and get you a list of all of the possible tags that the program should look for. Most of them aren't currently used by any skins, but a few guys from our project have made their way over here and are currently working on their first skin, so might as well have all the info just in case it gets used down the road.
A FEW OTHER NOTES:
-
crcindex.txt = This was a file that had the crc's for all of the games. It was a necessary part of getting the synopsis files running quickly on another system. It should just be avoided by the program.
-
skipped.txt = I'm not sure what this file was for, but it should be avoided by the program too.
-
I'd like to automatically add a line in these XML files that would add a Marquee image. For example we'll use the Atari 7800 game "Ace of Aces".
<marquee>/home/pi/RetroPie/Media/A7800/Artwork/Marquee/Ace Of Aces.jpg</marquee>
- I noticed that all "media" lines in the program are "/Media/A2600/". I know that you planned that at this stage. Just wanted to hilight that this is going to obviously be different from system to system and we have to decide whether we're actually searching for media in the program before making these media lines or just making them all like your program does now but we have to specify a different folder here per system at some point then.
EDIT:
I'll add other things I notice in this spot until you see the message and reply.I noticed on Ace of Aces.txt for the 7800 that it didn't carry over the "Genre" category.
The .txt file had this line: "Genre: Flight Simulator - Action"
The .xml file has this line: "<genre></genre>"
Maybe it doesn't like the " - "?
-
-
@Used2BeRX You did quite extensive testing !
The program uses original txt path & filename to write the xml: therefore if the original name includes "uncommon" characters it may generate the errors you got.
It uses a specific VB streamwrite command so I have to check if, as it seems, special characters cannot be used and if there is a workaround.
I'll come back to you on this when I have done specific tests on the files that actually generate errors.Concerning paths, as said, I'm going to add specific path fields the user can customize as needed.
Will look also into the missing <genre> issue.
-
@UDb23 Yep. :) I want to be really kicking the tires on this and making it as awesome as I think it can be.
I've got a question out to the guy who knows the most about our online synopsis system to see if there's an easy way to know for sure what fields are used for each system so I don't have to just open a bunch of games and try to find them. It looks as though the editor does not spit out empty fields, so it would be difficult to try to figure it out by hand. If I have to do that though, I will.
I'm also going to spend some more time with this hopefully later today or by tomorrow to see if I have any more suggestions.
Thanks again.
-
@Used2BeRX further testing on my side will take a couple of days; so no need to run to list all fields for you.
-
@UDb23 I have some free time the next few days so it's no problem. You're going to need that list eventually, so better to get it out of the way. The next few weeks are going to be a bit spotty for me after that, so I'll have less time to test this out, but it will be a top priority for me to get feedback to you ASAP so you can get back to it without waiting on me.
EDIT:
Looks like we're just going to make this easy (hopefully). I went into the synopsis system and from console to console the only fields they have for entering game info now are the following:
Xtras Name: (Extras name in the system means the file name and extension we use for each game)
Title: (Name of the game. The top line in all the TXT files)
Original Title: (Only used for Hacks or Translated games)
Alternate Title: (Only used for Hacks or Translated games)
Release Year:
Region:
Genre:
Gametype:
Developer:
Publisher:
Players:
Controller:
Media:
Hacked by:
Translated by:
Version: (Version of the hack/translation)I'd say just make sure the above fields are all accounted for, and leave in all of the fields you have currently. Any games they've added since they put the system online don't account for fields like musician or programmer, but it looks like if they were in there back when I was making this stuff by hand years ago that they've remained. No reason to take them out I guess.
If any problems come up or if I notice things we can talk about that when they occur.
-
@Used2BeRX Thanks for the "items/tag" list; with this I can update the txt parsing function in the program to catch all these.
I don't see the <genre> you mentioned previously; is it the same as <gametype> ? -
@UDb23 Whoops. I forgot Genre. That's different than gametype. The gametype specifies if it's an official release or a hack or translation or prototype etc.
Also, the "Xtras Name" won't be in any of the files. The txt file name IS the Xtras name 99.9% of the time. We're going to probably run into some problems down the road with this though since case sensitivity was not an issue on the other platform but will be here. Most of the time they're good, but for example a game like "Attack of the Mutant Camels.zip" might be "Attack Of The Mutant Camels.xxx" for one or more of the media pieces.
-
@Used2BeRX I can Camel Case (funny that you used Mutant Camels game as example!) the filenames of the xml if required. It's probably easier if you rename txt files and related media by using an utility like Bulk Rename to have all consistant camel cased names.
-
@UDb23 I thought Herb_Fergus was making a joke earlier in this thread or somewhere in another thread after I used that example. What does Camel Case mean?
I suppose if need be I can do a Bulk Rename on everything. My initial intention was to be able to use the exact same packs as were originally created and have been floating around the internet for the last 7-8 years, but I've had to make drastic changes to stuff for a Pi Zero pack anyhow.
What I'm really not looking forward to having to do is making romcenter datfiles with our naming conventions. I don't think that there is a way to make one file for each system that takes into account the subfolders. That would make life much easier, but I think I need to make a separate datfile for every sub-folder. Some of the systems like the NES have a crazy amount of nested folders.
I guess it really doesn't matter about the case now. I could just make every single file all upper or lower case since the game name on the emu will show as whatever is in the gamelist.xml and look nice.
When these sets were originally made, there were two things we had to deal with. None of the emulators on the other platform were set up initially to show anything other than the file name and extension (that was fixed years ago), and the other one was the filename limit was 42 characters including the extension.
-
@Used2BeRX Camel Case means having all first letters of words in a sentence in upper case;
lower case: star wars: a new hope
Camel Case:Star Wars: a New Hope
So you can keep those rom names with the initial word capitals and make all the names consistant by applying "camel case" to all txt and media files using an utility like "Bulk rename".
Also visual basic has a dedicated function to "camel case" a sentence. -
@UDb23 Ah... well for the most part I've got it Camel Case already then. But your example brings up the problem with that. I would have named the rom "Star Wars - A New Hope", but maybe somebody helping me put the packs together might have named a piece or two of the artwork "Star Wars - a New Hope". Like I said, it doesn't happen very often, but it went completely overlooked since nothing was case sensitive.
Too bad there isn't just a way to turn off case sensitivity. I know for sure everything matches up perfectly if that's not a factor.
EDIT: I did finally take the time to re-size all of my artwork to around 240x180 for the Pi Zero I'm working on. I was having problem with larger images with how many I had. Having the gamelist.xml for just the 4 atari systems was enough to keep emulators from launching before. I reduced the VMM to only 50MB like somebody suggested in another thread and the smaller images seem to scroll a lot smoother on the list. Time will tell if I can use images with the Pi Zero once we start making the XML's for larger collections like the NES and Gameboy.
I haven't re-"cased" anything yet. I'm kind of on a time frame to get this Pi back to my brother for a gift to his brother in law, so I don't want to be changing everything until I know one way or another whether he's going to get this Pi with artwork or not. I'll be getting a replacement to continue the work on when I give this one back though.
-
@UDb23 Case Sensitivity will not be an issue. I decided to make things easy and I just Bulk Renamed every file to all uppercase filenames and extensions. I haven't done that to the Atari systems that I've already made gamelist.xml files for already, but I'm sure I will once this program is working.
Any luck ironing out any of those issues I found earlier?
I don't know how hard it would be to do, but I was hoping that there could be a function in the program that would scan the roms directory and the sub-folders for the file match when it creates the line for where the rom exists. Every system I have comes with sub-folders that separate official titles from all sorts of hacks, translations, prototypes, homebrew and other categories. Putting them all in one folder would be a mess of a romlist that nobody would want to play, and having to go in and manually change almost 10,000 entries over all of the systems by hand would largely defeat the purpose of this program.
Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.
Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.