Making premade gamelist.xml with xtra media collections

Used2BeRX

@UDb23 Sweet. Just let me know if you need any more input before making the first compile. Otherwise I should be around to take it for a spin whenever you've got it ready.

Used2BeRX

@UDb23 Since I haven't heard back from you yet I figure you're still putting it together. I was hoping you could add a field in the first 10 <marquee></marquee> I don't have any marquee images myself, but some guys I know that are making a skin are using that field and making marquee style images by cropping from title screens.

UDb23

@Used2BeRX Sure I can add this additional tag.
Program development is progressing and I'll provide you a first working version over the weekend.

Used2BeRX

@UDb23 Nice.

I'm going to ask around here and see if I can get some answers on memory issues. I actually "broke" the Pi Zero with just the hand made XML files for only the 4 Atari systems. EmulationStation was taking 2-3 minutes to load and some emulators stopped functioning altogether. I'm just wondering if loading pointers to data that doesn't exist is draining memory resources or not. And I'm also wondering if every single picture you load is draining memory at one time. Seems terribly inefficient to me. I'll be able to test our work here one system at a time, but then I have to rename the XML file so it doesn't get loaded when I go on to the next system.

Anyways, maybe down the road there can be options made in your program to expand or limit exactly what fields the program adds to the XML file depending on what I find out about how things are loaded.

UDb23

@Used2BeRX First beta of the Application to convert your .txt to .xml is ready.
You can find it here.
Just unzip the MetaX.zip and launch Setup.exe.

Use the "Browse" button in the App to select the folder that contains the txt files.
Currently only one by one conversion works: select the file and click "convert" button.

I found that the game metadata source txt files are not consistent meaning some info may or may not be included; also same info (e.g. "developer" can be in different line in one txt and in another line in a different txt).

The application tries to find the specific "tags" regardless of line order; it also identifies the "description" by selecting the lines after the line with "________".

Could you let me know the complete list of "items/tags" the App should:

look for in the source txt files
write to the .xml files (on top of items found in the .txt)

Just want to first make sure everything is converted correctly as single files, and then implement "convert all" and "create global gamelist.xml" functionality.

App requires .net libraries installed (created in VB with Visual Studio community 2017); I suppose it will ask during installation (can't test on another PC).

note: if you convert the same txt multiple times, it will append lines to already existing corresponding .xml (we could call it a "known bug" ;-) ).

Note 2: App does not do anything else as converting txt to xml so it should be totally harmless. In any case keep a backup of your original txt files.

Used2BeRX

@UDb23 Awesome man. Thanks.

I downloaded it, but haven't had time to use it. I should have some feedback for you tomorrow or Tuesday night at the latest. Thanks so much for this again.

genericuserdude

@UDb23 I somehow got banned earlier this morning with no reason. I think it was a mistake. Just wanted to post here that I'm working on trying to get to the bottom of this. If somebody has it out for me for some reason and I get banned with this ID as well, please reach out to me as I think we're on to something really good here and I don't have any way of keeping contact with you right now. I couldn't figure out a way to send a PM, so here's the hail mary.

Please contact me on this online document if the ban doesn't get lifted until we come up with something better: https://docs.google.com/document/d/18cfXi3eO7Gj1piWbzans8zS5CQ2ATaUrxeM0jmdzal8/edit?usp=sharing

UDb23

@genericuserdude ok.
Maybe you should ask @Global-Moderators why this happened.

genericuserdude

@UDb23 said in Making premade gamelist.xml with xtra media collections:

@Global-Moderators

Okay. At least you and the other guy I have been working on another project has seen the link. Please save it in case I get banned again and they remove the posts so we can continue our work.

@Global-Moderators Could somebody please tell me what is going on with the ban?

mediamogul

@genericuserdude

Could somebody please tell me what is going on with the ban?

I just checked the ban history for @Used2BeRX and it says "no reason given" which is not usually done. Also, after a quick look over the recent post history for that account, I don't see anything that should be offensive. I guess it could have been a mistake, but it's a permanent ban and that takes a small amount of extra doing. It can still be reversed, but I'd like to see if there was any purposeful intention from any of the other moderators or admins first.

herb_fargus

@mediamogul I didn't ban him. If I did, it's because I'm on mobile and it was accidental with random button presses. I don't know if there are logs on who did what, @BuZz or @dankcushions ?

genericuserdude

@herb_fargus @mediamogul Thanks for looking into it guys. Sorry about creating a new account, but I couldn't figure out any other way to contact anybody here about the ban.

dankcushions

it wasn't me, officer! :)

mediamogul

Almost everyone is accounted for except for that suspicious new moderator, @ColonelMustard.

mediamogul

@genericuserdude

I'm going to take the liberty of unbanning your original account, as it has all the markings of an accident. Since every ban is added to your record, I also made a note that it seems to have been unintentional in some way.

genericuserdude

@mediamogul Thanks to you and everyone else for looking into it.

If you do find out why somebody did it and that it wasn't an accident, please let me know what I did and I will not do it again in the future.

Used2BeRX

@UDb23 I was in the middle of giving you a long reply about the program. It looks great. I tried 5-10 of the Atari 7800 txt files and there weren't any problems. I tried 5 NES files and 2 of them gave me errors. I had them all copied to send to you, but my laptop with all my pi work died on me. I think the motherboard or at least the GPU is fried. I don't know how long it's going to take me to get set up again. I don't have any of the work on this old PC I'm typing from and I can't afford a new laptop.

I might have time over the next few days to try to at least round up the synopsis files from other systems to give you a list, but I'm kind of depressed now. I just lost a few months of work.

UDb23

@Used2BeRX uh, that's bad! Really sorry about it.
You should still be able to access the hd of your laptop. There are quite cheap adaptors to connect laptop hd's to a PC. Hope you can fix your laptop and/or recover your files.

Used2BeRX

@UDb23 said in Making premade gamelist.xml with xtra media collections:

@Used2BeRX uh, that's bad! Really sorry about it.
You should still be able to access the hd of your laptop. There are quite cheap adaptors to connect laptop hd's to a PC. Hope you can fix your laptop and/or recover your files.

LOL... I don't think I'm going to be able to fix the laptop. It's now sitting in my garbage in about a thousand pieces :)

I'm sure I can get the data off of it. The failure was either the motherboard or GPU. I'm just seriously limited on funds right now and need to get a job. I have to wait on my brother to get me a 2TB external drive to get back into any serious work on this stuff and to get the data off of that drive. He loaned it out to somebody else a while back but they're only using it for Wii games.

This work is too important though and I didn't want to lose your interest by stalling so I pulled all the synopsis files for all of the systems so I could test this out again on another computer and give you initial feedback. I'm really impressed with it so far and I can't wait to see where this goes.

Brace yourself.... I've got a LOT of feedback for you, and I'm going to have more info for you hopefully later today. Here we go........

I started with the 7800 even though you made this with the 2600 in mind. It was a much smaller list of only 90 games so I was able to test every single file in about 10 minutes.

7800 TEST RUN (EVERY GAME TXT FILE TESTED):

I got an error and couldn't create the file for the following games:
Beef Drop - 10 new levels.txt
Beef Drop - More Beef Drop.txt
bonQ.txt
Centipede.txt
FailSafe.txt
Fatal Run.txt
Meltdown.txt
Ms Pac-Man.txt
Pac-Man Collection.txt
Robotron 2084.txt
Snowday Jr Pac-Man.txt
Tomcat F 14 Simulator.txt
Tower Toppler.txt

Here's the Error:

Unhandled exception has occurred in your application.  If you click Continue, the application will ignore this error and attempt to continue.  If you click Quit, the application will close immediately.

The given path's format is not supported.

DETAILS:

See the end of this message for details on invoking 
just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************
System.NotSupportedException: The given path's format is not supported.
   at System.Security.Permissions.FileIOPermission.EmulateFileIOPermissionChecks(String fullPath)
   at System.Security.Permissions.FileIOPermission.QuickDemand(FileIOPermissionAccess access, String fullPath, Boolean checkForDuplicates, Boolean needFullPath)
   at Microsoft.VisualBasic.FileIO.FileSystem.NormalizePath(String Path)
   at Microsoft.VisualBasic.FileIO.FileSystem.CombinePath(String baseDirectory, String relativePath)
   at Microsoft.VisualBasic.MyServices.FileSystemProxy.CombinePath(String baseDirectory, String relativePath)
   at MetaX.Form1.ConvertButton_Click(Object sender, EventArgs e)
   at System.Windows.Forms.Control.OnClick(EventArgs e)
   at System.Windows.Forms.Button.OnClick(EventArgs e)
   at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
   at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
   at System.Windows.Forms.Control.WndProc(Message& m)
   at System.Windows.Forms.ButtonBase.WndProc(Message& m)
   at System.Windows.Forms.Button.WndProc(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)


************** Loaded Assemblies **************
mscorlib
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1648.0 built by: NETFXREL3STAGE
    CodeBase: file:///C:/Windows/Microsoft.NET/Framework/v4.0.30319/mscorlib.dll
----------------------------------------
MetaX
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Users/The%20Dude/AppData/Local/Apps/2.0/Q5PQQAWH.RQH/9XWRQPVQ.KZM/meta..tion_9bae49a2533cd5b2_0001.0000_1813f733e3f252fe/MetaX.exe
----------------------------------------
Microsoft.VisualBasic
    Assembly Version: 10.0.0.0
    Win32 Version: 14.6.1586.0 built by: NETFXREL2
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/Microsoft.VisualBasic/v4.0_10.0.0.0__b03f5f7f11d50a3a/Microsoft.VisualBasic.dll
----------------------------------------
System.Windows.Forms
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1586.0 built by: NETFXREL2
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Windows.Forms/v4.0_4.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1647.0 built by: NETFXREL3STAGE
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System/v4.0_4.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Drawing
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1586.0 built by: NETFXREL2
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Drawing/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
System.Configuration
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1586.0 built by: NETFXREL2
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Configuration/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
----------------------------------------
System.Core
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1647.0 built by: NETFXREL3STAGE
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Core/v4.0_4.0.0.0__b77a5c561934e089/System.Core.dll
----------------------------------------
System.Xml
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1586.0 built by: NETFXREL2
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Xml/v4.0_4.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------
System.Runtime.Remoting
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1586.0 built by: NETFXREL2
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Runtime.Remoting/v4.0_4.0.0.0__b77a5c561934e089/System.Runtime.Remoting.dll
----------------------------------------
Accessibility
    Assembly Version: 4.0.0.0
    Win32 Version: 4.6.1586.0 built by: NETFXREL2
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/Accessibility/v4.0_4.0.0.0__b03f5f7f11d50a3a/Accessibility.dll
----------------------------------------

************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.

For example:

<configuration>
    <system.windows.forms jitDebugging="true" />
</configuration>

When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.

I took a look at the failed files, and I think I spotted the potential problems with them.

Beef Drop - 10 New Levels.txt, Beef Drop - More Beef Drop.txt, Robotron - 2084.txt, Snowday - Jr Pac-Man.txt, Tomcat F 14 Simulator.txt

These have a ":" in the title. There are a LOT of games over all the systems that will have this character as I thought it looks a lot better in the romlist than using the " - " between the title and sub-title. For example "Beef Drop: 10 New Levels".

(NOTE: Snowday Jr Pac-Man.txt also has a "." in the title, so probably would have failed for that as well)

bonQ.txt
The title has an asterisk in it. It is actually b**nQ. Would only be a problem with games like Q*bert and such.

Centipede.txt
A ";" in the Players line. Players: 1 or 2 VS; CO-OP
I believe there are a lot of instances of this over all the systems.

Ms Pac-Man.txt
Maybe the "." in "Ms. Pac-Man"???

Pac-Man Collection.txt
Maybe the "!" in "Pac-Man Collection!" or the "(" and ")" in "Region: Homebrew (USA)"???

Tower Toppler.txt
Maybe the ":" in the description? There's a section toward the bottom that has hints and says "Hints: "

I'm thinking that the program should avoid any colons while it's scraping text since a lot of the Game Titles will have them and they will probably creep up in other places like this example.

This would also explain the failures for Meltdown.txt, Fatal Run.txt and FailSafe.txt

I hope I'm right about all of these and saved you some time trying to figure out what went wrong.

I also tested the first 25 text files for the NES and had 10 failures. I didn't check why they failed yet. Here's the list of games:

10-Yard Fight.txt
1942.txt
1943.txt
3-D Battles of WorldRunner - Part II
720.txt
75 Bingo.txt
8 Eyes.txt
A Boy and His Blob.txt
A Nightmare On Elm Street.txt
Abadox.txt

Here's a full upload of the NES and 7800 synopsis files for you to check out:

https://www.sendspace.com/file/ueaeig

I will try to look at files from every system and get you a list of all of the possible tags that the program should look for. Most of them aren't currently used by any skins, but a few guys from our project have made their way over here and are currently working on their first skin, so might as well have all the info just in case it gets used down the road.

A FEW OTHER NOTES:

crcindex.txt = This was a file that had the crc's for all of the games. It was a necessary part of getting the synopsis files running quickly on another system. It should just be avoided by the program.
skipped.txt = I'm not sure what this file was for, but it should be avoided by the program too.
I'd like to automatically add a line in these XML files that would add a Marquee image. For example we'll use the Atari 7800 game "Ace of Aces".

<marquee>/home/pi/RetroPie/Media/A7800/Artwork/Marquee/Ace Of Aces.jpg</marquee>

I noticed that all "media" lines in the program are "/Media/A2600/". I know that you planned that at this stage. Just wanted to hilight that this is going to obviously be different from system to system and we have to decide whether we're actually searching for media in the program before making these media lines or just making them all like your program does now but we have to specify a different folder here per system at some point then.

EDIT:
I'll add other things I notice in this spot until you see the message and reply.

I noticed on Ace of Aces.txt for the 7800 that it didn't carry over the "Genre" category.

The .txt file had this line: "Genre: Flight Simulator - Action"

The .xml file has this line: "<genre></genre>"

Maybe it doesn't like the " - "?

UDb23

@Used2BeRX You did quite extensive testing !
The program uses original txt path & filename to write the xml: therefore if the original name includes "uncommon" characters it may generate the errors you got.
It uses a specific VB streamwrite command so I have to check if, as it seems, special characters cannot be used and if there is a workaround.
I'll come back to you on this when I have done specific tests on the files that actually generate errors.

Concerning paths, as said, I'm going to add specific path fields the user can customize as needed.

Will look also into the missing <genre> issue.