RetroPi Media - gamelist.xml generator
-
I love this kind of tools, so i did some tests. This is what i found so far:
- The window is so large that even maximized on 1920x1080 you have scrollbars horizontally and vertically. This makes it difficult to handle.
- Loading a full romset for fbneo (~6.000 roms) from a network drive is VERY slow (it takes ~15 minutes)
- I have the folders "snap", "video" and "wheel" under every system folder in the roms dir, so i leave the field "Media folder" empty and set "Video folder" to "video" and so on. With these settings the entries in the gamelist.xml look like "<image>.//snap/..." (double slash).
- A working gamelist entry like this:
<game> <path>./gtmro.zip</path> <name>1000 Miglia: Great 1000 Miles Rally (94/05/10)</name> <desc>Great 1000 Miles Rally © 1994 Kaneko. A racing game featuring classic cars from Ferrari, Alfa Romeo, Mercedes Benz and more.</desc> <rating>0.4</rating> <releasedate>19940101T000000</releasedate> <developer>Kaneko</developer> <publisher>Kaneko</publisher> <genre>Rennen, Fahren-Rennen, Fahren / Rennen</genre> <players>1-2</players> <image>./snap/gtmro.png</image> <video>./video/gtmro.mp4</video> <marquee>./wheel/gtmro.png</marquee> </game>
results after saving in this
<game> <path>./gtmro.zip</path> <name>1000 Miglia: Great 1000 Miles Rally (94/05/10)</name> <cover>.//covers/1000 Miglia: Great 1000 Miles Rally (94/05/10).png</cover> <image>.//snap/1000 Miglia: Great 1000 Miles Rally (94/05/10).png</image> <marquee>.//wheel/1000 Miglia: Great 1000 Miles Rally (94/05/10).png</marquee> <video>.//video/1000 Miglia: Great 1000 Miles Rally (94/05/10).mp4</video> <rating>0.9</rating> <desc>1000 Miglia: Great 1000 Miles Rally (94/05/10)</desc> <releasedate>19940101T000000</releasedate> <developer>Kaneko</developer> <publisher>Kaneko</publisher> <genre>Rennen, Fahren-Rennen, Fahren / Rennen</genre> <players>2</players> </game>
- The content of <desc> is replaced with the content of <name>
- The value of <rating> is wrong
- The value of <players> "1-2" is replaced with "2"
- The filenames for <cover>, <image>, <marquee> and <video> are generated based on <name>, not based on the name of the rom
-
@miwasp Thanks for the detail feedback...
@miwasp said in RetroPi Media - gamelist.xml generator:
I love this kind of tools, so i did some tests. This is what i found so far:
- The window is so large that even maximized on 1920x1080 you have scrollbars horizontally and vertically. This makes it difficult to handle.
RESPONSE: Interesting as I use 1920x1080 and have no scrolls bars, are you sure you are set to 1920x1080? - Loading a full romset for fbneo (~6.000 roms) from a network drive is VERY slow (it takes ~15 minutes)
RESPONSE: This app wasn't designed to be utilized over a network as that can get very complicated. A full-blown enterprise program would have to deal with that somehow but that's beyond my coding ability, so this is not something I will even attempt to address. HD speed, network speed / traffic will all slow it down. I ran this against my MAME with ~7500 and it only took 14 seconds to populate the list and 8 min to search the roms not found. For each file in the folder all~7500 it searches the xml file to see if there is a match. I will take a look at this code and see if there is a different approach I can take to speed it up. I will make this search optional / manual run after the list populates. Some people won't care if there are orphan roms. - I have the folders "snap", "video" and "wheel" under every system folder in the roms dir, so i leave the field "Media folder" empty and set "Video folder" to "video" and so on. With these settings the entries in the gamelist.xml look like "<image>.//snap/..." (double slash).
RESPONSE: The "/" is hard coded in the output so I can make an adjustment for when the folder is left blank. The trick is in the Windows it uses "" and in the Pi it uses "/". I'll have to create a workaround for this. - A working gamelist entry like this:
<game> <path>./gtmro.zip</path> <name>1000 Miglia: Great 1000 Miles Rally (94/05/10)</name> <desc>Great 1000 Miles Rally © 1994 Kaneko. A racing game featuring classic cars from Ferrari, Alfa Romeo, Mercedes Benz and more.</desc> <rating>0.4</rating> <releasedate>19940101T000000</releasedate> <developer>Kaneko</developer> <publisher>Kaneko</publisher> <genre>Rennen, Fahren-Rennen, Fahren / Rennen</genre> <players>1-2</players> <image>./snap/gtmro.png</image> <video>./video/gtmro.mp4</video> <marquee>./wheel/gtmro.png</marquee> </game>
results after saving in this
<game> <path>./gtmro.zip</path> <name>1000 Miglia: Great 1000 Miles Rally (94/05/10)</name> <cover>.//covers/1000 Miglia: Great 1000 Miles Rally (94/05/10).png</cover> <image>.//snap/1000 Miglia: Great 1000 Miles Rally (94/05/10).png</image> <marquee>.//wheel/1000 Miglia: Great 1000 Miles Rally (94/05/10).png</marquee> <video>.//video/1000 Miglia: Great 1000 Miles Rally (94/05/10).mp4</video> <rating>0.9</rating> <desc>1000 Miglia: Great 1000 Miles Rally (94/05/10)</desc> <releasedate>19940101T000000</releasedate> <developer>Kaneko</developer> <publisher>Kaneko</publisher> <genre>Rennen, Fahren-Rennen, Fahren / Rennen</genre> <players>2</players> </game>
- The content of <desc> is replaced with the content of <name>
RESPONSE: So if you import the gamelist.xml then the <desc> should be there and we can use it in the output. IF you use a hyperlist.xml it will not have the <desc> so we use the name again. - The value of <rating> is wrong
RESPONSE: same as the description. the rating can be used if there, hyperlist didn't use a numerical value, its is like "Other - NR (Not Rated)" - The value of <players> "1-2" is replaced with "2"
RESPONSE: similar to the <desc> issue, we can use the value if it came from the gamelist.xml otherwise either leave blank or use 2 - The filenames for <cover>, <image>, <marquee> and <video> are generated based on <name>, not based on the name of the rom
RESPONSE: you are correct, i will fix
So right now I'm adding the ability to search subfolders which really took me off track as it is more work than I really wanted to do. Once this is finished, I'll tackle the things you mentioned.
- The window is so large that even maximized on 1920x1080 you have scrollbars horizontally and vertically. This makes it difficult to handle.
-
@MotoGp I've made some really good progress tonight. I should have a new release ready tomorrow. It will included subfolder search as default and the issues brought up by @miwasp except the performance piece although I did made the search for orphans a manual button so you can choose to just not run that part as it can take awhile on large lists.
-
@MotoGp said in RetroPi Media - gamelist.xml generator:
- The window is so large that even maximized on 1920x1080 you have scrollbars horizontally and vertically. This makes it difficult to handle.
RESPONSE: Interesting as I use 1920x1080 and have no scrolls bars, are you sure you are set to 1920x1080?
Yes, resolution is 1920x1080 but scaling is set to 125%. I can disable scaling for this app only in the compatibility settings, so this is not really a problem.
@MotoGp said in RetroPi Media - gamelist.xml generator:
@MotoGp I've made some really good progress tonight. I should have a new release ready tomorrow. It will included subfolder search as default and the issues brought up by @miwasp except the performance piece although I did made the search for orphans a manual button so you can choose to just not run that part as it can take awhile on large lists.
That sounds great, thank you very much.
- The window is so large that even maximized on 1920x1080 you have scrollbars horizontally and vertically. This makes it difficult to handle.
-
OK here we go.
Here is a screenshot of the latest version. I moved a few things around.
By default it uses the gamelist.xml file but if you check the box it will use the Hyperlist file.
You can download the latest version from the first post
I've tested this on my system and everything appears to work properly, please backup your data before trying.
Please let me know how it works for you!Version 1.004
Fixes:
- For large rom sets, made the orphan tool optional / manual
- fixed the "//" double slash issue if some folder settings are left blank
- corrected the output XML file.
- fixed the file names in xml to match the ROM name not the <desc>
added features
- matching of ROMS will now included subfolders in match. This also applies to all videos, marquees, screenshots etc.
- ability to save your settings
- The rom list will now display all attributes that are imported from the gamelist.xml and/or the hyperlist.xml
- added version number to title at top so we now if we are working with the latest version
example xml of output
<game> <path>./10yard.zip</path> <name>10-Yard Fight (World, set 1)</name> <cover>./media/covers/10yard.png</cover> <image>./media/screenshots/10yard.png</image> <marquee>./media/marquees/10yard.png</marquee> <video>./media/videos/10yard.mp4</video> <rating>0.9</rating> <desc>Football Game)</desc> <releasedate>1983</releasedate> <developer>Irem</developer> <publisher>Irem</publisher> <genre>Sports / Amer. Football</genre> <players>2</players> </game>
-
I just tested version 1003 and then saw you changed it to version 1004. What are the differences?
According to version 1003:
- Loading a full romset for fbneo (~6.000 roms) from a network drive now takes 53 minutes (!) Maybe it has something to do with the new subfolder feature?
- Saving of settings does not seem to work. I always get the defaults after restart.
Edit:
Same for version 1004Edit 2:
@MotoGp said in RetroPi Media - gamelist.xml generator:- Loading a full romset for fbneo (~6.000 roms) from a network drive is VERY slow (it takes ~15 minutes)
RESPONSE: This app wasn't designed to be utilized over a network as that can get very complicated. A full-blown enterprise program would have to deal with that somehow but that's beyond my coding ability, so this is not something I will even attempt to address. HD speed, network speed / traffic will all slow it down. I ran this against my MAME with ~7500 and it only took 14 seconds to populate the list and 8 min to search the roms not found. For each file in the folder all~7500 it searches the xml file to see if there is a match. I will take a look at this code and see if there is a different approach I can take to speed it up. I will make this search optional / manual run after the list populates. Some people won't care if there are orphan roms.
I don't see why it is such a big difference (14 seconds on local disk compared to 53 minutes on network drive) to populate the list.
With my limited coding abilities i would:- read the xml file to an internal structure
- read the filelists for the folders box, video, snaps and marquee into internal arrays
Everything else could be done with the internal structures and arrays inside the program and access to any drive is not necessary any more. This would of course be slower on a network drive but by far not be that big difference. Reading a filelist into an array (even with several thousand files) is usually very quick on a network drive.
-
Regarding the whole over the network piece. As I mentioned before, I didn't create this to run over a network, was meant to be a "simple" local app to help me edit my gamelists. I mainly wanted to convert Hyperlists to gamelists.
To implement what you suggest would be a total rewrite of most of my code, so with that being said and my limited time to work on this, unfortunately I don't see this happening anytime soon. If you'd like to tackle this yourself, I'd be willing to share code with you.
I'm not sure why the subfolder piece would add time as it does not loop through all of the files for a match it simply does a check to see if the file exists once.
If (System.IO.File.Exists(sDir & "\" & FName)) Then
Regarding the saving, I tested this and it works for me. I saved restarted the app several times and even rebooted my PC and it saved. Now, if you saved in ver 1003 then opened 1004, the save will not be there. OR if you saved then moved or renamed the folder the app files are in, this will break the save as well.
regarding 1003 vs 1004 it was one small change with regards to inserting the orphans from the orphan list to the main list.
-
@MotoGp
Thank you for this great tool. I’m using v1.004 and have encountered a problem.I have a romset in zip format. Some of the roms are not on the imported gamelist.xml. After checking for orphans, these roms appear in the “ROM files not in Gamelist” section. I checked the “include” boxes and added them to roms list, then generated xml. Problem: The roms will appear in the generated xml, however, the extension “.zip” is being added to each path. Example output is below.
<cover>./media/Donkey Kong Country 3 - Dixie Kong's Double Trouble! (USA).zip.png</cover> <image>./media/image/Donkey Kong Country 3 - Dixie Kong's Double Trouble! (USA).zip.png</image> <marquee>./media/marquee/Donkey Kong Country 3 - Dixie Kong's Double Trouble! (USA).zip.png</marquee>
-
@SloTayHolmes said in RetroPi Media - gamelist.xml generator:
@MotoGp
Thank you for this great tool. I’m using v1.004 and have encountered a problem.I have a romset in zip format. Some of the roms are not on the imported gamelist.xml. After checking for orphans, these roms appear in the “ROM files not in Gamelist” section. I checked the “include” boxes and added them to roms list, then generated xml. Problem: The roms will appear in the generated xml, however, the extension “.zip” is being added to each path. Example output is below.
<cover>./media/Donkey Kong Country 3 - Dixie Kong's Double Trouble! (USA).zip.png</cover> <image>./media/image/Donkey Kong Country 3 - Dixie Kong's Double Trouble! (USA).zip.png</image> <marquee>./media/marquee/Donkey Kong Country 3 - Dixie Kong's Double Trouble! (USA).zip.png</marquee>
Ok, thanks I see why that's happening. I can fix it tomorrow. I combined some code for the gamelist and Hyperlist to make it more manageble. in gamelist the <path> or ROM file has the file extension where as in the Hyperlist for the ROM there is not file extension. When I add the ROM files not in Game list, I need to remove the extension as that name is also used in the path to the images and this is why the .zip is before the .png.
Thanks for letting me know. This is an easy fix I can tackle tomorrow.
Jay
-
@MotoGp that's a really cool program. Thanks for sharing!
-
Regarding the saving, I realized you were leaving fields blank and I edit the code to make that work. When the form opened it checked if a value was saved for the folders and if the value was " ", i used a default value so in your case it would not recall your settings. I've corrected that now. Sorry for the confusion.
-
@SloTayHolmes said in RetroPi Media - gamelist.xml generator:
@MotoGp
Thank you for this great tool. I’m using v1.004 and have encountered a problem.I have a romset in zip format. Some of the roms are not on the imported gamelist.xml. After checking for orphans, these roms appear in the “ROM files not in Gamelist” section. I checked the “include” boxes and added them to roms list, then generated xml. Problem: The roms will appear in the generated xml, however, the extension “.zip” is being added to each path. Example output is below.
<cover>./media/Donkey Kong Country 3 - Dixie Kong's Double Trouble! (USA).zip.png</cover> <image>./media/image/Donkey Kong Country 3 - Dixie Kong's Double Trouble! (USA).zip.png</image> <marquee>./media/marquee/Donkey Kong Country 3 - Dixie Kong's Double Trouble! (USA).zip.png</marquee>
ok Try version 1.005, link is in the first post.
Ver 1.005
- Fixed the saving issue if you left folder fields empty. It also saves your ROM folder and Hyperlist settings now.
- Fixed naming of media paths when roms are added from Orphan list.
-
@MotoGp
Thank you for posting the latest. Save settings and media path working. Problem: The game description data isn't outputting to <desc> in the generated xml, it's putting the game name instead. The game description data is populating correctly in the GUI however. Sample output below.<path>./ActRaiser (USA).zip</path> <name>Actraiser</name> <cover>./media/cover/ActRaiser (USA).png</cover> <image>./media/image/ActRaiser (USA).png</image> <marquee>./media/marquee/ActRaiser (USA).png</marquee> <video>./media/video/ActRaiser (USA).mp4</video> <rating>0.8</rating> <desc>Actraiser</desc> <releasedate>19911102T000000</releasedate> <developer>Quintet</developer> <publisher>Enix</publisher> <genre>Simulation</genre> <players>1-2</players>
-
@SloTayHolmes said in RetroPi Media - gamelist.xml generator:
@MotoGp
Thank you for posting the latest. Save settings and media path working. Problem: The game description data isn't outputting to <desc> in the generated xml, it's putting the game name instead. The game description data is populating correctly in the GUI however. Sample output below.<path>./ActRaiser (USA).zip</path> <name>Actraiser</name> <cover>./media/cover/ActRaiser (USA).png</cover> <image>./media/image/ActRaiser (USA).png</image> <marquee>./media/marquee/ActRaiser (USA).png</marquee> <video>./media/video/ActRaiser (USA).mp4</video> <rating>0.8</rating> <desc>Actraiser</desc> <releasedate>19911102T000000</releasedate> <developer>Quintet</developer> <publisher>Enix</publisher> <genre>Simulation</genre> <players>1-2</players>
since I was only using Hyperlist, I made the Desc the same as the game name for myself and forgot to change it back. its fixed now.
see first post for updated link Ver 1.006
-
@MotoGp I found a bug in the Generate XML code. It doesn't take into account the "include" status column. I'm working on the fix.
-
@MotoGp
Ver 1.007
Fixes the Generate Xml bug reported above. -
Thank you, Moto. Problem renaming matched orphans in v1.007 (using imported gamelist.xml). In the “Confirm File Rename” box:
CURRENT NAME: mslug3n.zip
NEW NAME: mslug3Result: Rom is renamed without the zip extension. The <path> in generated gamelist xml is correct.
-
@SloTayHolmes said in RetroPi Media - gamelist.xml generator:
Thank you, Moto. Problem renaming matched orphans in v1.007 (using imported gamelist.xml). In the “Confirm File Rename” box:
CURRENT NAME: mslug3n.zip
NEW NAME: mslug3Result: Rom is renamed without the zip extension. The <path> in generated gamelist xml is correct.
OK, so Ver 1.008 fixes that issue. I did realize that the find orphans does not search subfolders. This will take a little time to re-write.
-
@MotoGp Amazing job on the app MotoGP. I just used the latest version and it is working as expected. Retropie gamelist looks for gamelist.xml in the folder we select, would it be possible to just look for roms in a folder (like we give the rom extension zip and it lists all zips in folder without having gamelist.xml) or would that be beyond the scope of app?
-
This is an amazing tool! Thank you for the work. I had been trying to work through several others and there were always gaps but this is clean and very straight forward.
One question, is there a way to merge Gamelists? I have several that have descriptions in them and it would be great if I could merge their descriptions in with the hyperspin list this creates.
Thanks again!
Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.
Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.