Making premade gamelist.xml with xtra media collections
-
@meleu I had some time to sit down with the synopsis files and I believe that I figured out the common theme to any of the entries where the body text wasn't filled out with your script. In Notepad++, the ones that don't work right are listed as an "ANSI" file. All of the ones that work that I've checked seem to be listed as "UTF-8".
I don't know much about this other than they're two forms of character encoding and that UTF-8 is far more advanced than ANSI was. I have no clue how some of these files were saved differently. I'm going to try re-saving all of the non-working files and see if I can somehow save them as UTF-8.
Just wanted to let you know what that bug was.
EDIT: Yep. I believe that's what the problem is. In Notepad++ I can go to the "Encoding" menu and then select the second option "Encode in UTF-8". Some characters, like alternate apostrophes and quotes then change to weird things like "x92" and are highlighted. I can then just select them and put in a regular character and save it.
I'm going to do this to about a dozen of the bad ones and re-run the script before I sleep. They should all be working correctly with the new gamelist.xml.
-
@used2berx Awesome. That worked. Now I just need to go through and figure out which files had the ANSI format and convert them all to UTF-8 and the game descriptions will all be good. :)
-
Okay. Some weird things were happening. Lot's of characters that were in the synopsis files had bad characters that showed up as a floating box in both the XBox and the RetroPie. RP seems to have been able to display more of them though. What I can't figure out is why some files were ANSI when I opened them in Notepad++ and others were UTF-8, even though they had the same bad characters. Once they were fixed and saved though, they were automatically UTF-8 and I couldn't even manually save them as ANSI if I wanted to (which I didn't, but I was just testing it out).
I'm glad this issue came up now, actually. For years it has always bothered me that the strange characters looked so lousy on the XBox. The fact that they were actually breaking the script meant that I had to do something about them finally. I'll know for sure if I got them all after the new gamelist.xml is generated, but I think it's pretty safe to say that all 2,116 NES/FDS synopsis files are clean now. :)
I'm working with somebody to hopefully make a script to automate this process for future systems. I don't like automation unless I know exactly what it's doing, but after having spent 8 hours with this last night I feel pretty comfortable with it as long as it does exactly what I need it to do... which is the following:
Change all characters on the left side to the ones on the right side of the equal sign:
à = a
ā = a
é = e
ñ = n
ṇ = n
ø = o
ō = o
ǚ = u
ū = u™ = Nothing (remove it entirely)
– = -
— = -
‘ = '
’ = '
´ = '
… = ...
“ = "
” = "That's it.
There may be a few different characters that I find here and there as I move on to other systems down the line, but they could be added to the automation process as I discover them. This list should take care of 99% of any future issues. :)
Oh... and as a bonus, this also makes the gamelist.xml slightly smaller since every one of these characters were a long series of numbers in the XML files such as
𠄺nd”. It won't really make much of a difference on an XML file that is over 3.3MB, but every little bit helps... especially on the Pi Zero.Those numbers are actually how I found all of these problems. I just did a search in Notepad++ on the last generated gamelist.xml for

nd I found all of those one-off accented letters that I never would have found using my eyeballs. :) -
Thanks to my bud Nemesis I've learned a lot about the code in this script. I'm no coder, but using existing code I was able to make a few changes so far that I wanted in this script. I'm re-running it now, and all of the desired changes I made this time around are taking effect. :)
I was able to add code to add and find all data for Famicom Disk System games that are in NES subfolders without having to have manually changed the "Platform: " line to erroneously read "Nintendo Entertainment System" instead of the legitimate tag of "Famicom Disk System".
I was also able to add a
<boxfront>tag as a placeholder for all Box Front images, so now we have one for Box Front, Cart, Titles, Action and 3D Boxart images.I was also able to make the code put the Cart images in the
<marquee>tag, so now a skin that uses both tags can display both the Box art and the Cartridge art simultaneously.The next thing I want to do is a bit trickier. I've heard from somebody here that if you don't have a video for a game, the <video> tag can be used as a third image type. I'd like to add some code in there that will search for the video as it is already doing, but if it doesn't, it will add either the Title image or the Action image shots in that tag.
Once all this is taken care of, I'm hoping Nemesis and I can come up with a quick script that will change which images populate the <image>, <marquee> and <video> tags on the fly after the
gamelist.xmlis created. Right now, on the Pi Zero it takes about 3 hours for the script to run on the NES alone. But after it is created it should only take a few seconds to run a script that will change which folder each image tag points to for every game.Another thing on the list now is to add every single tag from the
synopsis.txtfiles to thegamelist.xmlfile so when I convert them and then use @mitu 's script to convert it to a spreadsheet, I can make all the edits I want in the spreadsheet and then convert it back to agamelist.xmlfile and finally back to thesynopsis.txtfiles without losing any of the original data.This is exciting. If we can get this working it's going to make editing game information for both the XBox and the RetroPie easier than it ever has been before.
Thanks again for all that hard work @meleu
-
Awesome.
I don't really know crap about coding, but I'm pretty good at pattern recognition and logic puzzles. Even though I don't know exactly what @meleu 's code is doing, I'm slowly figuring out ways to manipulate it.
So far I've been able to add a line for
<boxfront/>that's separate from<image/>. At least temporarily, I've pointed the secondary image type<marquee/>to the Cart location as well. This will make it much easier to switch between which image types are in the "hot spots" and are actually usable in RetroPie in an alternate script. The reason you'd want to do this is because it would literally take a few seconds to switch the file locations in the completedgamelist.xmlinstead of generating an entirely new one from scratch with different image parameters (which on a Pi Zero right now takes over 3 hours for just the NES gamelist).That being said, I'm going to want to figure out a better way to get the
<image/>and<marquee/>tags now that I've done all of this. I should be making it simply "copy" the file locations from<boxfront>for the image tag, and from<cart>for the marquee tag. I believe right now it's looking in the same location twice now, adding a lot of code run time to a process that already takes hours to finish. I'm sure this is a really simple thing to do. I'll figure it out eventually. It doesn't make any sense to do it the way that I'm doing it now.Meleu had to do some coding wizardry to make my
Players:andRelease Year:fields work in RetroPie as well. This was great, but in order to preserve all data when transferring thegamelist.xmlto a spreadsheet, doing massive editing on the spreadsheet, then converting back to a gamelist and further back into individualsynopsis.txtfiles for the XBox, I need to preserve the data exactly as it is found in those fields.By copying and pasting some existing code that populated other fields, I have managed to add an
<xtrasplayers>and<xtrasreleaseyear>tag to the gamelist. Though not useful at all in RetroPie, these tags will preserve the data in its entirety when converting from format to format. :)I've got some other various tags to add as well that I'm going to get around to sometime today hopefully. It's taking me a pretty long time since I'm documenting everything pretty heavily as I go just so I remember what I've been doing and I don't get lost.
No wonder why @meleu wanted out. This is a lot of stuff to keep straight. :)
Thanks again bud. Sorry it took me almost a year to get around to actually learning anything about the code. Probably better that you stopped so I would be forced to learn some of this on my own.
-
Does anybody know how to copy a location put in an XML tag into a 2nd tag?
I want to change @meleu's script to search for all image location tags first (
<boxfront>, <cart>, <titles>, <action>, <threedbox>, <xtrasmarquee>).Then, based off of which image types are selected when running the command, I would want the actual tags used by RetroPie to be populated by these image locations without having to re-scan for them. (Default locations pasted here would be the
"Box Front"folder for the<image>tag, and"Marquee"folder for the<marquee>tag.In this example, it would fill
<image>with whatever info was just scanned and put in the<boxfront>tag for the game. And it would fill the<marquee>tag with whatever info was just scanned and put in the<xtrasmarquee>tag.This will accomplish 3 things.
-
It will cut down the time needed to run the script quite a lot by not having to scan for the same image twice for two separate tags.
-
This will allow me to create a script that will (hopefully) easily be able to change which images are considered
<image>and<marquee>after the gamelist is made. It currently takes 3 to 4 hours to run this script on the Pi Zero for my NES/FDS collection of 2,116 games, so running it several different times to change these image types is foolish, and creating a script to do this on the fly would be much easier than what I've been doing, which is constantly going into Notepad++ after creation and manually running a few "find and replace all" commands to change the directories after the fact. -
This would allow me to hijack the
<marquee>tag for whatever secondary image I want to use in a particular skin easily. Ideally, we'll see more possible image types in RetroPie in the future, but with the addition of the Marquee tag at least a skin could display both the Box art and the Cart art at the same time.
I've managed to do quite a bit of things I wanted to do with this script now, but because I don't really know what the code is actually doing I'm lost at this point. I'm going to keep trying to Google the answer I'm looking for, but since I'm not even that familiar with Linux jargon and terminology, I'm having a difficult time finding anything relevant.
Thanks to anyone who reads this and has any ideas! :)
-
-
Interesting...
I finally put some random stuff over to the Pi in all of the media folders (including new ones that I added). All of them were filled out with the proper file in the location except for my new tag
<xtrasmarquee>. I didn't figure it out right away since the media was there and all of the code for the new tag seemed to be in order.That's when I figured out that I had improperly set the following code:
# xtrasmarquee : find it boxfront="$(find_file "Artwork/Marquee" "$file_name" png jpg)"It should have read like this:
# xtrasmarquee : find it xtrasmarquee="$(find_file "Artwork/Marquee" "$file_name" png jpg)"The "interesting" part to me was that rather than create a 2nd entry, it simply re-wrote the <boxfront> image location that it had previously just found with the Marquee image instead.
This:
<boxfront>/home/pi/RetroPie/Media/nes/Artwork/Marquee/8 Eyes.png</boxfront>Instead of This:
<boxfront>/home/pi/RetroPie/Media/nes/Artwork/Box Front/8 Eyes.png</boxfront>I'm actually happy to see this happen, and I'm learning things. Now I'm pretty sure if I can figure out how to code it correctly that it shouldn't be hard to copy the existing
<boxfront>tag into the<image>tag for each game since it seems to wrap up what it's doing for each game entry before moving on. I was worried this was going to be a pretty complex solution.Also, I fogot to mention that I added "PDF" images to the search for game Manuals as well as zip files. Last time I checked there is no way for RetroPie to actually view manuals, but in case it ever gets added, the script will look for either PDF files or Zip files that house PNG or JPG images of the manuals.
So the existing code will successfully output all media for a game now like this.
<boxfront>/home/pi/RetroPie/Media/nes/Artwork/Box Front/8 Eyes.jpg</boxfront> <cart>/home/pi/RetroPie/Media/nes/Artwork/Cart/8 Eyes.png</cart> <title>/home/pi/RetroPie/Media/nes/Artwork/Titles/8 Eyes.png</title> <action>/home/pi/RetroPie/Media/nes/Artwork/Action/8 Eyes.png</action> <threedbox>/home/pi/RetroPie/Media/nes/Artwork/3D Boxart/8 Eyes.png</threedbox> <xtrasmarquee>/home/pi/RetroPie/Media/nes/Artwork/Marquee/8 Eyes.png</xtrasmarquee> <gamefaq>/home/pi/RetroPie/Media/nes/GameFAQs/8 Eyes.zip</gamefaq> <manual>/home/pi/RetroPie/Media/nes/Manuals/8 Eyes.pdf</manual> <vgmap>/home/pi/RetroPie/Media/nes/VGMaps/8 Eyes.zip</vgmap>What I'm looking to do now is find a way to create the
<image>tag by simply copying the image location in the<boxfront>tag. And then create the<marquee>tag by copying the image location in the<marquee>tag. These would be the defaults, and would ultimately be able to be changed upongamelist.xmlgeneration and/or with the yet to be created script that could change all of the locations for these tags on the fly after it was generated.Though this isn't absolutely necessary to figure out, it would save a lot of time during the creation of the gamelist.xml file by not having to find the same image twice for two tags. This might be a moot issue on a Pi 3, but with how long it takes to make these files on a Pi Zero every little bit helps.
-
Got it! Finally....
Although, I'm not sure if I actually accomplished what I was trying to do, which is simply copy the output from one element
<xtrasmarquee>and paste it into another element<marquee>. It's quite possible that it's actually still looking for the actual file name and path again instead of copying the data. Anybody know how to verify this or know firsthand if it's doing it right?Here's the lines in question:
# xtrasmarquee : find it xtrasmarquee="$(find_file "Artwork/Marquee" "$file_name" png jpg)" marquee=$xtrasmarqueeSo it's first finding the actual file when generating
<xtrasmarquee>.Then the second line is either copying the
xtrasmarqueeoutput data and pasting it into the<marquee>element (which is what I want), or it is entirely re-running the first line, searching for the file and location, and then putting the data in the<marquee>element (which is exactly what I don't want).It's definitely doing one of the two things though. Earlier in the code I populate the
<marquee>tag with the file/location for the "Cart" image. If I run the command above it is properly showing the "Marquee" image file/location. If I comment out that line, then it will show the "Cart" image file/location instead.Progress..... I think. :)
-
Well.... looking at my last post here, I can tell just about exactly how long it took me to figure out how to do the next thing I wanted to do.
The script was writing in the first line as a line break for all the
<desc/>entries. Not a huge deal at the end of the day and not super noticible while scrolling through games since they all did it, but with screen real estate at a premium, having that useless blank line at the beginning of every game description was dumb.For 3 hours I tried so many different things and re-ran the script only to fail. I was totally on the wrong track. I was trying to get the code that recognized the first line of the Body text in the synopsis.txt files to also recognize line breaks as part of what was to be removed, basically by changing the highlighted text below to about 100 different combinations of crap that was never going to work in the first place....
desc="$(sed '
/^__________/,$!d' "$file" | tail -n +2 | tr -d '\r' | sed 's/&/&/g')"The secret was in
tailAll I had to do was change
tail -n +2totail -n +3and it got rid of that extra line.(What I believe) this command does, is starts writing from the line specified. (With "-" numbers actually writing everything up until those lines at the end, and "+" starting the writing at the line specified.
The beginning of every "Body" text for the descriptions starts with
____________________and then has a blank line before the text starts. This is because the entire file shows up when displayed on the XBox as simple text, and that line break looks better than not having one. In the case oftailhere in this line of code, the____________________line is considered line+1So with
tail -n +2, the script was starting at that blank line when it began writing. By changing it totail -n +3, it begins writing at the correct spot.Although that fixes the problem now, I would like to find a way to just start writing at the first Alpha-numeric character instead. I did a lot of heavy editing for the NES synopsis and know for a fact every single one is formatted this way. They weren't before I did all this work though. If that space is missing in any future entries on future systems then this script will cut off the first line of text. Now that I think about it, it would actually cut off the first paragraph since the paragraphs are considered a single line without any breaks.....
-
Well that worked, but if I was going to remove the URL links at the bottom of the body so I could put them in their own tags, that was going to be another story altogether. I thought I had it with a simple
head -n -2stuck in that line as well. I'd only been testing game synopsis files with it, and didn't even think what would happen when I tried converting the folder synopsis files that don't have urls at the bottom. Turns out I lose the last paragraph when I do it. :)Well here's what my <desc> code looks like now. Quite a bit more complex than it was before...
if [[ "$NO_DESC_FLAG" == 0 ]]; then desc="$(sed '/^__________/,$!d' "$file" | tail -n +3 | tr -d '\r' | sed 's/&/&/g' | sed -n '/http/{N;s/.*//;x;d;};x;p;${x;p;}' | sed '/^$/d' | sed '/^$/d;G')" fised -n '/http/{N;s/.*//;x;d;};x;p;${x;p;}' | sed '/^$/d'--- This code removes any line that begins with "http" as well as the line directly above it and below it. This way it knocks out both URLs if there are two as well as the blank line above the first one.sed '/^$/d;G'After that, I'm not exactly sure what happened with my code. All of the entries I had with more than one paragraph lost the space in between them. Adding this on the end of the line of code brought my spaces back. :)Here's the code to separate the URLs and give them their own tags:
# url : Synopsis cited link #1 - Citation link for info in all synopsis.txt files / Not usable in RetroPie (05/29/2018) if [[ "$NO_DESC_FLAG" == 0 ]]; then url="$(sed '/^http/,$!d' "$file" | head -1 | tr -d '\r' | sed 's/&/&/g')" url="$(grep -Ev '^(https?|ftp)://[^\s/$.?#].[^\s]*$' <<< "$url")" fi # urlalt: Synopsis cited link #2 (Typically only for Hacks and Translations) / Not usable in RetroPie (05/29/2018) if [[ "$NO_DESC_FLAG" == 0 ]]; then urlalt="$(sed '/^http/,$!d' "$file" | tail -n +2 | tr -d '\r' | sed 's/&/&/g')" urlalt="$(grep -Ev '^(https?|ftp)://[^\s/$.?#].[^\s]*$' <<< "$urlalt")" fiYou wouldn't hardly even recognize this code anymore man. I've been up about 30 hours straight messing with it. I'm going to have to let it run when I finally go to sleep so I can test the results out, but I think I might be able to actually get this thing to do everything I want to do with enough Google searching.
I still don't know what 3/4 of this code even means, but I've never let my complete ignorance on a subject keep me from being an expert on it before. Ain't about to start now. :)
-
So... I'm going to have to change some of that above code. When checking through all of the games, I noticed that there were 3 games that now didn't have any text to the body at all. It turns out that this was because I hadn't put a space between the single paragraph entry and the citation URL, and the code is telling the script to not copy over the URL line as well as the line above and below it.
So this pointed out all 3 entries I had out of 2,116 that had this problem, right? Wrong. :(
This only told me which ones had this typo that were single paragraph entries. Without opening every document and verifying this is correct on all files, I really have no way of knowing if this was a problem on other entries that had more than one paragraph. I only caught these because they were single paragraph entries and there was no body text at all in EmulationStation.
So I'm just going to change that code to delete the URL citation line and the line below it. It doesn't hurt anything to have a kilobyte or two of extra blank lines at the end of all the collective game entries in the gamelist.xml just to be on the safe side in case I made this typo in other spots. :)
-
I'm really starting to figure this stuff out now. It took me several hours here, but through a lot of trial and error I've completely re-written the code for scraping the
<desc>and<url> / <urlalt>tags, and I believe now they work perfectly. :)I'll post the revised code here, as well as the internal notes that I have inside the file that explain what everything is doing in case I look at all of this a few months from now and forgot everything I've done.
# desc : the content below "______", stopping before URL(s) / Usable element in RetroPie # NO_DESC_FLAG - If the "--no-desc" argument was used, the <desc>, <url>, and <urlalt> tags won't be filled. if [[ "$NO_DESC_FLAG" == 0 ]]; then desc="$(sed '/^__________/,$!d' "$file" | tail -n +2 | tr -d '\r' | sed 's/&/&/g' | grep -v ^http | sed '/^$/d' | sed '/^$/d;G')" # 1. sed '/^__________/,$!d' "$file" = Begins looking at any line that starts with 10 "_" characters. Writes to file using all # parameters following #1 # 2. tail -n +2 = Starts copying text from 1st line after "_" line above. If a space, it will be deleted with #6. # 3. tr -d '\r' = UNSURE. I believe this has to do with formatting line breaks? # 4. sed 's/&/&/g' = UNSURE. I believe this part of the line converts "&" into "&" # 5. grep -v ^http = This part of the line deletes any lines that begin with a URL (http and/or https) # Won't delete URLs in the middle of a line. # 6. sed '/^$/d' = Deletes all blank lines in text that's being copied over. # 7. sed '/^$/d;G' = Double Spaces the body output. (Added good double spacing after removing all blank lines for forced uniformity) fi # url : Synopsis cited link #1 - Citation link for info in all synopsis.txt files / Not usable in RetroPie (05/29/2018) if [[ "$NO_DESC_FLAG" == 0 ]]; then url="$(sed '/^__________/,$!d' "$file" | grep -m1 ^http | tr -d '\r' | sed 's/&/&/g')" # 1. sed '/^__________/,$!d' "$file" = Begins looking at any line that starts with 10 "_" characters. Writes to file using all # parameters following #1 # 2. grep -m1 ^http = Finds and writes only first URL line. "-m1" stops grep after first instance found. fi # urlalt: Synopsis cited link #2 (Typically only for Hacks and Translations) / Not usable in RetroPie (05/29/2018) if [[ "$NO_DESC_FLAG" == 0 ]]; then urlalt="$(sed '/^__________/,$!d' "$file" | awk ' f && NR==f+1; /^http/ {f=NR}' | tr -d '\r' | sed 's/&/&/g')" # 1. sed '/^__________/,$!d' "$file" = Begins looking at any line that starts with 10 "_" characters. Writes to file using all # parameters following #1 # 2. awk ' f && NR==f+1; /^http/ {f=NR}' = Searches for "http[s]" and writes the following line, even if it also begins with "http[s]", # Which in this case it always will if it exists. fiI've tested this on many different examples of my
synopsis.txtfiles that cover a vast array of possibilities and this doesn't seem to have any negative effects in any instance. Some more testing is required after a full re-run of the gamelist.xml, but I do believe that I've "perfected" the <desc> code and what I've been looking for here. Now there will be zero extra spaces where they aren't wanted, and there should be enough versatility in this code to allow for quite a few accidental variances in my synopsis.txt files that would be fixed when run through this script.Next step is to try to figure out the reverse code that it looks like Meleu actually had added or at least started to add. I had no idea that it has already been in there. I'm hoping that it was working or was really close to working and I can figure it out from there. :)
-
The Reversal code to convert the
gamelist.xmlto[synopsis].txtis working great now!Had to add a bunch of tags and tweak a lot of the output, but I can now create an exact duplicate of any inputted
[synopsis].txtfile by running the Reverse code after thegamelist.xmlis generated!Actually, because of the code I put in both the regular script and the Reverse code, running the files through both of these processes can actually clean up things like leading blank lines in the descriptions, missing blank lines between the description and the citation URL's and any accidental extra lines in between paragraphs. After running them back and forth they will all be completely standardized, which is great news for the XBox since it displays the txt files exactly as they are written.
Even better, I figured out how to create a
Reverse/[system]/directory and write them there to save on clutter. One day when everything is completely tested and sure to work, I will just direct these files be written over the old[synopsis].txtfiles that were used to generate thegamelist.xmlin the first place, but at this point I haven't tested it enough to feel comfortable with that idea.My big hurdle right now is trying to figure out how to generate
[synopsis].txtfiles for the Folders. This is something that @meleu didn't figure out and he even had notes in the script sayingTODO: not sure what to do if the gamelist.xml has a <folder> entry.With all I've done so far, I'm not confident I can figure this one out if Meleu didn't.
It's not imparative since there are only about 30 of them for the NES, and that's probably the system I have with the most sub-directories/rom categories. Some systems only have 5 or 6 of them. If I have to end up doing these manually, that won't be all that big a deal considering that in this case it's only about 30 files in nearly 2,150. It would just mean that I'd have to edit the txt files individually rather than editing them in the spreadsheet, which I actually might decide is the preferred method when doing any edits to the
<desc>field anyhow.I'm thinking that the spreadsheet editing process is really only going to be extremely beneficial for fields like
<genre>and<publisher>and such where I can alphabatize the results and fix anything that is a typo or doesn't conform to whatever standard(s) I decide at that time. -
Anybody here wouldn't know of a magical command that could change every instance of
&to&when I'm trying to reverse back to text files from thegamelist.xml, would they?All 46 entries that had an
&in the file name were ignored in the reversal process, so their[synopsis].txtfiles were not created at all.Also, I can't seem to figure out how to convert all cases of
&back to plain&in this code either. The reversal code seems to rely almost exclusively onxmlstarlet, which seems to be much different than what I've been learning elsewhere in the code and I'm not coming up with any searches online that are giving me any answers to these two questions. -
@used2berx Do you have an example ? I thing
xmlstarletshould be the first choice when extracting data from XML, the&thing is one of the XML escape characters that can possible appear in a rawCDATAtext section. -
@mitu Hey mitu.
I did figure it out. It was
xmlstarlettoo.@meleu had 6 lines that used the
xmlstarletcommand in the reverse code.I spent quite a few hours here banging my head against the wall. I finally decided to try to create a copy of the gamelist.xml, replace all instances of
&with&, run the reverse code and then delete the file and rename the copied file the original name.That worked out well until it didn't know what to do with the
&characters anymore and I totally broke the code.It turns out the solution was really simple. I just had to add
-Tto thexmlstarlet selto force it to handle the&as text.Example:
IFS=$'\n' names=($(xmlstarlet sel -t -v "/gameList/game/xtrasname" "$GAMELIST"))
Became:
IFS=$'\n' names=($(xmlstarlet sel
-T-t -v "/gameList/game/xtrasname" "$GAMELIST"))This solved both of my problems by putting this in all 6 instances. Now it will write files with the & when it needs to, and it also converts every instance of
&to&from the gamelist.xml to the [synopsis].txt files. :)It's going to take a few more days to re-run the reverse code for all the files now, but I've been occasionally checking a few things as it's going and so far , so good.
Thanks
-
@mitu Well.... nope....
It didn't work. My testing was flawed.
Although it seems to fix all writing instances of the
&problem, it's still ignoring the files themselves. I'm assuming it must have something to do with the path variable that states where the file is located. I didn't think to test that when I thought everything was good. As an example, here's the path for one of the D&D games:<path>/home/pi/RetroPie/roms/nes/(1) Licensed/(1_1) US Licensed/AD
&D Pool of Radiance.zip</path>I'm not concerned with any other occurance of this when it's pathing the images and videos and such. The reversal code is just writing information back to an individual text file per game that doesn't need or display any pathing info. The only reason I need this to be read correctly now is to get these synopsis.txt files generated.
I'm actually not even sure that this is the problem. I'm just spitballing right now.
-
Nevermind. I got it. :)
This pointed out an earlier flaw in the code that I wasn't even aware of since it didn't prevent creation of any entries in the gamelist.xml or any of the tags that RetroPie uses.
I have a new tag on there called <xtrasname> which is the file name. It turns out that the way I had it coded this tag wasn't being filled out for any game entry that had an
&in the file name when thegamelist.xmlwas being generated. The reversal code was re-written to rely on that data to start the new[synopsis].txtgeneration, so if that field is blank than it will just skip to the next entry. That's why all 46 games with an&in the filename weren't being created now.I had to change the following code:
xtrasname="${file_name%.*}"to read as this instead:
xtrasname="${file_name//&/&}"I'm going to re-run the
gamelist.xmlnow and when it's done I'll start the reversal code. If we end up with 2,116 files when it's done then this all finally worked. :) -
Awesome. All 2,116 synopsis files were successfully created with the reversal script!
I checked about 100-150 of them and they all look good so far. In fact, only 3 of them had any differences at all in Diffchecker from the original before the
gamelist.xmlwas created. :)I still want to check all of them to see if there are any examples of bad things happening, but the 4 differences that were found in the 3 files were all desired. They were the removal of an extra blank space at the end of file, Capitalization of two of the tags that weren't capitalized, and the removal of a tag that was blank in the original because it was unused for that game.
Rather than check all 2,116 of them in Diffchecker which was taking forever and not really worth it because most of them were identical, I'm going to get a folder matching program and only test the ones that come up with any differences. I've got a good feeling that everything is perfect this time.
Man.... this is going to save so much time in the future once I make a script that will automatically edit bad characters and other misc things that I may have missed when I put these things together. I spend so much time trying to make them perfect by hand, but there are still problems. Now I won't have to go back in and continually edit all of them manually anymore.
I should be ready to try out your script to convert the gamelist.xml to a spreadsheet. Your code will already convert it back when you're done editing?
Once I'm able to do that part, I can seriously go in and start editing fields like
Genreto make sure that everything is represented properly. :) -
@used2berx said in Making premade gamelist.xml with xtra media collections:
I should be ready to try out your script to convert the gamelist.xml to a spreadsheet. Your code will already convert it back when you're done editing?
No, my script only exports to Excel the gamelist.xml. To export back, you can export directly from Excel.
In fact, I think you can ditch my script altogether and import the XML directly into Excel, modify it, then export it back. Excel will generate an 'XML Schema' based on your document structure, when importing the data, then export it using the same Schema back to an XML file. See an example at
.If you'd like me to take a look, paste a gamelist.xml file with just 1 entry and I'll try to import it/export it directly with Excel.
Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.
Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.