Best way to update marquee, video and other tags in your gamelist.xml

darknior

Now EmulationStation has video support :D

It's so cool but it can be a pain to add all the path for image, marquee and video tags by hand in the gamelist.xml

<game id="3833" source="ScreenScraper">
	<path>/home/pi/RetroPie/roms/gba/Boktai - The Sun Is in Your Hand (Europe) (En,Fr,De,Es,It).zip</path>
	<name>Boktai - The Sun Is in Your Hand (Europe) (En,Fr,De,Es,It)</name>
	<desc>Boktai : The Sun is in Your Hand est un jeu de rôle et d'action sur GBA. Le joueur contrôle un chasseur de vampires équipé d'une arme qui fonctionne à l'énergie solaire. Il est donc nécessaire de jouer sous la lumière du soleil pour charger le Solar Gun, seul objet efficace contre les monstres des donjons.</desc>
	<image>/home/pi/RetroPie/medias/gba/box front/Boktai - The Sun Is in Your Hand (Europe) (En,Fr,De,Es,It).jpg</image>
	<marquee>/home/pi/RetroPie/medias/gba/logos/Boktai - The Sun Is in Your Hand (Europe) (En,Fr,De,Es,It).png</marquee>
	<video>/home/pi/RetroPie/medias/gba/videos/Boktai - The Sun Is in Your Hand (Europe) (En,Fr,De,Es,It).mp4</video>
	<thumbnail/>
	<rating>0.9</rating>
	<releasedate>20040514T000000</releasedate>
	<developer>Konami</developer>
	<publisher>Konami</publisher>
	<genre>Simulation</genre>
	<players>1-4</players>
</game>

Here is my easy way to do it on every systems ;)

To do it use an Editor like NotePad ++ (or NetBeans IDE)
Open the gamelist.xml
Open the Search menu and Replace menu
Select Search Mode : Regular Expression and not check at right.

Now you can use RegEx to work on your file very fast at do all you want easily.

You can also try your Regex there to see if you are correct or not ?

My exemple for you guys

Delete the EMPTY <image/><image/> in consecutive when there is an error in the XML

	Find 	: image( *?)/>\s+<image( *?)/>
	Replace : image />

Delete the <image/> (it's an error) after a good one <image> line

	Find 	: image>(.*?)\.jpg</image>\s+<image( *?)/>
	Replace : image>$1.jpg</image>

	Find 	: <image( *?)/>\s+<image>(.*?)\.jpg</image>
	Replace : <image>$2.jpg</image>

Remove all the ENTER from the <desc>

(<desc>.*?)[\r\n]+(.*?</desc>)
$1 $2

But the problem is that there are many ENTER in the text when you scrap them and it is complicate to correct them.
For my part i remove all the ENTER on the XML with this regex and after use an Online Beautifier to make the XML ok to use.

	Find 	:  \r\n
	Replace : !!NOTHING !!

https://codebeautify.org/xmlviewer#copy

Update empty <image/> with ROM NAME and YOUR MEDIA PATH
Here i use : /home/pi/RetroPie/medias/gba/box front/

	Find 	: (([^/]*)\.zip</path>)$\s+(.*?)\s+(.*?)\s+<image( *?)/>
	Replace : $1\r\n\t\t$3\r\n\t\t$4\r\n\t\t<image>/home/pi/RetroPie/medias/gba/box front/$2.jpg</image>

Update empty <marquee/> with same IMAGE PATH
And change box front to logos

	Find 	: ((/.*?)box front(/.*?)\.jpg</image>)$\s+<marquee( *?)/>
	Replace : $1\r\n\t\t<marquee>$2logos$3.png</marquee>

	Find 	: ((/.*?)box front(/.*?)\.jpg</image>)$\s+<video>
	Replace : $1\r\n\t\t<marquee>$2logos$3.png</marquee>\r\n\t\t<video>

Update empty <video/> with same MARQUEE PATH
And change logos to videos

	Find 	: ((/.*?)logos(/.*?)\.png</marquee>)$\s+<video( *?)/>
	Replace : $1\r\n\t\t<video>$2videos$3.mp4</video>

Add <thumbnail /> after <video/>

	Find 	: ((/.*?)videos(/.*?)\.mp4</video>)$\s+<rating
	Replace : $1\r\n\t\t<thumbnail/>\r\n\t\t<rating

After that there also many problem possible like some TAG that are not there ...

(</name>)$\s+<image>  to  $1\r\n\t\t<desc></desc>\r\n\t\t<image>

<thumbnail />  to  <thumbnail/>
<developer />  to  <developer/>
<publisher />  to  <publisher/>
<genre />      to  <genre/>
<players />    to  <players/>

(</image>)$\s+<releasedate>  to  $1\r\n\t\t<rating>0</rating>\r\n\t\t<releasedate>
(</image>)$\s+<developer>  to  $1\r\n\t\t<rating>0</rating>\r\n\t\t<releasedate>00000101T000000</releasedate>\r\n\t\t<developer>
(</genre>)$\s+</game>  to  $1\r\n\t\t<players></players>\r\n\t</game>

((/.*?)box front(/.*?)\.jpg</image>)$\s+<rating>  to  $1\r\n\t\t<marquee>$2logos$3.png</marquee>\r\n\t\t<rating>
((/.*?)logos(/.*?)\.png</marquee>)$\s+<rating>  to  $1\r\n\t\t<video>$2videos$3.mp4</video>\r\n\t\t<rating>
(</video>)$\s+<rating>  to  $1\r\n\t\t<thumbnail/>\r\n\t\t<rating>

Capeman

@darknior I love using regular expression to update my gamelists, i do mine in TextWrangler on Mac. It's amazing how powerful this tool can be! Good writeup!

darknior

@Capeman Yes regular expression are the best way to manage gamelist.ximl by hand.
Faster and easier if you understand what you do, because it is hard to understand how to make a good regex :p
I will try to make a video or screenshot for help peoples ...

meleu

I love sed and vim. My preferred ways to use regex.

meleu

By the way, xmlstarlet can be a good way to edit xml files via shell scripts. I use it here (step 9):

https://retropie.org.uk/forum/topic/10849/create-a-custom-es-system-able-to-launch-games-for-many-systems/

Used2BeRX

@darknior Hey darknoir. Thanks for the tip. I've been using Notepad ++ and I couldn't figure out how to do that, so I actually entered a marquee line by hand on over 1,000 entries a few weeks back.

I'm working with UDb23 on a program that will create gamelist.xml files automatically for us. I'm not sure how far you are deviating from the Rx1.0 setup, but in the end if he's able to code it that shouldn't matter for you. Here's the thread we're discussing it:

https://retropie.org.uk/forum/topic/10406/making-premade-gamelist-xml-with-xtra-media-collections/58

If you guys have other work to do like working on the skins and stuff, I would hold off on making those gamelist.xmls by hand until we get this done.

You're more than welcome to add your own input or suggestions, but I'd hold off on that for a little while until we get a little further into the development. He's kind of got his hands full with it at this stage.

darknior

@Used2BeRX said in Best way to update marquee, video and other tags in your gamelist.xml:

Db23 o

Thanks RX but my project is so different than yours ...
I don't like Rx 1.0 ... so many games so many bad games, clones ...
So old roms with some bugs ...

I make a "Best-of Roms" for each systems and it take us a long time to make it near perfect !
We remove all Mah-jong games, Japan no playable games ...
We add all the US translation of Jap games, and for us all the FR translations too.
Our pack is full US + FR set
We also add many excellent hacks for best games like Mario, Sonic, Metroid ...
I also Hack all the PSX games and other to remove protection. And some translations ...
We use last roms, the NoIntro Rom Pack, the best one, and the last CD ISO games with redump.org !

For the Xtra it is same, your Xtra are really OLD !
We have now many HD Covers, Some from ScreenScraper, some from EmuXtra, and other source ...
We also make some ourselves ... we work for Full HD 1080p project ! Skin and Xtra !
Same for videos, EmuMovies make fantastic videos, and i made some too, the Xtra 1.0 videos are now so crappy and not 640 x 480 !

But it is not the subject her :(
I will share all our stuff all our code our skins etc ...
You can use all we do, it is not a problem for me ... i like sharing my work.

Used2BeRX

@darknior Nah. There's no clones in the Rx1.0 set. It's completely clone free.

I already talked to Gilles and reclined the offer to work with you guys because you were removing a lot of games. I already knew we were working on two completely different projects.

I was just letting you know that you should wait to make your gamelist.xml files until we get the program done. Otherwise you're wasting a lot of time making those by hand.

darknior

@Used2BeRX said in Best way to update marquee, video and other tags in your gamelist.xml:

@darknior Nah. There's no clones in the Rx1.0 set. It's completely clone free.

I already talked to Gilles and reclined the offer to work with you guys because you were removing a lot of games. I already knew we were working on two completely different projects.

I was just letting you know that you should wait to make your gamelist.xml files until we get the program done. Otherwise you're wasting a lot of time making those by hand.

Thanks but no, with Universal XML Scraper it is very fast to make an gamelist.xml
Some Regex to add all Xtra and it is done.
The long task was to found all the stuff in Full HD for our RVGM project :(
We have it from xBox but it is long to work on it ... UXMLScraper have some (not all) bad stuff too ...