a gamelist.xml cleaner tool

theWinterDojer

@meleu Awesome. I will give this a try when I get home. Thanks a bunch!

theWinterDojer

@kaltinril @meleu

This worked perfectly, thank you. I deleted 826 images across 6 systems, all duplicates :)

xadox

The gamelist-cleaner.sh by @Kaltinril seems to be the newer one. Maybe this could be mergeded into the one by @meleu?

Is the images-cleaner.sh by @meleu already usable?

Vodkanakas

I an trying to use this to clean up the images of games that i have removed and all i get is errors. the comman im using is

./images-cleaner.sh -t -g /home/pi/RetroPie/roms/nes -i /home/pi/RetroPie/roms/nes/images

when it ends all i get is error could not find gamelist for system but i know the gamelist.xml is in that folder and the images are in the images folder.

Sano

@vodkanakas Are you sure your gamelist files are in /home/pi/RetroPie/roms/nes ?
Default is /home/pi/.emulationstation/gamelists/nes (that's the default for the script, too).
Edit : BTW if I remember, the script loops through the systems.
Have you just tried to launch it with the default folders ? : ./images-cleaner.sh -t

Vodkanakas

@sano yes seelphs scraper has an option to use the Rom folder for the gamelists and images so that's why I know they are there. It accesses the images folder just not the gamelist.xml

Edit: If i try just ./images-cleaner.sh -t i get a no such file or directory

AndersHP

I tried running the ./images-cleaner.sh -tcontaining the code that @meleu linked to here, after removing a bunch of roms from different system folders via SSH, but the script didn't report anything to delete.

I would suppose it both deleted the redundant artwork as well as cleaning up the gamelist files, right?

Does this script not work if roms are placed on USB? Art+gamelists are placed under emulationstation/gamelists and /downloaded_images

meleu

@andershp according to the --help you can specify an arbitrary directory with --images. Try it ;)

AndersHP

@meleu the images are in the default place - it's the ROMs that are on USB..

greenie

@meleu I've just been through and cleaned all my gamelist.xml files - congrats for creating such a time-saving tool!

However, I've just noticed that (although it says the entry is removed from the .xml file during processing) any entry with & in the name is not removed from the .xml file.

Is there any way around this?

Cheers.

greenie

@meleu That last comment was meant to say "& amp ;" but the post has changed it to "&".

greenie

@meleu I've now tried this on several PCs, (including the Windows Subsystem for Linux!) and it's the same result on all of them.

To reproduce...

Download one of the gamelist.xml and downloaded_images files/folders from the this master repository: https://github.com/recalbox/recalbox-gamelists/tree/master/gamelists .

Drop them in your system's ROMS folder. This is based on the no-intro romset, which my collection is based on.

You should now have the gamelist.xml file, downloaded_images folder and ROMs under your ROMS folder (in this example I'm using the NES system).

Run the gamelist-cleaner.sh script after editing the settings to point at your ROMS folder.

The script appears to run perfectly. I can watch the output in Terminal with glee!

But, after inspecting the new xml file, you'll see that entries such as Advanced Dungeons & Dragons - DragonStrike (USA).zip are still present even though, in my case, I don't have the ROM for that game.

Is this a formatting issue? Has anyone else experienced this?

Any advice would be hugely appreciated as this is the last step I need to nail before my system is finished. I'll be editing out the remaining entries in the mean time.

Thanks.

meleu

@greenie
Regarding the & char, it's a known issue and I posted about it here. In the linked post you can see a workaround to fix this issue (I agree that it isn't the ideal solution, though).

And thanks for this report. ;-)

greenie

@meleu Thanks for the response. I don't think this is exactly the same issue, I'm not sure what workaround you refer to on your post either.

The problem I was getting is that anything in the gamelist.xml file with "& amp ;" in the name was not getting removed. It could see the entry, and list it as removed in the terminal, but the entry remained.

I have managed to find my own workaround:

Find/Replace on gamelist.xml, changing all "& amp ;" to "ampersand".
Use A Better Finder Rename on the Mac (I'm sure there are similar tools out there, but I have this handy) to replace all "&" in ROM file names to "ampersand".
Run your script, then reverse the above steps.

Bish bash bosh. Done.

Tim

I'm new to RetroPie, but wanted to say thank you for this tool. My OCD thanks you as well. Sweet, sweet, clean gamelist!

AndersHP

@meleu could my issues here relate to the symlink issues I have in the other thread regarding launching images?

chipsnblip

i think i might have found the culprit regarding games that have an ampersand in the filename (& in the gamelist.xml)

line 221 in the script reads:

xmlstarlet ed -L -d "/gameList/game[path=\"$path\"]" "$clean_gamelist"

but if i change it to:

xmlstarlet ed -L -d "/gameList/game[path=\"$full_path\"]" "$clean_gamelist"

they are handled properly. i'm not a programmer by any means (just a computer enthusiast), so hopefully someone with more knowledge than myself can confirm that it's working.

duiz

I suffer from some rather messy gamelists, but I am a newbie when it comes to these commandline things... Is it possible to request a small step by step guide to get this up and running for a beginner with little experience in github and etc?

hansolo77

Wow, @meleu releases another script I didn't even know about! :) Can't wait to try it out tomorrow when I'm more awake.

Degsy

@chipsnblip Nearly right.. and that led me on to the fix

If you instead add in this above the xmlstarlet line, it will handle it correctly.

path="$(echo "$path" | sed 's@&amp;@\&@g')"

If you want to go further and want to see feedback onscreen when its triggered, then INSTEAD add in this block above the xmlstarlet line:

	#&amp; fix
	if [[ $path == *"&amp;"* ]]; then
		path="$(echo "$path" | sed 's@&amp;@\&@g')" 
		echo "&amp; fix applied"
	fi

The reason why it was failing was that either bash or xmlstarlet was translating it incorrectly.
The $full_path that you were using already has the & translated back to &, but that variable also includes the full path of the rom (/home/pi/RetroPie/roms/… etc) which does not match what is in the XML (didn't match mine anyway).