a gamelist.xml cleaner tool

hansolo77

@meleu said in a gamelist.xml cleaner tool:

NOTE: the script doesn't work if the gamelist.xml has invalid xml characters. Such as & in Track & Field. I will try to take a look at it soon.

Fortunately I implemented the --update feature, then the users can always try to run the script with that argument to get the latest version :-)

EDIT: HA! Looks like EmulationStation auto fix the gamelist.xml files with invalid xml characters as soon as it needs to update a gamelist.xml file. Then I'm not worried about this issue for a while! :D

This "bug" is bugging me. I have a huge Nintendo DS collection I've been going through. I have a lot of duplicate games as a result of different regions (USA, Europe, Japan). So after scraping, I'm going through my gamelist and removing all the extra non-USA versions where possible. I'm up to the letter P now, after about 4 days lol. I was editing the gamelist.xml manually until letter G before I remembered this script! Anyway, when I'm running the script, it's always missing some roms. I tried re-running it multiple times with the same result. I couldn't find a repo in your github to report it as an error, so I looked here. I discovered that any games with the & symbol in the filename (designated as & it would report as removed but wouldn't be). Sure enough, you already recognized it. So I'm just confirming that the issue still remains. I've even tried running the -u flag to update but it doesn't look like you've done anything with yet (cough 2 years). :)

I can still edit these out manually, but a fix would be nice for other people too. Hope you're doing well. Have a good one!

Example output of games NOT getting removed:

The game with <path> = "/home/pigaming/RetroPie/roms/nds/Gakken Chuugokugo Zanmai DS - Kiki &amp; Kaki Training (Japan) [b].zip" has been removed from xml.
The game with <path> = "/home/pigaming/RetroPie/roms/nds/Gakken Hangul Zanmai DS - Kiki &amp; Kaki Training (Japan) [b].zip" has been removed from xml.
The game with <path> = "/home/pigaming/RetroPie/roms/nds/Happy Party with Hello Kitty &amp; Friends! (Europe) (En,Fr,De,Es,It,Nl).zip" has been removed from xml.
The game with <path> = "/home/pigaming/RetroPie/roms/nds/My Baby 2 - Boy &amp; Girl (Europe) (En,Fr,De,Es,It,Nl).zip" has been removed from xml.
The game with <path> = "/home/pigaming/RetroPie/roms/nds/My Best Friends - Cats &amp; Dogs (Europe) (En,Fr,De,Es,It).zip" has been removed from xml.
The game with <path> = "/home/pigaming/RetroPie/roms/nds/Nintendogs - Labrador &amp; Friends (Europe) (En,Fr,De,Es,It).zip" has been removed from xml.
The game with <path> = "/home/pigaming/RetroPie/roms/nds/Nintendogs - Miniature Dachshund &amp; Friends (Japan).zip" has been removed from xml.
The game with <path> = "/home/pigaming/RetroPie/roms/nds/Nintendogs - Shiba &amp; Friends (Japan).zip" has been removed from xml.
The game with <path> = "/home/pigaming/RetroPie/roms/nds/Percy Jackson &amp; the Lightning Thief (Europe) (En,Fr,De,Es,It).zip" has been removed from xml.

shantigilbert

@hansolo77 I was having the same isue but I think I found a solution that works for me, I just need to confirm that it works on other setups, could you do a ls -la in your NDS rom folder and post the result to pastebin ? also paste your gamelist.xml

EDIT: I just opened a PR to deal with this issue

https://github.com/meleu/share/pull/12

hansolo77

Since YOu posted a pull request, do you still need me to give you the folder list and gamelist?

shantigilbert

@hansolo77 It would be helpful to test, or you can test it yourself see if that fixes your issue (in case you still have it)

meleu

@hansolo77 to test @shantigilbert's script you must have to download it:

wget https://raw.githubusercontent.com/shantigilbert/share/master/gamelist-cleaner.sh

hansolo77

Grrr... I was all excited and ready to test this, but it looks like I already manually cleaned my gamelist. I'll have to wait and see if it shows up on any new systems I add.

pjft

@meleu just saying that I'm late to the party, but this is spectacular. Thank you for putting this together! Hope you're doing great :)

Edit: actually, some feedback: this totally fails for games whose ROM is the folder - such as Daphne. It doesn't detect the folder and removes it from the gamelist. Other than that, all seems to be working well!

meleu

@pjft oh! I seriously need to update that script... I'll try it soon...

pjft

@meleu no prob, it is a great tool nonetheless :) It's just a matter of not replacing the Daphne gamelist :D

How are things? Good to hear from you!

bmulligan

Great tool. You've saved me hours of eye, back and neck strain. Thank you for this!