Duplicated games for various systems

jul**17001

Good afternoon everyone,

I have a problem that i didn't manage to solve reading topics on the web, hope you'll be able to help :)

I am using :

raspberry pi 3B V1.2 : retropie / emulationstation
SD card 64 Gb
Samsung EVO 1Tb SSD for roms storage
Power supply : 5.25V / 2.4A
Retropie version 4.5.10

I have been building my collection of roms onto my Pi this week and noticed that several systems are now showing duplicated games. (but are not duplicated on the ROM directory itself, only a visual thing in EMULATIONSTATION). This happened for ATARI LYNX, PC ENGINE, GAMEBOY, GAMEGEAR..(the full collection of ROMS is duplicated for those systems)
Considering the systems mentioned, it is not an issue of multi disks or PS1 games (Bin/cue).
If i select either of the 2 lines, the game is launching with no issue, doesn't matter.

IMG_0252 (1).jpg

I am wondering if my external hard drive can be an issue as I have to disconnect/reconnect it a lot (to transfer my ROMS on it). As i didn't find a way to eject the drive like on a regular PC, i am just wondering it that doesn't mess up things..
I was suspecting this because each time i disconnect the drive and connect it to my PC, i have a message saying that the drive may have errors and must be scanned. (despite the fact that i am always very careful to shutdown the system via the emulationstation menu, and wait the LED of the SSD to light off before to turn off the power of the pi and disconnect the SSD).

so here are my 2 questions :

How to delete duplicated titles in retropie / emulationstation (with ROM showing only 1 time in the system directory) ?
how to prevent that for the future ? if it is because of the external SSD, is there an easy way to disconnect it without creating errors like this ?

Thank a lot in advance for your answers. I have the feeling that i am not the only one having those issues, your answers will be greatly appreciated!

quicksilver

@jul-17001 Can you remove your external hard drive and then check to see if there are roms duplicated on the actual micro sd card?

jul**17001

@quicksilver

Tks for the advise, that was a good idea. Tried to disconnect the hard drive and launched the pie with the SD card only. 0 Games showed up, so definitely no game copies on the SD.

quicksilver

@jul-17001 Check your gameslist.xml and see if there are duplicate entries

jul**17001

so, here is an example of what i got for the game Awesome Golf for lynx system :
(This one appears twice in emulationstation)

looks like it is listed 5 times with the following extensions : 7z, LNX, ZIP, lnx, zip

<game id="" source="">
		<path>./Awesome Golf (USA, Europe).7z</path>
		<name>Awesome Golf</name>
		<desc>You are on the 18th tee, one stroke behind the defending champion. Do you take your caddy’s advice and play it safe, or do you risk the water? Awesome Golf provides great realism and a challenge for one to four players.</desc>
		<publisher>Atari</publisher>
		<developer>HandMade Software</developer>
		<image>./boxart/Awesome Golf (USA, Europe).png</image>
		<marquee>./wheel/Awesome Golf (USA, Europe).png</marquee>
		<video>./snap/Awesome Golf (USA, Europe).mp4</video>
		<releasedate>19910314T000000</releasedate>
		<genre>Sports</genre>
		<players>4+</players>
	</game>
	<game id="" source="">
		<path>./Awesome Golf (USA, Europe).LNX</path>
		<name>Awesome Golf</name>
		<desc>You are on the 18th tee, one stroke behind the defending champion. Do you take your caddy’s advice and play it safe, or do you risk the water? Awesome Golf provides great realism and a challenge for one to four players.</desc>
		<publisher>Atari</publisher>
		<developer>HandMade Software</developer>
		<image>./boxart/Awesome Golf (USA, Europe).png</image>
		<marquee>./wheel/Awesome Golf (USA, Europe).png</marquee>
		<video>./snap/Awesome Golf (USA, Europe).mp4</video>
		<releasedate>19910314T000000</releasedate>
		<genre>Sports</genre>
		<players>4+</players>
	</game>
	<game id="" source="">
		<path>./Awesome Golf (USA, Europe).ZIP</path>
		<name>Awesome Golf</name>
		<desc>You are on the 18th tee, one stroke behind the defending champion. Do you take your caddy’s advice and play it safe, or do you risk the water? Awesome Golf provides great realism and a challenge for one to four players.</desc>
		<publisher>Atari</publisher>
		<developer>HandMade Software</developer>
		<image>./boxart/Awesome Golf (USA, Europe).png</image>
		<marquee>./wheel/Awesome Golf (USA, Europe).png</marquee>
		<video>./snap/Awesome Golf (USA, Europe).mp4</video>
		<releasedate>19910314T000000</releasedate>
		<genre>Sports</genre>
		<players>4+</players>
	</game>
	<game id="" source="">
		<path>./Awesome Golf (USA, Europe).lnx</path>
		<name>Awesome Golf</name>
		<desc>You are on the 18th tee, one stroke behind the defending champion. Do you take your caddy’s advice and play it safe, or do you risk the water? Awesome Golf provides great realism and a challenge for one to four players.</desc>
		<publisher>Atari</publisher>
		<developer>HandMade Software</developer>
		<image>./boxart/Awesome Golf (USA, Europe).png</image>
		<marquee>./wheel/Awesome Golf (USA, Europe).png</marquee>
		<video>./snap/Awesome Golf (USA, Europe).mp4</video>
		<releasedate>19910314T000000</releasedate>
		<genre>Sports</genre>
		<players>4+</players>
	</game>
	<game id="" source="">
		<path>./Awesome Golf (USA, Europe).zip</path>
		<name>Awesome Golf</name>
		<desc>You are on the 18th tee, one stroke behind the defending champion. Do you take your caddy’s advice and play it safe, or do you risk the water? Awesome Golf provides great realism and a challenge for one to four players.</desc>
		<publisher>Atari</publisher>
		<developer>HandMade Software</developer>
		<image>./boxart/Awesome Golf (USA, Europe).png</image>
		<marquee>./wheel/Awesome Golf (USA, Europe).png</marquee>
		<video>./snap/Awesome Golf (USA, Europe).mp4</video>
		<releasedate>19910314T000000</releasedate>
		<genre>Sports</genre>
		<players>4+</players>
	</game>

quicksilver

What format are your lynx roms in? what scraper are you using?

jul**17001

@quicksilver

for the lynx i keep that in the zip files.
For the gameboy for example, who presents the same issue with duplicated games, it is unzipped, i am using the .gb

For the scrap, i tried many things and nothing was working as i wanted. So I just downloaded ROMS packages with artwork included to bypass the issue (they are all in files named snap, wheel, etc... within the system rom file itself).
i tried the 2 scrapers ThegamesDb and Screenscraper and nothing worked. Then i tried the Steven Selph's Scraper, was better but i wanted videos. So using now local artwork files at the source, in the rom files..

quicksilver

@jul-17001 so you downloaded pre-populated gameslist.xml files? The duplicate .zip entries are probably whats causing the issue. To test, first back up your gameslist.xml (so you can undo any changes). Then edit the gameslist.xml thats on your pi by removing one of the entries for .zip or .ZIP. Restart emulation station and see if the game is no longer duplicated. If that works then you know what you need to do to fix the other duplicate entries.

jul**17001

@quicksilver tks for the answer ! i will try that tonight and will let you know what happens. tks a lot for your help. kinda new with that so i am still learning a lot..

quicksilver

@jul-17001 This may be helpful as well https://retropie.org.uk/forum/topic/11074/a-gamelist-xml-cleaner-tool

IceChes1

Pull out your SD card and simply delete the duplicates on your laptop/PC.

quicksilver

@IceChes1 delete what? Why does he need to take out his SD card?

Clyde

@jul-17001 When cleaning up your roms, mind that Linux differentiates upper and lower case in filenames. Thus, the Awesome Golf (USA, Europe).lnx and Awesome Golf (USA, Europe).LNX in your gamelist.xml are two different files for Linux. In Windows, it would be one file which the other would've been overwritten by.

So, keep one of them and be sure to delete the xml entry maching the case of the actual file, or the remaining (wrong) entry may not be able to start the game anymore.

jul**17001

@IceChes1

That's good information. thank you. Just curious, do you have any ideas why my gamelist files are like that ? i will definitely clean up all that stuff but i would like to understand what happened in order to avoid that in the future.

Clyde

@jul-17001 Well, it looks like you have duplicate romfiles in different formats (7z, zip, lnx) in your rom folder, and whatever generated your gamelist.xml included all of them in it.

quicksilver

@jul-17001 you downloaded your gameslist.xml right? You didn't generate them yourself?

jul**17001

@quicksilver right. All my gamlists files were included with the rom / workart package. i will edit a gamelist file today as a test and will let you know if that solves the problem.

jul**17001

*Problem solved !

so I just tried to edit the gamelist.xml for the Lynx system (just used the regular notepad from windows). i deleted all the sections that i didnt want for 1 specific game (got rid of .7z, .lnx, LNX, ZIP) and just kept the .zip section that matches the extension type i decided to use for that specific system. It worked perfect ! now this game appeared only once in my collection. Thanks a lot for all your advises, i learnt a bit more with the gamelist files thanks to this issue.
Happy gaming