Used Steven Selph scraper and games multiplied (just on list).

nick1984

As mentioned in the Topic title I scraped all my systems, for covers and video samples and after that I fount in many systems that most of ROMS appear 2 3 or 4 times in the ROM lists. They were not copied (I checked with Filezilla) but I see more than one in the game list. Can anyone give me an advice on what happened?

Pi Model or other hardware: raspberry pi 3 b+
RetroPie Version Used: 4.1

mitu

Which system has the duplicates ? Do you have maybe .cue/.bin files that were scraped individually ?

nick1984

@mitu said in Used Steven Selph scraper and games multiplied (just on list).:

Which system has the duplicates ? Do you have maybe .cue/.bin files that were scraped individually ?

MAME, SNES, MEGADRIVE.
Files were scraped with Stephen Selph scraper running from console with ES closed.

mitu

You can check the gamelist.xml file generated by the scraper in \\retropie\configs\all\emulationstation\gamelists\snes and see what exactly is duplicated. You can post one such file on pastebin.com so we can take a look.

nick1984

I tried but I continuously have a 400 bad request error from pastebing; i guess the file is toooooo long

mitu

@nick1984 Post a relevant part of the file, which has some duplicates in it.

nick1984

@mitu
https://pastebin.com/TwUyZK61

This is from MEGADRIVE

mitu

The duplicates seem fine. You have multiple ROMs for the same game, from different regions, for which the scraper assigns the same title.
For instance - you have Donald in Maui Mallard (Brazil).zip and Donald in Maui Mallard (Europe).zip, which both appear as Donald in Maui Mallard. Or DJ Boy (Europe).zip and DJ Boy (Japan).zip, both appearing as DJ Boy.

nick1984

Maybe I took the wrong part of file or system, tomorrow I check since I am in SSH now.
I am sure FILES multiplied in list

mitu

@nick1984 said in Used Steven Selph scraper and games multiplied (just on list).:

I am sure FILES multiplied in list

Yes, they are - I can see it from the gamelist.xml fragment you posted. Did you read my previous reply to see why it's happening ?

nick1984

@mitu said in Used Steven Selph scraper and games multiplied (just on list).:

@nick1984 said in Used Steven Selph scraper and games multiplied (just on list).:

I am sure FILES multiplied in list

Yes, they are - I can see it from the gamelist.xml fragment you posted. Did you read my previous reply to see why it's happening ?

Dear mitu,
I know that some of my games are double, but I am talking about games that are not. Games that for example do not have EUROPE / USA Version.
Here following you can see photos of 3 games as I see in ES and also a screenshot of the ROM FOLDER from FILEZILLA.
I have 3 different games, with 3 ROMS in filezilla. But I see 7 total games in list.
2 for the 1st game, 2 for that 2nd game, 3 for the 3rd game.
Maybe my small knowledge about Retropie make me misunderstand somthing.
Thank you for your help

https://postimg.cc/gallery/hjghqnwc/

mitu

@nick1984 said in Used Steven Selph scraper and games multiplied (just on list).:

I know that some of my games are double, but I am talking about games that are not. Games that for example do not have EUROPE / USA Version.

I see that - but that's not what appears in the gamelist.xml file. Do you have an additional gamelist.xml file in your roms/megadrive folder ?

EDIT: you can get a more detailed log from EmulationStation if you're starting from the command line with emulationstation --debug. The log file should be in \\retropie\configs\all\emulationstation\es_log.txt - upload it on pastbin.com.

nick1984

@mitu said in Used Steven Selph scraper and games multiplied (just on list).:

emulationstation --debug

fount on /opt/retropie/configs/all/emulationstation

https://pastebin.com/CzNp4HEe

EDIT:
I do not have other gamelist in /home/pi/RetroPie/roms/megadrive
here I post the correct gamelist that I fount the exact directory you indicated me, I think before i made a mistake.
FROM
/opt/retropie/configs/all/emulationstation/gamelists/megadrive
GAMELIST
https://file.io/jAboY9

mitu

I don't see any errors or indication of why this duplicate behavior happens. Have you checked if you have an additional gamelist.xml file in the ROMs folder ?

nick1984

@mitu said in Used Steven Selph scraper and games multiplied (just on list).:

I don't see any errors or indication of why this duplicate behavior happens. Have you checked if you have an additional gamelist.xml file in the ROMs folder ?

I updated my èrevious post with an EDIT

mitu

OK, but this looks like the same issue - you have for instance Air Management - Oozora Ni Kakeru shown twice, for 2 different ROMs:

<game id="501" source="screenscraper.fr">
		<path>./Aerobiz (USA).zip</path>
		<name>Air Management - Oozora Ni Kakeru</name>
		<desc>...
	</game>

then also as

<game id="501" source="screenscraper.fr">
		<path>./Air Management - Oozora ni Kakeru (Japan).zip</path>
		<name>Air Management - Oozora Ni Kakeru</name>
		<desc>..
	</game>

Same thing for Air Management 2 - Koukuuou O Mezase - there are 2 ROMs with the same title:

Aerobiz Supersonic (USA).zip
Air Management II - Koukuuou o Mezase (Japan).zip

nick1984

Then it's a mystery ;-)

mitu

@nick1984 said in Used Steven Selph scraper and games multiplied (just on list).:

Then it's a mystery ;-)

No, it's the same thing I said yesterday - you have the same title for different files. You can re-scrape and choose the option to keep the filenames as titles, you should have the exact listing - both on disk and in EmulationStation.

nick1984

Sorry, maybe I am stupid.
Why I have 7 titles when I have only 3 ROM files?

I could understand if I have in ROMS folder :

SuperMario (USA).zip
SuperMario (EU).zip

and I see in ES:

Super Mario Bros
Super Mario Bros

with no specification since the Title is common.

But I have more titles than ROMS

mitu

@nick1984 said in Used Steven Selph scraper and games multiplied (just on list).:

Sorry, maybe I am stupid.
Why I have 7 titles when I have only 3 ROM files?

Look in the .xml file you posted for the duplicate title and see which files are associated with it. I've given you 2 examples above from the file you posted.