Can't Scrape & Save SNES Info

dsquared

Hello again:

So I am continuing my set up of my RPi 4B. The first 2 game collections I put on the RPi were NES & SNES. I am using the scraper in emulationstation. I successfully scraped and saved my NES collecion. However, I will note there is no /home/pi/.emulationstation/gamelists/nes/gamelist.xml directory/file.

I am not sure where scraping information is being saved. Could somebody please tell me where this information is located? - It may help me diagnose my problems with SNES.

The main problem is the SNES collection. I am unable to scrape the collection - information and images are not being retrieved, using either of the db's. Further, there is no /home/pi/.emulationstation/gamelists/snes/gamelist.xml directory/file. I have tried 1 gameboy rom in the roms/gb/ folder and successfully scraped and saved the information and image. It is located in /home/pi/.emulationstation/gamelists/gb/gamelist.xml. BTW I have tried the snes files in zip and smc formats - the result is the same. In the past I have been able to successfully scrape SNES files in either format.

I have created the /home/pi/.emulationstation/gamelists/snes/ folder and placed an empty gamelist.xml file in it - the result is the same.

Any ideas, questions or help in resolving this headache would be appreciated.

Obligatory Information:

Pi Model or other hardware: 4B+ 8GB
Power Supply: LABISTS Raspberry Pi 4 Power Supply 5.1V 3A
RetroPie Version Used: retropie-buster-4.7.12 (used to fix earlier network problem)
Retropie -Setup Script: V4.7.12
USB Devices connected: Logitech F310 Controller, wireless USB keyboard
Error messages received: None
Verbose log: (if relevant): Not Applicable
Guide used: None
File: config.txt - see attachment 1
Emulator: lr-snes9X
Attachment 1: config.txt: https://pastebin.com/nNPnL70m

mitu

What image did you use to install RetroPie ?

However, I will note there is no /home/pi/.emulationstation/gamelists/nes/gamelist.xml directory/file. [...]
Further, there is no /home/pi/.emulationstation/gamelists/snes/gamelist.xml directory/file

The gamelist.xml file(s) should be created regardless of whether you scraped or not the system, but if you had a prior gamelist.xml in the ROMs folder, this is where EmulationStation will save it. Did you disable metadata saving from EmulationStation's options ?

dsquared

Hello:

"What image did you use to install RetroPie ? "

As I said in my original post I used the "retropie-buster-4.7.12." I know this is not an "official release" but it was the only way I could fix my network problems that persisted over 4 days as I explained here.

Please Note: While switching to the terminal I saw the following message:

"Error parsing XML file '/home/pi/.emulationstation/gameslists/snes/gamelist.xml'!
No document element found"

This despite my having created the directory and an empty file.

mitu

@dsquared said in Can't Scrape & Save SNES Info:

Please Note: While switching to the terminal I saw the following message:
"Error parsing XML file '/home/pi/.emulationstation/gameslists/snes/gamelist.xml'!
No document elementfound"
This despite my having created the directory and an empty file.

The file should not be empty, hence the error logged. Do you have enough space on your sdcard ?
Run EmulationStation with the --debug parameter from the command line and reproduce the issue with the scraping, then post the log file from $HOME/.emulationstation/es_log.txt on pastebin.com.

dsquared

Thank you for getting back to me. Here is the es_log.txt file pastebin link. It looks like at line 242 - it stops checking for any rom directories after PSX.

mitu

@dsquared said in Can't Scrape & Save SNES Info:

It looks like at line 242 - it stops checking for any rom directories after PSX.

No, it doesn't. I don't see any errors related to scraping either and also, the nes gamelist file seems to be fine (whereas you reported earlier that's missing):

May 07 19:16:01 lvl2: 	Parsing XML file "/home/pi/.emulationstation/gamelists/nes/gamelist.xml"...
...
May 07 19:16:01 lvl1: 	File "/home/pi/RetroPie/roms/nes/Rollergames (Europe).nes" does not exist! Ignoring.
May 07 19:16:01 lvl1: 	File "/home/pi/RetroPie/roms/nes/U-Force Test (USA).nes" does not exist! Ignoring.

Run EmulationStation again, with the debug parameter, do a - unsuccessful - scrape for the SNES then exit EmulationStation. Take the log again and post it.

dsquared

Hello again:

Yes the NES gamelist.xml seems to have corrected itself. A partial victory!

I copied 20 SNES roms over to the MicroSD card. They are all in 7z format and in the /home/pi/RetroPie/roms/snes folder.

I then scraped the ROMS - this time I was successful in scrapping 18 of the 20 games. I consider this progress because I was not scraping anything related to SNES games before now.

I then restarted emulationstation - but no data/images were saved. (There is no gamelist.xml file in the /home/pi/.emulationstation/gamelists/snes/ folder - as I deleted the empty file I had placed in there.)

I am attaching my new es_log.txt on pastebin here. I do not see the log recording anything to do with my SNES scrape. Remember in my earlier post, that I said - it stops checking for any ROM directories after PSX. - I suspect something may have got tweaked in this part of ES.

mitu

@dsquared said in Can't Scrape & Save SNES Info:

I am attaching my new es_log.txt on pastebin here.

This doesn't contain the info I was asking - from the scraping and ES shutdown.

I do not see the log recording anything to do with my SNES scrape. Remember in my earlier post, that I said - it stops checking for any ROM directories after PSX.

You keep saying that, but there's nothing in the logs to support this.

I suspect something may have got tweaked in this part of ES.

I don't understand what that means. There's no difference between nes/snes as far as EmulationStation is concerned, they're both systems where the games are stored.

dsquared

"This doesn't contain the info I was asking - from the scraping and ES shutdown."

I realize that - nothing is being recorded regarding the scraping -despite my scraping the ROMS. - i don't know why it's not there...... I entered the command emulationstation --debug in the terminal and go to scraping the snes games.

"You keep saying that, but there's nothing in the logs to support this."

Sorry I have been looking at these files for so long - I am getting confused.

What I am trying to say is that ES examines each ROM folders - for example "n64" are checked for games. If no games are found ES responds with:

(from included file "/etc/emulationstation/themes/carbon/n64/../carbon.xml")
could not find file ""
May 08 10:07:14 lvl1: System "n64" has no games! Ignoring it.

When ES gets to "nes" it responds with:

May 08 10:07:14 lvl2: Parsing XML file "/home/pi/.emulationstation/gamelists/nes/gamelist.xml"...

ES seems to go along checking each game system folder and then goes along and scans "sg-1000" and ES responds with:

(from included file "/etc/emulationstation/themes/carbon/sg-1000/../carbon.xml")
could not find file ""
May 08 10:07:14 lvl1: System "sg-1000" has no games! Ignoring it.

The next ROM folder is "snes"

ES checks "snes" and responds with:

May 08 10:07:14 lvl1: Warning from theme "/etc/emulationstation/themes/carbon/snes/theme.xml"
(from included file "/etc/emulationstation/themes/carbon/snes/../carbon.xml")
could not find file "

There is no parsing of a snes/gamelist.xml file or "the System has no games ignoring it "message. It is like it just ignores this folder all together.

mitu

@dsquared said in Can't Scrape & Save SNES Info:

I realize that - nothing is being recorded regarding the scraping -despite my scraping the ROMS

The info I was most interested in is the exit messages, where the metadata saving (writing the info to the gamelist.xml file for a system) takes place.

There is no parsing of a snes/gamelist.xml file or "the System has no games ignoring it "message. It is like it just ignores this folder all together.

Since there is no System snes has no games message in the log, it means EmulationStation doesn't ignore the folder and there are games there. The lack of Parsing ... gamelist file means there is no gamelist saved yet. Nothing out of the ordinary on a first run, but exiting EmulationStation should save a gamelist file, that's why I asked for a complete log (including ES exiting).

dsquared

Thank you for being so patient with me. The problem is solved now. If I believed in gods, I would call you a retropie god. I guess Global Moderator will have to do.

"that's why I asked for a complete log (including ES exiting)." I was giving you what ES was producing.

I launched ES from terminal with the command emulationstation --debug
I went and scraped the snes ROMS - data and images were retrieved.
Exited ES and restarted it with just the emulationstation command
copied the es_log.txt file to a rom folder - so I could transfer it to my PC

How I fixed the problem:

Moved the SNES ROM's to a folder called SNESBAK
There was no /snes/gamelist.xml file
I found a bucket load of image files in /home/pi/.emulationstation/downloaded_images/snes/ I deleted them all
I rebooted retropie
Copied the snes ROMs from the SNESBAK folder to the snes folder
restarted ES
Tried scraping 10 of the snes ROMS and was succesful
Restarted ES and the 10 ROM's data and images were saved.
Deleted SNESBAK and ROMS