Skyscraper: problem with re-naming rom names without bracket info

muldjord

@mitu said in Skyscraper: problem with re-naming rom names without bracket info:

@muldjord cache invalidation is one of the hardest problems in CS :). But

are you sure it's the same problem here ?

CS?

I thought Skyscraper was already at version 3.x ?

Haha, yes, it is. I'm currently working on a different project that's on 2.x, so I got it mixed up. :D

mitu

@muldjord said in Skyscraper: problem with re-naming rom names without bracket info:

CS?

Computer science. Or you might just say programming/software engineering.

muldjord

@mitu said in Skyscraper: problem with re-naming rom names without bracket info:

Computer science. Or you might just say programming/software engineering.

Ah yes, the joys of supporting legacy formats. And after I just re-read OP, you might be right. He does not seem to be talking about renaming the files themselves. I'll need to think a bit more about this.

muldjord

@timmay Can you clear this up for me: Did you rename the files as well? I'm guessing you actually didn't.

Also, are you sure screenshots are gone from the gamelist.xml as well (please post an entry example)? Or is it just video? Then you might have forgotten to use the --videos option, otherwise they'll be ignored when regenerating (or in case of the RetroPie script, make sure videos are enabled). Remember that you can always set the videos=true in ~/.skyscraper/config.ini in the [main] section to always enable them.

If not, then there might be an issue here that I am not aware of and that I would like to look into.

timmay

Sorry, maybe I was a bit fuzzy - correct, I never renamed any rom files I just want to rename the game names shown in ES. To be precise in gamelist.xml it's the line

<name>GameName</name>

that has to be adjusted. I always used Skyscraper through Retropie-script and I'm sure videos are enabled since I never changed that option but I will double check on that and also post an entry example. I am not at home right now, will do that asap!

timmay

I did another run renaming the games for atari5200 and sega32x. The current settings in Skyscraper are:
gather source: Screenscraper
cache enabled for: screenshots, covers, marquees
rom names: source name
remove bracket info: enabled
use rom folders for gamelist+media: disabled
download videos: enabled

Somehow it worked for sega32x but did not for atari5200. The games of both systems have been renamed successfully but only for sega32x the paths for videos etc. have been kept in the gamelist.xml. Seems it works correctly for some systems but not all of them. So I have something wrong going on somewhere??
Here are the entry examples before and after renaming:

atari5200 before:

<game>
<path>/home/pi/RetroPie/roms/atari5200/Beamrider (USA).a52</path>
<name>Beamrider (USA)</name>
<cover />
<image>/home/pi/.emulationstation/downloaded_media/atari5200/screenshots/Beamrider (USA).png</image>
<marquee>/home/pi/.emulationstation/downloaded_media/atari5200/marquees/Beamrider (USA).png</marquee>
<video>/home/pi/.emulationstation/downloaded_media/atari5200/videos/Beamrider (USA).mp4</video>
<rating>0.8</rating>
<desc>Beamrider is a ...</desc>
<releasedate>19830101T000000</releasedate>
<developer>Activision</developer>
<publisher>Activision</publisher>
<genre>Action, Shoot'em Up</genre>
<players>4</players>
<lastplayed>20191030T190656</lastplayed>
<playcount>13</playcount>
</game>

atari5200 afterwards:

<game>
<path>/home/pi/RetroPie/roms/atari5200/Beamrider (USA).a52</path>
<name>Beamrider</name>
<cover />
<image />
<marquee />
<video />
<rating>0.8</rating>
<desc>Beamrider is a ...</desc>
<releasedate>19830101T000000T000000</releasedate>
<developer>Activision</developer>
<publisher>Activision</publisher>
<genre>Action, Shoot'em Up</genre>
<players>4</players>
<lastplayed>20191030T190656</lastplayed>
<playcount>13</playcount>
</game>

sega32x before:
(I post the first lines only, until '<video>' and leave out rating, description, publisher etc.)*

<game>
<path>/home/pi/RetroPie/roms/sega32x/After Burner Complete (Europe).zip</path>
<name>After Burner Complete (Europe)</name>
<cover />
<image>/home/pi/.emulationstation/downloaded_media/sega32x/screenshots/After Burner Complete (Europe).png</image>
<marquee>/home/pi/.emulationstation/downloaded_media/sega32x/marquees/After Burner Complete (Europe).png</marquee>
<video>/home/pi/.emulationstation/downloaded_media/sega32x/videos/After Burner Complete (Europe).mp4</video>
...
</game>

sega32x afterwards:

<game>
<path>/home/pi/RetroPie/roms/sega32x/After Burner Complete (Europe).zip</path>
<name>After Burner Complete</name>
<cover />
<image>/home/pi/.emulationstation/downloaded_media/sega32x/screenshots/After Burner Complete (Europe).png</image>
<marquee>/home/pi/.emulationstation/downloaded_media/sega32x/marquees/After Burner Complete (Europe).png</marquee>
<video>/home/pi/.emulationstation/downloaded_media/sega32x/videos/After Burner Complete (Europe).mp4</video>
...
</game>

Clyde

@muldjord said in Skyscraper: problem with re-naming rom names without bracket info:

EDIT: I might do this for Skyscraper 3.x though and introduce a "backup first and do it at your own risk" migration.

I would decidedly support this decision. :)

[end of offtopic, please carry on]

mitu

@timmay
The paths are the same before and after, but I don't understand why the video/image are skipped for the atari2600.

Can you try and re-generate the gamelist again, then post the last log file from /home/pi/RetroPie-Setup/logs to pastebin.com ? It should have the output from Skyscraper's operations.

timmay

Sure, here's the log after re-generating the atari5200 gamelist.
https://pastebin.com/adUsQ4Az

I also checked the previous log (renaming atari5200 and sega32x games) and noticed that with sega32x all paths for videos, screenshots, publisher etc. are logged and therefore included in the new gamelist.xml. However the log for atari5200 just lists a bunch of lines with

Resource is missing 'sha1' attribute, skipping...

I assume that is the issue, I'm just not sure what that means or how to correct it.

timmay

I also tried re-generating gamelists for fds, gb, gba, gbc - the log does not have the 'resource is missing' line as in the atari5200 log but it's the same result: all of those 4 are missing the paths for videos etc.
log is here:
https://pastebin.com/Kk2n4E2u

mitu

@timmay said in Skyscraper: problem with re-naming rom names without bracket info:

I assume that is the issue, I'm just not sure what that means or how to correct it.

Probably yes, try to re-scrape the atari5200 system, it looks like Skycraper cannot find anything in the cache.

timmay

Done. That seems to solve it. After re-scraping those systems re-generating the gamelists worked as it should.

I wonder why that issue occurred in the first place, because of course I checked the folders for videos, screenshots etc. and in both Skyscraper/cache and emulationstation/downloaded media all media files were present, same number of videos as roms in each systems rom folder, too. I never changed anything in there and used the same settings +source in Skyscraper when initially scraping, so that's odd?

Anyways, thanks a lot for helping out!
Gonna try it now for the remaining systems that need re-naming game names.

muldjord

@timmay said in Skyscraper: problem with re-naming rom names without bracket info:

Resource is missing 'sha1' attribute, skipping...

I assume that is the issue, I'm just not sure what that means or how to correct it.

That message only occurs while parsing the resource cache db.xml file for the system. And it indicates that the obsoleted sha1 attribute is missing. I think I know what is going on. Do you perhaps sometimes run Skyscraper from command-line and sometimes from the RetroPie script? And do you then maybe have two different versions? This would make sense, as the old version would look for the sha1 attribute. The newer version's give the message Resource is missing unique id, skipping...instead.

timmay

I had indeed run Skyscraper from Terminal once because from RP-script it wouldn't scrape the CPS1,2,3 systems. How is it possible to have two Skyscraper versions? If that's the case how to repair it- update Skyscraper? (Last update was about a week ago)

I now have successfully re-scraped and re-generated gamelists for a couple of systems like atari7800, gb, gba, mastersystem, megadrive, n64, nds, nes, psx, segacd, vectrex, zxspectrum.

Only Atari800 won't work, re-scraping skips all games with the line game not found and therefor the gamelist entries for video paths etc. are blank. Before all the media was downloaded and in the correct locations. In the beginning when setting up the systems scraping Atari800 caused no trouble at all. Since then I did not alter anything in the rom folder, so maybe something else got corrupted within this system.
Maybe completely set up atari800 again or if it makes sense is it possible to delete or replace the db.xml file?

timmay

@muldjord said in Skyscraper: problem with re-naming rom names without bracket info:

Do you perhaps sometimes run Skyscraper from command-line and sometimes from the RetroPie script? And do you then maybe have two different versions?

Maybe I misunderstood and by two versions you meant two different scrapes for a system instead two versions of Skyscraper?
In this case I'm pretty sure I only used Skyscraper from terminal for CPS systems as those were the only ones that originally didn't scrape from RP-script.