Skyscraper is back! (v3.17)

sleve_mcdichael

@Lolonois said in Skyscraper is back! (v3.9.2):

platforms_idmap.csv

Thanks. Apparently I had already known this at one point since I've already got a tg16 line added, but I guess I've forgotten it in the meantime.

Cheers, thanks!

windg

I remember in the past, when i was scrapping psx roms (bin/cue), to saw the metadata, i have to enter in the game folder in the ES menu. Did this behavior change in the recent releases ? Now i can see them without enter in the folder. It's a very welcome change!

Edit: I just saw the changelog.Thank you very much to add this feature!!

windg

I am reporting something that maybe is an issue of the current setup (v3.9.2).

With the multidisk psx games, if the .m3u text contents are :

Game_A (Disk1).cue
Game_A (Disk2).cue

Then, metadata, aren't visible without entering the folder. But they are visible if the .m3u file is :

Game_A (Disk1).CD1
Game_A (Disk2).CD2

Lolonois

@windg how is are directory/files structured?

I have tried with this layout inside ~/RetroPie/roms/psx:

$ find Alone\ in\ the\ Dark\:\ The\ New\ Nightmare*
Alone in the Dark: The New Nightmare/
Alone in the Dark: The New Nightmare/Game (Disk1).CD1
Alone in the Dark: The New Nightmare/Game (Disk1).CD2
Alone in the Dark: The New Nightmare.m3u

and this m3u content (also with cue extension for the disks):

$ cat Alone\ in\ the\ Dark\:\ The\ New\ Nightmare.m3u 
Alone in the Dark: The New Nightmare/Game (Disk1).CD1
Alone in the Dark: The New Nightmare/Game (Disk2).CD2

But in all four cases (Skyscraper 3.9.2 and current dev version and disc extension either cue or cd1/2 the metadata showed up instantly when in System psx (no need to navigate to subfolder).

I ran these commands:

Skyscraper -p psx --refresh -s screenscraper '/home/pi/RetroPie/roms/psx/Alone in the Dark: The New Nightmare.m3u'
Skyscraper -p psx

Could you verify on your end with this setup outlined above?

If it does not remediate the situation:
What is the setting of subdirs in your config.ini (in [main] as in [psx])?
which frontend are your using (see config.ini)?
what is the section "psx" showing in your peas.json?
I am running the emulationstation-dev version, what version do you use?
Can you share the part of your gamelist.xml for that section of <game> in question?

windg

@Lolonois

I am using Skyscraper v3.9.2 installed from binary.
Emulationstation frontend
Folder Structure :

Wild Arms 2 (USA)/
Wild Arms 2 (USA) (Disc 1).bin
Wild Arms 2 (USA) (Disc 1).cue
Wild Arms 2 (USA) (Disc 2).bin
Wild Arms 2 (USA) (Disc 2).cue
Wild Arms 2 (USA).m3u

Wild Arms 2 (USA).m3u :

Wild Arms 2 (USA) (Disc 1).cue
Wild Arms 2 (USA) (Disc 2).cue

config.ini file
peas.json :

},
    "psx": {
        "aliases": [
            "playstation",
            "sony playstation"
        ],
        "formats": [
            "*.cbn",
            "*.chd",
            "*.cue",
            "*.img",
            "*.iso",
            "*.m3u",
            "*.mdf",
            "*.pbp",
            "*.toc",
            "*.z",
            "*.znx"
        ],
        "scrapers": [
            "screenscraper",
            "thegamesdb"
        ]
    },

artwork.xml :

<?xml version="1.0" encoding="UTF-8"?>
<artwork>
  <output type="screenshot" resource="cover"/>
  <output type="marquee" resource="wheel"/>
</artwork>

gamelist.xml

Indeed, if i use .CDx extenstion instead of .cue, it will show the metadata without move into subfolder. But i haven't test multidisk games for other emulators that don't support the .CDx or if the user is using .chd files. Thanks to look into this !

Lolonois

@windg Thanks for the info. To narrow down the context, a few more questions: Which ES version are you running? What is the setting of Parse Gamelists Only in the ES UI Settings menu or in your ~/.emulationstation/es_settings.cfg ?

If I understand correctly, you have highlighted the folder in ES which is shown in your game list of the system PSX and then there is no metadata shown.

However, when you navigate into the folder you get metadata presented
a) regardless of the extension of the files in that folder? -or-
b) only when extension is CD1 / CD2?

Independently of this: Do you get metadata presented when you highlight the entry in ES which is the m3u file on your storage/filesystem?

windg

The Emulationstation version is v2.11.2RP, RetroPie Buster v4.8.6 .

Gamelist view is the default 'Automatic' . I am using the default theme .

"ParseGamelistOnly" value="false"

es_settings.cfg file

If the .m3u have .cue extentions, then :

No Metadata shown when i highlighted the folder in ES.
Metadata shown when navigate into the folder, for the .m3u, for disk1 (bin/cue) and for the disk2 (bin/cue), 3 times .

If the .m3u have .CDx extentions, then :

-Metadata shown when i highlighted the folder in ES.
-No metadata shown when i navigate into the folder. (only one entry is visible here(.m3u), not three like before)

gamelist.xml file with the both games that i test. The first is the Countdown Vampires, a two disks game - m3u file with .CDx extentions. And the other is Wild Arms 2, a two disk game - m3u file with .cue extentions.

To be more clear, this is happening only with the multi-disk games. Single disk shown correctly the metadata when i highlighted the folder in ES. Thanks again !!

(As a workaround, i am hiding the game folder and i have put the .m3u file in the psx rom folder .)

Lolonois

@windg Ah, now I can reproduce it.

The code line where the usage of "folder metadata" is decided is this one. I have inherited the logic from Lars, it is around since quite a while. Even when Lars was the maintainer.

While it is comprehensible that you put the m3u file into the game folder it is also unusual.

It is coincidence that Countdown Vampires shows the data at folder level.
The fact that inside the folder only one file is shown is most likely due to that the extension CD1/2 is not added in es_systems.cfg.
For the same reason (read: CD1/CD2 not in peas.json) the metadata is shown on folder level. The number of known files counted by Skyscraper is 1 in that folder (the file with the m3u extension).

The Wild Arms 2 does not show on folder level as the logic in Skyscraper only sets "folder metadata" when exactly one file with the extensions of peas.json is found in this folder. As cue and m3u are valid extensions it sums up to 3 files, thus no "folder metadata".

I am not sold if I should add another exception (see his comment more above at the code line) for m3u files and having the m3u file inside the game folder.
If there are more users demanding it I might consider it.

Additionally to your workaround I would also set subdirs=false in the config.ini (for [psx]). And set "Parse Gamelists Only" to true.

windg

I will use your suggetion for the multi-disk games. Thank you very much for your time and the detail expanation!

Lolonois

@windg Nvm. Thanks for tabling and challenging it.

One more thing (without the workaround): You still enter the folder of Countdown Vampires for example and then launch the m3u file or do you somehow manage to launch when you focus the folder Countdown Vampires?

windg

@Lolonois Yes, i have to enter into the folder, hit the m3u file to launch the game. I can avoid this if i throw the content of the game folder into the psx folder.

windg

New ppsspp v1.17.0 it's adding support for .chd files.
The current peas.json is missing this format. I add it, tested it and it's working.

},
    "psp": {
        "aliases": [
            "playstation portable",
            "sony playstation portable"
        ],
        "formats": [
            "*.cso",
            "*.iso",
            "*.pbp",
            "*.chd"
        ],
        "scrapers": [
            "screenscraper"
        ]
    }

Lolonois

@windg Thanks. Will be in the upcoming 3.10.0 release.

Lolonois

Version 3.10.0 is out.

One of the changes which may be beneficial for you is the added bash completion:

For example: Start typing Skyscraper - then Tab twice will give you all options, if you continue with Skyscraper -- then Tab twice you get only the long options. From there if you continue to type Skyscraper --c then Tab twice, you get Skyscraper --cache completed, if you again press Tab twice you get all cache options listed.

Same goes for the main options -p and -s. Type Skyscraper -p a Tab twice will give you for example ags amiga apple2 arcade atari2600 atari800 as completion options. Once only one option is left, it is fully expanded and you can continue with more options as outlined above.

And one important note: If you changed your local peas.json you may check the addition to peas.json.rp-dist in ~/.skyscraper/. In essence this this line has changed. You may want to add that to your peas.json. (see also previous posts on this change).

As always, all additions of this release are summarized here.

Clyde

Thanks from a long-term Linux user to whom Tab completion is as natural as breathing. 😉

sleve_mcdichael

@Lolonois said in Skyscraper is back! (v3.10.0):

Version 3.10.0 is out.

One of the changes which may be beneficial for you is the added bash completion:

This isn't working for me. Updated Ss (binary) via RP-Setup. I am running v3.10.0 as reported by --version. I have the ~/.bash_completion.d/Skyscraper.bash file. I even rebooted the RPi4.

Skyscraper --c[TAB][TAB]
Skyscraper -p sn[TAB][TAB]
/opt/retropie/supplementary/skyscraper/Skyscraper -p snes -s scr[TAB][TAB]
# etc.

...does nothing.

My RP-Setup is current, but underlying Raspbian Buster is not (251 packages can be upgraded. "If it ain't broke, don't fix it.") Should that make a difference?

Lolonois

@sleve_mcdichael this one is on me: I forgot to add a symlink command in the scriptmodule.

sudo ln -sf "/opt/retropie/supplementary/skyscraper/Skyscraper" "/usr/local/bin/Skyscraper" should remediate the situation.

I will file a PR soon for the scriptmodule.

sleve_mcdichael

@Lolonois I have that symlink too:

$ ls -l /usr/local/bin/Skyscraper 
lrwxrwxrwx 1 root root 49 Sep 27 21:41 /usr/local/bin/Skyscraper -> /opt/retropie/supplementary/skyscraper/Skyscraper

(Edit: to clarify, the Skyscraper command works fine, and has been, it's the tab-completion that doesn't seem to do anything.)

mitu

@sleve_mcdichael It works for me, so could be a local config issue. Can you post your skyscraper.ini, removing any personal/password fields ?

sleve_mcdichael

@mitu

[main]

;; define paths
; inputFolder="/home/pi/RetroPie/roms"
gameListFolder="/home/pi/.emulationstation/gamelists"
mediaFolder="/home/pi/.emulationstation/downloaded_media"
; cacheFolder="/home/pi/.skyscraper/cache"
; importFolder="/home/pi/.skyscraper/import"

;; cache settings
; cacheCovers="true"
; cacheScreenshots="true"
; cacheWheels="true"
cacheMarquees="false"
cacheTextures="false"

;; video settings
videos="true"
videoSizeLimit="42"
videoConvertCommand="videoconvert.sh %i %o"
videoConvertExtension="mp4"
symlink="true"

;; artwork
; artworkXml="artwork-wide.xml"
artworkXml="artwork-cover.xml"

;; generic
relativePaths="true"
brackets="false"
subdirs="false"
theInFront="true"
unattendSkip="true"
maxLength="10000"

[arcade]
; artworkXml="artwork-tall.xml"

[gb]
; artworkXml="artwork-tall.xml"
;; allow gbc titles in gb folder
addExtensions="*.gbc"

[gba]
; artworkXml="artwork-wide.xml"

[gbc]
; artworkXml="artwork-tall.xml"

[genesis]
;; allow sega cd titles in genesis folder
addExtensions="*.iso *.cue *.chd"

[pcengine]
;; use 'tg16' folders for pce titles
;; (deprecated - use custom system in peas.json/platforms_idmap.csv instead)
;inputFolder="/home/pi/RetroPie/roms/tg16"
;gameListFolder="/home/pi/.emulationstation/gamelists/tg16"
;mediaFolder="/home/pi/.emulationstation/downloaded_media/tg16"
;cacheFolder="/home/pi/.skyscraper/cache/tg16"
;importFolder="/home/pi/.skyscraper/import/tg16"

[screenscraper]
;; userCreds="user:pass"
userCreds="[user]:[pass]"

[esgamelist]
;; import customized ES textual data
cacheRefresh="true"
unattend="true"
videos="false"
cacheCovers="false"
cacheScreenshots="false"
cacheWheels="false"
cacheMarquees="false"

[import]
cacheRefresh="true"