Skyscraper is back! (v3.17)

mitu

@sleve_mcdichael then it's bug in the installation instructions - I think it's not working also on my system OOB.

@Lolonois are you sure that using $HOME/.bash_completion.d is the right place to copy user-defined Bash completion files ? It doesn't seem to sourced automatically by bash_completion.

Lolonois

Do you have a ~/.bash_completion @sleve_mcdichael ?

If not create one to pick the Skyscraper bash completion up:

cat <<'EOF' >~/.bash_completion
for bcfile in ~/.bash_completion.d/* ; do
    [ -f "$bcfile" ] && source "$bcfile"
done
EOF

Another bash -i should give you the completion and also this should survive reboots.

Apperently I forgot that I did this ~/.bash_completion creation for another bash completion already back in December 2021. :-/

However, I am still puzzled why the symlink noted above is already on your system @sleve_mcdichael as it is not created with the current skyscraper.sh scriptmodule, but is needed also for bash completion.

mitu

@Lolonois said in Skyscraper is back! (v3.10.0):

Apperently I forgot that I did this ~/.bash_completion creation for another bash completion already back in December 2021. :-/

Then maybe is better to install the completion file system-wide (/etc/bash_completion.d/ ?) if there's no standard user defined location for completion files.

Lolonois

@mitu ok for me to put it in the global /etc/bash_completion.d/ when you maintainers are. This reduces the logic in the scriptmodule where to put the completion file and do additional actions, depending on bash completion version and XDG compliant setups.

There is a lot of context here, FWIW.

mitu

@Lolonois said in Skyscraper is back! (v3.10.0):

There is a lot of context here, FWIW.

Oh, no, not the XDG_ police !

Lolonois

@mitu No worries. I will happily put it below /etc... to keep the XDG police / policy out. :-D

sleve_mcdichael

@Lolonois said in Skyscraper is back! (v3.10.0):

However, I am still puzzled why the symlink noted above is already on your system @sleve_mcdichael as it is not created with the current skyscraper.sh scriptmodule,

I have manually added this symlink a long time ago, as suggested in the CLIHELP.md document.

If you've installed Skyscraper through the RetroPie-Setup script, it is recommended to create a symbolic link to the executable. Do this by running sudo ln -s /opt/retropie/supplementary/skyscraper/Skyscraper /usr/local/bin/Skyscraper. This will allow you to just type Skyscraper when running it from command line.

sleve_mcdichael

@mitu @Lolonois seems to work if placed here, with no other modifications on my part:

~/.local/share/bash-completion/completions/Skyscraper.bash

Edit: https://github.com/RetroPie/RetroPie-Setup/pull/3872 is working for me (but am I understanding this might be system-dependent for different people?)

mitu

@sleve_mcdichael let's do the system install option.

YFZdude

@Lolonois
For the first time I used your updated version to scrape a system.
The resulting gamelist file started using absolute paths and sticking a bunch of <folder> stuff at the bottom of the file. This broke the artwork in that system (and any others that were re-scraped). It also caused there to be warnings left in the terminal window about not updating FileData for unknown items when exiting ES.

Am I just missing something simple like enabling the relative paths option in the config file? It never used to do this in the past. Old gamelists generated using 3.7 and lower had either relative paths, or paths that started with /home/<user>/RetroPie.

Also I do have a symlink in place of the standard RetroPie folder in the home folder so I can point it at a secondary hard disk. Not sure if that would affect this.

mitu

@YFZdude said in Skyscraper is back! (v3.10.0):

Am I just missing something simple like enabling the relative paths option in the config file? It never used to do this in the past. Old gamelists generated using 3.7 and lower had either relative paths, or paths that started with /home/<user>/RetroPie.

Hm, that's not good.
@Lolonois is this intended ? I don't see anything in the changelog about this change. This will break some setups where gamelists are saved in the ROMs folder and they rely on a relative path to work - this makes them non-portable.

The resulting gamelist file started using absolute paths and sticking a bunch of <folder> stuff at the bottom of the file ...

That's not good, stable ES doesn't know about this. @Lolonois is there a way to disable the generation of folder items by default and hide it behind a parameter/option ?

YFZdude

@mitu
For reference, I only symlinked the RetroPie folder in the home directory to a different drive. Anything in the /opt/retropie folder should be standard path.

If it helps I can upload some files/screenshots to show the things I see when it breaks.

mitu

@YFZdude said in Skyscraper is back! (v3.10.0):

For reference, I only symlinked the RetroPie folder in the home directory to a different drive. Anything in the /opt/retropie folder should be standard path.

That's not an issue. If it worked so far, it should continue to work.

If it helps I can upload some files/screenshots to show the things I see when it breaks.

I'm not sure if they're necessary, but let's wait for @Lolonois - maybe they'd like to see the logs.

Lolonois

I had a fix for the regression ready, just pushed out 3.10.2.

I fixed a issue when a symlink is used for single romfiles provided to Skyscraper, but apperently introduced this regression.

Anyhow, now both use cases should be fine. Esp. the one reported here and in #38 on Github.

I gave your setup @YFZdude a testdrive with 3.10.2 and it looked good:

$ pwd
/home/pi
$ ls -l RetroPie
lrwxrwxrwx 1 pi pi 27 Feb 27 19:16 RetroPie -> /mnt/yabba/dabba/_RetroPie/
# below RetroPie/ it is roms/, BIOS/ aso. as directories.

Gamelists get created as expected (depending on the relativePaths setting with or without the RetropIe platform rom path (/home/pi/RetroPie/roms/<platform>/)).

Pls let us know if it remediates the situation on your side.

mitu

Gamelists get created as expected (depending on the relativePaths setting with or without the RetropIe platform rom path (/home/pi/RetroPie/roms/<platform>/)).

The default seems to be still to output the full path, instead of a relative path as before. Why the change ? Also, folder entries are still generated.

YFZdude

@Lolonois This seems to straighten out my issue.
I moved the existing gamelist to generate a new one and got the expected paths containing /home/<user>/RetroPie/roms/ etc.

I did not see any <folder> entries at the bottom of my newly generated list either. I was also not using the relative paths setting in any config file.

Thanks

Lolonois

@mitu The folder generation is deliberately added as it is part of the gamelist spec [1] if one organizes roms in directories per platform.

Can you provide the parts and sections of the config.ini with relativePaths settings? Do you use videos=true and have symlink also set true?

And most importantly which symlinks are present up to /home/pi/RetroPie/roms/<platform>?

[1] https://github.com/RetroPie/EmulationStation/blob/master/GAMELISTS.md#folder

Lolonois

@YFZdude Thanks for reporting back. This means your case is ok?

YFZdude

@Lolonois
Yes I have it working fine on my setup.

mitu

@Lolonois said in Skyscraper is back! (v3.10.0):

@mitu The folder generation is deliberately added as it is part of the gamelist spec [1] if one organizes roms in directories per platform.

Yes, but that part is not present in most RetroPie installations - it's been only added to the -dev branch. Until this feature is present by default in RetroPie, I'm asking you to hide this under an option and default to not generate this info.

Can you provide the parts and sections of the config.ini with relativePaths settings? Do you use videos=true and have symlink also set true?

I have the default .ini, without anything else set. I didn't scrape for video. The default ini has:

[esgamelist]
cacheRefresh="true"
cacheScreenshots="false"

[import]
cacheRefresh="true"

enabled only.

And most importantly which symlinks are present up to /home/pi/RetroPie/roms/<platform>?

I don't have any symlinks that way, I only use $HOME/roms symlinked to $HOME/RetroPie/roms, but I don't see how that should affect scraping since the default RetroPie structure for ROM folders is intact.