Skyscraper now officially part of RetroPie, please test

muldjord

User at reddit reported having issues if his rom folder inside /home/pi/RetroPie/roms is symlinked to some other place such as /media/usb0/roms. The script reports "no populated rom folders were found in /home/pi/RetroPi/roms" and quits. Might be permission related, he's currently investigating that. Tested this with Skyscraper on command line and it works.

Links to thread: https://www.reddit.com/r/RetroPie/comments/ahs14m/skyscraper_with_symlinked_rom_folder_not_working/

mitu

@muldjord I've seen the thread, I'll check what's going on.
The new module is also ready, forgot to post the new UI screenshot. I've been re-testing the installation and the way it works with both versions. I have one more check to do and I'll probably submit the new version tonight.

muldjord

@mitu Looks really good! Really looking forward to getting this out there so I can start focusing on new features again.

Silent

So is there any reason for Skyscraper to still be in "experimental"?

muldjord

@Silent For the time being I think so yes. The module needs to be tested thoroughly for a while before I think it should be moved to Optional. And docs needs to be updated and so on (I have a lot of stuff I need to document). So while it's ready for use, there's some peripheral stuff that needs to solidify before it's ready for Optional. That's my opinion anyways.

Silent

Also, for the sake of "noob friendliness" I think the nice end goal would be to see Skyscraper as a backend for ES scraping. Keep in mind however that I am saying this without checking how that ES feature works at all, so perhaps it's never going to be possible.

mitu

@muldjord PR submitted - https://github.com/RetroPie/RetroPie-Setup/pull/2584. Since it should work with both versions, can be merged before you release 3.x.
Fixed both bugs ~~fixed~~ reported so far (the one with the ROM folder containing only sub-folders and the one with the symlink-ed ROMs folder).

muldjord

@mitu Great! I am working hard on the documentation at this very moment. I will try to squeeze in some time to test the script later tonight. I'll ping you if I run into anything that needs fixing.

mitu

@muldjord Take your time, documentation is always hard to write (for your own applications :) ).

muldjord

@mitu Just testing the script from the PR. For some reason the 3 Gather source: ... Option doesn't show up on my menu. Any idea why that is?

mitu

@muldjord I think I renamed it to 3 Gather source - <source> -->.

muldjord

@mitu Yeah, but it doesn't show up on my menu. I just have Option 1, 2, 4, 5 and so on. No 3.

mitu

@muldjord What's the md5 sum of the script ?

 $ md5sum `pwd`/scriptmodules/supplementary/skyscraper.sh 
7070a79beb0629a70478d36ff56e8742  /home/pi/RetroPie-Setup/scriptmodules/supplementary/skyscraper.sh

muldjord

@mitu

md5sum `pwd`/scriptmodules/supplementary/skyscraper.sh
84e97f7fd7fc6d4ab6701203d83e8482  /home/pi/RetroPie-Setup/scriptmodules/supplementary/skyscraper.sh

I got it by wgetting the raw file from the PR. Do I need to fetch it from somewhere else?

mitu

No, it should be ok. What do you have in the /opt/retropie/configs/all/skyscraper.cfg file ?

muldjord

@mitu

rom_name = "0"
use_rom_folder = "1"
download_videos = "1"
cache_marquees = "0"
cache_covers = "0"
cache_wheels = "0"
cache_screenshots = "1"
scrape_source = "localdb"
remove_brackets = "0"
force_refresh = "1"

It has "localdb" from an old installation. Might that be the reason?

EDIT: Yep, that's it. Changed it to "thegamesdb" and the menu option shows up. This should be handled somehow. Check if it has "localdb" and just set it to "screenscraper" or so if it is.

mitu

@muldjord Yes, because now localdb is no longer an option - I'll fix it. You can just nuke the .cfg file and start with the defaults (just need to handle this for upgrades with the fix).

muldjord

@mitu Tested the rest and it works really well. I think the workflow is a lot better now. In my opinion it is ready to be merged.

EDIT:

Minor. The ArcadeDB source is noted as just "Arcade" right now. It should be called "ArcadeDB".
IGDB should be removed. It has a 5 max rom limit in the vein of mobygames, so it's not really usable from the menu. The reasoning behind the limit is that it know has my user KEY added and it has a global limit of 10000 requests per month for all Skyscraper users.

mitu

@muldjord Should be fixed now - scraping source/removed IGDB/fixed spelling.

muldjord

Just a sec, found another error. Gonna describe it in a minute.

EDIT: Ah, nevermind. It's an error that I have already fixed in 3.0.0. It's not related to the script. Basically it's related to the --relative option when users have their roms in subfolders it would add the wrong rom location to the gamelist. But this is fixed in 3.0.0.

I have nothing further to add. Works really well! I'm pretty excited about this. :)