Skyscraper now officially part of RetroPie, please test

muldjord

@CapNinja said in Skyscraper now officially part of RetroPie, please test:

After scrapping from three sources, I just wanted to sort genres and to my surprise there was a ton of new genres. Is there any way to simplify?

Each source returns a bunch of genres related to the game. I have no control of each individual genre for each source. It comes as a package so to speak. You can prioritize which source you want to use the "genre package" from though. Read more here. I general, please check out the documents in this folder. It will explain pretty much all you need to know. Be aware that some of it is "under construction" as I am currently moving towards the 3.0.0 release.

1.a. I don't know if the miracle of tags multiplication was related to multiple scrape sources. But, how can you decide to description differences after 3 different scrapes (if this plural exists)? It's something like "first come, first served"?

See answer above.

Games with multiple covers (like the genesis - mega drive situation), there is a way to show both? Like, every 5-10 seconds change photos or something... I don't know if I made my self clear here

Not possible and not really part of "scraping". :) This would have to be handled by the frontend instead, which it doesn't at the moment as far as I know.

Suggestion. Select multiple sources to scrape in one go. So then I can sleep the sleep of the righteous and wake with all done.

You can always script it to do so. Skyscraper won't be supporting this as it's just as easy to simply create a script. You can use the "simple mode" or the RetroPie-Setup script for easier access if you don't like working on command line. :)

On the (well made) section of picture-cover-wheel structure, instead of change text, multiple examples to choose.

Check the artwork docs as answered above.

"Est. time left" is only for one system, right? Can we have another estimated time but for all systems selected?

Can't be done. Skyscraper only knows about the current running Skyscraper, not what happens after that. :)

Have fun with it!

muldjord

@mitu Just a heads up. Skyscraper 3.0.0 is pretty much ready for release. I've been testing it over the past week or so and I think it's ready for prime time.

muldjord

User at reddit reported having issues if his rom folder inside /home/pi/RetroPie/roms is symlinked to some other place such as /media/usb0/roms. The script reports "no populated rom folders were found in /home/pi/RetroPi/roms" and quits. Might be permission related, he's currently investigating that. Tested this with Skyscraper on command line and it works.

Links to thread: https://www.reddit.com/r/RetroPie/comments/ahs14m/skyscraper_with_symlinked_rom_folder_not_working/

mitu

@muldjord I've seen the thread, I'll check what's going on.
The new module is also ready, forgot to post the new UI screenshot. I've been re-testing the installation and the way it works with both versions. I have one more check to do and I'll probably submit the new version tonight.

muldjord

@mitu Looks really good! Really looking forward to getting this out there so I can start focusing on new features again.

Silent

So is there any reason for Skyscraper to still be in "experimental"?

muldjord

@Silent For the time being I think so yes. The module needs to be tested thoroughly for a while before I think it should be moved to Optional. And docs needs to be updated and so on (I have a lot of stuff I need to document). So while it's ready for use, there's some peripheral stuff that needs to solidify before it's ready for Optional. That's my opinion anyways.

Silent

Also, for the sake of "noob friendliness" I think the nice end goal would be to see Skyscraper as a backend for ES scraping. Keep in mind however that I am saying this without checking how that ES feature works at all, so perhaps it's never going to be possible.

mitu

@muldjord PR submitted - https://github.com/RetroPie/RetroPie-Setup/pull/2584. Since it should work with both versions, can be merged before you release 3.x.
Fixed both bugs ~~fixed~~ reported so far (the one with the ROM folder containing only sub-folders and the one with the symlink-ed ROMs folder).

muldjord

@mitu Great! I am working hard on the documentation at this very moment. I will try to squeeze in some time to test the script later tonight. I'll ping you if I run into anything that needs fixing.

mitu

@muldjord Take your time, documentation is always hard to write (for your own applications :) ).

muldjord

@mitu Just testing the script from the PR. For some reason the 3 Gather source: ... Option doesn't show up on my menu. Any idea why that is?

mitu

@muldjord I think I renamed it to 3 Gather source - <source> -->.

muldjord

@mitu Yeah, but it doesn't show up on my menu. I just have Option 1, 2, 4, 5 and so on. No 3.

mitu

@muldjord What's the md5 sum of the script ?

 $ md5sum `pwd`/scriptmodules/supplementary/skyscraper.sh 
7070a79beb0629a70478d36ff56e8742  /home/pi/RetroPie-Setup/scriptmodules/supplementary/skyscraper.sh

muldjord

@mitu

md5sum `pwd`/scriptmodules/supplementary/skyscraper.sh
84e97f7fd7fc6d4ab6701203d83e8482  /home/pi/RetroPie-Setup/scriptmodules/supplementary/skyscraper.sh

I got it by wgetting the raw file from the PR. Do I need to fetch it from somewhere else?

mitu

No, it should be ok. What do you have in the /opt/retropie/configs/all/skyscraper.cfg file ?

muldjord

@mitu

rom_name = "0"
use_rom_folder = "1"
download_videos = "1"
cache_marquees = "0"
cache_covers = "0"
cache_wheels = "0"
cache_screenshots = "1"
scrape_source = "localdb"
remove_brackets = "0"
force_refresh = "1"

It has "localdb" from an old installation. Might that be the reason?

EDIT: Yep, that's it. Changed it to "thegamesdb" and the menu option shows up. This should be handled somehow. Check if it has "localdb" and just set it to "screenscraper" or so if it is.

mitu

@muldjord Yes, because now localdb is no longer an option - I'll fix it. You can just nuke the .cfg file and start with the defaults (just need to handle this for upgrades with the fix).

muldjord

@mitu Tested the rest and it works really well. I think the workflow is a lot better now. In my opinion it is ready to be merged.

EDIT:

Minor. The ArcadeDB source is noted as just "Arcade" right now. It should be called "ArcadeDB".
IGDB should be removed. It has a 5 max rom limit in the vein of mobygames, so it's not really usable from the menu. The reasoning behind the limit is that it know has my user KEY added and it has a global limit of 10000 requests per month for all Skyscraper users.