Skyscraper now officially part of RetroPie, please test

mitu

@muldjord I've made some modifications to the scriptmodule, incorporating some of the feedback from the last discussion.

I removed the 'Gather for all platforms' entry from the menu, which should be now simpler. The gather action should now be similar to the gamelist generation.

I added a warning when starting scraping

Added the vacuum option under the Cache options and commands and switched the purge command from a straight delete to the corresponding --cache purge:all command.
Other small changes in the verbiage and included files.

I tried to test with having a single Select platforms option (for both gathering and generation), but it turned a bit complicated to pass on the parameters and frankly a bit cumbersone - you have to enter a menu, select something, go back, select another entry, then confirm you want to scrape, etc.

muldjord

@mitu This looks good, great work. It's clearer without the "Gather for all" option and the prompt clears up any confusion about how to stop the process, so thumbs up for that aswell.
Using the now included purge options is definitely a great addition too.

mitu

@muldjord Eh, it's mostly incremental, it's not that much.

I had one idea for Skyscraper - the ES built in scraper has a way to modify the search string when querying the scraping source (the Input button in the image below)

One option when modifying the search string is to use the syntax ID:number and the scraper would perform a search-by-id on the site.

I thought you could add this to the --query syntax:

--query id=<gameID>

and query by the ID. Of course, it will depend on the scraper source, but I think it's easier to enter than - for instance - the MD5/CRC when searching on ScreenScraper.

muldjord

@mitu This is already possible for the screenscraper source I believe - but it's probably called gameid or so. For the file name search based sources it is not though.

EDIT: No, that doesn't seem to work. I wonder why.

mitu

@muldjord I thought you might have implemented this and checked the docs (clihelp.md), but I didn't see it mentioned (and it doesn't appear in the --help output also).
I reported a bug in the V2 WebApi last week (the same forum thread on the ScreenScraper site) where searching by gameid didn't work just with the ID, so it might not have worked before - should work now, I tested it in the ES scraper. I think TGDB should also work with the ID - I know it worked before and I've seen it in the new added to the new ES TGDB scraper version (see here).

muldjord

@mitu, are you aware of this? https://github.com/muldjord/skyscraper/issues/134

I've tested the purge command and it works just fine. It shouldn't even be looking for artwork.xml if the command line is correct for it. I don't use the script myself so I can't test.

mitu

@muldjord No, but I'll take a look.

EDIT: I think I know why it fails - I'll submit a fix tomorrow. The old purge used simple rm commands, the new method uses the Skyscraper's command, but I forgot to make it run as the normal user. Since the RetroPie setup runs as root, the error is probably because that user doesn't have Skyscraper set-up properly.

muldjord

@mitu said in Skyscraper now officially part of RetroPie, please test:

EDIT: I think I know why it fails - I'll submit a fix tomorrow. The old purge used simple rm commands, the new method uses the Skyscraper's command, but I forgot to make it run as the normal user. Since the RetroPie setup runs as root, the error is probably because that user doesn't have Skyscraper set-up properly.

Sounds plausible. Thanks for looking into it.

2fst4u

@mitu hi there, submitter of that issue here. Thanks for looking into it and good to hear that it shouldn't be too difficult (relatively) to sort out.

In the meantime, is there anything manual that I can try to make it work or would it be best to just wait for a fix?

mitu

@2fst4u Hello and thank you for the bug report.
I've submitted a fix for the problem you reported (https://github.com/RetroPie/RetroPie-Setup/pull/2658) - I think you should wait to be added to the RetroPie-Setup script and then just update to get the fix.

2fst4u

@mitu ok thank you. How can I check that it's ready to be updated? Will you let us know?

mitu

@2fst4u Sure, I'll add a comment when the fix is added.

ZestyTS

Just found this thread, I believe I'm having trouble with the scriptmodule.

@mitu

Using Skyscraper (3.1.3) on RetroPie (4.4.9) I'm unable to select some of the options for Skyscraper from the RetroPie setup, either through command line or ssh. Example, upon loading up the GUI, clicking gather resources, I'm unable to select any of the platforms for resource gathering. I've tried reinstalling SkyScraper and RetroPie, but that hasn't fixed the issue.

Like if we look at the picture you uploaded earlier, I'm unable to select "Gather and cache resources" it just refreshes the screen as opposed to bringing me to a new screen about starting the scraping process.

mitu

@ZestyTS said in Skyscraper now officially part of RetroPie, please test:

Like if we look at the picture you uploaded earlier, I'm unable to select "Gather and cache resources" it just refreshes the screen as opposed to bringing me to a new screen about starting the scraping process

That's not an action, it's just a heading for the actions beneath it. It's supposed to act like that. Use the Gather Resources - the next menu beneath it.

ZestyTS

@mitu

I see that explains why the page refreshes. So I tried "Gather resources" before but I'm unable to check/select any of the platforms, I just tried it again and I'm still having the issue. I guess, the easy question is, "what are the key combinations required to select an option?"

mitu

@ZestyTS The Space key acts as a toggle/checkbox, that's how you select.

iainjh

Hi! when will that purge issue be addressed please? is there a manual workaround? My image is full up and I have no way to delete.

Its a fantastic scraper btw! - sorry for hassle

ZestyTS

@mitu Thanks for getting back to me, that works over ssh, but not with my keyboard plugged into the raspberry pi, so I'll have to figure out what that's configured as, but thanks!

mitu

@iainjh Use the command line parameters to purge/vacuum - https://github.com/muldjord/skyscraper/blob/master/docs/CLIHELP.md.

mitu

@2fst4u @iainjh The fix has been added to the module in the project's repository - update your RetroPie-Setup script and then try again the purge/vacuum functions to see if you still get the error.