Skyscraper now officially part of RetroPie, please test

mitu · 30 Mar 2019, 08:13

@muldjord So this is probably the reason I didn't implement a cache location detection - the ability to have multiple cache folders. I might be able to add a few modifications though to warn the user or to account for the situations where the global setting is present.

2fst4u · 30 Mar 2019, 08:17

@muldjord thanks for the heads-up, guys. I never knew this, I'll keep it in mind.

muldjord · 30 Mar 2019, 08:34

@mitu said in Skyscraper now officially part of RetroPie, please test:

@muldjord So this is probably the reason I didn't implement a cache location detection - the ability to have multiple cache folders. I might be able to add a few modifications though to warn the user or to account for the situations where the global setting is present.

The global cacheFolder setting is widely used, mostly by CLI users I would guess though. I don't think the platform setting is used all that often, if at all (it doesn't make much sense to use if you ask me).

2fst4u · 31 Mar 2019, 02:19

Another quick question that I can't find the answer to anywhere:

How do I get rid of data that has been scraped? Is purging the cache sufficient? I'm wanting to delete my downloaded video clips completely for a particular platform and then re-do my gamelist. After I purge, does regenerating the gamelist get rid of the videos in my ROMs folder? I've done a purge but the videos are still in my ROMs folder (where I've set the data to be located), which makes sense in a way, I know that data isn't the cache. I'm just curious if the regeneration of the gamelist will delete the old data when it finds it isn't in the cache anymore.

mitu · 31 Mar 2019, 04:31

@2fst4u Skyscraper will not remove the generated artwork (video/images). You can delete it yourself or just re-scrape and the files would be overwritten with the new data from cache.

2fst4u · 31 Mar 2019, 04:33

@mitu oh cool so it isn't as touchy about that stuff as it is with the cache data?

mitu · 31 Mar 2019, 04:44

@2fst4u said in Skyscraper now officially part of RetroPie, please test:

@mitu oh cool so it isn't as touchy about that stuff as it is with the cache data?

Why would it delete the generated data ? That's up to the user to do it. You don't see a browser deleting the downloaded files from the disk when you instruct it to clear the cache.

2fst4u · 31 Mar 2019, 04:50

@mitu true. I'm a big believer in "no stupid questions", hopefully someone else finds my questions and the answers you guys give helpful! Thanks.

Riverstorm · 10 Apr 2019, 21:05

@mitu - I couldn't quite get it to crash like it was the other day but ran out of time. I think I did an update between those times somewhere too though I did get an error in the middle of scraping.

Basically I have 271 games in the Arcade folder. Nothing excessive. I booted fresh and went to the scraper (no games ran or any other activity). I scraped them with thegamesdb with Arcade selected and no other systems. After scraping I viewed each one with the arrow down key.

At the end of the game selection list I pulled up the scraper again and changed the source to the other one (I can't remember the name--but there's only one other). I started a new scrape and around 90 or so I got this error message Error saving resized image. Out of memory? Disk full? I did click Retry and it continued but I had to go before it finished.

I have a 32GB card with about 17GB used and 12GB free. On the Pi 3 b I believe the default is 256MB. It's at the defaults.

I am running a Pi 3 b, RetroPie 4.4.11, RA 1.7.6, ES 2.8.2RP. The ES log had a lot of lvl1 messages similar to below but nothing really useful it seemed. I did see several memory parsing errors when shutting down where you briefly see the command line screen.

lvl1: 	System "vectrex" has no games! Ignoring it.
lvl1: 	  Warning from theme "/etc/emulationstation/themes/carbon/zxspectrum/theme.xml"
  (from included file "/etc/emulationstation/themes/carbon/zxspectrum/../carbon.xml")
    could not find file "" 
lvl1: 	System "zxspectrum" has no games! Ignoring it.
lvl1: 	  Warning from theme "/etc/emulationstation/themes/carbon/auto-allgames/theme.xml"
  (from included file "/etc/emulationstation/themes/carbon/auto-allgames/../carbon.xml")
    could not find file "" 
lvl1: 	  Warning from theme "/etc/emulationstation/themes/carbon/auto-favorites/theme.xml"
  (from included file "/etc/emulationstation/themes/carbon/auto-favorites/../carbon.xml")
    could not find file "" 
lvl1: 	  Warning from theme "/etc/emulationstation/themes/carbon/auto-lastplayed/theme.xml"
  (from included file "/etc/emulationstation/themes/carbon/auto-lastplayed/../carbon.xml")
    could not find file "" 
lvl1: 	  Warning from theme "/etc/emulationstation/themes/carbon/custom-collections/theme.xml"
  (from included file "/etc/emulationstation/themes/carbon/custom-collections/../carbon.xml")
    could not find file ""

muldjord · 4 Oct 2019, 22:13

@Riverstorm This doesn't seem to be related to the Skyscraper scraper script (which this thread is about). Perhaps this is about SSelph's scraper?

mitu · 4 Nov 2019, 05:06

@Riverstorm I think you're confusing Skyscraper(Lars's external game scraper) with the built-in Emulationstation scraper using the Screenscraper source.
The error you posted (Error saving resized image. Out of memory? Disk full?) is specific to the internal Emulationstation scraper and it happens when it receives a bad response from the site when an image is requested. If it's a scraper (ES) error, it would be useful to know if it always happens for a particular ROM (and what ROM is it), but what I think it happens is that the ScreenScraper site just returns a 'Busy now, closed for non-registered users' message instead of the requested image, thus confusing the scraper.
The other part of the log are just benign warnings, they don't happen because of having used the scraper.

EDIT: incidentally, I'm also 'responsible' for the ScreenScraper part of ES, so it would be useful to know if there's an error with a certain ROM or it's just the usual ScreenScraper 'busy now, try later' message that's happening.

Riverstorm · 10 Apr 2019, 21:14

@muldjord - Ah, ooops, is that sselph's scraper built into ES? Basically I just pulled up the scraper from the ES menu, first option on top.

Riverstorm · 10 Apr 2019, 21:20

@mitu - Ah ok, sorry for that false alarm and I appreciate the response guys. I'll run through it again later and post if it's a specific ROM or some pattern I see.