Scraper gets stuck when UI dims

[vbs]

I always had the problem that the scraper got stock after some time (like after ~10min). It keeps saying "Working" but nothing happens anymore. Then when I press a button it either continues right away or it says "Network error: Failure when receiving data from the peer". I can hit "Retry" and then it continues until it gets stuck again.

I think it is related to the UI dim feature The scraper immediately stops as soon as the screen dims. Then when I directly press a button the screen lits again and the scraper continues. When I wait longer before pressing a button then the network error message appears (seems to be some network timeout that occurs when the process gets stuck for too long).

Workaround seems to be to set the dim timeout to "0" which seems to mean "disabled".

Does this happen for you guys too? Then this is probably more a bug report.

Oh, I am on x86 Ubuntu 16.04!

Thanks!

Potential Fix:
I tried to fix this bug in Emulationstation. The problem was that ES just stops processing anything after entering dim mode. I tried to fix it by stopping ES going to sleep mode as long as scraping takes place. Seems to work so far.
Would be great if you guys would give it a try and report back if it fixed the issue for you and if you experienced any unexpected side effects. The fix can be found here:
https://github.com/verybadsoldier/EmulationStation/tree/no_sleep_when_processing

To use it:

1. Edit file RetroPie-Setup/scriptmodules/supplementary/emulationstation.sh

2. Go to line 37 and make replace this

    [[ -z "$repo" ]] && repo="https://github.com/retropie/EmulationStation"
    [[ -z "$branch" ]] && branch="master"
   

   by this

   [[ -z "$repo" ]] && repo="https://github.com/verybadsoldier/EmulationStation.git"
   [[ -z "$branch" ]] && branch="no_sleep_when_processing"
   

3. Start retropie_setup.sh

4. Choose Manage Packages -> core -> emulationstation

5. Choose Update from source

Emulationstation should not be recompiled using the patched sources. Restart ES/Retropie afterwards. While ES is scraping it should not enter sleep mode no more.

[herb_fargus]

@vbs use sselphs scraper.

https://github.com/retropie/retropie-setup/wiki/scraper

[vbs]

@herb_fargus
Thanks, I tried that and that is a great scraper! Sadly some (for me important) systems like Amiga and C64 are not supported:

An auto-scraper for EmulationStation written in Go using hashes. This currently works with NES, SNES, N64, GB, GBC, GBA, MD, SMS,     32X, GG, PCE, A2600, LNX, MAME/FBA(see below), Dreamcast(bin/gdi), PSX(bin/cue), ScummVM, SegaCD ROMs.

So I guess for those systems I am stuck with the ES built-in scraper. But I really think that its instability is just a bug related to screen dimming (at least in my case). After I disabled screen dimming I was able to scrape >22000 roms in one session without problems. Would be awesome if someone could look into it.

Thank you!

[DrMaxwell]

@vbs this is a great find!

[tehswiftone]

Yes this happens to me as well. I found shutting the screen saver off fixes the problem. I am not sure but I think the system goes into a hybernate type state when the screen saver kicks on.

[vbs]

I tried to implement a fix. You can find it in the first post. Would be great if you would give it a try. If its good then I would try to get it merged upstream. Thanks!

[g0nz0uk]

@vbs Aphex Twin, yes my favourite!

[DrMaxwell]

@vbs The systems that aren't represented need hash values and the corresponding media resources hosted. Sselphs scraper is great, but the database that he pulls media and metadata from isn't as efficient, nor complete, as his scraper is unfortunately. Also, I believe the database that the scraper pulls from is down right now.

[SirSoulBrother]

@herb_fargus Retropie Newb here. Was having the same issue with scraper stopping when screensaver came on. Tried the scraper you recommended and got a quick error about installation not being valid. Now when I run the Retropie scraper, one database, TheArchive, races through yielding no matches and TheGamesDB now moves 2-3 per minute and finds nothing. How do I restore my scraper without scraping my Retropie?