Skyscrapper / Screenscraper PSX issue

flifpront

Hi,

I'm using Retropie latest version.

Im a registered 6 thread Screenscraper member. I relocated cache folders to a NAS.
I scrapped all system quickly with no problem except psx which is extremely slow. 350 PBP games, takes 3/4 minutes per game, and always ends up bugging (broken pipe).

Tryed to delete/vaccum cache for the system, but nothing seems to work.

tanks a lot.

mitu

@flifpront said in Skyscrapper / Screenscraper PSX issue:

I scrapped all system quickly with no problem except psx which is extremely slow. 350 PBP games, takes 3/4 minutes per game, and always ends up bugging (broken pipe).

If the scraper hashes the files, then it makes sense that scraping a larger file takes longer. How much does it take to scrape one single .pbp file ?

Also, when opening a support topic, please add the info requested in https://retropie.org.uk/forum/topic/3/read-this-first

flifpront

Power Supply used: 5.25 V
RetroPie Version Used (eg 3.6, 3.8.1, 4.1 - do not write latest): RetroPie v4.6
Built From: Pre made SD Image on RetroPie website
USB Devices connected: none
Controller used: pS4 wireless
Guide used: First Installation Guide on RetroPie website

How much does it take to scrape one single .pbp file ?

it takes 1 to 3 minutes depending on the game.

I dont mind if it is slow, but the problem is the whole process just stops and freezes the system randomly after 100 or 200 scraps. either the system just freezes, or I get a message "broken pipe"

mitu

@flifpront said in Skyscrapper / Screenscraper PSX issue:

I dont mind if it is slow, but the problem is the whole process just stops and freezes the system randomly after 100 or 200 scraps. either the system just freezes, or I get a message "broken pipe"

If you're running the scraper from RetroPie-Setup, there should be a log in $HOME/RetroPie-Setup/logs from running the scraper - maybe you can get the log and post it on pastebin.com.

Can you try scraping using TheGamesDB site, instead of ScreenScraper ?

muldjord

This is undoubtedly because ScreenScraper requires the hash of the files. It take a looong time to hash those PSX files. Nothing that can be done about that. Luckily Skyscraper has a system I dubbed QuickId which saves the hashes and only rehashes when the file has changed on the disk. So for any game list regeneration it will be fast. But the first time any file is hashed, it will have to calculate it from the data inside the file.

EDIT: I spoke too quickly... My QuickId system is only used for the internal Skyscraper ID's... I should fix that, but I'm not actively developing Skyscraper right now, so you will have to live with it for PSX for the time being I'm afraid.

Optimally the Skyscraper internal ID's should use hashes. But i limited it to only hash files when they are smaller than 50 megs I think (because it would take too long for bigger files, which is also the issue here). So for anything above that, it simply hashes the filename. And it's one of those things where it could easily be made to always use hashes for the full files, but then I would break the caches of every single Skyscraper user. Can't do that... Ugh. I really should have given those ID's more thought back when I designed it two years ago. Keeps coming back to kick me in the behind. :(