Scraping totally fails

nexusone13

Hi @WhiteT982 ,
ARRM can scrape on this sites : Screenscraper, HFS-DB, TheGamesDb.net, LaunchBox, Steam, IGDB , GOG , MobyGames and GameTDB

beppi

Thanks all folks, I'm really getting crazy.
I tried to scrape using skraper, and it works. Only problem... it runs on Windows and I need it running on the raspberry. The only ption I see now is to scrape on Windows, create a script to change manually all the paths and informations that are missing and move everything manually, a huge work!
Furthermore, also other scrapers fail and I'm really styuck in a problem that looks to me as a sci-fi movie... I made a lot of configurations on my raspi and I consider myself able to dive in deep in linux configurations using my beloved vi, but I fail here.
Anything else I tried, unfortunately, failed, and now I'm trying to reach the author of Skyscraper on github, but I doubt I will be able to fix it.

Quackwalks

@beppi What I do Is back up my roms directory into a windows folder. In Skraper, you can change the settings to use the windows directory and it'll still output game lists that work for retropie. I just put my game lists in my rom directories, then a folder in each system for the snaps and box arts. After you're done scraping, transfer your game lists and scraped media to your raspberry pi where they should go.

Sorry I can't help with the details, but I know this method works with enough tinkering.

beppi

@Quackwalks this is indeed the only solution i found so far, but it's not a complete solution because some information is missing, for example the information about the location of videos and screenshots, and Skyscraper was also creating a composition with the box for the screenshot that was integrated with the theme. Of course it's not a big deal but I would like to understand why Skyscraper cannot connect from raspi.
I feel that I could fix this problem if I only had some kind of log, monitor, or anything else, do you know if there is a log somewhere? I could not find it.

YFZdude

@beppi
There should be logs anytime something fails to scrape with skyscraper in the /home/pi/.skyscraper folder.

Crush

@Quackwalks said in Scraping totally fails:

@beppi What I do Is back up my roms directory into a windows folder. In Skraper, you can change the settings to use the windows directory and it'll still output game lists that work for retropie. I just put my game lists in my rom directories, then a folder in each system for the snaps and box arts. After you're done scraping, transfer your game lists and scraped media to your raspberry pi where they should go.

Sorry I can't help with the details, but I know this method works with enough tinkering.

Just make a network drive on windows out of \\retropie\roms so you can scrape using screenscraper on windows directly instead of having to transfer back and forth.

beppi

@YFZdude thank you, unfortunately there's nothing, because there's no error in Skyscraper: it just receives the information "not found", appearantly. I watched by capturing in real time written files both on .skyscraper directory, and on /var/log, and no log is being generated.

beppi

Thanks all for answering me, unfortunately there's no solution right now.
I make you just one question: are you able to scrape with Skyscraper from inside raspberry, on screenscraper.fr or do you get the same error "game not found"? thank you

kiro

@beppi trybtjis, it may work...https://github.com/zayamatias/retroscraper

Floob

@beppi
FWIW I use Skraper and scrape on windows, then just copy across the files.
I do that so I can copy it across to any destination quickly - so you only need to scrape once really.
The paths are all relative, so you dont need to change them

For example:
<image>./media/boxart2d/3-D Tic-Tac-Toe (USA).png</image> <marquee>./media/wheel/3-D Tic-Tac-Toe (USA).png</marquee> <video>./media/videos/3-D Tic-Tac-Toe (USA).mp4</video>

So in Skraper, the video path would be:
%ROMROOTFOLDER%\media\videos

Same as @Quackwalks really.

beppi

@Floob i fixed the problem. On retropie it still fails, but I mounted a disk in a Linux distribution and using the bind option I recreated the needed directory tree and run Skyscraper from there. It works, so it's obvious that there is a problem on my raspberry but I don't have any tool to address it. Thank you

kiro

@beppi could it be a permissions issue in your rpi? Pi user does not have write rights on the mount ?

Quackwalks

@Crush That is also an option, but sometimes I'd rather get everything scraped on a separate hard drive. Just makes me feel more secure in various ways. I'm probably just silly.