ROM Scraping on a Network Share

JFamily

Pi Model: 3 B
RetroPie Version Used: Not sure at the moment. Latest Stable Build as of 7/23
Built From: Source
USB Devices connected: iBufallo SNES (x2), Logitech Wireless Keyboard
Controller used: iBufallo SNES (x2)
How to replicate the problem:

First, some backstory.... I've setup my RetroPie to mount a network share (//192.168.1.10/Storage/ROMs). After some trial, error and tinkering with config files, I got it to work and can play games that are hosted on another machine. I can also save files to that machine so I know I'm not having a permissions issue.

However, when I tell emulationstation to scrape my roms, it only scrapes the 3 or so folders that already have 'ROMs' added to them when emulationstation is installed. So I know, somewhere, the scraper is still looking in my ~/RetroPie/roms folder instead of the ~/RetroPie/smb folder that I've used to mount my share to.

I've tried scraping via the RetroPie menu and the one in retropie_setup.sh. I found parameters for /opt/retropie/supplementary/scraper/scraper which seems to have the options I need. I'm at work right now so can't test it out. At any rate, I would much rather incorporate the parameters into the application, if at all possible.

Can anyone give me some guidance to the file I would need to change paths in so I can get the scraping to work? Pictures and other meta data would make this system perfect!

I plan on posting a guide in the future to help other people set this up. I'm sure anyone could figure it out because I did and I'm no expert. But it may help someone. I combined 3-5 guides and general knowledge to get everything up and running. This is the last step and I can't seem to get passed it.

Edit: ~~Furthermore, it would be beautiful if I could store these images and data on the server rather than a local folder. My 8Gb SD card may not be able to handle 18+Gb of ROMs' data.~~ Nevermind, figured that one out :)

meleu

@JFamily I'm away from my RetroPie too, but I would start my investigation with:

grep -R 'RetroPie/roms' /opt/retropie/supplementary/scraper/* 2>/dev/null

JFamily

Brilliant, I'm not familiar enough with linux to think of simple things like that on my own. Lol!

I'll start there tonight. I was thinking, the way I plan to solve my desire to store metadata on the server involves copying that folder to the server and symlinking it to the correct location on the pi.

If I symlink my //192.168.1.10/Storage/ROMs to ~/RetroPie/roms and revert my config files, would this solve my problem? What about the ROMs that are stored in /roms by default? Will everything just merge seamlessly or will I have problems trying to link to folders that already have stuff in them? I would have wiped out the roms folder and did it this way to start with but some of the pre-installed ports don't want to copy to my windows computer. I've only tried copying/pasting, maybe I should try sFTP or something else...

sselph

A simple solution if you just want things to work would be to run

/opt/retropie/supplementary/scraper/scraper -scrape_all -thumb_only -workers 4`

That will parse the config the same as EmulationStation does then it will check every listed folder and scrape it placing the gamelist.xml in the rom folder for that system and the images in a folder called images in each system's rom folder. If the system isn't supported you may see a bunch of errors about not finding hashes or it might just take a while to not do anything.

The other option that is a little slower is to cd to each directory and run the scraper

cd /mnt/ROMS/nes
/opt/retropie/supplementary/scraper/scraper -thumb_only -workers 4

The other alternative you mentioned does work but you can't create a symlink with the same name as another file so you would have to delete the original roms folder first.

herb_fargus

@sselph pretty sure jools just added scraping into rom folders as an option with the scraper module too

JFamily

@sselph That looks great, I had no idea it would work that way!

Disregard the rest of this message. I found the solution on the Raspberry Pi Forums. Again, surprised it doesn't have anything to do with network shares but the solution will be the same.

In case someone else stumbles on this post and wants to know the solution I'll be going with; I'll be editing the savestate and savefile directory in the retroarch.cfg file. We'll make this a local destination and worry about space issues if/when the time comes.

I have another problem. I was apparently wrong about it being able to save to the share. The save files I saw in there were all from before I set it up this way and they don't update as we make progress.

When I hold Select + Right Shoulder, it says "Failed to save to /home/pi/RetroPie/smb" or something to that effect. I'll have to play with permissions and stuff on my Windows machine tonight but any advice would be greatly appreciated.

It's weird that I'm not finding anyone else that did a setup like this. It was one of the first ideas I had before the RasPi even came in the mail. Hopefully I'll get all the kinks worked out and write an all inclusive guide for it. Would love to see it put up on the wiki someday! :)

I take that back, I see where a couple of people touched on the subject and maybe more on Reddit but I can't access it at work :(. The one I was able to read barely got as far as I have and made no mention of issues with saving. I'm guessing it wasn't a problem because he was using NFS but I've never played with NFS shares so I can't speak to that.