I need some help scraping ROM data

bfollowell

I'm working on setting up a RetroPie build for my nieces and I've been having the hardest time scraping metadata.

All of my ROMs are from a No-Intro ROM set, and I've verified/fixed/renamed everything using RomCenter, so everything is named as expected. I still have thousands of ROMs that will not scrape. Absolutely nothing will scrape for PC Engine or SNES. Not the first ROM. And even for the systems that do scrape there are many, many ROMS, some very common ROMs like any all of the Super Mario World ROMs for NES, will not scrape.

I tried using Steven Selph's scraper and didn't have much luck at all. I finally started using the built-in scraper and had better luck, but still tons of misses. I ran the built-in scraper using TheGamesDb, then ran again using ScreenScraper. I found some under both, but like I said, some whole systems came up with nothing, and many common ROMs won't scrape. I have no idea what I may be doing wrong.

Does anyone have any ideas of something I can try, or some other method I may have more success with?

Thanks.

bfollowell

I probably should've mentioned that my system is setup on a Raspberry Pi 3b+ and I just downloaded the RetroPie image about a week ago, so I'm assuming it is the most recent version.

Clyde

I didn't have such problems at all with no-intro romsets for multiple systems including PC Engine and SNES. I also used RomCenter for renaming and filtering them. Maybe your Romsets aren't really no-intro or your processing did something with them.

I would try to feed the original files prior RomCenter processing to Selph's Scraper, and/or get another no-intro Romset from another source than the ones you have now.

How exactly did you process the roms with RomCenter?

bfollowell

I verified against No-Intro dat files from DAT-o-MATIC

Clyde

What did Selph's Scraper say exactly when it didn't find the roms? Were the hashes unknown or couldn't it connect to the chosen online database?

Additionally, you could try Selph's own guide about when Scraper can't find a game?

bfollowell

Most of the text rolled by so quickly, I couldn't tell you what any of them said. I'll try again this evening and see what they say.

I do know that most of them, especially the whole systems that are missing, seemed to go through just fine, with almost no missing or skipped ROMs. It's just that, when I went back into Emulation Station, nothing was there, as far as images and descriptions go.

I may install skraper and try that this evening.

MetalManTN

@bfollowell I could be wrong about this, but I'm thinking you have to reboot the Pi (or at least restart EmulationStation) before the scraped images and metadata will be saved and also appear on screen. You may have been doing this anyway, but since you didn't mention it I thought I would throw that out there.

IanDaemon

In addition to restarting ES, Skyscraper worked well for me. You can find it under the "experimental" section in the RetroPie
menus.

Clyde

@MetalManTN A reboot isn't necessary, just a restart of Emulation Station. That said, Selph's Scraper will refuse to scrape if ES is still running, because ES will overwrite the gamelist.xml on exiting, thus reversing any changes the scraper would make. So, you'll have to stop and restart it after scraping anyway.

@bfollowell Try removing or renaming the gamelist.xml in every rom directory in question. The scraper should rebuild it, so look for (and into) a new one after scraping. There should be entries for the artwork and information for every game then.

Skyscraper also worked well for me, but as I said, so did Selph's Scraper.

bfollowell

Thanks for everyone's help. I tried everything you guys suggested and, unfortunately, none of your suggestions helped at all in my case. I have no idea why neither the built-in nor the Selph's Scraper seemed to work for me. I downloaded Skraper and it is working like a charm. I'm scraping on my PC and then copying the files over my network to my Pi and everything is working perfectly. It's working easily enough, and well enough, that I really have no desire to find out why the other ways weren't working from my Pi any longer. I'm pretty sure I've found out how I will do all of my scraping moving forward.

I do appreciate everyone trying to help me though. Thank you!

SingingCoyote13

@bfollowell as far as i know of, you can best use skyscraper to do this, and do not forget to build the entire gameslists afterwards. You might want to set the scraping to thegamesdb if the title contains a ´[!]´ or like ´[ ´ ´]´ with those brackets, and to no-intro if the title lacks the brackets´ [´ ´]´ but has these ( ) like (English) or (US).

Clyde

@bfollowell Whatever works for you is fine, have fun!

IanDaemon

@Clyde https://media.discordapp.net/attachments/400348842832429066/616284091616329728/i84d7bl4c5x01.png

Clyde

@IanDaemon I think the „playing“ part is much too big in this diagram. 😉