Game List scraping consistency: Meta data, Snap, Logo, 3D boxart

[Dr.Groove]

Hi all,

I am fairly new to the community so wanted to ask if anyone has fixes for my issues before I start writing scrips or building any apps.

I have stared with a Retro Pi - Motion Blue image and love having a video preview, 3D box art, game details and the logo.
However I have found when obtaining ROM packs they are often inconsistent and some games are missing Meta data, logo's, snaps, etc.
As there are hundreds if not thousands of games it's not feasible to fix things manually.

I came across Steven Selph's Scraper which is fantastic how ever it creates a new gameslist which does not contain video or marquee nodes in the XML. So for a snap based theme, after running the scraper you won't have any video or logo links in the games list.

Does anyone know of a fix for this or am I perhaps missing a command in the scraper that would inject the video and marquee nodes?
If not I will write some scripts that will insert the video and marquee links after running Steven Steven Selph's Scraper using David Matrin's naming convention.

Cheers!

[sselph]

@Dr.Groove You can download videos and marquees from screenscraper by passing
-download_videos -download_marquees -console_src="ss"

If you already have videos that have been downloaded by some other method, I look for them locally but I expect for a game named rom.bin to have a video file named like rom-video.mp4. If you have things named differently there is a flag -video_suffix="-video" you can change to whatever.

[Dr.Groove]

@sselph Thank you so much for replying! Your scraper is fantastic and saves so much time getting ROM packs organised and consistent. I am a bit OCD with my build and want everything in it's place.

I will play around with those commands.

Cheers!

[mattrixk]

@sselph I already have screenshots, marquees and videos downloaded with UXS, but I don't like the screenscraper metadata much (I've found genre is often wrong, and lots of data is missing), so I want to use your SSelph Scraper to get the metadata from thegamesdb. What tags to I need to put in to make my gamelist populate with:

<game id="20228" source="theGamesDB.net">
    <path>./game.gb</path>
    <name>Game</name>
    <desc>This is a great game.</desc>
    <image>./media/game-screenshot.jpg</image>
    <marquee>./media/game-marquee.png</marquee>
    <video>./media/game-video.mp4</video>
    <releasedate>19900928T000000</releasedate>
    <developer>Nintendo</developer>
    <publisher>Nintendo</publisher>
    <genre>Shooter</genre>
    <players>1</players>
</game>

Considering I already have the 2 images and video locally within the media folder.

[sselph]

@mattrixk So it depends on if you are running this on the same system that the gamelist will live on(ie the RetroPie), Also assume you are running from the rom dir.

scraper -image_dir="media" -image_path="media" -marquee_dir="media" -marquee_path="media" -video_dir="media" -video_path="media" -image_suffix="-screenshot"

the *_dir flags tell the actual directory where you want the files to be downloaded. The *_path flags are what is stored in the gamelist.xml. If you are on the RetroPie, these are the same but if you aren't they could be different.

[mattrixk]

@sselph

Sorry, I was typing the question towards the end of my lunch break and missed some (probably quite important) information.

I do all my scraping on my PC. I just find it easier, faster and I have much more storage space.

I use this file structure:
D:\games\scrape\gb
Within the gb are:

• a folder called "media"
• 2 roms
• scraper.exe
• a file called "scrape.bat"

scrape.bat looks like this:

scraper.exe -image_dir="media" -image_path="media" -image_suffix="-screenshot" 

I use it because I'm not too good with console stuff. This works fine create a gamelist with links to the correct <image>, but it ignores the marquee and video.

However, whenever I use any of the "-marquee_" or "-video_" tags you mentioned, the console just spews out a "Usage of scraper:" list and doesn't do anything else. Am I missing some other tag that lets the scraper add the <marquee> and <video> elements to the gamelist? Do those tags only work when the "-console_src" is set to screenscraper?

On a side note, thank you for taking the time to help me out with this.

[sselph]

@mattrixk Ah I haven't done a release recently so if you are using my downloads you wouldn't have it. I'll do a release today.

[mattrixk]

@sselph cooool. It's well passed my bed time, so I'll check it out tomorrow (or whenever you get around to putting the release out. No rush).