(Another) skyscraper help topic!
-
Hi all,
I have looked around the forum but I couldn't see a similar thread, so apologies if this has been answered 1000 times before.
I started using this scraper as the built in one was giving me errors. I thought I would test it on a smaller collection (segacd). It seems to work really well and has only not matched 5 roms. What I am having trouble with is now trying to get these 5 found. If we use one example rom "RDF Global Conflict (USA).bin" is in skipped-cache as 'No returned matches'. It is on screenscraper here So how do i go about either adding this manually or creating an XML file with the missing entries?
I tried running this command ```
Skyscraper -p segacd -s screenscraper -m 30 --query "global" "/home/pi/RetroPie/roms/segacd/RDF Global Conflict (USA).bin"Can anyone please help/point me in the right direction please? Thanks. -
I am trying to edit the post but it says it is flagged as spam so here is the OP:
Hi all,
I have looked around the forum but I couldn't see a similar thread, so apologies if this has been answered 1000 times before.
I started using this scraper as the built in one was giving me errors. I thought I would test it on a smaller collection (segacd). It seems to work really well and has only not matched 5 roms. What I am having trouble with is now trying to get these 5 found. If we use one example rom "RDF Global Conflict (USA).bin" is in skipped-cache as 'No returned matches'. It is on screenscraper here So how do i go about either adding this manually or creating an XML file with the missing entries?
I tried running this command
Skyscraper -p segacd -s screenscraper -m 30 --query "global" "/home/pi/RetroPie/roms/segacd/RDF Global Conflict (USA).bin"but it still seems to skip. I have looked in the github guide but it isnt clear (to me) how i can generate the missing files.
Can anyone please help/point me in the right direction please?
Thanks.
-
Akismet, the forum spam plugin, it's usually picky when editing a post and you add code tags, happens to me all the time.
As for how to use the
--queryparameter - you need to use the checksum or theromnomto override the default search (by hash) done byskyscraper. For instance, to search byromnom:Skyscraper -p segacd -s screenscraper --query 'romnom=RDF - Global Conflict (USA).cue' ./RDF\ Global\ Conflict\ \(USA\).bin [...] Starting scraping run on 1 files using 1 threads. Sit back, relax and let me do the work! :) #1/1 (T1) Pass 1 ---- Game 'RDF Global Conflict (USA)' found! :) ---- Scraper: screenscraper From cache: NO Search match: 100 % Compare title: 'RDF Global Conflict' Result title: 'RDF: Global Conflict' () Platform: 'Mega-CD' () Release Date: '1994-01-01' () Developer: 'Imagineering' () Publisher: 'Absolute Entertainment' () Players: '1' () Ages: '6+' () Tags: 'Acción' () Rating (0-1): '0.6' () Cover: YES () Screenshot: YES () Wheel: YES () Marquee: YES () Description: () 'Using a thrilling combination of 3D rendering and explosive combat action, RDF: GLOBAL CONFLICT is an interactive multimedia experience that pushes the Sega CD system to its limits! As a tank commander in the RDF - the elite Rapid Deployment Force - you'll test your battle savvy in a realistic virtual world, from the war-torn countryside of Eastern Europe to Alaska's desolate tundra! But watch your back! An evil genius may pose a threat not only to the team - but to all of mankind!'You can also search by the checksum of the file, as published on the ScreenScraper site. For your game, you can see the list of files and hashes the scraping site knows for this game at https://www.screenscraper.fr/gameinfos.php?gameid=97374&action=onglet&zone=gameinfosroms. So you can use
Skyscraper -p segacd -s screenscraper --query 'md5=C872F62FDEE70C05ABA17AAE7B0FE731' ./RDF\ Global\ Conflict\ \(USA\).binNote that I took the checksum (md5) for the
RDF Global Conflict (USA) (DW0226).binfrom the page mentioned above. -
@mitu Thank you so much - i was looking everywhere to try and find the MD5 for the files!!
Is there a way i can put the missing files in an xml and then run a check - or is this a one-at-a-time process?
-
@showstopper You can query the scraping source any time you want, the results will be added to the cache and stay there, then later on when you generate the gamelists for Emulationstation they'll be written to the gamelist and media folders.
-
@mitu OK really sorry but i am trying this with my atari lynx game 'Ninja Gaiden III - Ancient Ship of Doom (USA, Europe).7z' that is coming up as unmatched.
I have tried searching with query and MD5 which is failing for me. This is what i am getting:
pi@retropie:~ $ Skyscraper -p atarilynx -s screenscraper --query 'md5=B263E945DA412BDCE0DC88697B629412' ./Ninja\ Gaiden\ III\ \(USA\).zip ------------------------------------------ Running Skyscraper v3.3.1 by Lars Muldjord ------------------------------------------ Filename: './Ninja Gaiden III (USA).zip' requested either on command line or with '--fromfile' not found! Please verify the filename and try again... pi@retropie:~ $Here is the link to the MD5
Sorry for the noob questions
EDIT - OK so i need the full path to my rom and then it worked. Thanks for all your help.
Skyscraper -p atarilynx -s screenscraper --query 'md5=B263E945DA412BDCE0DC88697B629412' "/home/pi/RetroPie/roms/atarilynx/Ninja Gaiden III - Ancient Ship of Doom (USA, Europe).7z" -
OK so i am now scraping my GBA roms and i have a few that dont scrape on screenscraper. I am trying to run a query in thegamesdb but it says it cant find it. Could someone tell me what i am doing wrong please?!
pi@retropie:~ $ Skyscraper -p gba -s thegamesdb --noscreenshots --nowheels --nom arquees --query 'Cartoon Network Block Party / Cartoon Network Speedway' "/home/ pi/RetroPie/roms/gba/2 Games in 1 - Cartoon Network Block Party + Cartoon Networ k Speedway (USA).7z" ------------------------------------------ Running Skyscraper v3.1.4 by Lars Muldjord ------------------------------------------ Platform: 'gba' Scraping module: 'thegamesdb' Input folder: '/home/pi/RetroPie/roms/gba' Game list folder: '/home/pi/RetroPie/roms/gba' Covers folder: '/home/pi/RetroPie/roms/gba/media/covers' Screenshots folder: '/home/pi/RetroPie/roms/gba/media/screenshots' Wheels folder: '/home/pi/RetroPie/roms/gba/media/wheels' Marquees folder: '/home/pi/RetroPie/roms/gba/media/marquees' Cache folder: 'cache/gba' DID YOU KNOW: Skyscraper caches all scraped data in the resource cache. This sav es bandwidth for the scraping sources as any rescaping will make use of this cac he instead of pounding the servers again. Reading and parsing resource cache, please wait... Successfully parsed 9869 resources! Looking for optional 'priorities.xml' file in cache folder... Not found, skippin g... Starting scraping run on 1 files using 4 threads. Sit back, relax and let me do the work! :) #1/1 (T1) Pass 1 ---- Game '2 Games in 1 - Cartoon Network Block Party + Cartoon Network Speedway (USA)' not found :( ---- 'thegamesdb' requests remaining: 1497 0/1/1 Elapsed time : 00:00:00 Est. time left : 00:00:00 ---- Resource gathering run completed! YAY! ---- Writing 9869 (0 new) resources to cache, please wait... Success! ---- And here are some neat stats :) ---- Total completion time: 00:00:01 Total number of games: 1 Successfully processed games: 0 Skipped games: 1 (Filenames saved to '~/.skyscraper/skipped-thegamesdb.txt') pi@retropie:~ $ -
I have issues with the code tags and getting spammed!!
This is what i am typing in:
Skyscraper -p gba -s thegamesdb --noscreenshots --nowheels --nomarquees --query 'Cartoon Network Block Party / Cartoon Network Speedway' "/home/pi/RetroPie/roms/gba/2 Games in 1 - Cartoon Network Block Party + Cartoon Network Speedway (USA).7z" -
I don't see any actual errors in your scraping command output. Maybe the game doesn't exist on the source site ?
-
@mitu It does exist as i copied the wording from thegamesdb exact since they dont seem to have CRC numbers.
No matter i just manually downloaded images and edited the db.xml. All good now. -
@showstopper, Just a side-note, you are using a really old version of Skyscraper. I would strongly suggest you update it. Also, I would like to point you to the Skyscraper documentation. It has examples for pretty much all of the available options. https://github.com/muldjord/skyscraper/blob/master/docs/CLIHELP.md
If you want to use the
--queryoption with thethegamesdbmodule, you'll need to know their query format. I believe it basically needs you to simplify things a bit so "Rick Dangerous" would become--query "rick+dangerous". It is a bit of an expert option and sometimes it is quite hard to get it right, especially with the name searched based modules such as TheGamesDb. So it expects users to know a bit more about the underlying API. But it is very useful to fill in the missing roms. -
@muldjord Thank you for this great application! I did an update today and so i'm surprised it is an old version. I'll try and update again.
EDIT - I did an update from within the app - looks like i need to do it outside the application. Thanks again for your hard work with this scraper.
-
EDIT - I did an update from within the app - looks like i need to do it outside the application. Thanks again for your hard work with this scraper.
You're welcome! Yes, you need to stick to one or the other (the one installed from the RP script or the one installed manually as describe on Github). Mixing them can cause problems.
-
I didnt want to make a new thread so....
I have this working for all my systems but i noticed that the Mame (Arcade) scrape for option Covers seems to get the artwork (from ArcadeDB) Flyer. Is there anyway to change this?
For example Super Street Fighter II uses the Flyer image and i would like it to use the Decal image (or other).
Is that possible?
-
@showstopper No, it's hardcoded to first try for the flyer and then the title artwork if it doesn't find a flyer. That's the problem with the arcade artworks, there are so many different ones and none of them match up well with an actual game box cover. So it comes down to a subjective thing and I personally like the flyer artwork best.
If you want, you can import them into the cache yourself by using the import module. Read more about that here.
-
@muldjord Thanks for replying. I totally agree that it is a subjective preference. Some of them look good as flyer. I manually checked a few and Decal seems to be quite a good one (for me).
Just reading your KB - i expect that i am reading it wrong but it looks like it would be a manual individual import - is that right? Is there no way to add 'Decal' as a bulk import option? -
@showstopper Well, I guess you could compile Skyscraper from source and just change the hardcoded "flyer" to "decal" in src/arcadedb.cpp. That would probably do it.
The import module is manual to the degree that you have to prepare each file. But once all files are placed correctly and named correctly, it's a bulk import.
-
@muldjord do you have a compile KB by any chance? I'm happy to give it a go.
-
@showstopper I don't know what a KB is, but the compile process is described in the documentation. :)
-
@muldjord KB = Knowledge Base (or documentation!)
I am running this on a Raspberry Pi so not sure how i would compile it on there. Can i change the aracdedb.cpp file on the Pi already - or does the app need compiling first with that change?EDIT - Actually could i do this by editing the artwork.xml file?
EDIT 2 - Ah no looks like it would need compiling
Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.
Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.