Versatile C++ game scraper: Skyscraper

quicksilver

# --------------------------------------------------------------------
# Skyscraper by Lars Muldjord (https://github.com/muldjord/skyscraper)
# --------------------------------------------------------------------

# This is an example config file for use with Skyscraper. Use it as a template for creating
#   your own. If you copy this file to 'config.ini' it will be used per default.
#   You can always copy config.ini.example to a filename of your choice and make Skyscraper
#   use it with the '-c' command line option.

# You can have several platform sections in one config file. Further down is an example of
#   a generic platform setup. Check '--help' for more info on supported platforms.

# You can have several scraping module sections in one config file. Further down is an example
#   of a generic module setup. Check '--help' for more info on supported modules.

# Remember that most of these settings can also be set as command line options. Doing so
#   will overrule the options seen here except for the scraping module configs which
#   overrules everything else.
# Platform specific configs overrules main configs.

# Uncomment the sections and lines you wish to use:

[main]
##Setting input, gamelist or media folder here, will automatically append '/[platform]' to path.
##If you need better control, use the same variables under a [<platform>] section instead.
#inputFolder="/home/pi/RetroPie/roms"
#gamelistFolder="/home/pi/RetroPie/roms"
#mediaFolder="/home/pi/RetroPie/roms"
#cacheFolder="/home/pi/.skyscraper/cache"
#cacheResize="false"
#cacheCovers="true"
#cacheScreenshots="true"
#cacheWheels="true"
#cacheMarquees="true"
#importFolder="/home/pi/.skyscraper/import"
#unpack="false"
#frontend="emulationstation"
#emulator=""
#launch=""
#videos="false"
#videoSizeLimit="42"
#symlink="false"
#brackets="true"
#maxLength="10000"
#threads="2"
#pretend="false"
#unattend="false"
#unattendSkip="false"
#interactive="false"
#forceFilename="false"
#verbosity="1"
#skipped="false"
#maxFails="30"
#lang="en"
#region="wor"
#langPrios="en,de,es"
#regionPrios="us,eu,ss,uk,wor,jp"
#minMatch="0"
#artworkXml="artwork.xml"
#relativePaths="false"
#addExtensions="*.zst"
#hints="false"
#subdirs="true"
#spaceCheck="true"
#scummIni="/full/path/to/scummvm.ini"

[psp]
artworkXml="artwork-psp.xml"

[vectrex]
artworkXml="artwork-vectrex.xml"

[ngpc]
artworkXml="artwork-ngpc.xml"

[arcade]
#inputFolder="/home/pi/RetroPie/roms/amiga"
#gamelistFolder="/home/pi/RetroPie/roms/amiga"
#mediaFolder="/home/pi/RetroPie/roms/amiga/media"
#cacheFolder="/home/pi/.skyscraper/cache/amiga"
#cacheResize="false"
#cacheCovers="true"
#cacheScreenshots="true"
#cacheWheels="true"
#cacheMarquees="true"
#importFolder="/home/pi/.skyscraper/import/amiga"
#unpack="false"
#emulator=""
#launch=""
#videos="false"
#videoSizeLimit="42"
#symlink="false"
#brackets="true"
#lang="en"
#region="wor"
#langPrios="en,de,es"
#regionPrios="eu,us,ss,uk,wor,jp"
#minMatch="0"
#maxLength="10000"
#threads="2"
#startAt="filename"
#endAt="filename"
#pretend="false"
#unattend="false"
#unattendSkip="false"
#interactive="false"
#forceFilename="false"
#verbosity="1"
#skipped="false"
artworkXml="artwork-arcade.xml"
#relativePaths="false"
#extensions="*.zip *.uae *.adf"
#addExtensions="*.zst *.rom"
#subdirs="true"

#[<FRONTEND, eg 'pegasus'>]
#artworkXml="artwork.xml"
#emulator=""
#launch=""
#gamelistFolder="/home/pi/RetroPie/roms/amiga"
#mediaFolder="/home/pi/RetroPie/roms/amiga/media"
#skipped="false"
#brackets="true"
#videos="false"
#symlink="false"
#startAt="filename"
#endAt="filename"
#unattend="false"
#unattendSkip="false"
#forceFilename="false"
#verbosity="1"
#maxLength="10000"

[screenscraper]
userCreds=":"
#threads="1"
#minMatch="0"
#maxLength="10000"
#interactive="false"
#cacheCovers="true"
#cacheScreenshots="true"
#cacheWheels="true"
#cacheMarquees="true"
#videos="false"
#videoSizeLimit="42"

Im running skyscraper directly through retropie-setup, latest version (3.5.5)

peligwe

This post is deleted!

peligwe

@muldjord Never mind. Figured it out. Played around with different heights to get the desired effect I needed. However the gamebox view for N64 is cutoff on the edges. Will play around with different width to see if I can fix.

muldjord

Hi guys, I'm currently working on implementing a full Pegasus game list parser into Skyscraper which will allow it to retain user-information from old game lists. This will bring the support for the Pegasus frontend up to speed with the other supported frontends.

When that parser is done and released with the next release of Skyscraper, I will probably leave the project for a while. I want to focus on other projects again (non-coding related, I'm also a musician) and Skyscraper is simply craving too much of my attention. I've noticed that my replies to certain users on Github have become somewhat aggravated and arrogant due to lack of motivation and personal pressure in general, and that tells me it's time to put the tools down and go do something else.

I'm not leaving the community - certainly not. My interest in retro-gaming and RetroPie is a strong as ever. But I've reached a point where Skyscraper does everything I want it to do. And the changes I, personally, want to work on, are so significant that they will need to be moved to a potential Skyscraper 4.x. And my motivation to start work on that is not on the drawing-board for me at the moment.

I will probably be keeping an eye on any scraper-breaking bugs and fix those for the current 3.x version. And I will probably still provide support here and there when I feel like it. And I know you good peeps also do your bit to help out other Skyscraper users (which I am incredibly thankful for!).

Thank you all for using and supporting Skyscraper. I appreciate you all!

quicksilver

@muldjord skyscraper is a awesome tool. Thank you so much for all your hard work and support. You certainly deserve a break!

seriema

@muldjord Thank you for the greatest scraper ever! It has inspired me to so many ideas. Including one I've been meaning to ask you about for some time but wanted to get further in my own project first. I guess it's now or never :) Check it out: https://retropie.org.uk/forum/topic/26709/could-scraper-data-be-cached-online

Clyde

@muldjord said in Versatile C++ game scraper: Skyscraper:

I've noticed that my replies to certain users on Github have become somewhat aggravated and arrogant due to lack of motivation and personal pressure in general, and that tells me it's time to put the tools down and go do something else.

Kudos for realising that by yourself and pulling in the reins.

I will probably be keeping an eye on any scraper-breaking bugs and fix those for the current 3.x version. And I will probably still provide support here and there when I feel like it.

Thanks for that, you lessen my almost immediate concern what will happen if a scraping source changes breaks Skyscraper by changing their API.

Thank you all for using and supporting Skyscraper. I appreciate you all!

Thank you for creating and maintaining this awesome tool!

gold64

@muldjord Thanks for all your work on this, enjoy your other interests!

Rion

@muldjord

Thank your for all the hard work and effort you put in to making this great Scraper.

It's good to know you are still around for maintenance if something went wrong with any of the api's.

Tusen tack! 😊

muldjord

@Rion Absolutely, I'm not going anywhere, just want to focus on other things in my life :D. And thank you all, I appreciate the nice comments.

peligwe

@muldjord So the problem not my height but the width.. How much width is there available?

muldjord

@peligwe As much as you want (depending on ram of course) :)

stevas

@muldjord
I'm trying to get skyscraper to import already existing media, and it just gets right through it and says it's skipped them all?
It's megadrive, I've set up the symbolic link, the media is in:
/Import/videos/megadrive
/Import/screenshots/megadrive
And I'm using:
Skyscraper -p megadrive -s import --flags video

Have I missed something in the ini file or...?

mitu

@stevas said in Versatile C++ game scraper: Skyscraper:

Have I missed something in the ini file or...?

Post your .ini file (or the relevant config related to the import module).

stevas

Do you mean config.ini? I'm not aware of anything specific to import, but I've actually commented literally everything out of config.ini except for my screensaver login (which, incidentally, also doesn't work - and I know it's valid, as it works in Skraper).

stevas

Erm... Okay.

I thought I was meant to be able to use subdirs with import?

It works as long as I have them in:
/Import/videos
/Import/screenshots

muldjord

@stevas You have them backwards. It should be:

/home/USER/.skyscraper/import/megadrive/videos
/home/USER/.skyscraper/import/megadrive/screenshots

That's why :)

And concerning your ScreenScraper login, it has to look exactly like this:

[screenscraper]
userCreds="USER:PASS"

If you type it in correctly with a : between the username and password, it will work. :)

peligwe

Trying to scrape Philip's CD-l. Would I need to under mame?

muldjord

@peligwe I'm not too knowledgable about that platform. But it seems to be lr-mess, which uses the mame naming convention I think. So maybe. :)

stevas

@muldjord

Gaaaaaaaaaaaah! Thanks, I did see what I'd done wrong once I dug into the config.ini!

As for the password, I do have the : in there, yeah. But it still seemed to be giving me messages at the start about it being an anonymous account. Lemme try it again this week sometime...