Versatile C++ game scraper: Skyscraper

SammyBoy

@muldjord Thank you for all your help.

muldjord

NOTE!!! The new version is not out yet! But below follows some notes I wanted to throw out there before release.

The new image compositor is now pretty much done, and I have to say, it's pretty awesome! Not only do Skyscraper now support wheel and marquee, but it also allows you to set up the output images EXACTLY the way you want it. It will use a format like this:

<?xml version="1.0" encoding="UTF-8"?>
<artwork>
  <!--
      <output>
      - Leave out 'width' and 'height' to use dimensions of first image resource
  -->
  <output type="screenshot" width="640" height="400">
    <!--
	<image>
	- Leave out 'width' or 'height' to keep aspect ratio.
	- Leave out both 'width' and 'height' to use original size.
	- 'align' can be 'left', 'center' or 'right'
	- 'valign' can be 'top', 'middle' or 'bottom'
	- 'x' and 'y' defines top-left coordinate. It is relative to 'align' and 'valign' if used.
	- Image nodes are rendered from bottom to top.
    -->
    <image resource="cover" valign="bottom" y="-15" height="200">
      <shadow distance="10" softness="10" opacity="75"/>
    </image>
    <image resource="screenshot" align="right" valign="top" width="520" height="390"/>
  </output>
  <!-- If an output node contains no image nodes, it will save it exactly as it was delivered by the scraping source. -->
  <output type="cover"/>
  <output type="wheel"/>
  <output type="marquee"/>
</artwork>

As you can see there are 4 different artwork outputs in this particular example. It will output 'cover', 'wheel' and 'marquee' exactly as they are delivered by the scraping source. 'screenshot' is composited of the 'screenshot' with a 'cover' with a drop shadow on top.

If an '<output>' node for any of the types ('cover', 'screenshot', 'wheel', 'marquee') is left out, it will simply not be exported while scraping. All data is still saved in the localdb cache, so you can change the artwork.xml any time you wish, and it will use all of the resources you have scraped and cached.

Any thoughts on this? It's a huge step from the old config.ini way of doing this I think.

Rion

@muldjord Great idé, i like it!

(Vilken jävla bra idé! Jag vill nästan gå så långt och säga att jag blir lite mörkrädd. Så bra var det!)

muldjord

@rion Mörkrädd!? :D Jeg håber ikke det bliver et horror release. :P

ETA for 2.2.0 won't be long - probably a week at the most. I just need to polish some rough edges and do a bunch of testing. Also need to document the new artwork functionality. :)

muldjord

So, I am pretty much ready to release Skyscraper 2.2.0. This is the biggest release since 2.0.0!

But, I am currently trying to figure out what pieces of artwork EmulationStation supports in the gamelist.xml files. I know about 'image' and 'video' but I can't seem to find proper documentation for whether it supports 'boxart', 'wheel' and 'marquee'. Do any of you guys know what xml nodes are needed for these? And are they supported at all by EmulationStation?

AnalogHero

@muldjord Dont know really. Depends on the theme used i guess.

Quick question: Can you now compose the image lets say with screenshot, boxart and marquee or wheel?

muldjord

@analoghero Yes, and you can do it any way you like. With as many layers as you want. You can even create the wheel graphics from cover and screenshot if you want (don't know why anyone would want that, but it's possible) :) It's one very simple xml file that needs to be edited and you are good to go.

Concerning the gamelist.xml types, I would assume the themes use them which means they would probably be listed in gamelist.xml somehow. So my question stands. There must be nodes for them like there is an <image> and <video> node. I just don't know what they are... Might be <wheel> or something...

EDIT: Ok, so I finally found some thread that mentioned a <marquee> gamelist.xml node. And to my surprise it seems that "wheel" and "marquee" are somewhat the same thing... I find it to be quite confusing.

EDIT2: I've decided to use the 'wheel' graphics and insert it into the '<marquee>' node in the EmulationStation gamelist.xml. Makes most sense to me.

SammyBoy

@muldjord is skyscraper 2.2.0 out?

SammyBoy

@sammyboy I'm only asking because I was just downloading skyscraper when I realised that you updated the github files 30mins ago but not the download instructions.

muldjord

@sammyboy Not yet, but almost... Currently trying to do a quick and dirty video on some of the new features.

It will be released in 20 minutes, at midnight where I am.

SammyBoy

If I download 2.1.1 how will I update the scraper?

muldjord

@sammyboy Wait 20 minutes and get 2.2.0 instead. :)

EDIT: Ah fuck it, I'll take a break from video making and release it now! :D

SammyBoy

@muldjord Thank you so much! But can you still tell me how to update the scraper after you finish releasing the scraper. And do you have a YouTube channel?

muldjord

@sammyboy You can't update it currently, just follow the installation instructions again after I've released it. The new instructions will include link to the 2.2.0 version.

SammyBoy

@muldjord Thank you for your hard work and effort. But still YouTube channel?

muldjord

Skyscraper version 2.2.0 released! And it's a big one!!! https://github.com/muldjord/skyscraper

Completely rewrote how artwork is handled. Now uses '~/.skyscraper/artwork.xml' for everything.
Implemented 'wheel' and 'marquee' media types and rewrote EVERYTHING to make use of them. :D (this was a damn long rewrite!!!!)
Added 'wheel' and 'marquee' media types to local dabatase
Added 'nocovers', 'noscreenshots', 'nowheels' and 'nomarquees' options to allow user to disable caching of those resources locally.
Added 'noresize' option to disable resizing artwork resources when saving them to localdb. NOTE! This it NOT directly related to how artwork is rendered. Check the artwork documentation if you want to disable them when exporting the frontend artwork.
Added support for 'fba' platform

Check this quick and dirty video for a rundown of most of the new features:

AnalogHero

@muldjord Thanks for the update. Played around with artwork.xml but couldnt get it to work.

What am i doing wrong? Try to get the Screenshot in the middle, cover to the left as it was before and wheel to the bottom right.

<?xml version="1.0" encoding="UTF-8"?>
<!-- This is the default 'artwork.xml' file that exports a screenshot composited from the cover and screenshot artwork and the wheel file. Please check 'artwork.xml.example' for a more thorough example of the possibilites. Also be sure to check the full artwork documentation at https://github.com/muldjord/skyscraper/blob/master/ARTWORK.md -->
<artwork>
  <output type="screenshot" width="640" height="400">
    <layer resource="cover" valign="bottom" y="-15" height="200">
      <shadow distance="10" softness="10" opacity="75"/>
        <layer resource="wheel" valign="bottom" align="right" x="5" y="5">
           <shadow distance="10" softness="10" opacity="75"/>
    </layer>
    <layer resource="screenshot" align="right" valign="top" width="520" height="390"/>
  </output>
  <!-- delete the next line to disable 'wheel entirely -->
  <output type="wheel" width="350"/>
  <!-- delete the next line to disable 'cover' entirely -->
  <output type="cover" height="390"/>
  <!-- delete the next line to disable 'marquee' entirely -->
  <output type="marquee" width="350"/>
</artwork>

muldjord

@analoghero Your xml is a bit messed up. You're missing a </layer> node after the first shadow node. You're basically nesting a layer inside another layer, which isn't possible.

Also, you have x="5" and y="5" on your second layer. That would move it out of frame. The coordinates are measured from top-left. So if you want to move it upwards to the left, you need to have x="-5" and y="-5" instead.

Here, I've corrected it for you. And I've also disabled shadow for the wheel since it doesn't work well with png's that have transparency at the moment (it works, but it looks like shite. I will fix this in the next release). I've centered the screenshot and I've also removed the last output nodes. There's no need for them if you don't use them. All they'll do is take up space.

<?xml version="1.0" encoding="UTF-8"?>
<artwork>
  <output type="screenshot" width="640" height="400">
    <layer resource="cover" valign="bottom" y="-15" height="200">
      <shadow distance="10" softness="10" opacity="75"/>
    </layer
    <layer resource="wheel" valign="bottom" align="right" x="-5" y="-5"/>
    <layer resource="screenshot" align="center" valign="top" width="520" height="390"/>
  </output>
</artwork>

SammyBoy

@muldjord Thank you for the update I have small favour to ask and that is if you can allow pc roms to be scraped.

muldjord

@sammyboy Sure, it will be in the next release :)