RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login

    Versatile C++ game scraper: Skyscraper

    Scheduled Pinned Locked Moved Ideas and Development
    skyscraperscrapergamelist.xmlscrapinggithub
    1.6k Posts 113 Posters 1.6m Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • RionR
      Rion @SammyBoy
      last edited by

      @sammyboy I think you can already scrape fba roms using the Arcade parameter because it's already incorporated by using adb.arcadeitalia.net?

      FBNeo rom filtering
      Mame2003 Arcade Bezels
      Fba Arcade Bezels
      Fba NeoGeo Bezels

      1 Reply Last reply Reply Quote 1
      • muldjordM
        muldjord @SammyBoy
        last edited by muldjord

        @sammyboy The artwork quality completely depends on what the scraping sources deliver. So I can't give you a clear answer to that as I am not the one creating them. I am planning a bit of a rewrite of the artwork part of Skyscraper, to allow for the largest possible artwork for those who needs it. Currently I resize them to save space, but I plan to allow users to disable this. Let me know what your thoughts are on this. Concerning 'fba' I will include it in 2.2.0. But as Rion suggests, it's already possible to scrape them with 'arcade'. But since you request it, of course I will implement it. :)

        To all of you, 2.2.0 is just around the corner. Some big stuff in this release. Most importantly Skyscraper now also scrapes for 'wheel' and 'marquee' artwork. Not only that, but I also rewrote the artwork handler completely. A byproduct of this means you will be able to save the artwork directly "as is", completely disabling the artwork compositor. This has been requested quite a few times, so I hope this will be appreciated. It has been one helluwa rewrite. :D

        Stay tuned... Need to test stuff, but it's almost ready for release.

        S 1 Reply Last reply Reply Quote 1
        • S
          SammyBoy @muldjord
          last edited by

          @muldjord Thank you for choosing to add FB Alpha and I will try Skyscraper out today and I know that it will be good. And one last question are you a able to choose what artwork gets scraped? And thank you for hard work and effort.

          muldjordM 1 Reply Last reply Reply Quote 0
          • muldjordM
            muldjord @SammyBoy
            last edited by

            @sammyboy Currently you can only enable/disable video scraping. All artwork types(cover, screenshot, wheel and marquee) will be scraped if it's available from the scraping source. I will try and find a way to enable/disable them individually.

            S 1 Reply Last reply Reply Quote 0
            • S
              SammyBoy @muldjord
              last edited by

              @muldjord Thank you for all your help.

              1 Reply Last reply Reply Quote 1
              • muldjordM
                muldjord
                last edited by muldjord

                NOTE!!! The new version is not out yet! But below follows some notes I wanted to throw out there before release.

                The new image compositor is now pretty much done, and I have to say, it's pretty awesome! Not only do Skyscraper now support wheel and marquee, but it also allows you to set up the output images EXACTLY the way you want it. It will use a format like this:

                <?xml version="1.0" encoding="UTF-8"?>
                <artwork>
                  <!--
                      <output>
                      - Leave out 'width' and 'height' to use dimensions of first image resource
                  -->
                  <output type="screenshot" width="640" height="400">
                    <!--
                	<image>
                	- Leave out 'width' or 'height' to keep aspect ratio.
                	- Leave out both 'width' and 'height' to use original size.
                	- 'align' can be 'left', 'center' or 'right'
                	- 'valign' can be 'top', 'middle' or 'bottom'
                	- 'x' and 'y' defines top-left coordinate. It is relative to 'align' and 'valign' if used.
                	- Image nodes are rendered from bottom to top.
                    -->
                    <image resource="cover" valign="bottom" y="-15" height="200">
                      <shadow distance="10" softness="10" opacity="75"/>
                    </image>
                    <image resource="screenshot" align="right" valign="top" width="520" height="390"/>
                  </output>
                  <!-- If an output node contains no image nodes, it will save it exactly as it was delivered by the scraping source. -->
                  <output type="cover"/>
                  <output type="wheel"/>
                  <output type="marquee"/>
                </artwork>
                

                As you can see there are 4 different artwork outputs in this particular example. It will output 'cover', 'wheel' and 'marquee' exactly as they are delivered by the scraping source. 'screenshot' is composited of the 'screenshot' with a 'cover' with a drop shadow on top.

                If an '<output>' node for any of the types ('cover', 'screenshot', 'wheel', 'marquee') is left out, it will simply not be exported while scraping. All data is still saved in the localdb cache, so you can change the artwork.xml any time you wish, and it will use all of the resources you have scraped and cached.

                Any thoughts on this? It's a huge step from the old config.ini way of doing this I think.

                RionR 1 Reply Last reply Reply Quote 1
                • RionR
                  Rion @muldjord
                  last edited by Rion

                  @muldjord Great idé, i like it!

                  (Vilken jävla bra idé! Jag vill nästan gå så långt och säga att jag blir lite mörkrädd. Så bra var det!)

                  FBNeo rom filtering
                  Mame2003 Arcade Bezels
                  Fba Arcade Bezels
                  Fba NeoGeo Bezels

                  muldjordM 1 Reply Last reply Reply Quote 1
                  • muldjordM
                    muldjord @Rion
                    last edited by

                    @rion Mörkrädd!? :D Jeg håber ikke det bliver et horror release. :P

                    ETA for 2.2.0 won't be long - probably a week at the most. I just need to polish some rough edges and do a bunch of testing. Also need to document the new artwork functionality. :)

                    1 Reply Last reply Reply Quote 1
                    • muldjordM
                      muldjord
                      last edited by

                      So, I am pretty much ready to release Skyscraper 2.2.0. This is the biggest release since 2.0.0!

                      But, I am currently trying to figure out what pieces of artwork EmulationStation supports in the gamelist.xml files. I know about 'image' and 'video' but I can't seem to find proper documentation for whether it supports 'boxart', 'wheel' and 'marquee'. Do any of you guys know what xml nodes are needed for these? And are they supported at all by EmulationStation?

                      1 Reply Last reply Reply Quote 0
                      • AnalogHeroA
                        AnalogHero
                        last edited by

                        @muldjord Dont know really. Depends on the theme used i guess.

                        Quick question: Can you now compose the image lets say with screenshot, boxart and marquee or wheel?

                        muldjordM 1 Reply Last reply Reply Quote 0
                        • muldjordM
                          muldjord @AnalogHero
                          last edited by muldjord

                          @analoghero Yes, and you can do it any way you like. With as many layers as you want. You can even create the wheel graphics from cover and screenshot if you want (don't know why anyone would want that, but it's possible) :) It's one very simple xml file that needs to be edited and you are good to go.

                          Concerning the gamelist.xml types, I would assume the themes use them which means they would probably be listed in gamelist.xml somehow. So my question stands. There must be nodes for them like there is an <image> and <video> node. I just don't know what they are... Might be <wheel> or something...

                          EDIT: Ok, so I finally found some thread that mentioned a <marquee> gamelist.xml node. And to my surprise it seems that "wheel" and "marquee" are somewhat the same thing... I find it to be quite confusing.

                          EDIT2: I've decided to use the 'wheel' graphics and insert it into the '<marquee>' node in the EmulationStation gamelist.xml. Makes most sense to me.

                          S 1 Reply Last reply Reply Quote 1
                          • S
                            SammyBoy @muldjord
                            last edited by

                            @muldjord is skyscraper 2.2.0 out?

                            S muldjordM 2 Replies Last reply Reply Quote 0
                            • S
                              SammyBoy @SammyBoy
                              last edited by

                              @sammyboy I'm only asking because I was just downloading skyscraper when I realised that you updated the github files 30mins ago but not the download instructions.

                              1 Reply Last reply Reply Quote 0
                              • muldjordM
                                muldjord @SammyBoy
                                last edited by muldjord

                                @sammyboy Not yet, but almost... Currently trying to do a quick and dirty video on some of the new features.

                                It will be released in 20 minutes, at midnight where I am.

                                1 Reply Last reply Reply Quote 0
                                • S
                                  SammyBoy
                                  last edited by

                                  If I download 2.1.1 how will I update the scraper?

                                  muldjordM 1 Reply Last reply Reply Quote 0
                                  • muldjordM
                                    muldjord @SammyBoy
                                    last edited by muldjord

                                    @sammyboy Wait 20 minutes and get 2.2.0 instead. :)

                                    EDIT: Ah fuck it, I'll take a break from video making and release it now! :D

                                    1 Reply Last reply Reply Quote 0
                                    • S
                                      SammyBoy
                                      last edited by

                                      @muldjord Thank you so much! But can you still tell me how to update the scraper after you finish releasing the scraper. And do you have a YouTube channel?

                                      muldjordM 1 Reply Last reply Reply Quote 0
                                      • muldjordM
                                        muldjord @SammyBoy
                                        last edited by

                                        @sammyboy You can't update it currently, just follow the installation instructions again after I've released it. The new instructions will include link to the 2.2.0 version.

                                        S 1 Reply Last reply Reply Quote 0
                                        • S
                                          SammyBoy @muldjord
                                          last edited by

                                          @muldjord Thank you for your hard work and effort. But still YouTube channel?

                                          1 Reply Last reply Reply Quote 0
                                          • muldjordM
                                            muldjord
                                            last edited by muldjord

                                            Skyscraper version 2.2.0 released! And it's a big one!!! https://github.com/muldjord/skyscraper

                                            • Completely rewrote how artwork is handled. Now uses '~/.skyscraper/artwork.xml' for everything.
                                            • Implemented 'wheel' and 'marquee' media types and rewrote EVERYTHING to make use of them. :D (this was a damn long rewrite!!!!)
                                            • Added 'wheel' and 'marquee' media types to local dabatase
                                            • Added 'nocovers', 'noscreenshots', 'nowheels' and 'nomarquees' options to allow user to disable caching of those resources locally.
                                            • Added 'noresize' option to disable resizing artwork resources when saving them to localdb. NOTE! This it NOT directly related to how artwork is rendered. Check the artwork documentation if you want to disable them when exporting the frontend artwork.
                                            • Added support for 'fba' platform

                                            Check this quick and dirty video for a rundown of most of the new features:

                                            1 Reply Last reply Reply Quote 5
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.