RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login

    Skyscraper now officially part of RetroPie, please test

    Scheduled Pinned Locked Moved Ideas and Development
    skyscraperscraperretropie-setupoptionalexperimental
    275 Posts 28 Posters 109.7k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • muldjordM
      muldjord @Silent
      last edited by

      @Silent For your examples, why not just use the command line? I don't see how a free text field would provide a user friendly way of running commands, quite the contrary. You don't have tab completion in the GUI, so you'd have to know the exact filename to do what you want (and not do any spelling errors). And for that you'd need to look them up on the command line beforehand. In which case you could just run the command there. If the user knows about these features you describe, they have already checked the output of --help where the commands are listed. So they even know how to run the executable from the correct location. So yeah, I don't think this makes much sense. I'd much rather put in some better help text telling people how to actually run it from command line. I mean, you've already exited EmulationStation to even open up the GUI. So the command line is right there.

      S 1 Reply Last reply Reply Quote 1
      • S
        spurlingD @muldjord
        last edited by

        @muldjord i have typed Skyscraper and it say command not found

        mituM 1 Reply Last reply Reply Quote 0
        • mituM
          mitu Global Moderator @spurlingD
          last edited by

          @spurlingD If you installed Skyscraper using the RetroPie scriptmodule, then the command is located in /opt/retropie/supplementary/skyscraper/Skyscraper. If you add the installation folder to your $PATH, then you don't have to type the full path to the command.

          S 1 Reply Last reply Reply Quote 0
          • S
            Silent @muldjord
            last edited by Silent

            @muldjord said in Skyscraper now officially part of RetroPie, please test:

            For your examples, why not just use the command line?

            Figured I'd post the way I "solved" my personal gripe I had with this approach:

            I never use "Use ROM folders" so I could just add those to config.ini as defaults:

            inputFolder="/home/pi/RetroPie/roms"
            gamelistFolder="/home/pi/.emulationstation/gamelists"
            mediaFolder="/home/pi/.emulationstation/downloaded_media"
            

            and now I don't need to provide those as -o and -g arguments, like RetroPie-Setup script does.

            Tested it when scraping for a single ROM and it worked flawlessly!

            EDIT:
            In setup like this, I hope that commandline parameters take precedence over what is in INI file? Just in case RetroPie-Setup ever provides paths different to these.

            EDIT2:
            Does --pretend imply --refresh? When scraping for single ROMs it's always redownloading from the website, bypassing the cache.

            1 Reply Last reply Reply Quote 0
            • S
              spurlingD @mitu
              last edited by

              @mitu how do i get it show it not in text from but the menu

              mituM 1 Reply Last reply Reply Quote 0
              • mituM
                mitu Global Moderator @spurlingD
                last edited by

                @spurlingD I don't understand what you mean. Can you give an example ?

                S 1 Reply Last reply Reply Quote 0
                • S
                  spurlingD @mitu
                  last edited by

                  @mitu like this 1539341719679-2d4d68da-bea6-4f9c-a44f-bee87d3d5ecd-image.png

                  mituM 1 Reply Last reply Reply Quote 0
                  • mituM
                    mitu Global Moderator @spurlingD
                    last edited by

                    @spurlingD Have you read the docs ? It's explained in there.

                    1 Reply Last reply Reply Quote 0
                    • B
                      brimby
                      last edited by brimby

                      I'm trying to scrape igdb, but it keeps rejecting my api key. I'm pasting the key directly in (not trying to type it out). Here is my command line entry:

                      /opt/retropie/supplementary/skyscraper/Skyscraper --unattend --skipped -g /home/pi/.emulationstation/gamelists/n64 -o /home/pi/.emulationstation/downloaded_media/n64 -p n64 -s igdb -u MYPASTEDAPIKEY --videos --nobrackets

                      Does anything look wrong with that line? The error message is: "The key you provided with either the '-u [key]' command line option or the 'userCreds="[key]" variable to config.ini does not seem to work. Now quitting..."

                      I'm on version 2.9.5

                      Update: I just tried generating a new key, but that one didn't work either.

                      1 Reply Last reply Reply Quote 0
                      • F
                        fuzznuzzbit
                        last edited by fuzznuzzbit

                        Would it be easy (or is there a way) to make a configuration to move the location of the cache? I run everything off of USB - roms, gamelist, etcetera, and would love have the Skyscraper cache and db in a folder on the usb as well.

                        Edit: Nevermind, it was as easy as editing the .ini file! Really like this application!

                        muldjordM 1 Reply Last reply Reply Quote 0
                        • muldjordM
                          muldjord @fuzznuzzbit
                          last edited by

                          @fuzznuzzbit Glad you found the solution and like the software :)

                          And happy new year everyone!

                          1 Reply Last reply Reply Quote 0
                          • muldjordM
                            muldjord
                            last edited by

                            @mitu In preparation for the 3.0.0 release of Skyscraper, it would be nice if any scraping ended with a dialog that said "All resources cached. Once you've gathered all the data you need, please remember to scrape again with the localdb module which will generate the game list" or similar. And obviously, when scraping with localdb it should say "Scraping complete, game list has been generated" or similar.

                            mituM 1 Reply Last reply Reply Quote 0
                            • mituM
                              mitu Global Moderator @muldjord
                              last edited by mitu

                              @muldjord when do you intend to release it ? The Wiki would also need to be updated.

                              muldjordM 1 Reply Last reply Reply Quote 0
                              • muldjordM
                                muldjord @mitu
                                last edited by

                                @mitu Hard to give an estimate as I am currently battling IGDB's unstable servers to integrate their new API. And I also have some stuff with cache entry editing I'd like to get in. So... Maybe 2 weeks? Rough estimate.

                                mituM 1 Reply Last reply Reply Quote 0
                                • mituM
                                  mitu Global Moderator @muldjord
                                  last edited by

                                  @muldjord Ok, then I should do some testing in the week-end with your current 3.0.0.
                                  I do have a question regarding the changes in 3.x.
                                  The new modus operandi will be that online scraping would only add resources to the cache, without modifying the users' gamelist. So let's say I scrape from X, Y, Z, etc.. and then with localdb - resulting in the gamelist.xml being updated (or whatever the AttractMode equivalent is).
                                  What happens if I do

                                  • scrape with X, then with localdb
                                  • scrape with Y, then with localdb.
                                    etc.
                                    Would I get the same results as before ?
                                  muldjordM 1 Reply Last reply Reply Quote 0
                                  • muldjordM
                                    muldjord @mitu
                                    last edited by muldjord

                                    @mitu Any module other than 'localdb' will only add data to the cache. So whenever the user wants a game list generated, they need to use 'localdb'. It's the gather and combine phases I am trying to clarify to the users. There's a lot of confusion as it is right now. Many users think they should just scrape with one module and then stop. But that was never the intention of course.

                                    So the proper way from 3.x forward is, as you describe, to do Z, Y, X or whatever order you want of the non-localdb sources. Then when you feel like you have all the data you need cached, you will do one last run with 'localdb' which generated the game list from all cached data. You can always go back and do Z, Y again if you want to, as long as you remember to add --refresh. And then, again, do one final run with localdb.

                                    EDIT: Actually, I think I'll start referring to it as the gather and generate phases instead. Makes more sense, right?

                                    mituM 1 Reply Last reply Reply Quote 0
                                    • mituM
                                      mitu Global Moderator @muldjord
                                      last edited by mitu

                                      @muldjord said in Skyscraper now officially part of RetroPie, please test:

                                      EDIT: Actually, I think I'll start referring to it as the gather and generate phases instead. Makes more sense, right?

                                      Well, it already made sense - for me - with the actual behavior :). The (CL)UI though refers to both as 'scraping', which I guess it's what you're trying to clarify more in the 3.x series (code and docs).

                                      muldjordM 1 Reply Last reply Reply Quote 0
                                      • muldjordM
                                        muldjord @mitu
                                        last edited by muldjord

                                        @mitu Yes, it would be cool to clarify it somehow. Perhaps move the localdb to the bottom of the list and add a title above it with "GENERATE GAMELIST:" and at the very top add a title with "GATHER RESOURCES:" or similar. the import and esgamelist modules and others should also be at the top to clearly remove them from the localdb module which is the one "GENERATE" module so to speak.

                                        EDIT: I think it would actually make sense to change this already pre-3.x as the process is the same. I've just forced it more in 3.x.

                                        mituM 1 Reply Last reply Reply Quote 0
                                        • goobatroopaG
                                          goobatroopa
                                          last edited by goobatroopa

                                          Used Skyscraper today for the first time. Great piece of software and worked like a charm. Thanks @muldjord

                                          muldjordM 1 Reply Last reply Reply Quote 1
                                          • muldjordM
                                            muldjord @goobatroopa
                                            last edited by

                                            @goobatroopa Awesome, thanks! Glad you like it :)

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.