RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login

    Scraper optimization

    Scheduled Pinned Locked Moved Ideas and Development
    scrap
    25 Posts 9 Posters 15.9k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Z
      Zigurana @Rion
      last edited by

      @Rion See here for a list of people who have created a fork of ES. Apparently its very difficult on Github to get a full list of forks, although you can seen them all when you actually want to do comparisons for pull-requests.

      I guess the point right now is not that there are no people willing to work on it, but more who will stand up to draw up a comprehensive action plan with a long term vision for ES, and is capable of pulling that particular cart for the foreseeable future?
      It will need to show these things before RetroPie can even consider switching upstream branches.

      The good thing about Herdingers project is that at least he/she is willing to publictly discuss the long-term goals. See here.

      If tetris has thought me anything, it's that errors pile up and that accomplishments dissappear.

      RionR 1 Reply Last reply Reply Quote 1
      • Z
        Zigurana @BuZz
        last edited by

        @BuZz @meleu
        I fully agree with that sentiment, if you want something different, its primarily up to you to educate yourself to do it.

        If tetris has thought me anything, it's that errors pile up and that accomplishments dissappear.

        1 Reply Last reply Reply Quote 0
        • RionR
          Rion @Zigurana
          last edited by

          @Zigurana said:

          @Rion See here for a list of people who have created a fork of ES. Apparently its very difficult on Github to get a full list of forks, although you can seen them all when you actually want to do comparisons for pull-requests.

          I guess the point right now is not that there are no people willing to work on it, but more who will stand up to draw up a comprehensive action plan with a long term vision for ES, and is capable of pulling that particular cart for the foreseeable future?
          It will need to show these things before RetroPie can even consider switching upstream branches.

          The good thing about Herdingers project is that at least he/she is willing to publictly discuss the long-term goals. See here.

          Wow that's more then i could imagine.
          But i see your point. To many forks and not many working together.

          It's a shame really.

          FBNeo rom filtering
          Mame2003 Arcade Bezels
          Fba Arcade Bezels
          Fba NeoGeo Bezels

          1 Reply Last reply Reply Quote 0
          • T
            Twitch0815
            last edited by

            Even other forks for example Recalbox scraping is more of a pain in the but then it is on Retropie.
            They do the same thing .fullscrape

            Until someone puts a massive amount of time into developing a gui wrapper for a scraper service, the better option is always going to be a command line scraper. And that is assuming thegaemesdb.net stays up. There are just some things cli clients do better.

            We need to keep ask's in perspective, we as a community put a huge ask on thegamesdb but their donations are not reflective of the demand we as a community put on them and the demand for their service is literally crushing their servers and we get vocal about our dislike of that feature being unavailable regardless of whether we have ever donated or making vocal and somewhat alarmist replace thegamesdb develop an alternative switch or code something better. Additionally asking the limited developers we have to focus on the one demand we have a working solution for its just a few more manual steps to get it working takes them away from kernel improvements new ports, features etc etc.

            Honestly as long as gamesdb.net scraping artwork with sspeh is a breeze and there is only one other frontend period I have seen do a better job easier with artwork and it is a mac OSX frontend. Mess with Hyperspin and emumovies accounts paid subscriptios download rate limits having to edit xml by hand renaming romsets its a pain in the but.

            And honestly its not going to be very long before Attract Mode or Meh replace Emulation Station as the default front-end for Retropie or alternatives at least and the artwork structure for those is going to be manual and manual for a good while. Luckily the Hyperspin and EmuMovies community have already done that heavy lifting though were going to have to deal with getting emumovie accounts and all the hoops that comes with that but the end product will look great. I would much rather the devs look forward or we put the ask to the meh and attract mode developers to make the art scraping and organizing process as user friendly as possible and let our devs keep focusing on what they want to work on as well as the good ideas we come forward with but we as a community need to be fair and not ask for major development on a platform that will be irrelevant in coming months.

            Z FinheadF 2 Replies Last reply Reply Quote 1
            • Z
              Zigurana @Twitch0815
              last edited by

              @Twitch0815 said:

              ...
              And honestly its not going to be very long before Attract Mode or Meh replace Emulation Station as the default front-end for Retropie or alternatives at least and the artwork structure for those is going to be manual and manual for a good while.
              ...
              I would much rather the devs look forward or we put the ask to the meh and attract mode developers to make the art scraping and organizing process as user friendly as possible and let our devs keep focusing on what they want to work on as well as the good ideas we come forward with but we as a community need to be fair and not ask for major development on a platform that will be irrelevant in coming months.

              Do you know these things, or are you speculating? I think there has been a lot of investments into ES, and there is a certain amount of momentum that needs to build up before any switch will be worth the hassle. I agree that we need to keep looking outside our bubble, and also not over-work the few developers we have, but switching to a completely new front-end is a large step.

              If tetris has thought me anything, it's that errors pile up and that accomplishments dissappear.

              1 Reply Last reply Reply Quote 1
              • FinheadF
                Finhead @Twitch0815
                last edited by

                @Twitch0815 said:

                Even other forks for example Recalbox scraping is more of a pain in the but then it is on Retropie.
                They do the same thing .fullscrape

                Until someone puts a massive amount of time into developing a gui wrapper for a scraper service, the better option is always going to be a command line scraper. And that is assuming thegaemesdb.net stays up. There are just some things cli clients do better.

                We need to keep ask's in perspective, we as a community put a huge ask on thegamesdb but their donations are not reflective of the demand we as a community put on them and the demand for their service is literally crushing their servers and we get vocal about our dislike of that feature being unavailable regardless of whether we have ever donated or making vocal and somewhat alarmist replace thegamesdb develop an alternative switch or code something better. Additionally asking the limited developers we have to focus on the one demand we have a working solution for its just a few more manual steps to get it working takes them away from kernel improvements new ports, features etc etc.

                Honestly as long as gamesdb.net scraping artwork with sspeh is a breeze and there is only one other frontend period I have seen do a better job easier with artwork and it is a mac OSX frontend. Mess with Hyperspin and emumovies accounts paid subscriptios download rate limits having to edit xml by hand renaming romsets its a pain in the but.

                And honestly its not going to be very long before Attract Mode or Meh replace Emulation Station as the default front-end for Retropie or alternatives at least and the artwork structure for those is going to be manual and manual for a good while. Luckily the Hyperspin and EmuMovies community have already done that heavy lifting though were going to have to deal with getting emumovie accounts and all the hoops that comes with that but the end product will look great. I would much rather the devs look forward or we put the ask to the meh and attract mode developers to make the art scraping and organizing process as user friendly as possible and let our devs keep focusing on what they want to work on as well as the good ideas we come forward with but we as a community need to be fair and not ask for major development on a platform that will be irrelevant in coming months.

                Nothing to do with the topic really, but there are no hoops to jump through for artwork on AM. You can get everything you need Right Here no membership etc. Then you place them in the correct folders, it's a good alternative to the scraper since it's not down.

                1 Reply Last reply Reply Quote 0
                • meleuM
                  meleu
                  last edited by

                  OK @BuZz , I agree that the fun is a pre-requisite. It was just a suggestion.

                  Going back to the topic (scraper optimization): Is there an automated way to backup/copy my scraps and put it on another fresh installed RetroPie?

                  If there isn't, I think it would be great.

                  • Useful topics
                  • joystick-selection tool
                  • rpie-art tool
                  • achievements I made
                  herb_fargusH 1 Reply Last reply Reply Quote 0
                  • herb_fargusH
                    herb_fargus administrators @meleu
                    last edited by

                    @meleu no automated way but you can easily copy the folders from samba shares to back up you games and scrapes and then just copy them over or you can create a backup image of your SD card altogether.

                    If you read the documentation it will answer 99% of your questions: https://retropie.org.uk/docs/

                    Also if you want a solution to your problems read this first: https://retropie.org.uk/forum/topic/3/read-this-first

                    meleuM 1 Reply Last reply Reply Quote 1
                    • meleuM
                      meleu @herb_fargus
                      last edited by

                      @herb_fargus yep, but the files are a little scattered.

                      I think I can do this, but I'm just suggesting an easy way for the n00bs.

                      • Useful topics
                      • joystick-selection tool
                      • rpie-art tool
                      • achievements I made
                      1 Reply Last reply Reply Quote 0
                      • BuZzB
                        BuZz administrators
                        last edited by

                        can also do it via a usb stick which can backup/restore configs.

                        To help us help you - please make sure you read the sticky topics before posting - https://retropie.org.uk/forum/topic/3/read-this-first

                        1 Reply Last reply Reply Quote 1
                        • First post
                          Last post

                        Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                        Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.