RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login
    Please do not post a support request without first reading and following the advice in https://retropie.org.uk/forum/topic/3/read-this-first

    Scraping no longer working due to old api

    Scheduled Pinned Locked Moved Help and Support
    retropiescraperthegamesdbapidevelopment
    90 Posts 21 Posters 27.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • mituM
      mitu Global Moderator @BuZz
      last edited by

      @BuZz I have something working with screenscraper.fr. Their API is a bit similar to the TheGamesDB's API, so it's simple to re-use the existing code and add a new scraper. It just needs more testing and I can submit a PR for review.

      N BuZzB 2 Replies Last reply Reply Quote 5
      • N
        naddel81
        last edited by

        being able to scrape games manually is crucial for the roms that have translations etc. and are not found through automatic scraping. I am currently missing several covers due to this on every system. would really appreciate the inbuilt scraper to be working again for its manual approach! that would be awesome!

        1 Reply Last reply Reply Quote 1
        • SmeltieS
          Smeltie @MosaicDragon
          last edited by

          @MosaicDragon i tried skyscraper but no matter what i try the output keeps outputting a compiled image consisting of the cover, screenshot and the wheel.
          I have tried the different included xml files but those all give error’s so i don’t know what now 😅.
          I’d just like to have covers only just like the built in scraper.

          mituM 1 Reply Last reply Reply Quote 0
          • mituM
            mitu Global Moderator @Smeltie
            last edited by

            @Smeltie You could have asked in the Skyscraper topic and I'm sure somebody would have helped. Have you looked in the documentation to see how the artwork.xml works ?

            SmeltieS 1 Reply Last reply Reply Quote 0
            • SmeltieS
              Smeltie @mitu
              last edited by

              @mitu have tried.
              To anyone still looking i tried a tool called "Skraper"
              And it works phenominally.
              it can really do it all, from combined art, to video's to 2d boxart it does it all.
              i highly reccomend everyone try it.
              i found it to work best when running on a windows computer with the configs and roms folders shared and added as drives under windows 10.

              1 Reply Last reply Reply Quote 1
              • N
                naddel81
                last edited by naddel81

                but skraper can't do it manually for missing games. :(

                1 Reply Last reply Reply Quote 0
                • N
                  naddel81 @mitu
                  last edited by

                  @mitu said in Scraping no longer working due to old api:

                  @BuZz I have something working with screenscraper.fr. Their API is a bit similar to the TheGamesDB's API, so it's simple to re-use the existing code and add a new scraper. It just needs more testing and I can submit a PR for review.

                  that would be awesome!

                  1 Reply Last reply Reply Quote 0
                  • BuZzB
                    BuZz administrators @mitu
                    last edited by

                    @mitu sounds great thanks. I think recalbox includes support for screenscraper.fr in their ES fork - might be useful for reference.

                    To help us help you - please make sure you read the sticky topics before posting - https://retropie.org.uk/forum/topic/3/read-this-first

                    1 Reply Last reply Reply Quote 1
                    • F
                      Failrunner
                      last edited by

                      Sweet it will be harder to use now. I haven't cursed at this os enough. I need to blow my voice screaming! 🤒

                      N 1 Reply Last reply Reply Quote 0
                      • N
                        naddel81 @Failrunner
                        last edited by

                        @Failrunner it will soon work again and maybe better than before with gamesDB.

                        1 Reply Last reply Reply Quote 0
                        • N
                          Nikorasu081 Banned
                          last edited by Nikorasu081

                          Can someone please make instructions on how to get the internal scraper "THEGAMEDB" working again? I have one game I need to scrape and I tried Skyscraper and Steven Selph's Scraper and it just didn't work out for me. I kept getting a bash error in the terminal with Skyscaper. With Steven Selph's Scraper I kept getting "It appears that thegamesdb.net isn't up. If your are sure it is use -skip_check to bypass this error. No sources available

                          O 1 Reply Last reply Reply Quote 0
                          • O
                            omen_peter @Nikorasu081
                            last edited by

                            @Nikorasu081 just look at the help and support section. Every other post today has been about scraping - in summary the in-built scraper isn't currently working, other alternatives are available, and there is a will it/won't it be fixed debate.

                            Personally, I'm surprised I've not seen it called "scrapping" today. Always laugh when I see that someone has tried to "scrap" games. At the end of the day fighting never solved anything.

                            Raspberry Pi3 Model B V1.2
                            16 GB Micro SD Card (OS), 4TB WD Elements Portable HDD (Roms)
                            5V 3.0A Power Supply
                            RetroPie V4.7.7, EmulationStation V2.9.6RP, RetroArch V1.8.8
                            1 x Sonnics i8 Mini Keyboard (Wireless), 4 x Wii U Pro Controllers (Bluetooth)

                            N 1 Reply Last reply Reply Quote 0
                            • N
                              Nikorasu081 Banned @omen_peter
                              last edited by

                              @omen_peter As you can see in my above post nothing worked for me.

                              1 Reply Last reply Reply Quote 0
                              • N
                                naddel81
                                last edited by

                                we have to wait for @mitu to fix the scraper. really looking forward to it.

                                SmeltieS 1 Reply Last reply Reply Quote 0
                                • SmeltieS
                                  Smeltie @naddel81
                                  last edited by

                                  This post is deleted!
                                  1 Reply Last reply Reply Quote 0
                                  • R
                                    retrogamer0897
                                    last edited by

                                    As simple mortal end user, I just wanted to add my 2 cents in and say that the work of developers is much appreciated!

                                    I also wanted to chime in and say that I also appreciate the built-in scraper for being able to manually search and add individual games. I usually use the Selph scraper to take care of the bulk, then manually add the ones it doesn't find.

                                    Thank you!

                                    1 Reply Last reply Reply Quote 0
                                    • TPRT
                                      TPR @BuZz
                                      last edited by

                                      @BuZz While I understand that the internal scraper may be far from the best tool, it is the only one that I've found that works (worked) fantastic for scraping just a single ROM that I've added to my list. As someone who is a bit less on the technical side, having this tool has been a great help in many situations. Even if it is not high on the priority list to fix, I for one would be greatly appreciative of it was somewhere on the list down the road. Thanks for all the time and hard work put into this!

                                      quicksilverQ 1 Reply Last reply Reply Quote 1
                                      • quicksilverQ
                                        quicksilver @TPR
                                        last edited by

                                        Since there seem to be a lot of people interested in this feature I would suggest waiting to see what mitu is able to come up with. He stated he is working on a fix to have the internal scraper switched to a different website. If that fix doesn't come then those that want it should probably consider putting together a bounty to get it fixed. This is a good way for those of us with no coding skills to motivate those who have the skills.

                                        mituM 1 Reply Last reply Reply Quote 0
                                        • mituM
                                          mitu Global Moderator @quicksilver
                                          last edited by mitu

                                          @quicksilver For now, I don't think a bounty is needed. A couple of patches have been posted on the ES RetroPie repository to enable scraping again - one of them is mine -, they're just waiting for review and merge. I avoided adding an update on the forum so that the github page wouldn't be paged with unnecessary 'please fix/me too' messages.

                                          quicksilverQ R 2 Replies Last reply Reply Quote 7
                                          • quicksilverQ
                                            quicksilver @mitu
                                            last edited by

                                            @mitu That is awesome! Thanks for your hard work on this and all your troubleshooting support as well!

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.