• Recent
  • Tags
  • Popular
  • Home
  • Docs
  • Register
  • Login
RetroPie forum home
  • Recent
  • Tags
  • Popular
  • Home
  • Docs
  • Register
  • Login
Please do not post a support request without first reading and following the advice in https://retropie.org.uk/forum/topic/3/read-this-first

Scraping totally fails

Scheduled Pinned Locked Moved Help and Support
skyscraperscraperthegamesdbscreenscraper
17 Posts 9 Posters 2.8k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • B
    beppi @max
    last edited by beppi 10 Jun 2022, 19:42 6 Oct 2022, 18:40

    Thanks all folks, I'm really getting crazy.
    I tried to scrape using skraper, and it works. Only problem... it runs on Windows and I need it running on the raspberry. The only ption I see now is to scrape on Windows, create a script to change manually all the paths and informations that are missing and move everything manually, a huge work!
    Furthermore, also other scrapers fail and I'm really styuck in a problem that looks to me as a sci-fi movie... I made a lot of configurations on my raspi and I consider myself able to dive in deep in linux configurations using my beloved vi, but I fail here.
    Anything else I tried, unfortunately, failed, and now I'm trying to reach the author of Skyscraper on github, but I doubt I will be able to fix it.

    Q F 2 Replies Last reply 6 Oct 2022, 22:06 Reply Quote 0
    • Q
      Quackwalks @beppi
      last edited by 6 Oct 2022, 22:06

      @beppi What I do Is back up my roms directory into a windows folder. In Skraper, you can change the settings to use the windows directory and it'll still output game lists that work for retropie. I just put my game lists in my rom directories, then a folder in each system for the snaps and box arts. After you're done scraping, transfer your game lists and scraped media to your raspberry pi where they should go.

      Sorry I can't help with the details, but I know this method works with enough tinkering.

      B C 2 Replies Last reply 7 Oct 2022, 16:38 Reply Quote 1
      • B
        beppi @Quackwalks
        last edited by beppi 10 Jul 2022, 17:39 7 Oct 2022, 16:38

        @Quackwalks this is indeed the only solution i found so far, but it's not a complete solution because some information is missing, for example the information about the location of videos and screenshots, and Skyscraper was also creating a composition with the box for the screenshot that was integrated with the theme. Of course it's not a big deal but I would like to understand why Skyscraper cannot connect from raspi.
        I feel that I could fix this problem if I only had some kind of log, monitor, or anything else, do you know if there is a log somewhere? I could not find it.

        Y 1 Reply Last reply 8 Oct 2022, 15:36 Reply Quote 0
        • Y
          YFZdude @beppi
          last edited by YFZdude 10 Sept 2022, 00:28 8 Oct 2022, 15:36

          @beppi
          There should be logs anytime something fails to scrape with skyscraper in the /home/pi/.skyscraper folder.

          B 1 Reply Last reply 9 Oct 2022, 12:33 Reply Quote 0
          • C
            Crush @Quackwalks
            last edited by Crush 10 Aug 2022, 19:25 8 Oct 2022, 18:25

            @Quackwalks said in Scraping totally fails:

            @beppi What I do Is back up my roms directory into a windows folder. In Skraper, you can change the settings to use the windows directory and it'll still output game lists that work for retropie. I just put my game lists in my rom directories, then a folder in each system for the snaps and box arts. After you're done scraping, transfer your game lists and scraped media to your raspberry pi where they should go.

            Sorry I can't help with the details, but I know this method works with enough tinkering.

            Just make a network drive on windows out of \\retropie\roms so you can scrape using screenscraper on windows directly instead of having to transfer back and forth.

            Q 1 Reply Last reply 13 Oct 2022, 02:09 Reply Quote 0
            • B
              beppi @YFZdude
              last edited by beppi 10 Sept 2022, 13:36 9 Oct 2022, 12:33

              @YFZdude thank you, unfortunately there's nothing, because there's no error in Skyscraper: it just receives the information "not found", appearantly. I watched by capturing in real time written files both on .skyscraper directory, and on /var/log, and no log is being generated.

              B 1 Reply Last reply 9 Oct 2022, 12:37 Reply Quote 0
              • B
                beppi @beppi
                last edited by 9 Oct 2022, 12:37

                Thanks all for answering me, unfortunately there's no solution right now.
                I make you just one question: are you able to scrape with Skyscraper from inside raspberry, on screenscraper.fr or do you get the same error "game not found"? thank you

                K 1 Reply Last reply 9 Oct 2022, 14:35 Reply Quote 0
                • K
                  kiro @beppi
                  last edited by 9 Oct 2022, 14:35

                  @beppi trybtjis, it may work...https://github.com/zayamatias/retroscraper

                  1 Reply Last reply Reply Quote 0
                  • F
                    Floob @beppi
                    last edited by 10 Oct 2022, 16:52

                    @beppi
                    FWIW I use Skraper and scrape on windows, then just copy across the files.
                    I do that so I can copy it across to any destination quickly - so you only need to scrape once really.
                    The paths are all relative, so you dont need to change them

                    For example:
                    <image>./media/boxart2d/3-D Tic-Tac-Toe (USA).png</image> <marquee>./media/wheel/3-D Tic-Tac-Toe (USA).png</marquee> <video>./media/videos/3-D Tic-Tac-Toe (USA).mp4</video>

                    So in Skraper, the video path would be:
                    %ROMROOTFOLDER%\media\videos

                    Same as @Quackwalks really.

                    Please read the Docs before asking a new question.
                    RetroPie Help Guides: https://goo.gl/3gcNsT

                    B 1 Reply Last reply 12 Oct 2022, 12:48 Reply Quote 1
                    • B
                      beppi @Floob
                      last edited by 12 Oct 2022, 12:48

                      @Floob i fixed the problem. On retropie it still fails, but I mounted a disk in a Linux distribution and using the bind option I recreated the needed directory tree and run Skyscraper from there. It works, so it's obvious that there is a problem on my raspberry but I don't have any tool to address it. Thank you

                      K 1 Reply Last reply 12 Oct 2022, 12:52 Reply Quote 0
                      • K
                        kiro @beppi
                        last edited by 12 Oct 2022, 12:52

                        @beppi could it be a permissions issue in your rpi? Pi user does not have write rights on the mount ?

                        1 Reply Last reply Reply Quote 0
                        • Q
                          Quackwalks @Crush
                          last edited by 13 Oct 2022, 02:09

                          @Crush That is also an option, but sometimes I'd rather get everything scraped on a separate hard drive. Just makes me feel more secure in various ways. I'm probably just silly.

                          1 Reply Last reply Reply Quote 0
                          17 out of 17
                          • First post
                            17/17
                            Last post

                          Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                          Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.

                            This community forum collects and processes your personal information.
                            consent.not_received