RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login
    Please do not post a support request without first reading and following the advice in https://retropie.org.uk/forum/topic/3/read-this-first

    The installation of scraper package is failling.

    Scheduled Pinned Locked Moved Help and Support
    scraper
    23 Posts 4 Posters 2.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • windgW
      windg
      last edited by windg

      RPi4
      Official Psu
      Sd-card 32 gb (2yr old)
      RetroPie 4.8.4 (manual installation on top of Buster legacy)

      I try to install this scraper package(Steven Selph) but it is failing. I don't know if there is a conflit with the skyscraper that i have already on the system. Thanks

      log file : https://pastebin.com/AkDtTbNd

      I just find that is a upstream issue.

      My English isn't at a good level.

      LolonoisL 1 Reply Last reply Reply Quote 0
      • LolonoisL
        Lolonois @windg
        last edited by

        @windg could you be more specific abt the upstream?

        AFAIK math/bits (the missing include error in the pastebin-log) was introduced with golang 1.9 onwards.

        The golang.sh script module does a sloppy version comparison, thus go1.8.7 may be enforced/installed. I propose a quick patch, which pulls in golang 1.11 (which is currently used in Buster) and does proper version comparison:

        diff --git a/scriptmodules/supplementary/golang.sh b/scriptmodules/supplementary/golang.sh
        index 0ac552df..58fa4589 100644
        --- a/scriptmodules/supplementary/golang.sh
        +++ b/scriptmodules/supplementary/golang.sh
        @@ -18,14 +18,15 @@ function _get_goroot_golang() {
         }
         
         function install_bin_golang() {
        -    local version
        +    local version="0"
        +    local min_version="1.11.13"
        +
             if [[ -e "$md_inst/bin/go" ]]; then
        -        local version=$(GOROOT="$md_inst" "$md_inst/bin/go" version | sed 's/.*\(go1[^ ]*\).*/\1/')
        +        local version=$(GOROOT="$md_inst" "$md_inst/bin/go" version | sed 's/.*go\(.*\)\s.*/\1/')
             fi
        +
             printMsgs "console" "Current Go version: $version"
        -    if [[ ! "${version}" < "go1.8.7" ]]; then
        -        return 0
        -    fi
        +    compareVersions "${version}" ge "${min_version}" && return 0
         
             rm -rf "$md_inst"
             mkdir -p "$md_inst"
        @@ -40,6 +41,6 @@ function install_bin_golang() {
             if isPlatform "aarch64"; then
                 arch="arm64"
             fi
        -    printMsgs "console" "Downloading go1.8.7.linux-$arch.tar.gz"
        -    downloadAndExtract "https://storage.googleapis.com/golang/go1.8.7.linux-$arch.tar.gz" "$md_inst" --strip-components 1
        +    printMsgs "console" "Downloading go${min_version}.linux-$arch.tar.gz"
        +    downloadAndExtract "https://storage.googleapis.com/golang/go${min_version}.linux-$arch.tar.gz" "$md_inst" --strip-components 1
         }
        
        windgW 1 Reply Last reply Reply Quote 1
        • windgW
          windg @Lolonois
          last edited by

          @Lolonois I find this older topic but doesn't have details : https://retropie.org.uk/forum/topic/32847/could-not-install-scraper-steven-selph?_=1690218810409

          My English isn't at a good level.

          1 Reply Last reply Reply Quote 0
          • mituM
            mitu Global Moderator
            last edited by

            @windg Update RetroPie-Setup and re-try the installation.

            windgW 1 Reply Last reply Reply Quote 0
            • windgW
              windg @mitu
              last edited by windg

              @mitu Thanks to looking into this!

              I try to reinstal, after update the setup-script and it's failing again :

              Log started at: Mon 24 Jul 2023 09:26:04 PM EEST
              
              RetroPie-Setup version: 4.8.4 (5fceb82)
              System: rpi4 (armv7l) - Raspbian GNU/Linux 10 (buster) - Linux retropie 5.10.103-v7l+ #1529 SMP Tue Mar 8 12:24:00 GMT 2022 armv7l GNU/Linux
              
              = = = = = = = = = = = = = = = = = = = = =
              Installing dependencies for 'scraper' : Scraper for EmulationStation by Steven Selph
              = = = = = = = = = = = = = = = = = = = = =
              
              
              = = = = = = = = = = = = = = = = = = = = =
              Installing (binary) 'golang' : Golang binary install
              = = = = = = = = = = = = = = = = = = = = =
              
              Current Go version: go1.8.7
              /home/pi/RetroPie-Setup/tmp/build/scraper /home/pi
              
              = = = = = = = = = = = = = = = = = = = = =
              Getting sources for 'scraper' : Scraper for EmulationStation by Steven Selph
              = = = = = = = = = = = = = = = = = = = = =
              
              package math/bits: unrecognized import path "math/bits" (import path does not begin with hostname)
              Note: checking out '43ed8a0b364ed2d8521d0'.
              
              You are in 'detached HEAD' state. You can look around, make experimental
              changes and commit them, and you can discard any commits you make in this
              state without impacting any branches by performing another checkout.
              
              If you want to create a new branch to retain commits you create, you may
              do so (now or later) by using -b with the checkout command again. Example:
              
                git checkout -b <new-branch-name>
              
              HEAD is now at 43ed8a0 Add 403 errors
              /home/pi
              /home/pi/RetroPie-Setup/tmp/build/scraper /home/pi
              
              = = = = = = = = = = = = = = = = = = = = =
              Building 'scraper' : Scraper for EmulationStation by Steven Selph
              = = = = = = = = = = = = = = = = = = = = =
              
              src/google.golang.org/protobuf/encoding/protowire/wire.go:15:2: cannot find package "math/bits" in any of:
              	/opt/retropie/supplementary/golang/src/math/bits (from $GOROOT)
              	/home/pi/RetroPie-Setup/tmp/build/scraper/src/math/bits (from $GOPATH)
              /home/pi
              /home/pi/RetroPie-Setup/tmp/build/scraper /home/pi
              Could not successfully install Scraper for EmulationStation by Steven Selph (/home/pi/RetroPie-Setup/tmp/build/scraper/scraper not found).
              /home/pi
              
              Log ended at: Mon 24 Jul 2023 09:26:31 PM EEST
              Total running time: 0 hours, 0 mins, 27 secs
              

              My English isn't at a good level.

              1 Reply Last reply Reply Quote 0
              • mituM
                mitu Global Moderator
                last edited by

                Try removing the golang package first.

                1 Reply Last reply Reply Quote 0
                • windgW
                  windg
                  last edited by

                  I try sudo apt-get remove go1.8.7 and sudo apt-get remove golang but it says that this package is not installed.

                  My English isn't at a good level.

                  mituM 1 Reply Last reply Reply Quote 0
                  • mituM
                    mitu Global Moderator @windg
                    last edited by

                    @windg The RetroPie golang package, not the system package.

                    windgW 1 Reply Last reply Reply Quote 1
                    • windgW
                      windg @mitu
                      last edited by

                      @mitu Many thanks is working now !

                      My English isn't at a good level.

                      1 Reply Last reply Reply Quote 0
                      • M
                        Mathes75
                        last edited by

                        Have the same issue, sorry to ask - but what's the difference between the RetroPie golang package and the system package? How to remove the RetroPie golang package?

                        windgW 1 Reply Last reply Reply Quote 0
                        • windgW
                          windg @Mathes75
                          last edited by windg

                          @Mathes75 To uninstall the golang :

                          First update RetroPie-Setup script, then, go to RetroPie Menu >> RetroPie Setup >> Manage packages >> Manage dependency packages >> uninstall and reinstall golang package. After that try to install again the scraper .

                          My English isn't at a good level.

                          M 1 Reply Last reply Reply Quote 0
                          • M
                            Mathes75 @windg
                            last edited by Mathes75

                            @windg OK. I was able to locate the package, uninstall and reinstalls go1.11.13, but still no luck with scraper :(

                            Here is the interesting part from the log...

                            # cd /home/pi/RetroPie-Setup/tmp/build/scraper/src/github.com/J-Swift/thegamesdb-swagger-client-go; git pull --ff-only
                            You are not currently on a branch.
                            Please specify which branch you want to merge with.
                            See git-pull(1) for details.
                            
                                git pull <remote> <branch>
                            
                            package github.com/J-Swift/thegamesdb-swagger-client-go: exit status 1
                            HEAD is now at 43ed8a0 Add 403 errors
                            error: patch failed: ds/ss.go:230
                            error: ds/ss.go: patch does not apply
                            error: patch failed: ds/ss_mame.go:2
                            error: ds/ss_mame.go: patch does not apply
                            error: patch failed: ss/ss.go:14
                            error: ss/ss.go: patch does not apply
                            /home/pi
                            /home/pi/RetroPie-Setup/tmp/build/scraper /home/pi
                            
                            = = = = = = = = = = = = = = = = = = = = =
                            Building 'scraper' : Scraper for EmulationStation by Steven Selph
                            = = = = = = = = = = = = = = = = = = = = =
                            
                            # golang.org/x/oauth2/internal
                            src/golang.org/x/oauth2/internal/token.go:140:10: lc.v.CompareAndSwap undefined (type atomic.Value has no field or method CompareAndSwap)
                            /home/pi
                            /home/pi/RetroPie-Setup/tmp/build/scraper /home/pi
                            Could not successfully install Scraper for EmulationStation by Steven Selph (/home/pi/RetroPie-Setup/tmp/build/scraper/scraper not found).
                            /home/pi
                            
                            LolonoisL 1 Reply Last reply Reply Quote 0
                            • windgW
                              windg
                              last edited by

                              I can confirm this, it looks that something it is not working correct.

                              My English isn't at a good level.

                              1 Reply Last reply Reply Quote 0
                              • M
                                Mathes75
                                last edited by

                                OK. First switch brain on, then start actions, then complain.

                                I got Scraper installed by copying scraper files from githib.com folder in temp to folder that is not found above, but now it doesn't work. No new ROM info is fetched. Error message is referring to /opt/retropie/supplementary/scraper/scraper not found. All files seem to be there...

                                1 Reply Last reply Reply Quote 0
                                • LolonoisL
                                  Lolonois @Mathes75
                                  last edited by

                                  The issue is twofold.

                                  @Mathes75 said in The installation of scraper package is failling.:

                                  error: patch failed: ds/ss.go:230

                                  This can be remediated by running sudo RetroPie-Setup/retropie_packages.sh scraper clean and then sudo RetroPie-Setup/retropie_packages.sh scraper

                                  But hold your horses: The second issue needs attention too.

                                  @Mathes75 said in The installation of scraper package is failling.:

                                  src/golang.org/x/oauth2/internal/token.go:140:10: lc.v.CompareAndSwap undefined (ty...

                                  This requires a newer golang version (was introduced in 1.17 AFAIK). My bad on suggesting 1.11 in the first place. However, since 1.17 some commands have been deprecated, which also needed to be reflected in the scraper.sh scriptmodule.

                                  TL;DR: Here is a patch.

                                  Save the patchfile somewhere on your Pi, then run in folder /home/pi/RetroPie-Setup this command: git apply /path/to/patchfile. No worries, you can undo the patch with git apply --reverse /path/to/patchfile

                                  Then run in this order:

                                  sudo RetroPie-Setup/retropie_packages.sh golang
                                  sudo RetroPie-Setup/retropie_packages.sh scraper clean
                                  sudo RetroPie-Setup/retropie_packages.sh scraper
                                  

                                  It should provide you a scraper binary. I did not try to scrape but you pls try and report back.

                                  HTH

                                  PS: Once more this is only life-support for the Scraper. The upstream git repo of Scraper hat it's last.update in 2019 the retropie scriptmodule uses some duct tape (read patches) to keep it alive. I suggest using skyscraper. This skyscraper branch is from late 2022.

                                  1 Reply Last reply Reply Quote 2
                                  • windgW
                                    windg
                                    last edited by windg

                                    Thanks for your time to looking into this.
                                    I try the patch but i have some errors when i try to apply it.

                                    pi@retropie:~/RetroPie-Setup $ git apply /home/pi/rp_modules_golang_scraper.patch
                                    error: patch failed: scriptmodules/supplementary/golang.sh:18
                                    error: scriptmodules/supplementary/golang.sh: patch does not apply
                                    error: patch failed: scriptmodules/supplementary/scraper.sh:22
                                    error: scriptmodules/supplementary/scraper.sh: patch does not apply
                                    

                                    I believe that is not worth your time to try to fix it, there better alternatives out there. Thanks again !

                                    My English isn't at a good level.

                                    mituM 1 Reply Last reply Reply Quote 0
                                    • mituM
                                      mitu Global Moderator @windg
                                      last edited by

                                      @windg Try cleaning up the scraper sources first, before attempting to re-build.

                                      windgW 1 Reply Last reply Reply Quote 0
                                      • windgW
                                        windg @mitu
                                        last edited by

                                        @mitu I have remove both scraper and golang packages but it's still failing.

                                        = = = = = = = = = = = = = = = = = = = = =
                                        Getting sources for 'scraper' : Scraper for EmulationStation by Steven Selph
                                        = = = = = = = = = = = = = = = = = = = = =
                                        
                                        go: go.mod file not found in current directory or any parent directory.
                                        	'go get' is no longer supported outside a module.
                                        	To build and install a command, use 'go install' with a version,
                                        	like 'go install example.com/cmd@latest'
                                        	For more information, see https://golang.org/doc/go-get-install-deprecation
                                        	or run 'go help get' or 'go help install'.
                                        fatal: cannot change to '/home/pi/RetroPie-Setup/tmp/build/scraper/src/github.com/J-Swift/thegamesdb-swagger-client-go/': No such file or directory
                                        fatal: cannot change to '/home/pi/RetroPie-Setup/tmp/build/scraper/src/github.com/sselph/scraper': No such file or directory
                                        /home/pi
                                        
                                        = = = = = = = = = = = = = = = = = = = = =
                                        Building 'scraper' : Scraper for EmulationStation by Steven Selph
                                        = = = = = = = = = = = = = = = = = = = = =
                                        
                                        no required module provides package github.com/sselph/scraper: go.mod file not found in current directory or any parent directory; see 'go help modules'
                                        Could not successfully install Scraper for EmulationStation by Steven Selph (/home/pi/RetroPie-Setup/tmp/build/scraper/scraper not found).
                                        

                                        My English isn't at a good level.

                                        mituM M 2 Replies Last reply Reply Quote 0
                                        • mituM
                                          mitu Global Moderator @windg
                                          last edited by

                                          @windg Yeah, it won't work. I'll take a look later on, but that may be the last patchwork for this package.

                                          1 Reply Last reply Reply Quote 2
                                          • windgW
                                            windg
                                            last edited by

                                            I believe even if you fix it it will be broken again soon or later. Maybe it's time to drop this package and save your time.

                                            My English isn't at a good level.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.