The installation of scraper package is failling.

Mathes75

Have the same issue, sorry to ask - but what's the difference between the RetroPie golang package and the system package? How to remove the RetroPie golang package?

windg

@Mathes75 To uninstall the golang :

First update RetroPie-Setup script, then, go to RetroPie Menu >> RetroPie Setup >> Manage packages >> Manage dependency packages >> uninstall and reinstall golang package. After that try to install again the scraper .

Mathes75

@windg OK. I was able to locate the package, uninstall and reinstalls go1.11.13, but still no luck with scraper :(

Here is the interesting part from the log...

# cd /home/pi/RetroPie-Setup/tmp/build/scraper/src/github.com/J-Swift/thegamesdb-swagger-client-go; git pull --ff-only
You are not currently on a branch.
Please specify which branch you want to merge with.
See git-pull(1) for details.

    git pull <remote> <branch>

package github.com/J-Swift/thegamesdb-swagger-client-go: exit status 1
HEAD is now at 43ed8a0 Add 403 errors
error: patch failed: ds/ss.go:230
error: ds/ss.go: patch does not apply
error: patch failed: ds/ss_mame.go:2
error: ds/ss_mame.go: patch does not apply
error: patch failed: ss/ss.go:14
error: ss/ss.go: patch does not apply
/home/pi
/home/pi/RetroPie-Setup/tmp/build/scraper /home/pi

= = = = = = = = = = = = = = = = = = = = =
Building 'scraper' : Scraper for EmulationStation by Steven Selph
= = = = = = = = = = = = = = = = = = = = =

# golang.org/x/oauth2/internal
src/golang.org/x/oauth2/internal/token.go:140:10: lc.v.CompareAndSwap undefined (type atomic.Value has no field or method CompareAndSwap)
/home/pi
/home/pi/RetroPie-Setup/tmp/build/scraper /home/pi
Could not successfully install Scraper for EmulationStation by Steven Selph (/home/pi/RetroPie-Setup/tmp/build/scraper/scraper not found).
/home/pi

windg

I can confirm this, it looks that something it is not working correct.

Mathes75

OK. First switch brain on, then start actions, then complain.

I got Scraper installed by copying scraper files from githib.com folder in temp to folder that is not found above, but now it doesn't work. No new ROM info is fetched. Error message is referring to /opt/retropie/supplementary/scraper/scraper not found. All files seem to be there...

Lolonois

The issue is twofold.

@Mathes75 said in The installation of scraper package is failling.:

error: patch failed: ds/ss.go:230

This can be remediated by running sudo RetroPie-Setup/retropie_packages.sh scraper clean and then sudo RetroPie-Setup/retropie_packages.sh scraper

But hold your horses: The second issue needs attention too.

@Mathes75 said in The installation of scraper package is failling.:

src/golang.org/x/oauth2/internal/token.go:140:10: lc.v.CompareAndSwap undefined (ty...

This requires a newer golang version (was introduced in 1.17 AFAIK). My bad on suggesting 1.11 in the first place. However, since 1.17 some commands have been deprecated, which also needed to be reflected in the scraper.sh scriptmodule.

TL;DR: Here is a patch.

Save the patchfile somewhere on your Pi, then run in folder /home/pi/RetroPie-Setup this command: git apply /path/to/patchfile. No worries, you can undo the patch with git apply --reverse /path/to/patchfile

Then run in this order:

sudo RetroPie-Setup/retropie_packages.sh golang
sudo RetroPie-Setup/retropie_packages.sh scraper clean
sudo RetroPie-Setup/retropie_packages.sh scraper

It should provide you a scraper binary. I did not try to scrape but you pls try and report back.

HTH

PS: Once more this is only life-support for the Scraper. The upstream git repo of Scraper hat it's last.update in 2019 the retropie scriptmodule uses some duct tape (read patches) to keep it alive. I suggest using skyscraper. This skyscraper branch is from late 2022.

windg

Thanks for your time to looking into this.
I try the patch but i have some errors when i try to apply it.

pi@retropie:~/RetroPie-Setup $ git apply /home/pi/rp_modules_golang_scraper.patch
error: patch failed: scriptmodules/supplementary/golang.sh:18
error: scriptmodules/supplementary/golang.sh: patch does not apply
error: patch failed: scriptmodules/supplementary/scraper.sh:22
error: scriptmodules/supplementary/scraper.sh: patch does not apply

I believe that is not worth your time to try to fix it, there better alternatives out there. Thanks again !

mitu

@windg Try cleaning up the scraper sources first, before attempting to re-build.

windg

@mitu I have remove both scraper and golang packages but it's still failing.

= = = = = = = = = = = = = = = = = = = = =
Getting sources for 'scraper' : Scraper for EmulationStation by Steven Selph
= = = = = = = = = = = = = = = = = = = = =

go: go.mod file not found in current directory or any parent directory.
	'go get' is no longer supported outside a module.
	To build and install a command, use 'go install' with a version,
	like 'go install example.com/cmd@latest'
	For more information, see https://golang.org/doc/go-get-install-deprecation
	or run 'go help get' or 'go help install'.
fatal: cannot change to '/home/pi/RetroPie-Setup/tmp/build/scraper/src/github.com/J-Swift/thegamesdb-swagger-client-go/': No such file or directory
fatal: cannot change to '/home/pi/RetroPie-Setup/tmp/build/scraper/src/github.com/sselph/scraper': No such file or directory
/home/pi

= = = = = = = = = = = = = = = = = = = = =
Building 'scraper' : Scraper for EmulationStation by Steven Selph
= = = = = = = = = = = = = = = = = = = = =

no required module provides package github.com/sselph/scraper: go.mod file not found in current directory or any parent directory; see 'go help modules'
Could not successfully install Scraper for EmulationStation by Steven Selph (/home/pi/RetroPie-Setup/tmp/build/scraper/scraper not found).

mitu

@windg Yeah, it won't work. I'll take a look later on, but that may be the last patchwork for this package.

windg

I believe even if you fix it it will be broken again soon or later. Maybe it's time to drop this package and save your time.

Mathes75

@windg What I did to get across this is to fetch the repositorys by hand opening a second SSH connection (no keyboard attached to cabinet). Will give it a try tonight again!
I think there is a folder mismatch in the script. I couldn't find it to fix it, that's why I downloaded it during golang install.

mitu

From a cursory look, @Lolonois is correct - one of the dependencies requires (at least) Golang 1.17 and compilation will fail with the version we install now with the golang dependency scriptmodule (v1.11.x).

Bumping the required Golang version may be enough now, but in future if other dependency (direct or indirect) is upgraded to work with a newer Golang we'd be facing the same issue. One solution is to pin the dependencies to a certain version (using a go.mod file) or maybe just provide a binary installation option - the resulting scraper binary doesn't even need golang installed and has no other system dependencies. I'll see how complicated the former solution is, since I have no golang experience.

windg

Many thanks mitu Scraper is installable again !