Test a new data source for sselph/scraper

robertybob

@Nightfire said in Test a new data source for sselph/scraper:

Hi @sselph, happy to hear you're working on the scraper again, I've used it every time I've worked with RetroPie. Not sure if you had seen recent development news of RetroPie, but they are working on emulators for PSP (ppsspp), Nintendo DS (desmune) and GameCube (dolphin). So, I was wondering, will you be adding support for scraping PSP/NDS/GameCube games? Or is there a way to scrape for those systems that I'm just not aware of?

PSP:
https://github.com/sselph/scraper/issues/52

NDS:
https://github.com/sselph/scraper/issues/28

Gamecube would be possible, I think, as the hashes are listed on redump

Nightfire

@robertybob ahh, ok, thanks for the info, guess I never went to the 'issues' when I was there.

spannernick1

Will this scraper have UK box art as well,the ones that come with ES are not good at finding Mega Drive and CD Pal box art,had to end up adding them manually ,sometimes it adds a completely different game when its not that game.

robertybob

@spannernick1 Don't bother with the ES scraper, use Sselph's one.

Sselph's scraper should match (Europe) roms to Mega Drive games and (USA) roms to Genesis games. If not, first check whether the image uploaded to thegamesdb.net is actually for the correct region (more often that not it isn't). http://thegamesdb.net/browse/36/ If the Mega Drive game is listed there and the artwork is for the Europe copy, then check which region your roms are.

Failing that, segaretro.org has all the best quality Mega Drive scans :)

paradadf

If you use the flag -region with eu in the first place, it will find the megadrive box art, assuming it exists and, of course, it is pressent on www.screenscraper.fr
SS either has added to his database the media from the best possible sources, or the people has contributed creating it.

paradadf

I'm not really sure how retropie handles scummvm. I saw on the wiki that you use either .sh or .svm extensions (?). Anyway... this information is valid for recalbox, I suppose that you use files as shortname.svm (or .sh). If you put the fullname inside those files as shown here, you will be able to scrape scummvm using SS as well. I'm assuming that both extensions are already known by the scraper. @sselph Would you mind adding .scummvm to the list, please?

tien_huu_1408

@sselph : I have downloaded scraper code (rpi2 version). On Raspberry Pi 3.
How to use it by terminal?
Hope you could help me step by step, especially for scraping 3D image.
Thanks a lot. Hope to hear from you asap.

paradadf

hi @sselph! I found something weird with one specific Game Boy rom.
http://screenscraper.fr/rominfos.php?romid=309753

the scraper isn't finding the game at all... I asked about it on IRC and the API is apparently returning the right gameid, but still nothing

2016-10-29 13:15:42 | ip: ##.##.##.### | Aladdin (USA) (SGB Enhanced).gb -> gameid=3025

the sha1 is correct also.

Nismo

hi @sselph, the scraper works ok except doesn't downloading any image for any game. What i'm doing wrong?

I use windows version, this is my command line:

scraper -add_not_found=true -append=true -download_images=true -image_dir="P:\emulationstation\roms\megadrive\images" -image_path="P:\emulationstation\roms\megadrive\images" -image_suffix="" -img_format="png" -no_thumb=true -thumb_only=false -thumb_suffix="" -use_gdb=true -use_ss=true -use_ovgdb=true -use_nointro_name=false -workers=4 -missing="missing.txt" -console_img="3b" -region="us,eu,jp,fr,xx" -lang="en"

All metadata is ok on gamelist.xml but in missing.txt all games are "missing image" like this:

AAAHH!!! Real Monsters (UE) [!].zip,,fa579a7e47ee4a1bdb080bbdec1eef480a85293e,missing image
3 Ninjas Kick Back (U) [!].zip,,d634e3f04672b8a01c3a6e13f8722d5cbbc6b900,missing image

Any help whould be appreciated, thanks.

enderandrew

Your path doesn't look correct:

P:.emulationstation\roms\megadrive\images

Shouldn't that be something more like:

P:\emulationstation\roms\megadrive\images

Nismo

@enderandrew No, the path is correct, even if I change the path to C:/images it's the same... missing image for all games.

Thanks for help.

Edit: I edit my last post to avoid confussions.

Nismo

I solved by myself. For who interested I solved downloading fastscraper script and edit some parameters to my like.

You can found here: https://forum.recalbox.com/topic/2594/batch-scrape-your-roms-on-your-pc-fastscraper

Regards.

relative

Hello,

How do I turn off gdb is becoming nuisance with server going offline whenever I want something done.

How do I enable the use of SS ?

I used this and it continues to default to gdb [ console srcs : gdb ]
scraper -use_gdb=false -use_ss=true

relative

@relative nvm, I changed it through the front-end... adding that string in the command line has no effect whatsoever.

sselph

@relative the -use_ flags were not great so a while back I changed these to -console_src="ss,gdb" where you pass an ordered list of sources. There is also -mame_src and you can see the list of accepted flags and documentation with -h

superjoe

How do you specify the user and password for screenscraper.fr on the arguments of the sselph scrapper? So far I've been using the Universal Xml Scrapper V2 but it requires registration.

relative

@superjoe
-ss_username -ss_password

They are not mandatory as far as I I know. I was able to scrape without them.

relative

@sselph okay thanks.

sselph

@relative Yeah they haven't typically been mandatory but if there is any congestion on the server, you will be told that you can't access it. Also they limit the number of threads to 1 unless you are registered, but that isn't too bad.

craig0r

@sselph can you tell me how I might add my credentials to the scraper.sh script? (I'm indeed running into that congestion.)