Scraper

Zodiark12

Hey everyone,

I've been trying to start using Steven Selph Scraper instead of the built-in ES Scraper and I've been having issue. At first it was scraping but nothing was downloading, then I found out that Screenscraper.fr closed the API for anonymous users. So I created an account but I'm still not seeing anywhere to enter the account credentials. Is there any config file anywhere I can edit to add these credentials so that the scraper would actually work with screenscraper.fr?

It work fine with thegamesDB but I wanted to use screenscraper.fr for the videos.

Thank in advance.

mitu

I think user/pass are available only as command line parameters. If you wish to use a registered user for ScreenScraper, you can try Skyscraper, which has the same features (and more) as Steven Selph's scraper and which allows you to set the user/pass in the configuration file.

Zodiark12

@mitu
Thanks, I'll check out Skyscraper instead.

muldjord

@zodiark12 Here's a link to the Skyscraper docs explaining how to add your user credentials to the Skyscraper config: https://github.com/muldjord/skyscraper/blob/master/docs/CONFIGINI.md#usercredscredentials-or-key

You can edit the Skyscraper config.ini from the Skyscraper RetroPie menu.

Zodiark12

@muldjord
Would be hard to argue with you over this. Thank you very much for taking the time. I'll give it a go as soon as I have a chance.

Zodiark12

Well I gave Skyscraper a try and it worked. However the result was kinda unexpected...

I've scraped a PSX rom to try it. It downloaded the video but nothing else. No cover, screenshot, wheel of marquee. The rom that I've scraped now only work 1 time out of 4. The other 3 times it freeze retropie completely and I need to do a hard shutdown. The rom only started doing so after scraping it and no other rom do this on my build.

Finally the 1 time I was able to launch the rom the runcommand splashscreen that I had was replaced with the game cover which is odd because skyscraper did not download anythinf else beside the video. I had custom art for the splashscreen and the file is still in place but the game cover was shown instead. Probably just a setting to change somewhere but I'm not sure where.

The fact that it did not scrape everything and now that the rom is freezing the system is more concerning.

muldjord

@zodiark12 said in Scraper:

The rom that I've scraped now only work 1 time out of 4. The other 3 times it freeze retropie completely and I need to do a hard shutdown.

Something else on your system is causing this - maybe you are running out of disk space or your SD card is going bad. Skyscraper does not change how games are launched at all.

I'm pretty sure Skyscraper did scrape some of the other artwork, you just don't see it. Could you provide your gamelist.xml for the psxplatform, please? Then I can take a look at what it has done.

Zodiark12

@muldjord
My message was maybe badly formated but I do not blame Skyscraper. I perfectly understand that it would not cause such issue. The issue did start however after I scraped the rom. The rom could be corrupted, video could be corrupted or an issue with the theme I use or something else.

However, is it normal that Skyscraper did not download anything other than the video? Could it have changed the launch splashscreen? I doubt it, but I found it weird that after scraping my personal art was replaced by the game cover (Which was not downloaded by Skyscraper) when starting the rom.

Zodiark12

@muldjord
Sorry, seemed you edited your poste while I was replying to you.
Here is the XML file content :

<?xml version="1.0"?>
<gameList>
  <game>
    <path>./alundra.chd</path>
    <name>Alundra</name>
    <thumbnail />
    <image />
    <marquee />
    <video>./media/videos/alundra.mp4</video>
    <rating>0.9</rating>
    <desc>Alundra is a boy who can step into other people&apos;s dreams. His own dreams tell him to go to the village Inoa. The ship on which Alundra is coming to the village goes down, but Alundra himself is washed up on a beach near the village. There, a blacksmith Jess takes care of him and treats him like his own son. But Alundra, stepping into the dreams of the village people, understands there&apos;s a danger somewhere. The people have horrible nightmares, and only Alundra can find out what they mean...

Alundra is an action role playing game (RPG) somewhat similar to the famous Zelda games. It uses real-time combat and has a lot of physical and environmental puzzles. A big part of the game is spent in dungeons, where the player has to solve puzzles in order to proceed.</desc>
    <releasedate>19980605T000000</releasedate>
    <developer>Sony</developer>
    <publisher>Psygnosis</publisher>
    <genre>Role playing games</genre>
    <players>1</players>
    <lastplayed>20201111T013040</lastplayed>
    <playcount>1</playcount>
  </game>
</gameList>

muldjord

@zodiark12 From the looks of that gamelist I'm guessing you disabled the cover, marquee and so on in the Skyscraper menu. Then it won't use them. By default Skyscraper downloads and caches screenshot, cover, marquee and wheel. It outputs a marquee, wheel and screenshot when generating for EmulationStation by default I think, I can't quite remember as I have my own set up differently.

Zodiark12

@muldjord
I did disable the cache of these because to my understanding Skyscraper would be addind the medias to the rom folder of the system (As I've set it to do) and I didn't want it to have a duplicate elsewhere in the cache. Reading your last post it seem that disabling the cache means it won't download any of these at all?

If so I will enable them but after scraping can I empty the cache without affecting the medias in the rom folders?

mitu

@zodiark12 said in Scraper:

If so I will enable them but after scraping can I empty the cache without affecting the medias in the rom folders?

Skyscraper first downloads the media to the cache, then it will copy it to the media folder where EmulatonStation reads it (being it in the ROM folder or not) in a subsequent step - when you generate the gamelists. Once you finish scraping, yes, you can clear the cache and have only 1 copy of the artwork/video/etc.