Test integration of sselph/scraper and ES

[sselph]

Hi Everyone,

I think I have this to a point that it can be tested. I have compiled my scraper to a C library that ES can import. This means you can call my scraper code directly from ES's builtin scraper. It is simple and mimics the current builtin scraper options. thegamesdb seems to be having some issues tonight but there is screenscraper or the mamedb mirror you can try.

Installation instructions:

1. Download a fresh copy of retropie and install
2. Connect the raspberry pi to the network
3. Boot the raspberry pi and exit ES back to the terminal
4. Download and install the new ES and new libraries

wget https://storage.googleapis.com/stevenselph.appspot.com/es.tar.gz
tar -xvzf es.tar.gz
./ssinstall.sh

5. Copy some ROMs, reboot and scrape something using the builtin scrapers that start with (Auto)

Edit: clarify what was done

[sselph]

I was up late finishing this up so forgot to give a few known issues

• There is a pause before scraping the first game. This is the scraper updating its csv file and loading it in to memory. Ideally there would be some UI notification of it so it doesn't appear frozen.
• When scraping ScreenScraper it doesn't update with the text and image between games. This is a side effect of how ES builds the save path and the way ScreenScraper URLs are done. It results in images like rom-image.php?id=1234&format=png&foo=bar I can reliably know the image type from parsing the HTML response but couldn't figure out how to relay that back to the code that writes the XML so just took over that entirely in my code. The result is ES doesn't see anything until it sees everything so no partial update.
• I forgot to disable some logging so you will see a bunch of crap on the screen when you exit.

[recio.alex]

When i get some sparse time at home, i'll try this with some romsets and report back

[lilbud]

@sselph Does this mean we can now use your scraper without having to exit emulationstation?

[sselph]

@lilbud Actually reading my post again, I wasn't very clear. I would phrase it as I improved the builtin scraper using my code. So that using one of the new scrapers listed in the list as (Auto)GamesDB, (Auto)ScreenScraper, (Auto)MameDB are actually triggering my code. So yeah you don't have to close ES because you are scraping using ES. (I'll update my original post)

Another improvement I will make after this is submitted is updating the manual gamesdb scraper to not ask if there is hash verified match then fall back to the search.

[herb_fargus]

@sselph also we should remove the archive source at some point as that has never worked

[sselph]

@herb_fargus I can easily remove that scraper. I can send out a PR for that.

I noticed in another post that people are compiling RetroPie/EmulationStation for windows. I assumed this was only meant to run on linux and so this would potentially break linking on windows. In Go 1.9 they seem to be working on adding the ability to generate dll's but that doesn't exist yet. One option is for me to write a simple C file that has all the correct function calls but doesn't do anything that ES can substitute if it can't find my scraper library.

[BuZz]

@sselph Thanks!