[SOFT] New Scraper in the works

kiro

@Folly said in [SOFT] New Scraper in the works:

NO ROMS IN DIRECTORY

Ok, should be fixed, interesting. To avoid scanning through images, I remove anything whose path is the same as the 'media' path, of course, if the mediapath is the same as the system path, well guess what.... I just added the system name to the media path so it should be ok now.

Folly

@kiro
Did a second update !
I works ok.

kiro

@Folly Thanks for that, would've never thought of doing that path combination!

Folly

@kiro

So you like it ?

kiro

@Folly I like the testing being done, usually when developing there are so many things you give by 'understood' that you tend to forget some of the possibilities a user may input...

Folly

@kiro

It's a good process,
It's more fun/productive when there is some interaction, don't you agree ?

Btw.
How do we add media that is not recognized right now for our project ?

kiro

@Folly Well, here's the difficult question... how do we do that.. ok there are 2 things to consider:

1- I'll have to manually add the images to the DB, that is not really a concern as long as I understand to which game each image corresponds, which should be fairly easy (thanks to filenames).

2-What do we do if there's already a screenshot available for the game in question, for example, Archimedes. (although I see that most of your images correspond rather to 'systems' than games? And some images seem to be fairly similar (game&watch for instance)

So, if you want to make a small test, pack all the all_in_1 images, which I see are missing from my backend, into a file and I'll try to put them in the right place, once done, we can try to see if scraping gets these images. makes sense?

Folly

@kiro
I see your point.
Perhaps we can start with an easy one, tigerrz (Tiger R-Zone) which has just 3 games.
I scraped it earlier with your scraper and, as far as I know there are no marquees, images and videos in your DB.

kiro

@Folly Let's try it :-)

Folly

@kiro

Here is the googledrive of DTEAM with predefined gamelists, which is used by our project :
https://drive.google.com/drive/folders/1f_jXMG0XMBdyOOBpz8CHM6AFj9vC1R6m

You can just download the file tigerrz.

I just tested my files with your scraper and these have no media and no description and full name etc :
rzbatfor.zip rzindy500.zip rztoshden.zip

Don't know if .7z files have media, etc in your DB.

kiro

@Folly thx!

But, when I try to download (I guess right click then download on folder), I get

Download failed...do I need permissions or am I not downloading what I'm supposed to be downloading?

Folly

@kiro

I just tested, and it works here so it should work for you too.
You should just click on the tigerrz and a zip file is created and downloaded.

I recently had the same problem working on an open network, though I think you are not on an open network now, right.

kiro

@Folly Not sure, I tried an incognito window and it seems to work... maybe because since I'm logged in with my Gmail account to chrome it somehow messes things up? not sure...Anyway, It's a bit on the late side so I'll be importing the images tomorrow first thing and will let you know as soon as I'm done. I see there's a screenshot, a marquee, and a video (almost for every game) correct?

Folly

@kiro

Aha, so it's working now.
Yes indeed, all should be there.

Good evening !

kiro

Morning @Folly, I've been checking on the files, couple of questions:

All three games have 4 files in them, of these files, 2 are .mp4 (just .mp4 and .ia.mp4) when I try to play the ia.mp4 it does not show properly..should I consider the .mp4 one for the video?

Then, I have the logo, which I suppose should be the marque, and then the last one which looks like a cartridge, should this be the screenshot?

Have a great day!

Folly

@kiro

Good morning.

You can take the regular .mp4 ones, I see that the .ia.pm4 aren't used in the accompanied gamelist.
(As it should be, game seems to have no audio ~~seems 1 files has no audio, perhaps something to fix later~~)

The picture with the name should be the logo/marquee (*m.png).
Perhaps you can add the R-Zone logo sideways, just like with your other marquees, you can find logos in this theme :
https://github.com/DTEAM-1/cygnus-blue-flames

Indeed, the cartridge should be the screenshot/image.

Have a great day too !

kiro

@Folly I've updated the medias and they should be working (cannot really test without having the roms myself). I think I'll have to work on creating a script to extract the info from your gamelists and import it into the DB, shouldn't be very hard to do.

kiro

@Folly In the meantime I've also added the option to chose path from the retrropie_setup script. The funny thing is if you chose '.' as your media path, it will write in the same folder as the roms... (it covers a request made by someone in the forum)

Folly

@kiro said in [SOFT] New Scraper in the works:

@Folly I've updated the medias and they should be working (cannot really test without having the roms myself).

Tested, but it doesn't work.
No descriptions and no media or do I have to update the script ?

I think I'll have to work on creating a script to extract the info from your gamelists and import it into the DB, shouldn't be very hard to do.

Could be a good idea.

kiro

@Folly Hmm...

I've tested myself and it does work..

       <game>
                <rating></rating>
                <name>R-Zone: Batman Forever</name>
                <marquee>/home/pi/RetroPie/roms/arcade/marquees/rzbatfor-marquee.png</marquee>
                <image>/home/pi/RetroPie/roms/arcade/images/rzbatfor-image.png</image>
                <publisher>Unknown</publisher>
                <releasedate></releasedate>
                <players></players>
                <video>/home/pi/RetroPie/roms/arcade/videos/rzbatfor-video.mp4</video>
                <genre></genre>
                <path>/home/pi/RetroPie/roms/arcade/rzbatfor.7z</path>
                <developer>Unknown</developer>
                <thumbnail/>
                <desc>Batman Forever (handheld)</desc>
                <playcount>0</playcount>
                <lastplayed>0</lastplayed>
        </game>
        <game>
                <rating></rating>
                <name>R-Zone: Battle Arena Toshinden</name>
                <marquee>/home/pi/RetroPie/roms/arcade/marquees/rztoshden-marquee.png</marquee>
                <image>/home/pi/RetroPie/roms/arcade/images/rztoshden-image.png</image>
                <publisher>Unknown</publisher>
                <releasedate></releasedate>
                <players></players>
                <video>/home/pi/RetroPie/roms/arcade/videos/rztoshden-video.mp4</video>
                <genre></genre>
                <path>/home/pi/RetroPie/roms/arcade/rztoshden.7z</path>
                <developer>Unknown</developer>
                <thumbnail/>
                <desc>Battle Arena (handheld)</desc>
                <playcount>0</playcount>
                <lastplayed>0</lastplayed>
        </game>
        <game>
                <rating></rating>
                <name>R-Zone: Indy 500</name>
                <marquee>/home/pi/RetroPie/roms/arcade/marquees/rzindy500-marquee.png</marquee>
                <image>/home/pi/RetroPie/roms/arcade/images/rzindy500-image.png</image>
                <publisher>Unknown</publisher>
                <releasedate></releasedate>
                <players></players>
                <video>/home/pi/RetroPie/roms/arcade/videos/rzindy500-video.mp4</video>
                <genre></genre>
                <path>/home/pi/RetroPie/roms/arcade/rzindy500.7z</path>
                <developer>Unknown</developer>
                <thumbnail/>
                <desc>Indy 500 (handheld)</desc>
                <playcount>0</playcount>
                <lastplayed>0</lastplayed>
        </game>

So not sure what could be the issue???