[SOFT] New Scraper in the works
-
@Folly maybe have a look at the logs? I know this could happen due to permission issues, sometimes... it should ouptut something like :
CANNOT CREATE DIRECTORY [Your/directory] - ERROR [Explanation] - PLS VERIFY AND TRY AGAIN
And then exit ... so strange is not exiting...have you got he latest version?
-
@kiro said in [SOFT] New Scraper in the works:
so strange is not exiting...have you got he latest version?
It does exit now always, so no problem.
I checked the --mediadir option again with absolute path.
The issue is that when using a path with the folder, for example,RetroPie
(R and P are uppercase) RetroPie isn't used/created butretropie
is. (everything with lowercase)I see you have fixed the other issues ;-)
-
@Folly About the folders, didn't quite understand, so if you put /RetroPie/ will it create /retropie/ or you put /retropie/ and it will not be recognized as /RetroPie/ ? The latter being totally ok as directories under unix/linux are case sensitive..
-
@kiro said in [SOFT] New Scraper in the works:
if you put /RetroPie/ will it create /retropie/
This.
-
@Folly Ok, then probably I'm putting everything lowercase where I should not, going to check right now..
Should be fixed and uploaded to git...
-
I get this :
python3 /opt/retropie/supplementary/retroscraper/retroscraper.py --mediadir /home/pi/RetroPie/roms/upright --systems upright --cleanmedia Starting scraping Scanning Files Scanning Directory ERROR SYSTEM upright: NO ROMS IN DIRECTORY /home/pi/RetroPie/roms/upright/ Scan Done! SCRAPPING ENDED --- Thank you for using retroscraper!!
-
@Folly said in [SOFT] New Scraper in the works:
NO ROMS IN DIRECTORY
Ok, should be fixed, interesting. To avoid scanning through images, I remove anything whose path is the same as the 'media' path, of course, if the mediapath is the same as the system path, well guess what.... I just added the system name to the media path so it should be ok now.
-
@kiro
Did a second update !
I works ok. -
@Folly Thanks for that, would've never thought of doing that path combination!
-
So you like it ?
-
@Folly I like the testing being done, usually when developing there are so many things you give by 'understood' that you tend to forget some of the possibilities a user may input...
-
It's a good process,
It's more fun/productive when there is some interaction, don't you agree ?Btw.
How do we add media that is not recognized right now for our project ? -
@Folly Well, here's the difficult question... how do we do that.. ok there are 2 things to consider:
1- I'll have to manually add the images to the DB, that is not really a concern as long as I understand to which game each image corresponds, which should be fairly easy (thanks to filenames).
2-What do we do if there's already a screenshot available for the game in question, for example, Archimedes. (although I see that most of your images correspond rather to 'systems' than games? And some images seem to be fairly similar (game&watch for instance)
So, if you want to make a small test, pack all the all_in_1 images, which I see are missing from my backend, into a file and I'll try to put them in the right place, once done, we can try to see if scraping gets these images. makes sense?
-
@kiro
I see your point.
Perhaps we can start with an easy one, tigerrz (Tiger R-Zone) which has just 3 games.
I scraped it earlier with your scraper and, as far as I know there are no marquees, images and videos in your DB. -
@Folly Let's try it :-)
-
Here is the googledrive of DTEAM with predefined gamelists, which is used by our project :
https://drive.google.com/drive/folders/1f_jXMG0XMBdyOOBpz8CHM6AFj9vC1R6mYou can just download the file tigerrz.
I just tested my files with your scraper and these have no media and no description and full name etc :
rzbatfor.zip rzindy500.zip rztoshden.zipDon't know if .7z files have media, etc in your DB.
-
@Folly thx!
But, when I try to download (I guess right click then download on folder), I get
Download failed...do I need permissions or am I not downloading what I'm supposed to be downloading?
-
I just tested, and it works here so it should work for you too.
You should just click on the tigerrz and a zip file is created and downloaded.I recently had the same problem working on an open network, though I think you are not on an open network now, right.
-
@Folly Not sure, I tried an incognito window and it seems to work... maybe because since I'm logged in with my Gmail account to chrome it somehow messes things up? not sure...Anyway, It's a bit on the late side so I'll be importing the images tomorrow first thing and will let you know as soon as I'm done. I see there's a screenshot, a marquee, and a video (almost for every game) correct?
-
Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.
Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.