Scraper optimization

herb_fargus

@schmulkfuster it needs work before we'd do that.

Zigurana

There are many forks of ES, herdinger is not the first to try and continue the work.

BuZz

@schmulkfuster there is nothing significant in the new fork yet. I am keeping an eye on things. We currently run from our own fork, as we have our own customisations. No plans to rebase over a new upstream yet

Rion

@BuZz said:

@schmulkfuster there is nothing significant in the new fork yet. I am keeping an eye on things. We currently run from our own fork, as we have our own customisations. No plans to rebase over a new upstream yet

Any plans to have a look at what the other forks are doing for example Recalbox and incorporate that into your fork?

@Zigurana said:

There are many forks of ES, herdinger is not the first to try and continue the work.

That's news to me. Do you have any links to the other forks? Kind of interested to see what they are doing with Emulationstation.

BuZz

@Rion I already do this, however I don't currently want to spend my time working on Emulation Station.

meleu

You guys have to agree that the emulationstation is an important part of RetroPie.

I don't want to be that boring dude saying what the developers have to do, but I would like to see some little improvements on emulationstation.

What do you developers think about a donating system where the donors could say where they would like to see improvements? (Just as a suggestion!)

I think it will increase the number of donations ;-)

meleu

Another idea: the developer that is used to play with the emulationstation code say here "if I get X bucks I will make some effort to implement the Y feature".
Something like the kickstarter.
I don't know the details of how to do this. It's just a brainstorm, OK? :D

BuZz

I wouldn't be interested - I do this for fun, and not as a job.

Nothing is stopping you from contributing back improvements yourself of course - but bear in mind that I got involved in RetroPie because I wanted to see some improvements as a RetroPie user. I do listen to and implement many requests from users, but I don't want this to turn into something where I get told what I should be working on - that would no longer be something I would like to do.

Zigurana

@Rion See here for a list of people who have created a fork of ES. Apparently its very difficult on Github to get a full list of forks, although you can seen them all when you actually want to do comparisons for pull-requests.

I guess the point right now is not that there are no people willing to work on it, but more who will stand up to draw up a comprehensive action plan with a long term vision for ES, and is capable of pulling that particular cart for the foreseeable future?
It will need to show these things before RetroPie can even consider switching upstream branches.

The good thing about Herdingers project is that at least he/she is willing to publictly discuss the long-term goals. See here.

Zigurana

@BuZz @meleu
I fully agree with that sentiment, if you want something different, its primarily up to you to educate yourself to do it.

Rion

@Zigurana said:

@Rion See here for a list of people who have created a fork of ES. Apparently its very difficult on Github to get a full list of forks, although you can seen them all when you actually want to do comparisons for pull-requests.

I guess the point right now is not that there are no people willing to work on it, but more who will stand up to draw up a comprehensive action plan with a long term vision for ES, and is capable of pulling that particular cart for the foreseeable future?
It will need to show these things before RetroPie can even consider switching upstream branches.

The good thing about Herdingers project is that at least he/she is willing to publictly discuss the long-term goals. See here.

Wow that's more then i could imagine.
But i see your point. To many forks and not many working together.

It's a shame really.

Twitch0815

Even other forks for example Recalbox scraping is more of a pain in the but then it is on Retropie.
They do the same thing .fullscrape

Until someone puts a massive amount of time into developing a gui wrapper for a scraper service, the better option is always going to be a command line scraper. And that is assuming thegaemesdb.net stays up. There are just some things cli clients do better.

We need to keep ask's in perspective, we as a community put a huge ask on thegamesdb but their donations are not reflective of the demand we as a community put on them and the demand for their service is literally crushing their servers and we get vocal about our dislike of that feature being unavailable regardless of whether we have ever donated or making vocal and somewhat alarmist replace thegamesdb develop an alternative switch or code something better. Additionally asking the limited developers we have to focus on the one demand we have a working solution for its just a few more manual steps to get it working takes them away from kernel improvements new ports, features etc etc.

Honestly as long as gamesdb.net scraping artwork with sspeh is a breeze and there is only one other frontend period I have seen do a better job easier with artwork and it is a mac OSX frontend. Mess with Hyperspin and emumovies accounts paid subscriptios download rate limits having to edit xml by hand renaming romsets its a pain in the but.

And honestly its not going to be very long before Attract Mode or Meh replace Emulation Station as the default front-end for Retropie or alternatives at least and the artwork structure for those is going to be manual and manual for a good while. Luckily the Hyperspin and EmuMovies community have already done that heavy lifting though were going to have to deal with getting emumovie accounts and all the hoops that comes with that but the end product will look great. I would much rather the devs look forward or we put the ask to the meh and attract mode developers to make the art scraping and organizing process as user friendly as possible and let our devs keep focusing on what they want to work on as well as the good ideas we come forward with but we as a community need to be fair and not ask for major development on a platform that will be irrelevant in coming months.

Zigurana

@Twitch0815 said:

...
And honestly its not going to be very long before Attract Mode or Meh replace Emulation Station as the default front-end for Retropie or alternatives at least and the artwork structure for those is going to be manual and manual for a good while.
...
I would much rather the devs look forward or we put the ask to the meh and attract mode developers to make the art scraping and organizing process as user friendly as possible and let our devs keep focusing on what they want to work on as well as the good ideas we come forward with but we as a community need to be fair and not ask for major development on a platform that will be irrelevant in coming months.

Do you know these things, or are you speculating? I think there has been a lot of investments into ES, and there is a certain amount of momentum that needs to build up before any switch will be worth the hassle. I agree that we need to keep looking outside our bubble, and also not over-work the few developers we have, but switching to a completely new front-end is a large step.

Finhead

@Twitch0815 said:

Even other forks for example Recalbox scraping is more of a pain in the but then it is on Retropie.
They do the same thing .fullscrape

Until someone puts a massive amount of time into developing a gui wrapper for a scraper service, the better option is always going to be a command line scraper. And that is assuming thegaemesdb.net stays up. There are just some things cli clients do better.

We need to keep ask's in perspective, we as a community put a huge ask on thegamesdb but their donations are not reflective of the demand we as a community put on them and the demand for their service is literally crushing their servers and we get vocal about our dislike of that feature being unavailable regardless of whether we have ever donated or making vocal and somewhat alarmist replace thegamesdb develop an alternative switch or code something better. Additionally asking the limited developers we have to focus on the one demand we have a working solution for its just a few more manual steps to get it working takes them away from kernel improvements new ports, features etc etc.

Honestly as long as gamesdb.net scraping artwork with sspeh is a breeze and there is only one other frontend period I have seen do a better job easier with artwork and it is a mac OSX frontend. Mess with Hyperspin and emumovies accounts paid subscriptios download rate limits having to edit xml by hand renaming romsets its a pain in the but.

And honestly its not going to be very long before Attract Mode or Meh replace Emulation Station as the default front-end for Retropie or alternatives at least and the artwork structure for those is going to be manual and manual for a good while. Luckily the Hyperspin and EmuMovies community have already done that heavy lifting though were going to have to deal with getting emumovie accounts and all the hoops that comes with that but the end product will look great. I would much rather the devs look forward or we put the ask to the meh and attract mode developers to make the art scraping and organizing process as user friendly as possible and let our devs keep focusing on what they want to work on as well as the good ideas we come forward with but we as a community need to be fair and not ask for major development on a platform that will be irrelevant in coming months.

Nothing to do with the topic really, but there are no hoops to jump through for artwork on AM. You can get everything you need Right Here no membership etc. Then you place them in the correct folders, it's a good alternative to the scraper since it's not down.

meleu

OK @BuZz , I agree that the fun is a pre-requisite. It was just a suggestion.

Going back to the topic (scraper optimization): Is there an automated way to backup/copy my scraps and put it on another fresh installed RetroPie?

If there isn't, I think it would be great.

herb_fargus

@meleu no automated way but you can easily copy the folders from samba shares to back up you games and scrapes and then just copy them over or you can create a backup image of your SD card altogether.

meleu

@herb_fargus yep, but the files are a little scattered.

I think I can do this, but I'm just suggesting an easy way for the n00bs.

BuZz

can also do it via a usb stick which can backup/restore configs.