EmulationStation mod
-
@meleu Are most of your games that are in those big lists (1000+ roms) not scraped or have metadata?
-
@jacobfk20 man! really really sorry! I forgot to give you an important feedback! The ES from Metadata branch didn't even touch the gamelist.xml.
I saw the last changed time with
ls -land all the gamelist.xml was last changed when I last tested the default RetroPie ES with "save metada on exit" turned on. And it was in 2016-Aug-15. After I installed the Metadata these files didn't change and I'm sure that the "save metadata on exit" is on.Now, answering your question:
@jacobfk20 said in EmulationStation mod:
Are most of your games that are in those big lists (1000+ roms) not scraped or have metadata?
The romsets are scraped but some ROMs failed to get scraped. I don't have the exact number...
-
@meleu Yes, if nothing changed in that system's metadata it will not bother saving anything, so gameList will not be touched. A gamelist will only be saved to if you change any metadata on any of that system's games.
-
@jacobfk20 but I played a bunch of games to make the tests (nes, snes, megadrive and mastersystem). I'll test it again detailing each steps. More info tonight.
-
In that build playing a game doesn't justify saving metadata. Times played, and last time played do not trigger metadata to save. Editing game metadata and scraping will trigger a save.
Anyway, would you be willing to run two tests for me? It will require running two different builds though. I think I may have found the biggest cause of what makes saving metadata take so long and I would like to have a before and after comparison.
-
@jacobfk20 said in EmulationStation mod:
In that build playing a game doesn't justify saving metadata. Times played, and last time played do not trigger metadata to save. Editing game metadata and scraping will trigger a save.
Sorry if I misunderstanding, but then I think it's almost useless :-)
I only scrape and/or edit gamelist.xml when ES is not running. I only see use cases for it when using the ES scraper and using the trick to scrape screenshots.Anyway, would you be willing to run two tests for me? It will require running two different builds though. I think I may have found the biggest cause of what makes saving metadata take so long and I would like to have a before and after comparison.
Aye aye Captain!
Since I don't have enough C++ skills to contribute with the code, I'm happy to help as a tester. :-)
-
@meleu lol, I could have it save when "times played" and "last time played" are updated.
These two builds I want you to run save everything no matter what (unless you have "save on exit" turned off) but this time they log how long in seconds it took per system and how many loop cycles to terminal and es_log. So when you run the first build, save the es_log (~/.emulationstation) then run the next build and save it's log somewhere and send them both to me.
The second build could potentially lower the save time by quite a bit.
I'll post the builds and full instructions I want you to run here in a few. -
@meleu Alright, The two builds are ready. First thing: They are pretty safe but PLEASE backup your gamelists just to be 100% safe. I've ran these a tests a good chunk of times but you can never be too sure.
- Copy over the emulationstation in test1 and run it. (you can run this anywhere on your pi it doesn't have to overwrite what is in /opt/retropie/supplementary/emulationstation) You'll need to set it as executable with 'chmod +x emulationstation' (no quotes)
- After ES loads, make sure you have "Save on Exit" turned on and exit ES. It'll save normally (this will take the original wait time you normally would expect)
- After its fully closed, grab a copy of the es_log and name it something like "es_log-1" or something (Log is located in ~/.emulationstation)
- Copy over the emulationstation in test2, make it executable, and run it.
- After loaded, Make sure save on exit is still enabled and close ES. ( This time should be reduced. It reports times as well.)
- Copy and rename the new es_log.
- Send me the two es_logs you saved.
Grab the two builds from my drive
You can email me the logs at my gmail. (My gmail is the same as my username here)
No hurries :)
-
@jacobfk20 Done! The test2 build is much faster in updating the metadata! Tested both with just exiting ES, and exiting after playing 2 or 3 games. I hope it helps.
-
@meleu Thank you! It looks like I was correct. You went from a hair over 5 minutes to 9-10 seconds on exit now. It wasn't having much impact on my exit time (2 seconds to 1.4 seconds) but I'm glad to see it works on someone that has long exit times.
I did the math with the data given from the es_logs and the new method saves you 2,967,762 cycles in a while loop. That's insane!!
Is 10 seconds manageable? lol, I don't think I can get it any faster.
Anyway, I'm going to clean this branch up and may do a PR. Thanks again for doing the test. :)
-
Where would be able to theme the carousel bar rank on your to do list?
-
@lilbud I have a branch that I work on occasion that will allow that. So far all it does is just allow you to move the little bar beneath that shows game count and change its color. As for priority... I'm not sure. GridView has calmed down a bit and I just got OSK running good so it may get moved up.
-
@jacobfk20 said in EmulationStation mod:
Is 10 seconds manageable? lol, I don't think I can get it any faster.
10 seconds is great! :D
Could you put a message saying to the user that ES is updating the metadata or something similar? The first time I faced this problem I thought the RetroPie crashed ("Hey! It is not shutting down!"). A simple message would be enough to calm down the user. :-)
-
@meleu I will consider adding a message on shutdown. I'd like for you to do one final test for right now if you wouldn't mind. I cleaned the branch up and changed how it works slightly and I just want to make sure it still gets you down to 10 seconds. It doesn't keep time this time but it keeps cycles.
You can get this test build in the usual place.
Please send a copy of the es_log when done :)
-
@jacobfk20 Done! It took 8 seconds to exit and the lastplayed/timesplayed were correctly updated.
I have noticed some lvl1 warnings about data type:
lvl1: requested mismatched theme type for [detailed.md_lastplayed] - expected "datetime", got "text"But the lastplayed info displayed in ES is correct.
You can see more details in the files I've sent.
-
@meleu I get a mismatch warning from the official retropie build but it's to do with ratings. This could be from the theme you're using. Does it report that warning when you run the official retropie branch?
-
@jacobfk20
I checked more carefully now, with 3 different themes: Pixel, Simple and Carbon. The mismatch occurs only on Carbon. All the tests were made with the official RetroPie ES. So, it's not an issue with your branch. Phew~ -
I caused the error with carbon - will fix.
-
Looking forward to all the features being worked on here! Keep up the great work.
-
@meleu please can you update the carbon theme and let me know if the error is gone now ?
To help us help you - please make sure you read the sticky topics before posting - https://retropie.org.uk/forum/topic/3/read-this-first
Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.
Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.