EmulationStation mod
-
@jacobfk20 I think that may be wise (albeit likely more work for you). Anyways this forum post is getting pretty long I prefer to keep development discussion on github where its closer to the code. I'll log an issue for this on your RPI repo and carry it on there.
-
@herb_fargus I'm working mostly from https://github.com/jacobfk20/EmulationStation and not the RPiE fork.
-
@jacobfk20 If you'd like to enable the issue tracker on your emulationstation fork I'd be happy to log the issues there instead
-
I didn't know it was disabled. It's enabled now.
-
I hope this makes it into RetroPie 4 in some form, the grid view will keep me happy until Attract-Mode is finally, officially, included as a choice alongside ES!
-
@robertybob given that RC images are out, I don't think that's likely, but I don't know the release procedure. I think it's more likely to get a script to change ES versions
-
Just for curiosity:
Could changing from xml to another database avoid/minimize this problem?
(exit emulationstation take ages when "save metadata on exit" is on) -
@meleu Probably. Wouldn't have to dump metadata to a file on exit anymore. A db would keep everything up to date as it is modified.
-
@jacobfk20
Then you guys have here one more vote to go ahead with this changing... :-) -
@meleu It would be interesting to test that out.
-
@jacobfk20 there are a couple things to consider on that front, in the past a bit of testing has been done and response times are somewhat comparable and it seemed that the only real benefit for a SQL backend came for the rom hoarders who have like 20,000 games. Im also not sure if gamelists/scrapes are included in that or still stay as separate xmls? Anyways there is more testing to be done on that front and would be interesting to see the results.
The other semi-related thing is the white screen of death issue that is prolific with any themes that are 1080p with unique wallpapers. If there were a way to load each system dynamically kinda like you've tried to do with the grid boxart that would open up more variety of themes but I don't know how extensive code changes for that would be
-
@herb_fargus I'm glad to see that it has been attempted. I need to take a closer look at metadata sometime still. GridView's dynamic loading will get better and I will see what happens when I apply something similar to system view.
-
How much resources do SQLite/MariaDB consume? Memory, CPU, Power? Should the daemon be kept running in the background when firing up an emulator, or should it be started, shut down, started, shut down?
DBs are mainly for very large datasets (some of them have 100 different algorithms for sorting data fast, depending on how you want the data sorted). We don't need that?
DBs are built with transactional integrity in mind. We don't need that?
DBs are built for multiple concurrent connections. We don't need that?
DBs are built for network access, with security, access control, logins and such. We don't need that?
When I develop on Windows .Net has really great features for dealing with XML and datasets. I vote for finding such a library to work with if you want to load static data, change something and write it back.
DBs consume too much resources because they are build for very heavy and complicated situations.
Writing 1 MB of XML data from a dataset should take no more than 10 ms. Just like any other 1 MB file.
-
I took a quick look at Metadata and I love how it is formatted. I see that on exit MetaData will append everything to xml regardless if it has been modified or not. It could be made to where metadata only saves to file when something has been modified, that would be a simple approach and it could make it to where you don't have to go re-enable save meta data in the menu when you want to save a change.
It could even keep an array of indexes and then only modify those nodes on the xml doc on exit but that may not actually speed it up because it will now have to search the xml nodes for the correct child to change. I don't know fast the pugixml library would do that.
I actually want to know exact time it takes for someone with long exit times. How big are their game lists?
I don't know how much SQLite uses in resources as I've never used it in anything. If it has been looked into before and it has shown to not have any real improvements then maybe time should be spent looking for a different solution than going to a sql db.
-
@jacobfk20 Often, you load the XML into memory as a tree of data. You do search against that tree with XPath or other methods. Algorithms for doing so are very optimized. Changing data is just like changing a value for any variable. When writing to disk, you write the full file with XML-serialized data tree. If you do not need table locking, concurrent users, security, joining multiple tables, group by, then XML is fast and small footprint (Or the developed applications and frameworks dealing with data trees). In .Net on Windows you can load XML into memory and query that dataset with Linq, more extensively than SQL.
-
@jacobfk20 said in EmulationStation mod:
@meleu Probably. Wouldn't have to dump metadata to a file on exit anymore. A db would keep everything up to date as it is modified.
Can't this updating method be done with xml?
@jacobfk20 said in EmulationStation mod:
I actually want to know exact time it takes for someone with long exit times. How big are their game lists?
I timed it here: exactly 4 minutes and 58 seconds. But we can say 5 minutes.
game list size:
Atari2600: 1443
FBA: 1
GameBoy Advance: 10
Master System: 545
Mega Drive: 1239
NeoGeo: 7
NES: 2141
PC Engine: 2
SNES: 2271 -
@meleu something's wrong, how can we troubleshoot? XML is just like any text file with a strict defined way of representing tree data. Databases are by no means faster... They do solve some particular problems, but that comes with a price.
-
@meleu thank you for doing that test. How about we just save gamelist/metadata on systems that have been flagged with data that has been modified? Like if you only change something in Master System then it will only take 30ish seconds to shut down. Or if you change nothing it'll shut down in normal time.
@Arcuza I'm not fully sure, but it looks like MetaData creates a complete new node tree from scratch every time it saves. I could be missing something though.
-
@jacobfk20 just for reference someone added a PR this morning to address this issue somewhat. Doesn't solve it completely but should make it faster, I still need to test it.
If you read the documentation it will answer 99% of your questions: https://retropie.org.uk/docs/
Also if you want a solution to your problems read this first: https://retropie.org.uk/forum/topic/3/read-this-first
-
@Arcuza
I'm not the first to have this issue. It's in the RetroPie's FAQ: https://github.com/RetroPie/RetroPie-Setup/wiki/FAQ#why-does-shut-down-and-reboot-take-agesIf I understood right, it happens because emulationstation dumps all the metadata (no matter if some change was made or not) to a file on exit. When I got these 5 minutes, I just turned on my raspi2, waited for emulationstation start, and then "exit emulationstation". I didn't change anything and it took 5 long minutes to exit.
Well, I'm not an DB approach advocate, but I see some emulationstation hackers tending to it. As @jacobfk20 said: "A db would keep everything up to date as it is modified." And I saw @Zigurana arguing in favor of a DB approach in other posts [edit: not exactly, as he says below].
This discussion reminded me an old MAME frontend I used years ago on Windows. They had several options of filtering the game list (no clones, beat'em'up, fighting, 2 players, 4 players, and many many others). I think they use a db for it.
Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.
Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.