Metadata Improvements in EmulationStation

Nismo

@MWGemini Yes I supposed that it's a lot of work, but I don't loose my hope...

mattrixk

@screech: I appreciate all the work you've put into this excellent scraper. While the feature of downloading media in different sizes would be fantastic, for the time being I have a work around of just downloading everything in a larger size and then resizing them manually using FastStone Image Resizer. It's a bit more work, but it does the job.

Also, there is no real rush on that, as I mainly want this ability for when/if we get these new image metadata elements added into ES.

@MWGemini: that sounds like it would be exceedingly awesome.

MWGemini

@mattrixk Which part, the automated builds, the multi-language support, or the visual theme editor?

lilbud

@MWGemini theme editor for sure

mattrixk

@MWGemini I was mainly thinking about the automated builds. I personally don't have any use for the language support (but I do realise how great it would be for people who need/want it). I'm also not really sure how the visual theme editor would work, but if you want to take a crack at it, I'll gladly try testing it out/giving some input.

MWGemini

@mattrixk @lilbud right now my priority is in helping @fieldofcows with the stuff he is working on. He's the hero of this thread and I think he's doing a lot of great development on EmulationStation. I've only been testing the work he's done, I haven't done any development on ES, so I don't want to steal his thunder.

If/when I get a bit more free time, I want to help @fieldofcows implement the changes mentioned in this thread (assuming he wants/needs the help), and after that, I'll probably try to pull in language support, grid view, theme system improvements, and then I'll work on the visual editor (assuming someone hasn't beaten me to it by that point).

Since the thread has gone a bit of topic, I'll try to steer it back. @fieldofcows, I've modified the database design document I sent you, thanks to input from @mattrixk , @sselph, and @screech. I think we have a very solid design in place, and if you agree with it, I can start writing some of the SQL for it (or build/test automation, or something else, if you have a request/idea) this week, if you want.

mattrixk

@MWGemini said in Metadata Improvements in EmulationStation:

@fieldofcows [is] the hero of this thread and I think he's doing a lot of great development on EmulationStation.

Hear hear!

fieldofcows

@MWGemini said in Metadata Improvements in EmulationStation:

He's the hero of this thread

Aww thanks! I'm letting you lot discuss these issues for a little while while I finish the WSOD issue. I'll get back to the metadata in due course...

MWGemini

@herb_fargus @Buzz I'd love to get your input on the design doc I've been working on as well. I'm assuming you're both able to send private messages, so if you'd like to PM me your email address, I can share the Google doc with you. I've been getting the input of some of the community's biggest contributors (scraper authors, theme designers, etc.), and it's still a work-in-progress, but I think we've got a very robust design.

BuZz

@MWGemini no pm on forum. Can't you just post a link? No need to discuss in private.

darknior

Woaw ... really impressive work here ... i dream about this option from a long time :D
More METADATA, FILTERS, VIEW ... FASTER ... so incredible ;)

sp00k

has anyone experimented with different xml parsers?
i use perl for hacky things, and reading my largest gamelist.xml went from 48 -> 7 seconds by using a different parser.
eg: sax instead of dom

Nismo

@sp00k interesting... how do you do that?

More info about sax and dom: http://stackoverflow.com/questions/6828703/what-is-the-difference-between-sax-and-dom

jdrassa

@sp00k I am not sure that switch the xml parser would have much impact here. Any speed improvements would only effect startup. Also the speed improvements you saw may not even be realized in EmulationStation. ES needs to read every element in the xml file. If your tests were only reading specific elements, then they wouldn't be indicative of the performance we would see in ES.

sp00k

@jdrassa
my hacky perl involved reading the rating and path of every game
it then creates launch scripts for any game with non-zero ratings under a new emulator "ichiban"
i'm pretty sure the xml is fully parsed

i'll dig a little deeper when i get a chance
but saving gamelists is a cpu-bound operation
ie, the bottleneck is not:
reading the xml (which are only a few megs at most)
querying the filesystem