what about adding a cheevos flag in gamelist.xml ?

hansolo77

REQUEST - Update the tool to spit out additional texts. Specifically, I'd like to see what games exist with cheevos that I don't have, so I know to go out and find them.

REQUEST - Update the tool to keep a log of the date you last checked for cheevo games, then compare the dates with TODAY to see if it needs re-scraped.

QUESTION - I'm happy with the way it works now, but what if I choose to run the scraper again.. will it make duplicate copies in the texts, thus making me have multiple copies of the same game when I run the tool to auto-add it to a custom system?

PokeEngineer

@robertybob

I second this.

meleu

@hansolo77 said in what about adding a cheevos flag in gamelist.xml ?:

REQUEST - Update the tool to spit out additional texts. Specifically, I'd like to see what games exist with cheevos that I don't have, so I know to go out and find them.

I would love this feature too, but I'm afraid it's not going to happen.

Why?

Because with the [currently?] RetroAchievements.org isn't smart enough to provide a list with all games that have cheevos. Implementing what you're suggesting with the current "technology" would stress the RA server (and would take ~~ages~~ several hours to accomplish the task).

REQUEST - Update the tool to keep a log of the date you last checked for cheevo games, then compare the dates with TODAY to see if it needs re-scraped.

Not sure what you mean. Can you expand that?

QUESTION - I'm happy with the way it works now, but what if I choose to run the scraper again.. will it make duplicate copies in the texts, thus making me have multiple copies of the same game when I run the tool to auto-add it to a custom system?

No. The add-game-to-custom-system.sh doesn't create duplicated links.

hansolo77

@meleu said in what about adding a cheevos flag in gamelist.xml ?:

@hansolo77 said in what about adding a cheevos flag in gamelist.xml ?:

REQUEST - Update the tool to keep a log of the date you last checked for cheevo games, then compare the dates with TODAY to see if it needs re-scraped.

Not sure what you mean. Can you expand that?

Just as means to know if I should scrape again.. Like in the Retropie-Setup script, it tells you when the last time was that you last updated the script. I'd like to have a similar system in place, that checks when the last time was you last scraped. Even better would be a simple version number check. See if your version is the same you have in your github databases. That way I'm not needlessly re-scraping all my games again a week later if you haven't updated the databases.

This actually brings up another thing I noticed. When it was going through checking each rom, it was actually going back and saying something like "checking on retroachievements". Is it doing that, or is that just a bit of bad wording since you have it scraping from github?

meleu

@hansolo77 said in what about adding a cheevos flag in gamelist.xml ?:

Just as means to know if I should scrape again.

OK, I got it. I'll try to think in something to avoid re-scraping everything. Would you be kind enough to make those requests on the repository issue tracker? I think I'll forget if this stays here on the forums. :-)

https://github.com/meleu/hascheevos/issues

When it was going through checking each rom, it was actually going back and saying something like "checking on retroachievements".

It's really doing that. If the script fails to find info in the github files, it tries to get from RetroAchievements.org.

hansolo77

@meleu said in what about adding a cheevos flag in gamelist.xml ?:

OK, I got it. I'll try to think in something to avoid re-scraping everything. Would you be kind enough to make those requests on the repository issue tracker? I think I'll forget if this stays here on the forums. :-)

https://github.com/meleu/hascheevos/issues

Will Do!

It's really doing that. If the script fails to find info in the github files, it tries to get from RetroAchievements.org.

Well it was doing it for what appeared to be ALL of them. So if the server crashed yesterday or was slow, it was probably my fault.

lilbud

I think cheevos are fine, but what would really be a worthwhile addition would be backloggery support: http://www.backloggery.com/

meleu

Hello friends!

I've got distracted with PS Vita hacking and hunting RetroAchievements for some Mega Man games. But now I finally made an useful progress on my "cheevos scraper".

You can check it in this repository: https://github.com/meleu/hascheevos

The README has instructions on how to use it.

An example of how useful the tool can be: In order to update your NES' gamelist.xml adding <achievements>true</achievements> for every game that has cheevos, you just need to use this command:

$ ./hascheevos.sh -u USER -p PASSWORD --system nes --scrape

Now let's wait if @pjft can find sometime to work on EmulationStation code in order to make the <achievements> flag filterable. :-)

hansolo77

Exciting progress! So just to be clear.. in order to access the new feature (adding the flag), I will need to rebuild the binary correct? I'm not particularly eager to get this specific feature right now, I'd just like to know. Quick question.. is there a reason a binary hasn't been made yet, to skip having to build it? 2nd quick question.. any progress been made on checking to see what new roms exist without having to completely rescrape?

Kudos!

cyperghost

@hansolo77

in order to access the new feature (adding the flag), I will need to rebuild the binary correct?

Yes!

Quick question.. is there a reason a binary hasn't been made yet, to skip having to build it?

As @pjft is the develeoper for adding new flags to XMLs (as he did by adding favorite selections) it's up to him for a release.

So things will surely be done, just let them grow. I highly appreciate such a possibility as the RetroAchievements makes the gaming experience really better ... :)

@meleu
I will point to your latest thread with my RA-automate-script

pjft

@hansolo77 This hasn't been developed yet. I do not have an ETA for this, it can still be weeks or months, unfortunately.

meleu

@hansolo77 said in what about adding a cheevos flag in gamelist.xml ?:

in order to access the new feature (adding the flag), I will need to rebuild the binary correct?

Not sure if you're talking about EmulationStation or the hascheevos tool. If it's about EmulationStation, there's nothing related to achievements implemented yet.

If it's about the hascheevos tool I suggest you to delete your current hascheevos directory and then git clone it again.

rm -rf hascheevos
git clone --depth 1 https://github.com/meleu/hascheevos
cd hascheevos
make

You won't need to do it everytime you want to update the tool, it's just now. Because I implemented some new features since you downloaded the tool, including an option to --update. ;-)

Quick question.. is there a reason a binary hasn't been made yet, to skip having to build it?

Again, if you're talking about the hascheevos, it's not an actual binary, it's a bash script. But I implemented the --update feature to avoid typing many commands.

I also implemented some features that will help to keep the repository synchronized with RetroAchievements.org database. Maybe you, @grenade44 and other cheevos enthusiasts could help me with this task. I will talk more about these features later. ;-)

2nd quick question.. any progress been made on checking to see what new roms exist without having to completely rescrape?

In theory yes. I have some ideas on how to do it and did some tests. But nothing has been implemented in hascheevos yet.

cyperghost

@meleu Thx for clarification @hansolo77 - I think we were on wrong track as there was talked about binaries
Actually the cheevos-scraper is a script - not a binary ;)

meleu

@cyperghost there's just a small C program in the repo (cheevoshash.c - the hash calculator) that needs to be compiled to a binary. But it's really small, compiles almost instantaneously. It is done and working exactly as expected. No need to receive updates anymore.

pjft

@meleu Hi!

So, I am yet to get everything set up for a proper development environment on my end, alas.

Still, I tried to put something together on my free time. I have NOT tested it, and worst of all, I have not been even able to try to compile it :(

That being said, it doesn't seem to be tremendously difficult so if you'd want to try to compile it on your end and see if it does what's expected, here's the branch:

https://github.com/pjft/EmulationStation/tree/Achievements

I'll try to compile it later in the week as soon as I can.

Hope this helps.

meleu

@pjft Nice blind coding bro! Time to remember that couplet:

couplet

I didn't test on a Raspberry Pi, but compiled fine here on my RetroPie x86 (Linux Mint). Just noticed this warning while compiling (not sure if it's related with the "achievements" changes):

After compiling I tested with a gamelist.xml populated with some <achievements>true</achievements> and it works as expected.

Huge thank you for this!!!

cyperghost

@meleu Haha! your new toy is ready setted ;)
Freshly baked by @pjft awesome C++ cuisine - Que aproveche!

pjft

@meleu @cyperghost Thanks - it's always good to remember that couplet :)

If you could thoroughly test that, it'd be great as then I'd submit the PR. I believe there are plans to have a code freeze for a bit, so I'd like to get this small change through beforehand.

Thanks.

meleu

@pjft Hello friend. Finally took some time to compile it on my pi2, and it runs just fine. Exactly as expected. I also tried to edit metadata of a game enabling/disabling achievements. Really cool.

I did a very well-behaved test. I didn't try to break anything. If you think it needs a more "hardcore" test, I think I will have some free time later in this week. Any particular test you want me to do?

So far I'm really happy with what you made. You hit my feature request in the bull's-eye! Huge thanks for that! :D

pjft

@meleu thank you good sir. No need to try to break it, I didn't do anything really out of the ordinary, just followed the lead from other such attributes. Just wanted to get your hands on testing to confirm that it worked as intended and that I didn't miss anything.

Did you have a chance to release the scraper, or did you per chance had any further discussions about getting this into @sselph 's scraper in one way or the other?

I'll try to submit the PR later this week. If anyone wants to test it further that'd be very much appreciated.

Thanks!