hascheevos.sh - a tool for RetroPie/RetroAchievements users

hansolo77

I've been occasionally running the tool against my sets (about once a month) and have started to notice that my rom count (for all in one list) has been steadily decreasing rather than increasing. I backed up the most recent list, then updated the list and ran a diff against the two. There are a bunch of roms no longer in the list. I checked my games and they don't pop with a list anymore, and the RA site no longer lists them. Is this a typical behavior, that games will suddenly disappear? Or is there some pruning going on? It's just alarming to see my list dwindling.

Here's the output of my latest diff.
https://pastebin.com/K6GtSqUv

meleu

@hansolo77 said in hascheevos.sh - a tool for RetroPie/RetroAchievements users:

is there some pruning going on? I

this.

We had so many wrong ROMs linked to games on our database and in the past few weeks we did a big cleanup.

Also, I think I need to update some stuff on my hascheevos tool. It currently isn't able to check if Arcade ROMs have cheevos, for example...

hansolo77

@meleu I know. Just keeping you on your toes. :) I'm liking all the advancements though.

meleu

@hansolo77 I've updated some stuff on the hascheevos code. Now I think it's able to check ROMs for all supported systems. BUT... be aware that to check arcade ROMs you MUST use the script with the --arcade option.

hansolo77

Awesome. I'll give it a whirl! :)

hansolo77

I don't think the arcade system is working. I tried a bunch of different things, and got nothing:

pi@retropie:~ $ ./hascheevos/bin/hascheevos.sh --arcade
Checking JSON hash libraries...
Done!
pi@retropie:~ $ ./hascheevos/bin/hascheevos.sh --single-collection --arcade
Checking JSON hash libraries...
Done!
pi@retropie:~ $ ./hascheevos/bin/hascheevos.sh --system arcade
Checking JSON hash libraries...
Done!

Am I missing something? I also ran the --update prior to do things.

meleu

@hansolo77

Am I missing something?

yeah, you're missing something. But it was my fault, I didn't give enough instructions.

TL;DR: you most likely want this command:

./hascheevos.sh --arcade --system fba

Explaining:

The --arcade option says to hascheevos that we are going to process the zip files by themselves (in other words, there's no need to unzip and check the actual rom file as the script does for another systems), that's the way we deal with Arcade files at RetroAchievements.org.
The libretro core used to have achievements for arcade games is fbalpha, then --system fba makes hascheevos "scrape" data for the ROMs in ~/RetroPie/roms/fba. Of course if you put all your arcade ROMs in the arcade folder, you should use --system arcade, but keep in mind that only those you run with fbalpha are able to support cheevos.

hansolo77

Sounds good. Trying it again now..

Doesn't seem to be working. Went by pretty quickly (a lot less time than any other system).. It didn't find any cheevo games, just ran a lot of :

Checking JSON hash libraries...
Done!
Checking "/home/pi/RetroPie/roms/fba/1941.zip"...
--- 7af6266cc52234b5aa339b16695f7fc4
--- unable to get game ID.

"/home/pi/RetroPie/roms/fba/1941.zip" has no cheevos. :(

Checking "/home/pi/RetroPie/roms/fba/1942.zip"...
--- 519c84155964659375821f7ca576f095
--- unable to get game ID.

"/home/pi/RetroPie/roms/fba/1942.zip" has no cheevos. :(

Checking "/home/pi/RetroPie/roms/fba/1943.zip"...
--- c3395dd46c34fa7fd8d729d8cf88b7a8
--- unable to get game ID.

"/home/pi/RetroPie/roms/fba/1943.zip" has no cheevos. :(

Checking "/home/pi/RetroPie/roms/fba/1943kai.zip"...
--- 46bc905cd32b5819e62b966f6b677ada
--- unable to get game ID.

"/home/pi/RetroPie/roms/fba/1943kai.zip" has no cheevos. :(

etc.

meleu

@hansolo77 keep in mind that we currently (23/May/2018) only have cheevos for the 13 Arcade games listed here.

lilbud

@meleu said in hascheevos.sh - a tool for RetroPie/RetroAchievements users:

only have cheevos for the 13 Arcade games

14 if I'm not burnt out by the end of the week

hansolo77

@meleu said in hascheevos.sh - a tool for RetroPie/RetroAchievements users:

@hansolo77 keep in mind that we currently (23/May/2018) only have cheevos for the 13 Arcade games listed here.

Yeah I know, but still. It didn't generate a new xml for the Arcade system. It's like it didn't detect the hash on ANY games. I re-did my set though, as the original one I had been using was in a split format. I did a merge to reduce all the extra copies of roms on my list. The games still work (Cadillacs and Dinosaurs for instance), they just didn't get added to the list. I don't think it should matter, as I also have a merged set for my other systems too, and they detect fine. RetroArch like the roms fine too. I only mentioned it as a report that it didn't like anything I had, or didn't appear to actually scrape against a hash (like it didn't download an Arcade has library or something).

meleu

@hansolo77 Whoops! My bad bro! I forgot to commit some changes I had only locally.

Could you update and then try it again?

hansolo77

@meleu said in hascheevos.sh - a tool for RetroPie/RetroAchievements users:

@hansolo77 Whoops! My bad bro! I forgot to commit some changes I had only locally.

Could you update and then try it again?

No worries! We're all on the same team. ^_^ I'll give it a go here shortly. I might not be able to report back until tomorrow though. I just got off work today at 2:00pm, and have to be back at work tomorrow at 3:00am. They're having a special "presidential" corporate visit to make sure our store is set and ready for Memorial Day. SMH, they already did that last week, and they're expected to just visit and say "Thanks" tomorrow, but still, Management is all up our butts wanting things to be perfect. The holiday ain't even until Monday lol.

Quick mention... When I run it now, I'm getting a lot of errors of this variety:

Checking "/home/pi/RetroPie/roms/fba/aliensyn.zip"...
--- 0103136b3bf5ad61e4f5187168797bcb
--- game ID: 11972
WARNING: undefined RetroAchievements.org user (see "--user" option).
"/home/pi/RetroPie/roms/fba/aliensyn.zip" has no cheevos. :(

But it does look like it's identifying a game id now.

hansolo77

Lookie what I made. My first ever RetroPie script! Granted, it's not usable for everybody. I use the hascheevos tool in a method most likely different from everybody else. I generate 1 list of all games together (that way I can use the random function to pick a game).

This script I wrote backs up the current list in case something goes wrong. It then updates the tool, then scans the basic console roms. Now with the arcade addition being different, it renames the created list, then scans the arcade roms, then combines the arcade list to the console list. All done. It then prompts if you want to reboot or not (since you need to reload EmulationStation to see the updated cheevo list).

Feel free to use it, hack it up, improve it, whatever. I just wanted to share what I accomplished all on my own, with ample Google searching. :)

https://pastebin.com/7uge2spX

To use: Place in your /home/pi/ path. Execute chmod +x updatecheevos.sh from the terminal. Then as needed, exit EmulationStation via F4 and run ./updatecheevos.sh.

stt

well should've known achievement capable game collections/playlists was an active concern among achievement hunters, google-fu failed to pick this up couple days ago when I was interested so instead hacked this together over lunch
https://gist.github.com/stt/a328ba4c5413a1d9e4dd3db3cc4bf85d
Requires py3+lxml (pip3 install lxml)

Given a retroarch .lpl playlist file it fetches achievement supporting games from ra.org and produces a new playlist of that subset by string matching game names.
The result is obviously just a rough approximation as the names differ in punctuation etc, but it was good enough for me for the moment, so maybe somebody else finds some use for it as well.

meleu

@stt cool.
Now that I have access "to the other side" (I mean, not just a RetroAchievements consumer), I plan to make some improvements on the API so we can get more valuable info from those http requests. :)

hansolo77

Hey @meleu
Not sure if you've got a moment to answer a quick question, but here goes:

Does the hascheevos tool only search for roms in specific folder paths?

I'm in the process of moving out all my hacked roms to their own dedicated folders. Using @TMNTturtlguy's awesome ComicBook theme. He's designed it so you could create a folder like ~/roms/nesh and that would correspond to NES Hacks. The way I had my system set up before, I had my hacks inside the same folder with my non-hacked roms. This worked out just fine with the hascheevos tool, but it wasn't very well organized. I liked the idea of the separate folder idea. I just wasn't sure if the tool can accommodate it or not. If it's not able to currently, is there any way to add this capability? I thought about maybe modifying the script but I'm not sure where to begin. I know I'll need at least gbah, gbh, genh, nesh and snesh. There are other systems that have hacks too, such as PCE, FDS, GBC, N64, SEGA32X, SEGA GAMEGEAR and SEGA MASTER SYSTEM, but the ComicBook theme doesn't support those (yet). I'm sure RetroAchievements will be developing cheevos on these games in time. In fact, I didn't even know about hacks until I saw cheevos for them.

mitu

@hansolo77 Since the program looks at the ROM itself, regardless of where it's stored, I don't think there is a problem if you have a separate folder for hacks.

hansolo77

I think my concern stems from the program being able to find the roms in the first place. Like, does it look for roms in specific folders based on what systems are currently available at RetroAchievements? When I've watched it run, it looks like it downloads a rom list for a given system, then checks the files located in each system's folder one by one. It doesn't appear to be doing a scan of EVERY rom, just the ones in folders designated as a compatible system. So say for instance that I have completely renamed my megadrive folder as genesis (I haven't, this is just an example). The hascheevos tool then goes to scan but skips the Genesis folder completely because it's looking for Megadrive. I have the roms, but not in the same folder it wants, so it doesn't scan for it. Does that make sense?

mitu

@hansolo77 The script has a --system parameter for specifying which system under $HOME/RetroPie/roms/ should be analysed. I'm sure you can also modify the script to extend the list of default folders it looks, it's just a wrapper around the binary (which looks only at a ROM at a time).