RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login

    Game statistics

    Scheduled Pinned Locked Moved Projects and Themes
    script
    44 Posts 7 Posters 5.6k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • langestL
      langest
      last edited by langest

      Hi,

      I came up with a fun idea, to get statistics on which games you play and for how long you play them.
      I added the following to runcommand-onstart.sh

      echo $(date),start,$1,$2,$3,$4 >> ~/RetroPie/game_stats.log
      

      and runcommand-onend.sh

      echo $(date),end,$1,$2,$3,$4 >> ~/RetroPie/game_stats.log
      

      As long as the games doesn't crash I think this will be a good enough log to get some aggregate data on which games you play and average session length for each game and total play time et.c.

      Haven't built a script to convert the data yet, but as you can see it will be logged in a csv format so it should be easy to write a python program that parses the file.

      Just thought I would share this in case someone is interested. I'll update if I ever write a program to parse the data.

      EDIT:
      I made the program that parses the log. Here is the code: https://github.com/langest/RetroStats

      1 Reply Last reply Reply Quote 5
      • B
        backstander
        last edited by

        @langest
        I'm going to try this out when I get home! Thanks!

        1 Reply Last reply Reply Quote 0
        • ClydeC
          Clyde
          last edited by

          Sharing is caring, thanks!

          1 Reply Last reply Reply Quote 0
          • langestL
            langest
            last edited by langest

            I started implementing the python script I mentioned before and the comma doesn't make a good delimiter in the script since that is a quite common character in rom names.

            echo $(date)'|'start'|'$1'|'$2'|'$3'|'$4 >> ~/RetroPie/game_stats.log
            echo $(date)'|'end'|'$1'|'$2'|'$3'|'$4 >> ~/RetroPie/game_stats.log
            

            I updated the delimiter to pipe symbol. So update your run command to use this instead and remove the old game stats log. I'll make sure to share my python script when it's ready.

            Also, just to be sure, pipe symbol |is quite uncommon in rom names right? If not, which character would make a good delimiter?

            1 Reply Last reply Reply Quote 1
            • langestL
              langest
              last edited by langest

              Here is the code: https://github.com/langest/RetroStats
              How to run:

              $ python retro-stats/game_stats.py -h                                                                                                                                            
              usage: game_stats.py [-h] -f FILE [-s SKIP_SHORTER_THAN]                                                                                                                                                         
              
              Process some integers.
              
              optional arguments:
                -h, --help            show this help message and exit
                -f FILE, --file FILE  path to the stats file
                -s SKIP_SHORTER_THAN, --skip-shorter-than SKIP_SHORTER_THAN
                                      skip sessions shorter than this number of seconds,
                                      defaults to 120
              
              

              or simply

              $ python retro-stats/game_stats.py -f game_stats.log -s 120
              

              It will print something like:

              {'gbc': {'Wario Land 3 (World) (En,Ja).gbc': {'times_played': 3, 'total_time_played': '0.0h 7.0m', 'average_session_length': '0.0h 2.0m', 'median_session_length': '0.0h 2.0m'}}, 'nes': {'Smurfs, The.7z': {'times_played': 2, 'total_time_played': '0.0h 10.0m', 'average_session_length': '0.0h 5.0m', 'median_session_length': '0.0h 5.0m'}}, 'kodi': {'': {'times_played': 1, 'total_time_played': '0.0h 25.0m', 'average_session_length': '0.0h 25.0m', 'median_session_length': '0.0h 25.0m'}}}
              

              As of now it prints number of times played, average session length, median session length and total time played.

              What I am planning to add in the future:

              • Global and per console top list
              • Time since last played
              • Prettier print
              • Choose delimiter via argument

              Also clean up the code and make a proper package of it.

              cyperghostC 1 Reply Last reply Reply Quote 1
              • cyperghostC
                cyperghost @langest
                last edited by

                @langest so an entry with less of 120s of playtime is ignored by default I assume.

                langestL 1 Reply Last reply Reply Quote 0
                • langestL
                  langest @cyperghost
                  last edited by

                  @cyperghost Yes that is correct. In case you accidentally started the wrong game we don't want don't count it. But you can just pass in -s 0 and every entry will be counted.

                  1 Reply Last reply Reply Quote 0
                  • WeirdHW
                    WeirdH
                    last edited by

                    Very cool, this would make an awesome addition to a theme, if it could be incorporated.

                    langestL 1 Reply Last reply Reply Quote 1
                    • langestL
                      langest
                      last edited by langest

                      I have put in more or less the features I want in the tool now. I will probably add some more documentation on how to use it.
                      If you want to try you can use it like this:

                      $ python3 retro-stats/game_stats.py -f game_stats.log -s nes -c times -m 120                                                                                                         
                      1 Smurfs, The.7z for nes, played 2 times, time played: 0:08:33, avg: 0:08:33, median: 0:08:33                                                                                                                    
                      2 Balloon Fight (USA).zip for nes, played 1 times, time played: 0:11:05, avg: 0:11:05, median: 0:11:05                                                                                                           
                      3 Adventures of Lolo (U) [!].nes for nes, played 0 times, time played: 0:00:00, avg: 0:00:00, median: 0:00:00
                      
                      $ python3 retro-stats/game_stats.py -f game_stats.log                                         
                      1  for kodi, played 2 times, time played: 1:13:47, avg: 0:36:53.500000, median: 0:36:53.500000                                                                                                                   
                      2 Wario Land 3 (World) (En,Ja).gbc for gbc, played 3 times, time played: 0:43:21, avg: 0:14:27, median: 0:06:03                                                                                                  
                      3 Crash Team Racing [SCUS-94426].cue for psx, played 1 times, time played: 0:16:20, avg: 0:16:20, median: 0:16:20                                                                                                
                      4 Balloon Fight (USA).zip for nes, played 1 times, time played: 0:11:05, avg: 0:11:05, median: 0:11:05                                                                                                           
                      5 Wild Arms.cue for psx, played 1 times, time played: 0:10:49, avg: 0:10:49, median: 0:10:49                                                                                                                     
                      6 Super Mario 64 (U) [!].z64 for n64, played 1 times, time played: 0:10:31, avg: 0:10:31, median: 0:10:31                                                                                                        
                      7 Smurfs, The.7z for nes, played 1 times, time played: 0:08:33, avg: 0:08:33, median: 0:08:33                                                                                                                    
                      8 Crash Bandicoot 3.cue for psx, played 1 times, time played: 0:05:30, avg: 0:05:30, median: 0:05:30                                                                                                             
                      9 Super Smash Bros. (U) [!].z64 for n64, played 1 times, time played: 0:03:27, avg: 0:03:27, median: 0:03:27                                                                                                     
                      10 Super Mario Bros. Deluxe (U) (V1.1) [C][!].gbc for gbc, played 0 times, time played: 0:00:00, avg: 0:00:00, median: 0:00:00                                                                                   
                      11 Adventures of Lolo (U) [!].nes for nes, played 0 times, time played: 0:00:00, avg: 0:00:00, median: 0:00:00                                                                                                   
                      12 Bubble Ghost (U) [!].gb for gb, played 0 times, time played: 0:00:00, avg: 0:00:00, median: 0:00:00
                      

                      For more information use -h, ask in this thread or raise an github issue.

                      $ python3 retro-stats/game_stats.py -h                                                                                                                                               
                      usage: game_stats.py [-h] -f FILE [-c CRITERIA] [-s SYSTEM]                                                                                                                                                      
                                           [-m MINIMUM_SESSION_LENGTH]
                      
                      Calculate some play statistics for your retro gaming
                      
                      optional arguments:
                        -h, --help            show this help message and exit
                        -f FILE, --file FILE  path to the stats file
                        -c CRITERIA, --criteria CRITERIA
                                              which criteria to order by, available options are:
                                              total (time), times (played), average (session
                                              length), median (session length), defaults to total
                        -s SYSTEM, --system SYSTEM
                                              the system you want statistics for, if omitted, will
                                              use all systems
                        -m MINIMUM_SESSION_LENGTH, --minimum-session-length MINIMUM_SESSION_LENGTH
                                              skip sessions shorter than this number of seconds,
                                              defaults to 120
                      
                      cyperghostC 1 Reply Last reply Reply Quote 0
                      • langestL
                        langest @WeirdH
                        last edited by

                        @WeirdH said in Game statistics:

                        this would make an awesome addition to a theme, if it could be incorporated.

                        I totally agree. I don't know what support themes have to add custom data fields. I guess that is up to the theme, emulation station or pegasus devs et.c. to figure out.

                        1 Reply Last reply Reply Quote 0
                        • langestL
                          langest
                          last edited by

                          @mitu Do you think this would fit somehow into retropie? And maybe you know who else might have a say if it could be part of the system.
                          Maybe it could fit into a menu entry in the emulation station start menu that lists your statistics?

                          mituM 1 Reply Last reply Reply Quote 0
                          • mituM
                            mitu Global Moderator @langest
                            last edited by

                            @langest If you'd like to add something to RetroPie, just create a Pull Request on the Github's project.
                            Personally, I think it's nice, but it's not something that I see included in the base image. Don't let this discourage you from developing it further, though :).

                            1 Reply Last reply Reply Quote 2
                            • langestL
                              langest
                              last edited by

                              I added support for using skyscraper cache for getting rom names to get a prettier print. However, it is much slower. I haven't profiled the implementation, but I might optimize this at some point. Also, it doesn't seem to work for all the systems. Probably has to do with which files skyscraper uses to calculate the sha1 sum.
                              If you want to try it:

                              retro-stats/game_stats.py -f ../../RetroPie/game_stats.log -t ~/.skyscraper/cache/
                              

                              Pass the path to skyscraper cache to the -t parameter.

                              1 Reply Last reply Reply Quote 0
                              • cyperghostC
                                cyperghost @langest
                                last edited by cyperghost

                                @langest I see this as usefull information for the MAME RoWs ;)

                                Hmmm.... I think this could be made better by using bash dialogs.
                                So you parse your output to info box and you can scroll and select with joypad.
                                Then I'm sure you will hit a wider audience. If you can do some coding in python then you can also set some bash scripts ;)

                                langestL 1 Reply Last reply Reply Quote 1
                                • langestL
                                  langest @cyperghost
                                  last edited by langest

                                  @cyperghost
                                  Yeah, that could be the next step. You don't really need to rewrite the script in bash though.
                                  You can call the python script from bash and just format the output nicely into an info box. Would be a nice project to learn some bash.

                                  Seems like the sha1 sum is different for the arcade games as well. Maybe has something to do with them being ziped.
                                  I think I will try to fix this problem before I create a bash module.

                                  Maybe @muldjord can point me in the right direction to figure out how and for which files skyscraper calculates the sha1? (compressed files and .cue are the ones that doesn't work)

                                  cyperghostC muldjordM 2 Replies Last reply Reply Quote 1
                                  • cyperghostC
                                    cyperghost @langest
                                    last edited by

                                    @langest said in Game statistics:

                                    You don't really need to rewrite the script in bash though.

                                    This wasn't my intention at all ;)
                                    You explainded later the correct situation

                                    You can call the python script from bash and just format the output nicely into an info box. Would be a nice project to learn some bash.

                                    👍

                                    1 Reply Last reply Reply Quote 1
                                    • muldjordM
                                      muldjord @langest
                                      last edited by muldjord

                                      @langest Depends on the module. For ScreenScraper it does a sha1, md5, crc32 and exact filename search. If either of these return a unique result, ScreenScraper will hand back the game data. For the other modules it is filename search based. So if you have a good filename, you will have pretty good results aswell. So, for instance, a filename of "abrq10.zip" is a poor name, as it makes no sense to search for. If instead it is "Game_name_1.zip" is will work a lot better. But there are some nifty tricks on top of that which I do. For both mame and scumm names I have a lookup which converts the names such as "Golden Axe" instead of "goldnaxe" (or whatever it's called).

                                      All of the modules are described here, which should provide at least some insight as to how each work when looking up roms.

                                      Lastly, the CD based platforms are notoriously hard to scrape, as they often have multifiles per game and / or are based on .cue files which can vary since they are often user generated. So for these platforms even ScreenScraper is highly reliant on the exact filenames instead. So it will need to exactly match, for instance, "game_name.cue" inside of ScreenScrapers database to work.

                                      langestL 1 Reply Last reply Reply Quote 0
                                      • langestL
                                        langest @muldjord
                                        last edited by

                                        @muldjord
                                        Thanks for the informative answer.

                                        What I am doing is parsing the db.xml files. And in there the entries are stored based on sha1sums. When I use sha1sum on a rom (and when I use it in my python program I get the same result):

                                        $ sha1sum roms/arcade/joust.zip
                                        48c27d4b623b83e3e724ed348b40ff6193b92b1e  roms/arcade/joust.zip
                                        

                                        It is different from the sum in cache/arcade/db.xml

                                        <resource sha1="ed1455f34143e6781dc443c064ca7d69e4093cff" type="title" source="openretro" timestamp="1562952656836">Joust</resource>
                                        

                                        Same with .cue (understandably)

                                        $ sha1sum roms/psx/Bishi\ Bashi\ Special\ 2\ \(Japan\).cue
                                        3d1fb64cd75afd208f7eedb3059a0b0a09fbb78a  roms/psx/Bishi Bashi Special 2 (Japan).cue
                                        
                                        <resource sha1="cdd966e49743a5571eeb81d249d6a3abefbfb408" type="title" source="thegamesdb" timestamp="1563723954281">Bishi Bashi Special 2</resource>
                                        

                                        It works fine for files that are "plain" roms:

                                        $ sha1sum roms/gbc/Wario\ Land\ 3\ \(World\)\ \(En\,Ja\).gbc
                                        bb7877309834441fd03adb7fa65738e5d5b2d7ba  roms/gbc/Wario Land 3 (World) (En,Ja).gbc
                                        
                                        <resource sha1="bb7877309834441fd03adb7fa65738e5d5b2d7ba" type="title" source="openretro" timestamp="1563722070330">Wario Land 3</resource>
                                        

                                        Do you know why this happens?

                                        cyperghostC muldjordM 2 Replies Last reply Reply Quote 0
                                        • cyperghostC
                                          cyperghost @langest
                                          last edited by

                                          @langest imho the FBN/MAME games or fixed by their names.
                                          pacman.zip is always PACMAN independent from ROM version.

                                          About PSX games .... if you check the bin/img file. Did you then get a result?

                                          1 Reply Last reply Reply Quote 0
                                          • langestL
                                            langest
                                            last edited by

                                            I also figured out why using skyscraper data is "slow". It is because you might be reading and hashing a big file, dreamcast rom et.c.
                                            And the IO is what takes the most time. I have added a function cache to reduce this problem. But it would probably be best to cache all the results on disk so you don't have to recalculate it every time you run restart the program.

                                            mituM muldjordM 2 Replies Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.