RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login

    YARMan Web (BETA)

    Scheduled Pinned Locked Moved Ideas and Development
    metadataeditorweb
    84 Posts 15 Posters 34.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • daeksD
      daeks
      last edited by

      @Arcuza I dont think so. It is a local installation with some connections to the internet (reading retropie news, scraping, etc.).

      @pjft / @herb_fargus For the time being I have changed the gamelist.xml search mechanism. I looks in the rom folder first and after that in the emulationstation/gamelists folder.

      Nevertheless with the implementation of SQLite, I will change the whole gamelist/rom behavior:

      • Romlist will be parsed to internal SQLite database on first startup
      • Gamelist will be parsed to internal SQLite database on first emulator select
      • Changes will be saved into internal SQLite database instead of the gamelist file directly
      • New button "Sync" will update/import the gamelist.xml/roms to the database (in case of any external changes)
      • New button "Export to" will trigger a save to the gamelist.xml with some kind of dialog to be able to select which fields should be exported

      Beside increasing the performance for large romsets, another example would be that users are able to maintain all metadata fields like image AND video at the same time, but can choose time by time if they want to include for video tag in the gamelist.xml or not.

      A 1 Reply Last reply Reply Quote 1
      • A
        Arcuza @daeks
        last edited by

        @daeks In your web front, people would be able to set game rating and emulator compability. If we would be able to collect that information from all users into one single location together with background data like times played, time spent, days since last played, emulator used we would have a super great database were we could find issues and make game recommendations to users.

        I love data mining and I have already written a recommendation algorithm that could be used for that purpose.

        And it would be perfect to use the same authentication (login) as for this forum.

        I just cannot see why we're not capitalizing on this opportunity.

        daeksD 1 Reply Last reply Reply Quote 0
        • daeksD
          daeks @Arcuza
          last edited by daeks

          @Arcuza Personally I am no friend of data mining but I understand your purpose and the idea itself. It is an interesting approach to get features ready like "proofed to be compatible with...", "recommended / rated by community" but that should be discussed also with @herb_fargus and @BuZz. From my opinion there are enough places out there collecting such information already on a base level.

          From my point of view talking about YARMan, the source code is open source and contributions are welcome. So if someone (you?) find some time for it, go ahead and prepare a pull request for it. Currently I am concentrating on the basic features listed here.

          1 Reply Last reply Reply Quote 0
          • daeksD
            daeks
            last edited by daeks

            New modules added:

            • Configuration Editor
              • Support of general config
              • Support of Emulator's config (was a wish on github)
              • Each save results in a .bak file
            • Log Viewer
              • System logs
              • Support of viewing RetroPie-Setup's gz archives
            G 1 Reply Last reply Reply Quote 1
            • G
              grenade44 @daeks
              last edited by

              @daeks said in Rebranded: YARMan Web (former RetroPie WebGUI) + Metadata Editor (BETA):

              New modules added:

              • Configuration Editor
                • Support of general config
                • Support of Emulator's config (was a wish on github)
                • Each save results in a .bak file
              • Log Viewer
                • System logs
                • Support of viewing RetroPie-Setup's gz archives

              Thanks for the emulator config. πŸ˜€
              I love all the hard work you are putting into this. Can't wait to get scrapper support and game management. Wish I could code and help.

              1 Reply Last reply Reply Quote 0
              • meleuM
                meleu @daeks
                last edited by

                @daeks said in Rebranded: YARMan Web (former RetroPie WebGUI) + Metadata Editor (BETA):

                I am not very familiar with the integrated webserver. If it supports all the stuff like apache does, we can also implement that. Currently I am more focusing on the application itself instead of the setup in retropie. I will try to figure out how the integrated one could be used but would really appreciate any help on that.

                Great tool man! Thanks for sharing.

                I didn't delve into the yarman apache settings, but I'm curious about what features would you need to see in the webserver?

                • Useful topics
                • joystick-selection tool
                • rpie-art tool
                • achievements I made
                daeksD 1 Reply Last reply Reply Quote 0
                • daeksD
                  daeks @meleu
                  last edited by

                  @meleu The retropie module script is already changed to internal php webserver :)

                  1 Reply Last reply Reply Quote 1
                  • J
                    Jeck11
                    last edited by Jeck11

                    @daeks - Great rework! Love the look and very useful.

                    I've noticed something and wanted to share to see if it's something I'm doing wrong or a bug. I'm not getting my previews to show up:

                    screenshot examp

                    I have a feeling it's because I used @screech's Universal XML Scraper V2. This puts the images in the roms folder (ie - \\retropie\roms\psx\downloaded_images\Crash Bandicoot-image.png) instead of under configs.

                    daeksD ShakzS 2 Replies Last reply Reply Quote 0
                    • daeksD
                      daeks @Jeck11
                      last edited by

                      @Jeck11 yes, that is the issue. Thanks for the input, as soon as I have reworked everything for the SQLite implementation, I will fix that :)

                      J 1 Reply Last reply Reply Quote 0
                      • J
                        Jeck11 @daeks
                        last edited by Jeck11

                        @daeks - NP, wish I could help with the code to actually fix the issue. Don't have the skills.

                        Also a recommendation - It would be great to have the ability to filter the game listing based on specific text as well to find games faster:

                        current filter options

                        daeksD 1 Reply Last reply Reply Quote 1
                        • daeksD
                          daeks @Jeck11
                          last edited by daeks

                          @Jeck11 You're welcome but that fix is quite simple. Would do it right now, but have to rewrite a couple things as I am struggling a bit with performance issues on large gamelists ( > 4000 entries). So it will take a while :(

                          Regarding the filter, I have noted that. I am thinking currently on including some kind of searchbar or something else.

                          J 1 Reply Last reply Reply Quote 0
                          • J
                            Jeck11 @daeks
                            last edited by

                            @daeks - I think that would probably work even better.

                            I was experiencing crash when I tried to run the scrape over my entire collection 5k+. That's probably what you're referring to.

                            1 Reply Last reply Reply Quote 0
                            • daeksD
                              daeks
                              last edited by daeks

                              This would be my complete rewrite of the metadata editor:

                              • Emulator has to be synced with YARMan by new button on the left side of the emulator selection (that includes parsing folder and existing gamelist.xml)
                              • (This will be done on first startup automatically)
                              • Each change will be saved to internal database first (not to gamelist.xml)
                              • Gamelist.xml has to be exported by new button on the right side of the emulator selection
                              • Single Rom can be synced with existing gamelist.xml entry by a new button on the right side (near the save button)
                              • (Fields to be exported can be selected)
                              1 Reply Last reply Reply Quote 0
                              • daeksD
                                daeks
                                last edited by

                                Currently I am facing some issues while restarting the emulationstation from yarman. Anyone can help?
                                After restarting with export HOME=/home/pi && emulationstation > /dev/null 2>/dev/null & I can not start any game.
                                No runcommand loading screen will show up and I will be returned to ES.

                                Nothing is in the es_log or runcommand.log:

                                lvl2: 	 req sound [detailed.launch]
                                lvl2: 	   (missing)
                                lvl2: 	Attempting to launch game...
                                lvl2: 		/opt/retropie/supplementary/runcommand/runcommand.sh 0 _SYS_ atari7800 /home/pi/RetroPie/roms/atari7800/Alien\ Brigade\ \(Europe\).a78
                                lvl2: 	Creating surface...
                                lvl2: 	Created window successfully.
                                lvl2: 	Added known joystick 8Bitdo NES30 Pro (instance ID: 0, device index: 0)
                                
                                Parameters: 
                                Executing: /opt/retropie/emulators/retroarch/bin/retroarch -L /opt/retropie/libretrocores/lr-prosystem/prosystem_libretro.so --config /opt/retropie/configs/atari7800/retroarch.cfg "/home/pi/RetroPie/roms/atari7800/Alien Brigade (Europe).a78" --appendconfig /dev/shm/retroarch.cfg
                                
                                meleuM 1 Reply Last reply Reply Quote 0
                                • meleuM
                                  meleu @daeks
                                  last edited by

                                  @daeks What's the point on restarting emulationstation via yarm?! I can't see a use case for it...

                                  • Useful topics
                                  • joystick-selection tool
                                  • rpie-art tool
                                  • achievements I made
                                  daeksD 1 Reply Last reply Reply Quote 0
                                  • daeksD
                                    daeks @meleu
                                    last edited by

                                    @meleu running es overwrites gamelist.xml, therefore it needs to be quited. Anyhow, nobody has to use it if he does not need it ;) - Nevertheless, I am struggling to start a game after restarting es as mentioned above.

                                    1 Reply Last reply Reply Quote 1
                                    • meleuM
                                      meleu @daeks
                                      last edited by

                                      @daeks As I reported on yarman issue tracker, the I had some issues when trying to install it on RetroPie x86 (Linux Mint 18.1).

                                      • Useful topics
                                      • joystick-selection tool
                                      • rpie-art tool
                                      • achievements I made
                                      1 Reply Last reply Reply Quote 0
                                      • daeksD
                                        daeks
                                        last edited by daeks

                                        sqlite implementation is finished so far.
                                        Currently doing some deeper tests before releasing it. It can be already tested on the sqlite branch.

                                        The biggest issue I have is the execution time for hashing roms all at once during the sync/setup process. Tests on a very large romset resulted in script timeouts and the general process takes too long - Thinking on dat support later on, I currently struggle a bit how to solve that. Maybe some cron job?

                                        1 Reply Last reply Reply Quote 1
                                        • ShakzS
                                          Shakz @Jeck11
                                          last edited by

                                          @Jeck11 You can change that setting in the config menu so that it puts the art in the correct folder rather than the roms directory.

                                          RetroPie tips, tricks, and tweaks: https://www.youtube.com/eazyhax

                                          1 Reply Last reply Reply Quote 0
                                          • daeksD
                                            daeks
                                            last edited by daeks

                                            I have released a new version of YARMan Web. Please update the scriptmodule before updating.
                                            Please be aware that some modules are still BETA.

                                            Note: YARMan Web is currently only tested on Raspberry 3B with latest RetroPie installation.

                                            Changelog:

                                            -- [FIX] Config Editor strips content in some circumstances
                                            -- [FIX] Image / Video preview uses proper path from xml tag
                                            -- [FIX] Better HTML5 upload dialog & error handling
                                            -- [FIX] Restarting ES didnt work

                                            -- [NEW] SQLite Implementation
                                            -- [NEW] Quick Setup on first startup
                                            -- [NEW] Dynamic Administration Panel
                                            -- [NEW] Emulator Configuration
                                            -- [NEW] Database Management

                                            -- [CHANGE] Dashboard rewrite with dynamically loaded widgets
                                            -- [CHANGE] New Metadata Editor (no direct save on gamelist.xml)
                                            -- [CHANGE] Rom hashing preparation for DAT support
                                            -- [CHANGE] Performance Improvements and several minor points

                                            Some impressions:
                                            0_1489056216350_dashboard.png
                                            0_1489055503204_metadata_editr.png
                                            0_1489055514904_export.png
                                            0_1489055530974_editor.png
                                            0_1489055591401_logsviewer.png
                                            0_1489055578196_config.png

                                            R 1 Reply Last reply Reply Quote 5
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.