RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login

    Development of module-script generator for lr-mess, lr-mame and mame standalone

    Scheduled Pinned Locked Moved Ideas and Development
    developmentlr-messmamelr-mamescripts
    2.3k Posts 37 Posters 7.4m Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • FollyF
      Folly @DTEAM
      last edited by Folly

      @dteam

      I have a solution already ;-) (too bad, not the solution after all ! :-( )

      This is the code to get all 162 files the artwork files which are then placed in the correct directory ( still testing but seems to be ok / damn stuck at 114 files ! ) :

      wget -nv -O /tmp/gdrivedl.py https://raw.githubusercontent.com/matthuisman/gdrivedl/master/gdrivedl.py
      curl https://drive.google.com/embeddedfolderview?id=1sm6gdOcaaQaNUtQ9tZ5Q5WQ6m1OD2QY3#list | egrep -o '(\w|-){26,}' | grep entry | sed 's/entry-//g' | while read fileid ; do python /tmp/gdrivedl.py https://drive.google.com/file/d/$fileid -P $HOME/RetroPie/roms/mame/artwork ; done
      rm /tmp/gdrivedl.py
      
      

      This code should be working but google drive seems to block after a while.
      (see https://retropie.org.uk/forum/topic/29682/development-of-module-script-generator-for-lr-mess-and-mame-standalone/41 )

      This is how it works :

      • with https://drive.google.com/embeddedfolderview?id=1sm6gdOcaaQaNUtQ9tZ5Q5WQ6m1OD2QY3#list you can list all the files in one page.
      • with egrep -o '(\w|-){26,}' | grep entry | sed 's/entry-//g' you filter out only the file-id's.
      • then it uses these file-id's every time in the loop, creating a command one by one for each file.
      DTEAMD 1 Reply Last reply Reply Quote 0
      • DTEAMD
        DTEAM @Folly
        last edited by

        @folly
        Some files are for arcade folder, is It for that you stuck at 114?

        FollyF 1 Reply Last reply Reply Quote 0
        • FollyF
          Folly @DTEAM
          last edited by Folly

          @dteam

          There is another reason why this is not working.

          Haha, I think I tested too much, keep getting => 403 : Forbidden.

          Read the output of one file that I was trying.

          Sorry...Sorry... We're sorry... ... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.

          Can you get all the files with the code ?

          DTEAMD 1 Reply Last reply Reply Quote 0
          • DTEAMD
            DTEAM @Folly
            last edited by DTEAM

            @folly

            #gets the cheat.7z and places it in the correct path
            #works perfect
            wget -N -P /tmp http://cheat.retrogames.com/download/cheat0221.zip
            unzip -o /tmp/cheat0221.zip cheat.7z -d $HOME/RetroPie/BIOS/mame/cheat
            rm /tmp/cheat0221.zip

            Like you said, It works perfect. It's great

            wget -nv -O /tmp/gdrivedl.py https://raw.githubusercontent.com/matthuisman/gdrivedl/master/gdrivedl.py

            curl https://drive.google.com/embeddedfolderview?id=1sm6gdOcaaQaNUtQ9tZ5Q5WQ6m1OD2QY3#list | egrep -o '(\w|-){26,}' | grep entry | sed 's/entry-//g' | while read fileid ; do python /tmp/gdrivedl.py https://drive.google.com/file/d/$fileid -P $HOME/RetroPie/roms/mame/artwork ; done

            rm /tmp/gdrivedl.pyrm /tmp/gdrivedl.py

            I got only 38 files ( 271 Mb). I'm gonna try another run (same thing)
            Example of Errors I had:
            2.jpg

            FollyF 1 Reply Last reply Reply Quote 1
            • FollyF
              Folly @DTEAM
              last edited by

              @dteam

              Thanks for trying.

              Basically it also says : HTTP Error 403: Forbidden.

              It looks like google detects it and then will blocks you for a time.
              ;-(

              I will search again for a solution on the internet.

              DTEAMD 1 Reply Last reply Reply Quote 0
              • DTEAMD
                DTEAM @Folly
                last edited by

                @folly
                If I zip all those files in one file to download, and after that, you manage It on the Pi. Could It be a solution?

                FollyF 2 Replies Last reply Reply Quote 0
                • FollyF
                  Folly @DTEAM
                  last edited by Folly

                  @dteam said in Development of module-script generator for lr-mess and mame standalone:

                  @folly
                  If I zip all those files in one file to download, and after that, you manage It on the Pi. Could It be a solution?

                  If everything else fails, then we can try that ^^, but large files have other problems too solve..

                  Whoow, I just had a full test run and all files are downloaded :-)
                  I Basically used the sample script on this page just using 2 curl lines :
                  https://gist.github.com/tanaikech/f0f2d122e05bf5f971611258c22c110f
                  I injected the filenames and the file-id's from a handmade comma seperate file (.csv) into this sample script.

                  I have to rewrite this a bit.
                  Now I am hopefull !!!

                  1 Reply Last reply Reply Quote 0
                  • FollyF
                    Folly @DTEAM
                    last edited by Folly

                    @dteam

                    Done rewriting.
                    Can you test this for me ? :

                    fileids=(); filenames=()
                    weblist=$(curl https://drive.google.com/embeddedfolderview?id=1sm6gdOcaaQaNUtQ9tZ5Q5WQ6m1OD2QY3#list)
                    IFS=$'\n' GLOBIGNORE='*' command eval  'fileids=($(echo $weblist | egrep -o "(\w|-){26,}" | grep entry | sed "s/entry-//g"))'
                    IFS=$'\n' GLOBIGNORE='*' command eval  'filenames=($(echo $weblist | tr ">" "\n" | tr "<" "\n" | grep zip | grep -v application | grep -v : ))'
                    mkdir -p $HOME/RetroPie/roms/mame/artwork
                    cd $HOME/RetroPie/roms/mame/artwork
                    for index in "${!filenames[@]}"; do 
                    echo downloading : ${filenames[$index]}
                    curl -c ./.cookie -s -L "https://drive.google.com/uc?export=download&id=${fileids[$index]}" > /dev/null
                    curl -Lb ./.cookie "https://drive.google.com/uc?export=download&confirm=`awk '/download/ {print $NF}' ./.cookie`&id=${fileids[$index]}" -o ${filenames[$index]}
                    done
                    cd $HOME
                    
                    

                    How does it work :

                    • create empty array's (see it as an excel column with more cell's with data)
                    • get all data of the website and put it in the string : weblist
                      ( using embedded folder view : http://stackoverflow.com/questions/20681974/ddg#20681975 )
                    • IFS is used to get all data it reads, when encountering a newline (\n), and puts it in the array
                    • sed, tr and grep commands filter the website so IFS will read the data
                    • all data, gathered in the array's, is injected in the curl commands by the for loop
                    DTEAMD 1 Reply Last reply Reply Quote 0
                    • DTEAMD
                      DTEAM @Folly
                      last edited by DTEAM

                      @folly
                      It running right now. It seems to be faster than the other script to get the files.
                      new edit
                      162 zip files + 1 cookie file ... It works !!

                      FollyF 1 Reply Last reply Reply Quote 0
                      • FollyF
                        Folly @DTEAM
                        last edited by Folly

                        @dteam

                        Yeahhh.. That is great !

                        I am now using a normal curl example and no scripts from others here,
                        perhaps faster that way.

                        Maybe I have to improve it a bit :

                        • remove cookie afterwards or place in /tmp directory.
                        • try to improve writing directly in the artwork directory, without going to the directory

                        But for now, it will do.

                        Are all files coming in the correct place or do some files have to be moved ?

                        DTEAMD 1 Reply Last reply Reply Quote 0
                        • DTEAMD
                          DTEAM @Folly
                          last edited by DTEAM

                          @folly said in Development of module-script generator for lr-mess and mame standalone:

                          Are all files coming in the correct place or do some files have to be moved ?

                          Mame artwork go in /home/pi/RetroPie/roms/mame/artwork

                          .cfg files for overlays are generated in propers folders

                          like: /home/pi/RetroPie/overlays/konamih/kgradius.cfg

                          kgradius.cfg

                          overlays = 1
                          overlay0_overlay = kgradius.png
                          overlay0_full_screen = false
                          overlay0_descs = 0
                          

                          kgradius.png is not automatically generated

                          In the MAME artworks kgradius.zip we found
                          4 .png files and only one is the backgroung.

                          Do you want to add the entire handheld overlay or just get the background? I,m saying that because I saw this picture for MAME in your previous post .

                          Personally, with the background only It's fine for me. I made some of the default.lay files that we find on google drive (svmu, pockstat, gp32, mc_tv200, sy889, etc.) and I don't know how to turn a default.lay file into a .cfg for retroarch overlay with position, size, etc.

                          gradius

                          FollyF 1 Reply Last reply Reply Quote 0
                          • FollyF
                            Folly @DTEAM
                            last edited by Folly

                            @dteam said in Development of module-script generator for lr-mess and mame standalone:

                            kgradius.png is not automatically generated

                            The code you used here is only for downloading the artwork files :
                            https://retropie.org.uk/forum/topic/29682/development-of-module-script-generator-for-lr-mess-and-mame-standalone/46

                            I still have to program the extraction of the .png files , but it seems it is doable / quite easy. ;-)
                            EDIT : I just added extraction of Background.png's (classich, gameandwatch, konamih, tigerh) in : generate-overlay-configs.sh / get-cheats-artwork-overlays.sh
                            I cloned it also to get-cheats-artwork-overlays.sh so we can add the download stuff later into that script.

                            For some artwork files there are no "Background.png" files. So we will have to find a solution for that.

                            Do you want to add the entire handheld overlay or just get the background? I,m saying that because I saw this picture for MAME in your previous post .
                            Personally, with the background only It's fine for me.

                            I only want to do the background.

                            DTEAMD 1 Reply Last reply Reply Quote 0
                            • DTEAMD
                              DTEAM @Folly
                              last edited by

                              @folly

                              For cheat codes with MAME, I think you will have to add

                              cheatpath /home/pi/RetroPie/roms/mame/cheat
                              cheat 1
                              

                              in mame.ini.

                              FollyF 2 Replies Last reply Reply Quote 0
                              • FollyF
                                Folly @DTEAM
                                last edited by Folly

                                @dteam

                                Added in the generate-systems-lr-mess_mame-1v8-alpha.sh script.
                                It will be added to the mame.ini config if installing/updating an install-<RPname>-cmd.sh module-script in the RetroPie-setup.

                                1 Reply Last reply Reply Quote 0
                                • FollyF
                                  Folly @DTEAM
                                  last edited by Folly

                                  @dteam

                                  How is it going with the backgrounds ?

                                  These games have bg.jpg and can work in lr-mess (but not in the script yet).
                                  kdribble
                                  kloneran
                                  knfl
                                  taddams
                                  tddragon3
                                  tgaiden
                                  tinday
                                  tmchammer
                                  trobocop2

                                  We have to find a solution for these games and probably more.
                                  1 - So I can try to change the script so it will also detect bg.jpg and alter the config accordingly.
                                  But this can conflict/be a problem in the long run if more games have to be added with also different background filenames.

                                  An uniform approach would be better, I think.
                                  Something like this :

                                  2 - So it would be nicer,I think, if bg.jpg is converted to a Background.png and that both files are kept in the archive.
                                  3 - Or the layout is altered for Background.pngand then keeping only Background.pngis also a posibility.

                                  My intuition says option 2 .
                                  Keeping original files and make it compatible for our project.

                                  What do you think ?

                                  DTEAMD 1 Reply Last reply Reply Quote 0
                                  • DTEAMD
                                    DTEAM @Folly
                                    last edited by

                                    @folly
                                    I'm gonna do the conversion for files in Google Drive

                                    FollyF 1 Reply Last reply Reply Quote 1
                                    • FollyF
                                      Folly @DTEAM
                                      last edited by

                                      @dteam said in Development of module-script generator for lr-mess and mame standalone:

                                      @folly
                                      I'm gonna do the conversion for files in Google Drive

                                      Are you letting me know when it's ready ?

                                      DTEAMD 1 Reply Last reply Reply Quote 1
                                      • DTEAMD
                                        DTEAM @Folly
                                        last edited by DTEAM

                                        @folly said in Development of module-script generator for lr-mess and mame standalone:

                                        Are you letting me know when it's ready ?

                                        done
                                        I tested It, and It Works. Background.png files are converted with the appropriate namings in**/home/pi/RetroPie/overlays/home/pi/RetroPie/overlays/** "system name"

                                        FollyF 1 Reply Last reply Reply Quote 0
                                        • FollyF
                                          Folly @DTEAM
                                          last edited by Folly

                                          @dteam

                                          Cool, isn't it ;-)

                                          Next thing to do is, checking the games that don't need an overlay.
                                          Then I will remove these games from the generating overlay script so no loose config files are generated.

                                          And there are games with 2 screens.
                                          With these the TOP/BOTTOM or LEFT/RIGHT backgrounds have to be joined I think.
                                          We will have to try on how this will look.

                                          DTEAMD 1 Reply Last reply Reply Quote 0
                                          • DTEAMD
                                            DTEAM @Folly
                                            last edited by

                                            @folly said in Development of module-script generator for lr-mess and mame standalone:

                                            Cool, isn't it ;-)

                                            👍

                                            When you load a game with Lr-Mess, do you have to select the overlay or it,s done automatically. On my side, I have to load It via retroarch UI

                                            FollyF 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.