Development of module-script generator for lr-mess, lr-mame and mame standalone

[Folly]

@DTEAM @valerino

As a test-3, I added the option to install handhelds.

It's not directly the structure on how we want it in the future.
But I did this to see how the array's have to look if I want to make a choice on a description.
I also wanted to see if, perhaps, I could make things simpler.

It is not going very fast.
I will take a large amount of small steps to get it how we want it.

wget -q -nv -O /home/$(ls /home)/RetroPie-Setup/scriptmodules/supplementary/add-mamedev-systems.sh https://raw.githubusercontent.com/FollyMaddy/RetroPie-Share/main/00-workdir-00/add-mamedev-systems-test3.sh
 

Here you can see most changes of test-2 vs test-3 :
https://github.com/FollyMaddy/RetroPie-Share/commit/d48643325fc39c3bf808dadc0d5eeea9e5682740

[Folly]

@valerino

While making things in test3 i learned a bit more on how the array's are structured.
As you can see, it isn't a nice structure.
In the function choose_system_add() you will also see that all info read into $system_read will be inserted in both systems_read() and options().
So in this example, basically doubling the info, which will, presumably, take more time to execute.
Also injecting the numbers seems to be a bit odd, we already have the cell numbers in the array.
So it would also be nice if we could use those cell numbers, without injecting them again.
So I am trying to find a solution to this problem ( you can see how it is structured in this manual approach in function choose_dteam_add() ) :

local systems_read=( "ablmini" "alnattck" "gnw_ball" "jak_batm" "kbilly" "taddams" "rzbatfor" )
local options=( "0" "All in One Handheld and Plug and Play" "1" "Classic Handheld Systems" "2" "Game and Watch" "3" "JAKKS Pacific TV Games" "4" "Konami Handheld" "5" "Tiger Handheld Electronics" "6" "Tiger R-Zone" )
.
.
local choice=$("${cmd[@]}" "${options[@]}" 2>&1 >/dev/tty)

I want the arrays to be arranged like this :
(representation : converting it into a csv)
ROW1,ROW2
All in One Handheld and Plug and Play,ablmini
Classic Handheld Systems,alnattck

As you can see it's, right now it is arranged like this, so info doesn't match :
ROW1|ROW2
All in One Handheld and Plug and Play,1
Classic Handheld Systems,ablmini
Game and Watch,2

Perhaps too difficult, but it would be nice if we could extract into 2 dimensional array's or make fixed 2 dimensional array's.
Something like this :
https://stackoverflow.com/questions/16487258/how-to-declare-2d-array-in-bash
Then we would be able to link the information better, put all the extracted info into one 2-dimensional-array and have a better overview.

Hopefully you understand what I am talking about.

Do you have an idea ?

@DTEAM , Ofcourse, you are welcome to look at it too.
If you want some explanation about how basic array's work, just let me know.

[DTEAM]

@folly

Hi Folly,

From now on, it will be difficult for me to help you on this project. I will continue to test and read what you are doing, but I am not a programmer. I understand most of what you do, but I cannot contribute more than what I have done. Hope for you that some programmers ( @valerino , @2Play, and other ) will dive into this project with you because it has huge potential to add new working systems on RetroPie and add MESS / MAME for existing systems in Retropie.

Regards

[Folly]

@dteam

Hi,

You are already doing a lot, that is much appreciated.
I understand it when you say that the programming part becomes more difficult for you now.
That is no problem, I hope too that others can help.
I must admit I also have some trouble understanding it all, sometimes.
And if you asked me 3 years ago, I didn't understand it either.
But in these situations I experience every-time a learning boost when I find a solution for a problem like described in my last post.
Then it becomes more clear to me, so I hope I get the aha so we can program this script faster.

Regards

[Folly]

EDIT : If there are spaces in the descriptions then this doesn't work, I will have to find a better solution

@DTEAM @valerino ,

I think I found a better way of dealing with the dialog api.
The dialog api wants 2 options added in the commandline.
(choice and what to choose from)
Just did a small test with 2 arrays :
(you will see it will output the data of both arrays in sequence)

a=()
b=()
a=( "1" "2" "3" )
b=( "z" "c" "e" )
for i in ${!a[@]}; do echo "${a[$i]} ${b[$i]}";done
 

Then I did a line with the dialog api to test :
(now you will see it will display in the same sequence)

dialog --backtitle "test" --menu "RetroPie-Setup Mamedev Systems Adder" 22 76 16 $(for i in ${!a[@]}; do echo "${a[$i]} ${b[$i]}";done)
 

I hope this solution will work in the script.
It should make the script far better.😊

[Folly]

EDIT : If there are spaces in the descriptions then this doesn't work, I will have to find a better solution

@DTEAM @valerino

This does the same as the commands from the last post.
I reduced the method, extracting the number from the for loop instead of a separate array :

b=()
b=( "z" "c" "e" )
dialog --backtitle "test" --menu "RetroPie-Setup Mamedev Systems Adder" 22 76 16 $(for i in ${!b[@]}; do echo "$i" "${b[$i]}";done)
 

Much better, this way

Edit : With spaces in the example it will not work because it will present this as one option or as too many options depending on the situation, no idea how to fix this. ☹️

b=()
b=( "choice one" "choice two" "choice three" )
dialog --backtitle "test" --menu "RetroPie-Setup Mamedev Systems Adder" 22 76 16 $(for i in ${!b[@]}; do echo "$i" "${b[$i]}";done)
 

[Folly]

@DTEAM @valerino

I have found a solution that makes adding fixed arrays simpler and in that way it's also possible to "connect" between multiple arrays.

The for loop creates a separate options array, setting the choice number and the choice behind each other in the array.
This is how dialog wants it.
For now, there is no way around this.

local systems_read=( "ablmini" "alnattck" "gnw_ball" "jak_batm" "kbilly" "taddams" "rzbatfor" ) 
local descriptions_read=( "All in One Handheld and Plug and Play" "Classic Handheld Systems" "Game and Watch" "JAKKS Pacific TV Games" "Konami Handheld" "Tiger Handheld Electronics" "Tiger R-Zone" )  
for i in ${!descriptions_read[@]}; do options+=("$i" "${descriptions_read[$i]}");done

Test4 is ready :

wget -q -nv -O /home/$(ls /home)/RetroPie-Setup/scriptmodules/supplementary/add-mamedev-systems.sh https://raw.githubusercontent.com/FollyMaddy/RetroPie-Share/main/00-workdir-00/add-mamedev-systems-test4.sh
 

Perhaps it doesn't seem much, but it's a huge update !
Still turned off, but I also added some basic test lines for importing faster from text files.
Not usable for now because you have to have the data in a text file, but if I can implement this, it's 3 or 4 times faster.

Changes test3 vs test4 :
https://github.com/FollyMaddy/RetroPie-Share/commit/da8b4eeff83328df94e00fa1d7bb02606e22f4b4
https://github.com/FollyMaddy/RetroPie-Share/commit/e871c35399aa72af95de2a7f843334949dc49ffa#diff-23cb2bcf9c8990a5889800ad28f7bbd66c91a0fad95249958916aaa10b629c15
https://github.com/FollyMaddy/RetroPie-Share/commit/285b8aa6b18cfa67f5c7bd3c4ca9302fd32b1f3e#diff-23cb2bcf9c8990a5889800ad28f7bbd66c91a0fad95249958916aaa10b629c15

[DTEAM]

@folly

Wow!!! and wow again

I like your download menu and I like the way you presented All systems with real system names and with MAMEsystem names .

You are not far from the ultimate goal.
Really nice what you have done Folly

[Folly]

@dteam

I boosted this weekend.
Quite happy, for now. 😉

I just had a look at your schematics.
I will try to work towards the schematics if I can.

We will see how it goes.

[Folly]

Edit :
This part is removed -> will be added to a new script !

Just an idea that I wanted to add :
I added a submenu in test4 for downloading module-scripts from different external repositories.
It will only download if the file doesn't exist in the normal path.
It will put them in the appropriate directories in the ext path.
After downloading, just go to the normal things in RetroPie-Setup.

Script is restored to the original state.

[DTEAM]

@folly

At the end, It must be simple for the user. I think we should go with your script only and optimize some systems (most in demand). We could create a new thread for that. When It will be like the schematics (or almost), We could ask the permission to put that on the official RetroPie. The systems optimization should come after from us and also other contributors. Are you agree?

What is the advantage for a user to use 3 module scripts for Lr-Mess?

[Folly]

@dteam said in Development of module-script generator for lr-mess and mame standalone:

@folly

At the end, It must be simple for the user. I think we should go with your script only and optimize some systems (most in demand). We could create a new thread for that. When It will be like the schematics (or almost), We could ask the permission to put that on the official RetroPie. The systems optimization should come after from us and also other contributors. Are you agree?

What is the advantage for a user to use 3 module scripts for Lr-Mess?

Yes, I agree.
I have the same feeling about it now
I didn't realize it, while I was busy.

I will separate it.

[Folly]

@dteam

Now that I have separated the repository part from the script.
I worked more on this and learned again some stuff that can be used again, later on.
I also get a better overview over the whole hierarchy of all scripts.
So an overall improvement, I think.

Friday we go on vacation for a week. 😊
So no updates then.

btw :
I had a look at 2 dimensional array's.
I think we have to use it for sorting and filtering.
Because this way we keep the information locked to each other,
at least that is my hope.
Sorting and filtering could get much easier this way.
This a small example of a 2 dimensional array (2 rows x 2 columns):

declare -A arr
arr[0,0]="system"
arr[0,1]="The System Description"
echo "${arr[0,0]} <-> ${arr[0,1]}"
 

[Folly]

@valerino

Very busy ?

[Folly]

@DTEAM

Just found nice way of running the gdrivedl.py script online, without downloading.
If we use this in the script we can use just 1 line instead of 3, as we recently did :

curl https://raw.githubusercontent.com/matthuisman/gdrivedl/master/gdrivedl.py | python - https://drive.google.com/drive/folders/<enter_the_folder_id_here> -P <destiny path>

[valerino]

@folly yep, i don 't have time to dedicate to retropie now :(

anyway, seems you made a really big jump ahead alone, you're making a really good work, really congratulations !

when you think it's enough stable and working as intended, we'll see how to do to join mine and your repo.....

v.

[Folly]

@valerino said in Development of module-script generator for lr-mess and mame standalone:

@folly yep, i don 't have time to dedicate to retropie now :(

anyway, seems you made a really big jump ahead alone, you're making a really good work, really congratulations !

when you think it's enough stable and working as intended, we'll see how to do to join mine and your repo.....

v.

Thanks for your message.
It's understood, not everyone can spend that much time on this.

We will continue to work on it, keeping the joining in mind.

[Folly]

@dteam @valerino

I have done some menu restructuring tests on the add_ext_repos.sh script, which basically used the dialog api in the same way as our add-mamedev-systems.sh

I did this on this script because it still contained just 1 main-menu.
So this was an easy way to test my idea to use comma separate values in the array.

I wanted to make a structure so :

• the code is easy to read
• implementation of data is easy (now using csv style, which could also be considered as a 2 dimensional array in some way)
• it's possible to use exel/calc and cut/paste from a csv file for implementing the data, if needed
• data can be imported directly from a csv file, if needed
• not every menu needs a menu script (every menu/submenu should be built by the same function part)
• lesser code that can do the same thing

I hope and think it's a good idea for the base structure of our script and more scripts.

This is the code before and after, just to give you an idea on what I have done :
Before :
https://raw.githubusercontent.com/FollyMaddy/RetroPie-Share/d66f3c78ac1239f724868dbdcaa4d8ec1e5cfeb4/00-workdir-00/add-ext-repos.sh
After :
https://raw.githubusercontent.com/FollyMaddy/RetroPie-Share/main/00-workdir-00/add-ext-repos.sh

ps.

I did a second test to see if I could install packages directly from my menu structure, and this works,
showing a new window-menu with install/update/remove/package help etc.

I just added a few lines to the csv_array to test the ability to install packages directly from our menu structure :
(so now you can also see that it's possible to run different things per line in one menu)

local csv=()
#the first value should be an empty value so the menu begins with 1
#make sure every line begins and ends with quotes because of possible spaces
#just use the first and last colum in excel/calc for the quotes and you should be fine
csv=(
",,,"
",FollyMaddy/RetroPie-Share,FollyMaddy/RetroPie-Share/tree/main/00-scriptmodules-00,download_ext_module_scripts,"
",GeorgeMcMullen/rp-box86wine,GeorgeMcMullen/rp-box86wine/tree/main/scriptmodules,download_ext_module_scripts,"
",zerojay/RetroPie-Extra,zerojay/RetroPie-Extra/tree/master/scriptmodules,download_ext_module_scripts,"
",valerino/RetroPie-Setup,valerino/RetroPie-Setup/tree/master/scriptmodules,download_ext_module_scripts,"
",install mame",,"package_setup mame,"
",install goonies",,"package_setup goonies,"
    )

Knowing above, I think it's time to rebuild the front-end script.

[RussellB]

@folly At Folly's request I'm posting the changes I made to the run_mess.sh script which allows for each game that uses it to be started with a command file that instead of having it named 'tmpmess.cmd' it is named after the ROM that it is starting, e.g. "Alice in Wonderland (USA).cmd". This way each game is unique and RetroArch/mess will save custom game configs to that instead of constantly overwriting 'tmpmess.cfg'.

These are the changes:

_system="$4"
_biosdir="$5"
<<NEW CODE FOLLOWS>>
#
# use the name of the rom to generate the 'tmpmess.cmd' file so RetroArch can save a custom game config$
#
_fname="$(basename "${*: -3:1}")"
_fbname="${_fname%.*}"
<<END NEW CODE>>

# generate mess.cmd
echo "\t/tmp/mess.cmd content: ${_cmdarr[@]}"
<<NEW CODE FOLLOWS>>
# _tmpcmd="$_romdir/tmpmess.cmd"
#
# Now use the root name of the rom file as the name of the cmd file
#
_tmpcmd="$_romdir/$_fbname.cmd"
<<END NEW CODE>>

Using this with mess for Philips CD-i roms now every game has a bezel and I changed the screen offset to move the "CD-i" out of the view. Looks cleaner that way to me.

Hope this helps!

RussellB

[Folly]

@russellb

Thanks for posting it here.