RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login

    Create custom-collections from txt file [v1.3, final]

    Scheduled Pinned Locked Moved Ideas and Development
    custom systemcyperghost
    26 Posts 4 Posters 6.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • cyperghostC
      cyperghost @meleu
      last edited by cyperghost

      @meleu Hi mate ;) Didn't know you are not using custom-collections;)
      Maybe now? Btw: Do you know a good manual for RetroAchivments?

      @pjft Wonderfull work ;) One question: Are spaces in custom-collections allowed?

      I think this tool can be quite usefull because you can create collections:

      • No need of keyboard to Pie
      • No need for custom-.cfg - I always forget this to add
      • You can create an "overview" of collections because file content will not be changed, so no need to "maintain" the list! Just add a new entry and run the script!

      Suggestion 1: Thank you ;) That is usefull for multiuser mashines

      Suggestion 2: I think @pjft removed empty characters vie ES keyboard input but filename with space work. So this can be well... explained as "expect the unexpected". Your solution is surly better but will remove spaces ;) So what to do?

      Suggestion 3: No, it's already created by default if ES 2.6.0 is updated via RetroPie_setup

      Suggestion 4: Seems to me just for ... speed reasons? Because I read file in variable and then rewrite it with trim command. You do this in one slash ;)

      So everything is clear for me ... but:
      Is the CAT command usefull here? Because it will extract single strings in file?

      meleuM 1 Reply Last reply Reply Quote 0
      • meleuM
        meleu @cyperghost
        last edited by

        @cyperghost said in Create custom-collections from txt file:

        @meleu Hi mate ;) Didn't know you are not using custom-collections;)
        Maybe now?

        Yes. I wanna use it!

        Suggestion 3: No, it's already created by default if ES 2.6.0 is updated via RetroPie_setup

        But if the user try the script without updating ES he/she will get a bunch of "no such file or directory" errors.

        Is the CAT command usefull here? Because it will extract single strings in file?

        Well, I used it because tr can't read from a file name. It reads only the stdin. But we can avoid the cat using tr -d '\r' < "$txtfile".

        Btw: Do you know a good manual for RetroAchivments?

        There's a FAQ on their site. I think it's simple, but I agree the info is a little scattered. I'll try to write two or three paragraphs about it and post here in the forum.

        But just to let you start I suggest you to:

        1. Cretate an account at RetroAchievements.org

        2. Configure your RetroArch following the instructions on the docs.

        3. Play something. :-)

        • Useful topics
        • joystick-selection tool
        • rpie-art tool
        • achievements I made
        1 Reply Last reply Reply Quote 1
        • pjftP
          pjft
          last edited by

          This is the regular expression of things I allow/exclude:

          [^A-Za-z0-9-[]()\s']

          So, we should allow:

          • letters (lower and upper case)
          • numbers
          • spaces
          • '

          I believe none of these would pose problems with any file system. That was the main criteria.

          meleuM 1 Reply Last reply Reply Quote 1
          • cyperghostC
            cyperghost
            last edited by cyperghost

            @meleu As whitespaces are working (even if they are not allowed!) I would suggest following

            if [[ "$line" =~ ^[^\/]+$ ]] && [[ -n "$line" ]]; then
            

            remove the whitespace in the regex but check $line if empty or not.

            Maybe pjft can tell more ;)
            Goodnight mate


            EDIT: Was to late
            So we should allow whitespaces ;)

            pjftP meleuM 2 Replies Last reply Reply Quote 0
            • pjftP
              pjft @cyperghost
              last edited by

              @cyperghost I think in concept everything should work as long as it's accepted by the file system. I just wanted to play it safe and create a minimum common denominator set of characters.

              1 Reply Last reply Reply Quote 1
              • cyperghostC
                cyperghost
                last edited by cyperghost

                @pjft I think so but as I'm not a C++ coder I can't exactly tell. Thank you for taking part!

                The touch command is a smooth way to leave files unchanged so this script seems to be save even if we update with new collections. The exclusion that @meleu made ... the two slashes are correct. But I wan't avoid that LF entries create custom-.cfg files therefore a check of $line parameter is needed.

                I will update first post and it's codes as @meleu agrees to changes - maybe he gots a better idea ;)

                1 Reply Last reply Reply Quote 0
                • meleuM
                  meleu @pjft
                  last edited by

                  @pjft said in Create custom-collections from txt file:

                  This is the regular expression of things I allow/exclude:

                  [^A-Za-z0-9-[]()\s']

                  So, we should allow:

                  • letters (lower and upper case)
                  • numbers
                  • spaces
                  • '

                  Are you saying that this is what currently works with your custom collections feature? Or is it your personal opinion?

                  • Useful topics
                  • joystick-selection tool
                  • rpie-art tool
                  • achievements I made
                  pjftP 1 Reply Last reply Reply Quote 0
                  • meleuM
                    meleu @cyperghost
                    last edited by meleu

                    @cyperghost said in Create custom-collections from txt file:

                    @meleu As whitespaces are working (even if they are not allowed!) I would suggest following

                    if [[ "$line" =~ ^[^\/]+$ ]] && [[ -n "$line" ]]; then
                    

                    remove the whitespace in the regex but check $line if empty or not.

                    looks like you didn't test that... :-) that regex don't match empty strings. Try it and see.

                    prompt$ [[ "" =~ ^[^\/]+$ ]] && echo valid || echo invalid
                    invalid
                    

                    Translating that regex:

                    • ^ - start of the line
                    • [^\ /] - any character that is NOT a space '\ ' or a slash '/'.
                    • + - one or more occurences (here is the trick for not allowing empty strings)
                    • $ - end of the line
                    • Useful topics
                    • joystick-selection tool
                    • rpie-art tool
                    • achievements I made
                    cyperghostC 1 Reply Last reply Reply Quote 1
                    • pjftP
                      pjft @meleu
                      last edited by

                      @meleu those are the checks I implemented for when the user can type a collection name.

                      1 Reply Last reply Reply Quote 2
                      • cyperghostC
                        cyperghost @meleu
                        last edited by

                        @meleu You did forget the timeshift ;)
                        I updated first posting with script!
                        It works flawless...
                        Whitespaces in filenames of collections were never the problem

                        @pjft I think the simple RegEx method inside ES is okay and covers all BE/AE usecases

                        meleuM 1 Reply Last reply Reply Quote 0
                        • meleuM
                          meleu @cyperghost
                          last edited by

                          @cyperghost just to match the same limits @pjft imposed to what the user can type for a collection name, I suggest this regex in the script: ^[][()\ \'A-Za-z0-9-]+$.

                          Then that if inside the while becomes this:

                          if [[ "$line" =~ ^[][()\ \'A-Za-z0-9-]+$ ]]; then
                          

                          @pjft out of curiosity: is the underscore _ invalid?

                          • Useful topics
                          • joystick-selection tool
                          • rpie-art tool
                          • achievements I made
                          pjftP cyperghostC 3 Replies Last reply Reply Quote 0
                          • pjftP
                            pjft @meleu
                            last edited by

                            @meleu I suppose it is, by omission, not deliberately. As I said, I tried to get a regular expression that would match the majority of names people would try to enter - it was not extensive, but had the main concern of not allowing people to enter characters that would fail miserably in some OSs ('/', '*', '?', etc).

                            Underscores could certainly be added. My recommendation is that if you manage to try it out and confirm it works via your script in ES, then by all means keep it. As I mentioned, these checks are only used at text entry time, so they shouldn't affect in any way how ES would work if you create them elsewhere. But if they fail, don't hold me against it :)

                            1 Reply Last reply Reply Quote 2
                            • cyperghostC
                              cyperghost @meleu
                              last edited by cyperghost

                              @meleu I updated!
                              You restrict underscore with this but well ... I think that enough

                              Thx for your help and in creating generic solutions ;)

                              @pjft

                              But if they fail, don't hold me against it :)

                              Always ;)

                              1 Reply Last reply Reply Quote 0
                              • cyperghostC
                                cyperghost @meleu
                                last edited by

                                @meleu @pjft
                                I updated previous post to give tributes.
                                Well I think there are other characters that can be included. But I think space and "-" and () are good way.
                                The underscore works btw ;)
                                What do you both think.... Is v1.3 with RegEx ^[][()\ \'A-Za-z0-9-]+$ final? or should it be extended. For my personal usage it's enough

                                We can add \_ and then the underscore works, what's next? #,&,!,?,~.....

                                pjftP meleuM 2 Replies Last reply Reply Quote 0
                                • pjftP
                                  pjft @cyperghost
                                  last edited by

                                  @cyperghost No, please stay away from most of the characters you just mentioned, as some of them are special characters in different file systems (~, ?, etc).

                                  I'd rather keep the REGEXP exactly the same for compatibility purposes. I don't like to give people tools for them to easily shoot themselves in the foot :)

                                  1 Reply Last reply Reply Quote 1
                                  • cyperghostC
                                    cyperghost
                                    last edited by

                                    Okay... I update script (hopefully last time) and add the underscore.

                                    1 Reply Last reply Reply Quote 0
                                    • meleuM
                                      meleu @cyperghost
                                      last edited by

                                      @cyperghost said in Create custom-collections from txt file [v1.3, final]:

                                      We can add _and then the underscore works

                                      There's no need to scape underscore. Then the regex is ^[][()\ \'_A-Za-z0-9-]+$

                                      • Useful topics
                                      • joystick-selection tool
                                      • rpie-art tool
                                      • achievements I made
                                      cyperghostC 1 Reply Last reply Reply Quote 0
                                      • W
                                        w0lv3r1nix
                                        last edited by

                                        Hi

                                        Really nice idea ! Thanks

                                        Suggestion :
                                        Instead of using if/else, just sanitize the string with the regex used in the if:
                                        sline=${$line//^[][()\ '_A-Za-z0-9-]/}

                                        1 Reply Last reply Reply Quote 1
                                        • cyperghostC
                                          cyperghost @meleu
                                          last edited by cyperghost

                                          @meleu updated
                                          Can we discuss the RegEx things on the bash script thread?
                                          It's a very powerfull feature indeed ;)

                                          @w0lv3r1nix
                                          Well that's also a suggestions there a many ways to skin a cat ;)
                                          As I'm still learning I'm thankfull for every little bash lesson.
                                          But I think the if/else clause is easier for code readings and beginners like me.
                                          See if you stumple after years on your piece of code.... you would ask... what was my intentions there.

                                          With if you have a expression with a clear result ... no if without then and maybe an else >> good result ;)

                                          meleuM 1 Reply Last reply Reply Quote 0
                                          • meleuM
                                            meleu @cyperghost
                                            last edited by

                                            @cyperghost said in Create custom-collections from txt file [v1.3, final]:

                                            Can we discuss the RegEx things on the bash script thread?

                                            Sure. Go ahead. :-)

                                            • Useful topics
                                            • joystick-selection tool
                                            • rpie-art tool
                                            • achievements I made
                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post

                                            Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                            Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.