RetroPie forum home
    • Recent
    • Tags
    • Popular
    • Home
    • Docs
    • Register
    • Login

    Adding AI like Alexa or Google Assistant to retropie?

    Scheduled Pinned Locked Moved Ideas and Development
    artificial
    16 Posts 5 Posters 6.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      alaxmofo12
      last edited by

      I was searching for AI for raspberry pi 3 and came across this

      "https://www.hackster.io/xtools/assistantpi-74b772"

      And it says It requires raspbian jessie lite.. which retropie does run. So it should be possible to have an ai alongside retropie. I would test this myself but I have no USB mic. Anybody willing to try?

      1 Reply Last reply Reply Quote 0
      • Z
        Zigurana
        last edited by

        Can you describe the use case here? What would this do?

        If tetris has thought me anything, it's that errors pile up and that accomplishments dissappear.

        A 1 Reply Last reply Reply Quote 0
        • A
          alaxmofo12 @Zigurana
          last edited by

          @zigurana Well, do you not think it would be cool to have an ai on retropie? Like you could tell it to go to a game system or certain game

          1 Reply Last reply Reply Quote 0
          • Z
            Zigurana
            last edited by

            Ah, I see, you would like a voice-controller for EmulationStation?

            If tetris has thought me anything, it's that errors pile up and that accomplishments dissappear.

            A 1 Reply Last reply Reply Quote 0
            • A
              alaxmofo12 @Zigurana
              last edited by

              @zigurana yea pretty much put you could do much more with it like playing music from Spotify, and Pandora, etc. Also the Ai's like Google Assistant and Alexa have custom commands so you could tell it go to a game in said console. So many possibilities if it could work. If no testers I'm waiting for USB mic in the mail.

              pjftP 1 Reply Last reply Reply Quote 0
              • L
                linkreincarnate
                last edited by linkreincarnate

                It kinda works, I just finished integrating google assistant but there are a few kinks to fix.

                It doesn't work currently in anything but desktop.mode. I think this has something to do with how emulation station uses audio.
                When I try to ssh in and start the google assistant demo it runs but is unable to take control of the audio to output anything. I think emulation station may hog the resource without letting it go. It works perfect in desktop mode though. I was even able to use ifttt to add new functionality to it once it was installed.

                I've also read that you can get GA to control the GPIO pins and run bash scripts by setting up an https server on your pi and using nodered and ifttt to get GA to send commands to it. I'm working on implementing that bit now.

                I have no idea how to get around the audio issue. If any linux audio guru's want to chime in I'm all ears.

                Oh, here is the link to the walkthrough in the google assistant sdk docs for setting it up on the raspberry
                pi.
                https://developers.google.com/assistant/sdk/develop/python/hardware/setup

                1 Reply Last reply Reply Quote 1
                • pjftP
                  pjft @alaxmofo12
                  last edited by

                  @alaxmofo12 you're talking about voice control, not necessarily AI. :)

                  AI would be nice for personalized game recommendations based on the ones you like, crowdsourcing best settings, performance tweaks, etc. Even adding an AI controlled second player for co op or competitive games would be a great - albeit extremely difficult - effort!

                  @linkreincarnate i imagine it might be related to the way we're using SDL for audio. There's an open effort to improve some of that, unsure if related.

                  Tell me: do you have to create a new paid Google Cloud account to play and customize this? Or can we use the API as is for free?

                  L 1 Reply Last reply Reply Quote 0
                  • L
                    linkreincarnate @pjft
                    last edited by

                    @pjft You have to create an account but it's free (you still need a card to sign up though) and they give you a 300 dollar credit for cloud stuff. Nothing I have done with it has costed me any money though. Maybe if i launched something publicly but just getting it up and running on my pie was free.

                    1 Reply Last reply Reply Quote 0
                    • L
                      linkreincarnate
                      last edited by

                      Ok so I've made no progress with this. Here is exactly what happens when I try to run this with emulation station up after boot instead of the desktop. Can anyone direct me to some things to check? I really don't know where to start.

                      pi@Iron-ManMk1Fightstick:~ $ source env/bin/activate
                      (env) pi@Iron-ManMk1Fightstick:~ $ google-assistant-demo
                      ON_MUTED_CHANGED:
                      {'is_muted': False}
                      ON_START_FINISHED

                      ON_CONVERSATION_TURN_STARTED
                      ON_END_OF_UTTERANCE
                      ON_RECOGNIZING_SPEECH_FINISHED:
                      {'text': 'tell me a joke'}
                      ON_RESPONDING_STARTED:
                      {'is_error_response': False}
                      OpenAlsaOutputStream PcmOpen: Device or resource busy
                      Start: pcm could not be opened...
                      [1307:1317:ERROR:audio_output_stream.cc(109)] An error occurred when playing audio output
                      ON_RESPONDING_FINISHED
                      ON_CONVERSATION_TURN_FINISHED:
                      {'with_follow_on_turn': False}

                      pjftP 1 Reply Last reply Reply Quote 0
                      • pjftP
                        pjft @linkreincarnate
                        last edited by

                        @linkreincarnate the pcm device is busy with ES.

                        You may want to try, for the sake of hacking, to change the ES audio device in the sound settings and maybe restart ES. Try others. They won't work, meaning you won't have ES sound, but hopefully for this test it'll allow you to load the assistant up.

                        1 Reply Last reply Reply Quote 0
                        • A
                          alaxmofo12
                          last edited by

                          I would test this myself but still haven't received my USB mic... But anyways I may have found a lead , by splitting the audio from retropie and the ai to play at the same time. Here (https://raspberrypi.stackexchange.com/questions/40310/different-audio-output-on-hdmi-and-jack)

                          Near the end of the page

                          1 Reply Last reply Reply Quote 0
                          • L
                            linkreincarnate
                            last edited by linkreincarnate

                            I found another tutorial on this but have been unable to get the install scripts to run.

                            YouTube Music Streaming for Google Assistant on Raspberry Pi

                            when I get to step 4.2 Execute the installers and I run this command

                            sudo /home/pi/GassistPi/scripts/gassist-installer-pi-zero.sh
                            it says

                            sudo: pip3: command not found

                            Any idea what is causing this? Running apt-get install python 3 shows that it is installed

                            running man pip shows and entry for pip3

                            I am a complete linux noob so I'm running out of ideas to fix this.

                            1 Reply Last reply Reply Quote 0
                            • mituM
                              mitu Global Moderator
                              last edited by

                              @linkreincarnate Try to install python3-pip and see if the pip3 command becomes available.

                              1 Reply Last reply Reply Quote 0
                              • L
                                linkreincarnate
                                last edited by

                                It's installing now. So pip is a python compiler?

                                mituM 1 Reply Last reply Reply Quote 0
                                • mituM
                                  mitu Global Moderator @linkreincarnate
                                  last edited by

                                  @linkreincarnate No, pip is the Python package installer.

                                  There are different versions for pip, depending on the Python version you're using. Although different versions can co-exist in the system at the same time, I think in Jessie the default is Python version 2, hence pip is the version 2 package installer.
                                  That's why you need the version 3 installed explicitely.

                                  1 Reply Last reply Reply Quote 0
                                  • L
                                    linkreincarnate
                                    last edited by linkreincarnate

                                    Ok thanks. Well I got this working right thanks to you, not only did it compile and install, but it's audio works in emulationstation as a bonus! Also you can set actions to be performed locally without having to use webhooks and a sever. Also custom wakewords. If people just install pip manually for python 3 then they can follow that entire tutorial and it will work.

                                    I'm going to start on adding macro recording and playback (probably over my head but that's how I learn...)activated by google voice commands next.

                                    After I make it respond to "Jarvis".

                                    1 Reply Last reply Reply Quote 0
                                    • First post
                                      Last post

                                    Contributions to the project are always appreciated, so if you would like to support us with a donation you can do so here.

                                    Hosting provided by Mythic-Beasts. See the Hosting Information page for more information.