Adding AI like Alexa or Google Assistant to retropie?

alaxmofo12

I was searching for AI for raspberry pi 3 and came across this

"https://www.hackster.io/xtools/assistantpi-74b772"

And it says It requires raspbian jessie lite.. which retropie does run. So it should be possible to have an ai alongside retropie. I would test this myself but I have no USB mic. Anybody willing to try?

Zigurana

Can you describe the use case here? What would this do?

alaxmofo12

@zigurana Well, do you not think it would be cool to have an ai on retropie? Like you could tell it to go to a game system or certain game

Zigurana

Ah, I see, you would like a voice-controller for EmulationStation?

alaxmofo12

@zigurana yea pretty much put you could do much more with it like playing music from Spotify, and Pandora, etc. Also the Ai's like Google Assistant and Alexa have custom commands so you could tell it go to a game in said console. So many possibilities if it could work. If no testers I'm waiting for USB mic in the mail.

linkreincarnate

It kinda works, I just finished integrating google assistant but there are a few kinks to fix.

It doesn't work currently in anything but desktop.mode. I think this has something to do with how emulation station uses audio.
When I try to ssh in and start the google assistant demo it runs but is unable to take control of the audio to output anything. I think emulation station may hog the resource without letting it go. It works perfect in desktop mode though. I was even able to use ifttt to add new functionality to it once it was installed.

I've also read that you can get GA to control the GPIO pins and run bash scripts by setting up an https server on your pi and using nodered and ifttt to get GA to send commands to it. I'm working on implementing that bit now.

I have no idea how to get around the audio issue. If any linux audio guru's want to chime in I'm all ears.

Oh, here is the link to the walkthrough in the google assistant sdk docs for setting it up on the raspberry
pi.
https://developers.google.com/assistant/sdk/develop/python/hardware/setup

pjft

@alaxmofo12 you're talking about voice control, not necessarily AI. :)

AI would be nice for personalized game recommendations based on the ones you like, crowdsourcing best settings, performance tweaks, etc. Even adding an AI controlled second player for co op or competitive games would be a great - albeit extremely difficult - effort!

@linkreincarnate i imagine it might be related to the way we're using SDL for audio. There's an open effort to improve some of that, unsure if related.

Tell me: do you have to create a new paid Google Cloud account to play and customize this? Or can we use the API as is for free?

linkreincarnate

@pjft You have to create an account but it's free (you still need a card to sign up though) and they give you a 300 dollar credit for cloud stuff. Nothing I have done with it has costed me any money though. Maybe if i launched something publicly but just getting it up and running on my pie was free.

linkreincarnate

Ok so I've made no progress with this. Here is exactly what happens when I try to run this with emulation station up after boot instead of the desktop. Can anyone direct me to some things to check? I really don't know where to start.

pi@Iron-ManMk1Fightstick:~ $ source env/bin/activate
(env) pi@Iron-ManMk1Fightstick:~ $ google-assistant-demo
ON_MUTED_CHANGED:
{'is_muted': False}
ON_START_FINISHED

ON_CONVERSATION_TURN_STARTED
ON_END_OF_UTTERANCE
ON_RECOGNIZING_SPEECH_FINISHED:
{'text': 'tell me a joke'}
ON_RESPONDING_STARTED:
{'is_error_response': False}
OpenAlsaOutputStream PcmOpen: Device or resource busy
Start: pcm could not be opened...
[1307:1317:ERROR:audio_output_stream.cc(109)] An error occurred when playing audio output
ON_RESPONDING_FINISHED
ON_CONVERSATION_TURN_FINISHED:
{'with_follow_on_turn': False}

pjft

@linkreincarnate the pcm device is busy with ES.

You may want to try, for the sake of hacking, to change the ES audio device in the sound settings and maybe restart ES. Try others. They won't work, meaning you won't have ES sound, but hopefully for this test it'll allow you to load the assistant up.

alaxmofo12

I would test this myself but still haven't received my USB mic... But anyways I may have found a lead , by splitting the audio from retropie and the ai to play at the same time. Here (https://raspberrypi.stackexchange.com/questions/40310/different-audio-output-on-hdmi-and-jack)

Near the end of the page