Integrate Oswald with Google Assistant to create a powerful voice assistant
This week, we released an update to Oswald that was all about integrations (read about it in our previous blog). Today, I want to guide you through one specific integration: Google Assistant.
What is Google Assistant?
Google Assistant is the voice assistant of Google. With Google’s experience with natural language processing and search, it’s probably the smartest voice assistant available on the market. It has three main features:
- Available on many platforms
- Available in many languages
- Possible to integrate your own chatbots
Available on many platforms
Google Assistant is natively available on mobile devices 📱, like Android smartphones, or through a mobile application for iPhones. That’s nice, but it doesn’t stop there. It is available on smart speakers 🔊, like the complete collection of Google Home devices, but also third-party speakers like the Sonos One (at this moment only available for selected test users). And there are even more ways to talk to Google Assistant, like headphones 🎧 (Google Pixel Buds, Bose QuietComfort 35 II and many more), watches ⌚️, TV’s 📺, cars 🚗… We can conclude with the fact that many users might already have access to Google Assistant.
Available in many languages
One of the main reasons we started with Google Assistant as the first voice assistant integration for Oswald, was its support for the Dutch 🇧🇪 language. But there are many more languages like Danish 🇩🇰, English 🇺🇸, French 🇫🇷, German 🇩🇪, Hindi 🇮🇳, Italian 🇮🇹, Japanese 🇯🇵, Korean 🇰🇷, Norwegian 🇳🇴, Portuguese 🇧🇷, Spanish 🇪🇸 and Swedish 🇸🇪.
Possible to integrate your own chatbots
As an admirer of all Apple products and services and a heavy Siri user, this feels a little bit achy to write. The main problem with Siri is that it doesn’t integrate well with third-party applications. Yes, some applications already have a Siri integration, but it’s very limited compared to what Google and Amazon offer developers as a development platform for respectively Assistant and Alexa. Alexa offers ‘Alexa Skills’ to developers, Google offers ‘Google Actions’. These are development platforms to create your own chatbots, using the tools you want, and make them available through Alexa and Assistant. You basically only use the platform (and the devices), the speech to text and the text to speech from Google Assistant. Everything else is handled by your favourite chatbot platform. Of course, you will be using Oswald as chatbot platform, because of its easy to use interface and many extra features 😉
Let’s create a voice assistant
First of all, you need to create a chatbot, called “Frank The Weather Chatbot” (Belgians are going to get it 😃) using Oswald. In this blog, I’m not going to cover all Oswald features and methodologies. I already created a simple chatbot that knows all about the weather. It has the following intents:
- Clouds: ask about the cloud cover in a city
- Temperature: ask about the temperature in a city
- Weather info: ask about the weather in a city
- Wind: ask about the wind in a city
- Sun: ask about sunrise and sunset in a city
I uploaded a list of cities in the world as a ‘city’ entity in Oswald.
In every scenario, I created an advanced code response connecting to the Openweathermap API. Take a look at an example of such an advanced code response:
from responses.responsemessage import ResponseMessage
from responses.responsetype import ResponseType
from responses.responsedata import ResponseData
from responses.response import Response
import json
import requests
class WeatherInfo(Response):
baseUri = "http://api.openweathermap.org/data/2.5/"
appId = "XXX"
lang = "en"
units = "metric"
def getResponse(self, sentence, context, options={}):
cities = sentence.getEntitiesWithLabel("city")
if len(cities) == 0:
cities = context.getEntitiesWithLabel("city")
city = cities[0].value
params = {
"lang": self.lang,
"units": self.units,
"appid": self.appId
}
r = requests.get("{}weather?q={}".format(self.baseUri, city), params=params)
weatherInfo = json.loads(r.content)
description = weatherInfo["weather"][0]["description"]
return ResponseMessage(
data=[
ResponseData(
message="There is currently {} in {}.".format(description, city.title()),
type=ResponseType.text
)
]
)Now take a look at a conversation with the chatbot. Note the context sensitivity and the new ‘test your chatbot in your own webpage’ feature 😉
Connect to Google Assistant
Okay, that was a little introduction to my demo chatbot. Now let’s start with the real interesting part: the Google Assistant integration.
First of all, we’re going to click on ‘Integrations’ in the left menu and add Google Actions as a new integration.
You now see some steps to configure the configuration. Don’t worry I will guide you through it.
First, we’re going to log in to the Google Actions console and create a new project.
Now we’re going to the bottom of the page and click on the ‘Conversational’ card.
That was part one! Now we’re going to download the Google Actions command line interface. Now, I’m going to create a directory on my local machine with this Google Actions CLI file. Then, I’m going to cd into that directory and run the following commands:
mv gactions.dms gactions
chmod +x gactionsThe first command is because I don’t like the .dms extension 🤓 The second line is to make the gactions file executable. Next, we go to the integrations page and we click the ‘Download an actions.json file per language’ button. It downloads an actions file per language we defined for the chatbot. In this case, we’re only going to implement the English language. When the file is downloaded, we’re going to add it to our local directory. Then, we’re going to add it to our Google Actions project. We need our Google Actions project ID. You can find it in the Google Actions project settings page. For me, the Project ID is ‘frank-the-weather-chatbot’.
Okay, it’s time to run cd into our local directory and run the following command.
./gactions update --action_package actions.en-US.json --project frank-the-weather-chatbotWe get this response:
Gactions needs access to your Google account. Please copy & paste the URL below into a web browser and follow the instructions there. Then copy and paste the authorization code from the browser back here.
Visit this URL:
https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=XXX&redirect_uri=XXX&response_type=XXX&scope=XXX&state=XXX
Enter authorization code:Easy enough, copy/paste the URL in your browser, log in to your Google account and copy the authorization code. Paste the authorization code in your terminal and hit the return key!
When we go to the Actions page in the Google Actions console, we see there is a new actions.intent.MAIN action for the English language.
That’s it! We’ve successfully integrated our Frank The Weather Chatbot with Google Assistant. Now let’s activate the Google Actions integration in Oswald by activating the toggle.
Then, we’re going to try the Google Actions simulator.
Cool, right? Once the connection is made, you can make any changes you like to the Oswald chatbot. Once you publish your changes, your Google Assistant chatbot automatically gets updated. And it gets even beter! Because I have a Google Home Mini, which is logged in to the same Google Account, I can test my Oswald chatbot using my Google Home. Watch the video below to see the results!
