Enable your Assistant action to send data to any nearby device.
Zero setup, configuration or pairing.
Thanks to our talented engineer Dinu, Chirp is now easily integrated with Google Assistant allowing developers to create actions which can broadcast data to any nearby device with a microphone.
The ability to send data using audio from any Assistant device opens up a wide range of possibilities. Engineers can implement Chirp data transfer in a flexible way within their product in a way that suits their particular use case. For example:
- An application can now send authentication or configuration data to a users secondary device making login more seamless.
‘Ok Google, log me into App X on my phone’
‘Ok Google, setup my new Product Y device’
‘Ok Google, send Wi-Fi credentials to my Product Z device’
- An action can send data about a product or event with every nearby device, regardless of whether those devices are paired, setup or otherwise associated with the Google Assistant device.
‘Ok Google, share my Product X profile with everyone here’
‘Ok Google, send directions to Place Y to everyone in the room’
- New second-screen creative experiences can be created.
‘Ok Google, we decide to turn left into the enchanted forest in Game Z’
At Chirp, we work hard to make sure that our SDKs and APIs are flexible so that they can be applied to brand new challenges in all manner of different products and services.
With our Google Assistant support this is no different - we are excited to see what developers can create by integrating our technology, leveraging Google Assistant to send data to any nearby device.
In this simple example tutorial we will implement a simple use case. We will ask Google Assistant to share our Wi-Fi credentials via Chirp.
Here's what we are aiming for :
First we'll invoke the Assistant action, then the Google Assistant will respond and, using the Chirp Audio REST API, send Wi-Fi credentials encoded as audio. Any nearby device running the Chirp SDK will be able to hear and decode the broadcast, and can then use the credentials to access the local wireless network.
1. In order to be able to test the Action we are going to build, make sure you have enabled the following permissions on the Activity Controls page.
- Web & App Activity (you should also enable the option to ‘Include Chrome history')
- Device Information
- Voice & Audio Activity
2. From the “Actions console” page, create a new “Actions project”. This name is only for your internal reference.
3. Choose the type of action you want to build. In our case, we’ll choose
Conversational (Build customised conversational experiences).
4. Click Build your Action to expand the options and select
Add your first action.
6. On the Create Action dialog, select
Custom Intent and click
Build. This will open the Dialogflow Console in another tab.
7. On the Dialogflow Console, click the
Create button to save the Action.
8. Click on the
Intents tab and delete all the intents (if you have any), except Default Welcome Intent.
9. Open Default Welcome Intent and rename it 'Send' to indicate our intent script is intended for sending the credentials when intent will be triggered.
10. Remove all the training phrases you have and replace them with “Ask wifi manager to send wifi credentials”.
Also, remove all responses, as we are going to generate responses in a script.
11. In the Fulfillment section click on
Enable webhook call for this intent in order to indicate our Action should trigger a script when this intent is requested.
12. Save the changes and go to the Fulfillment tab to indicate the Action the script to trigger when the intent is invoked.
13. Here we have two options, either to use a Webhook or the Inline Editor which will host the script on Firebase in the end.
Because the Firebase free tier only allows outbound networking to Google services we need to make a network call in order to generate an Authentication Token for the Chirp Audio API. We are therefore going to build our own server which will generate a response for our Action allowing us to make authenticated calls to the Chirp audio API.
If you are on the paid tier of Firebase, or already have infrastructure available to request an authentication token you can upload the script directly into Inline Editor rather than building your own server.
So, here we go with the Action script :
14. In the example source files, which you can clone from this Github repository you have to update the Chirp
APP_SECRET which you can get from the Chirp developers website.
Also, you may want to update the
WIFI_PASSWORD which represent your Wi-Fi credentials.
15. There are a couple of things here to note before going forward:
- The Audio API URL points to the
/default/protocol which you can choose from the developer's applications page.
For more info about Audio API see documentation here.
- The payload which is generated from your
buildCredentialsPayload function) should not exceed the maximum payload size defined by the protocol you have chosen on the developer's applications page.
- Google assistant is not capable of sending ultrasonic data (at least with the Android phone we tested here) and for that reason, I recommend using the 16kHz mono protocol which is capable of sending 32 bytes payloads, enough for most WiFI credentials.
16. Once you are done with the script, we need to start it from a secure (https) server in order for google assistant to be able to reach it. If you don’t have any available secured servers, I recommend using ngrok which will generate a tunnel through “ngrok” secured host which will make your server publicly available and accessible for google assistant.
node index.jsto start the local server
./ngrok http 3000to start “ngrok” tunnel on local port 3000.
17. If you are done with the local server and you have the script running, go back to the Fulfillment tab in your Dialogflow editor and enable the Webhook option.
18. Paste the webhook url into the URL field, click save button and we are ready for testing.
19. In the Dialogflow Console left navigation, click on Integrations. Then, click on Google Assistant > Integration Settings.
20. Click Test to update your Actions project and load it into the Actions Console simulator.
21. In the Actions console type in your utterance “Ask WiFi manager to send WiFi credentials” and check if you get the expected response with chirp being played.
22. Now you can go ahead and ask your Google Assistant to send Wi-Fi credentials with Wi-Fi manager.
Note that you have to use the devices with the same account as you set up your Action and since the Action. Also, on mobile devices I suggest using Google Assistant app rather then default google assistant widget since this didn’t work straight away (I assume this is because the Action is not publicly available yet at this stage).