How to create your Smart Toy with bluemix & 7688 Duo board

© 2014 IBM Corporation

How to Create your Smart Toy with IBM bluemix & MTK 7688 Duo

Tommy Wu , IBM Cloud.

Senior Cloud Architect

Document number


準備清單(Prerequisite)

MTK 7688 Duo 一塊

USB Sound Card 外接音效卡

Speaker 喇叭

Mic 麥克風

Ragdoll 布娃娃

SD卡

2


架構示意圖 -Device Publish STT Result To Cloud Q&A Logic

iot_client.publish('iot-2/evt/status/fmt/json',

'{"d":{"status": "connected" }}');

3

7688 Duo DEIVCE

Node-RED

你好

我是ooo MQTT

Q&A Logic

小腦

大腦


架構示意圖 -Device 監聽雲端 Q&A Response Data to Speaker Out

4

speaker

Node-RED

7688 Duo

device MQTT

大腦

小腦

iot_client.on("message",

function(topic,payload){

console.log('received topic:'+topic+',

payload:'+payload);});

Q&A Logic


Bluemix 準備工作


BLUEMIX Registration 事前準備!

6

前往BLUEMIX官網申請帳號


建立Speech To Text 服務

Bluemix > Category > Watson > Speech To Text > Create Services

Bluemix > Dashboard > Servies > Speech To Text-ro – Get Your Username/Password

7


建立IOT應用程式

8


Go To Bluemix Dashboard – Create Your application

9


Choose Template - 選擇樣板

10


Choose IOT Starter Template 應用程式

11


完成建立IOT AP 應用程式

12

建立完成後，Go To Bluemix > Dashboard > Application > Your IOT Application


選擇 IOT 服務

13


GO To IOT Service Dashboard -前往儀表板

14


Select Devices Tab -點選裝置

15


Add Your Device -新增裝置

16

Ex :7688duo robot device 2016年6月10日下午10:20:55


Please keep the device token -請保留這張資訊

17


Device 端 (7866 Duo) 邏輯開發


Initialize MTK LinkIt 7688 Duo

19

參考CAVEDU 基本設置http://blog.cavedu.com/%E7%89%A9%E8%81%AF%E7%B6%B2/linkit-

7688duo%E6%9D%BF%E4%BB%8B%E7%B4%B9%E8%88%87arduino-

%E8%A8%AD%E7%BD%AE%E6%95%99%E5%AD%B8/

安裝所需npm package : mqtt/watson-developer-cloud/node-aplay/request/fs/querystring

http://blog.cavedu.com/%E7%89%A9%E8%81%AF%E7%B6%B2/linkit-7688duo%E6%9D%BF%E4%BB%8B%E7%B4%B9%E8%88%87arduino-%E8%A8%AD%E7%BD%AE%E6%95%99%E5%AD%B8/









Cross-Compiler for NPM Packages

20

本身只有 32mb flash 以及 128 mb Memory,所以需要Cross-Compiler來加快速度，有二個方案：

https://iamblue.gitbooks.io/linkit-smart-nodejs/content/zh-TW/emulator/ 開個7688模擬器

https://medium.com/maker-cup/mt7688-%E6%8A%97%E7%97%9B%E5%AF%A7-

%E4%BA%A4%E5%8F%89%E7%B7%A8%E8%AD%AF-node-js-

%E5%8E%9F%E7%94%9F%E6%A8%A1%E7%B5%84-69abc6258bce#.ggasla7qu建立7688 cross-compiler環境

安裝所需npm package : mqtt/watson-developer-cloud/node-aplay/request/fs/querystring

安裝 madplay for playing TTS voice.

空間使用資源如下：可以把所有東西移到SD卡，下次複製比較方便 (Put all your resource

under SD card for easier replication)

https://iamblue.gitbooks.io/linkit-smart-nodejs/content/zh-TW/emulator/







https://medium.com/maker-cup/mt7688-%E6%8A%97%E7%97%9B%E5%AF%A7-%E4%BA%A4%E5%8F%89%E7%B7%A8%E8%AD%AF-node-js-%E5%8E%9F%E7%94%9F%E6%A8%A1%E7%B5%84-69abc6258bce



















以npm安裝mqtt package 並撰寫code

var clientId = [‘d’, “(組織ID)”, “robot”, “(裝置ID)”].join(‘:’); //7688 Duo Device Client ID

iot_client = mqtt.connect("mqtt://(組織ID).messaging.internetofthings.ibmcloud.com:1883",//

連結mqtt伺服器的認證

{

"clientId" : clientId,

"keepalive" : 30,

"username" : "use-token-auth",

"password" : "(鑑別記號)"

});

21


傳送一個Event至BLUEMIX的Status節點

22

iot_client.on('connect', function() {//如果連接上的話要做的事情

console.log(‘STT client connected to IBM IoT Cloud.');

iot_client.publish('iot-2/evt/status/fmt/json', '{"d":{"status": "connected" }}');

//發布一個event至伺服器, event的名子叫做status, format為json，其中值為 {“status”，”connected”}

iot_client.subscribe('iot-2/cmd/+/fmt/+', function(err, granted){

console.log('subscribed command, granted: '+ JSON.stringify(granted)); });

//訂閱來自Cloud的response command, 允許為任何format


23

7688 Duo Device 端程式邏輯 (1)

- Watson STT Enablement

Use 8bit 22k recording to avoid overrun & delay

Enable Watson Speech-to-Text Streaming Service


7688 Duo Device 端程式邏輯 (2) -Watson STT Listen Event & IOT Publish

7688 Duo subscribe Cloud Q&A Logic and Response in Google TTS (中文)

24

Websocket listen for STT results and Publish to Bluemix IOT


Device端 7688 Duo 執行命令及結果檢視(monitor)

啟動Node.js AP

語音問答 (Q&A Results)

25


雲端(Bluemix) 對談流程設計


前往IOT application - Node-RED based Application

27


Design your Simple Q&A Logic with Node-RED Flow

28


執行程式碼就可以看到資訊

29

You should see the status connected sent from 7688 Duo


雲端大腦 Simple Q&A Editor

Node-Red : 4 Nodes to create your own dialog

30

Voice STT-In

Google TTS-Out

自行編輯欲對話的簡單QA


Next Step

Connect To Watson Service : Dialog, Natural Language Classifier , Conversation.

Add Grove Pi Sensors to 7688 Duo Arduino MCU board

31


Resources

Bluemix Registration : https://console.ng.bluemix.net/registration

Source Code on Github : https://github.com/tommywu052/7688duoDoll

Demo Video on Youtube : https://youtu.be/aQ24_Zdb9Ps

32

https://console.ng.bluemix.net/registration

https://github.com/tommywu052/7688duoDoll

https://youtu.be/aQ24_Zdb9Ps


謝謝觀賞!

33

How to create your Smart Toy with bluemix & 7688 Duo board

Technology

Transcript of How to create your Smart Toy with bluemix & 7688 Duo board