Speech to text in Twilio over websocket using Google Speech API
npm install twilio-stt-gspeech
https://{host}/voice
set GOOGLE_APPLICATION_CREDENTIALS=D:\path\to\credentials\file.json
export GOOGLE_APPLICATION_CREDENTIALS=D:\path\to\credentials\file.json
$ mkdir twilio-transcribe-test
$ cd twilio-transcribe-test
$ touch index.js
$ npm i express @google-cloud/speech twilio ws twilio-stt-gspeech
`
Now we need to setup the server and configure the twilio speech to text using google speech API.
Open the index.js file and add the following code.
`
const express = require('express');
const http = require("http");
const speech = require('@google-cloud/speech');
const WebSocket = require('ws');
const { init } = require('twilio-stt-gspeech');
const app = express();
const server = http.createServer(app);
wss = new WebSocket.Server({ server });
init(wss, new speech.SpeechClient(), true);
app.use(express.json());
app.use(express.urlencoded({
extended: true
}))
app.use('/voice', require('./router'));
server.listen(8080, () => {
console.log("server running at 8080");
});
`
In the above code, init method has to be imported from twilio-stt-gspeech package. Next the parameters for the init method includes created websocket, google speech client, and an optional boolean parameter to display debug messages.
Next, a listener is added for the route /voice.
Create a folder with name as router and a file inside the folder and name it as index.js.
`
$ mkdir router
$ cd router
$ touch index.js
`
Open the index.js file and add the following code
`
const { VoiceResponse } = require('twilio').twiml;
exports.init = (host, { CallSid }) => {
const twiml = new VoiceResponse();
twiml.start()
.stream({
url: wss://${host}/${CallSid},
name: CallSid
});
twiml.redirect('/voice/stream');
return twiml.toString();
}
exports.stream = ({ CallSid }) => {
const twiml = new VoiceResponse();
twiml.stop()
.stream({
name: CallSid
});
return new Promise( (resolve, reject) => {
require('twilio-stt-gspeech').listen(CallSid)
.then( data => {
twiml.say("You said "+data);
resolve(twiml.toString());
})
.catch((err) => {
console.log("Error occurred", err);
reject();
})
});
}
`
When a call is made to a twilio number, and as per the twilio webhook endpoint which is configured to https://{host}/voice/ the init method is invoked.
The init method opens up a websocket stream for the caller and expects the user to speak. The websocket is opened using the twilio API start and inside it stream API is invoked using the following parameters that has to be mentioned explicitly. url is the the websocket url with paths containing the caller's SID and language with country code (Eg. en-IN for Indian English). By default the language is en-US. The next parameter is the name the value should be the caller's SID.
And finally it completes the request by redirecting to another route i.e., https://{host}/voice/stream/.
The redirected route will listen for the speech to text value from the package's(twilio-stt-gspeech) listen API. This API can be configured with a timeout so that for how long the route can wait to receive the speech to text value.
Hence, when the user speaks, it is received via websocket, and when the user pauses after speaking, then the Speech to Text value is resolved via the listen` method's promise.