HeyGen SDK for interactive avatar
npm install @heygen/streaming-avatarpushToTalkStart and pushToTalkStop. voiceChatTransport: VoiceChatTransport.LIVEKIT, and requires activityIdleTimeout to control time before session ending after last activity.keepAlive method to continue session in a controlled way.disableIdleTimeout, activityIdleTimeout and keepAlive should be used instead.eleven_flash_v2_5 and eleven_multilingual_v2 models.duration_ms in the avatar_stop_talking event1. The API now supports interrupt current streaming
1. The API now supports start/stop event callbacks
2. The API now supports setting jitter buffer target (https://developer.mozilla.org/en-US/docs/Web/API/RTCRtpReceiver/jitterBufferTarget)
This SDK provides a convenient way for developers to better interface with HeyGen's Interactive Avatar.
``
npm install @heygen/streaming-avatar
To build and compile the typescript sources to javascript run install then:
`
npm run build
For demo of this SDK and how it is used when installed in an app, please refer to the following: https://github.com/HeyGen-Official/InteractiveAvatarNextJSDemo.
`JS
import StreamingAvatar, { AvatarQuality, StreamingEvents } from '@heygen/streaming-avatar';
let streamingAvatar;
async function startChatCreation(){
streamingAvatar = new StreamingAvatar({token: 'ENTER_ACCESS_TOKEN_HERE'});
// some events
streamingAvatar.on(StreamingEvents.AVATAR_START_TALKING, (e) => {});
streamingAvatar.on(StreamingEvents.STREAM_DISCONNECTED, () => {});
streamingAvatar.on(StreamingEvents.STREAM_READY, (event) => {});
const sessionInfo = await streamingAvatar.createStartAvatar({
quality: AvatarQuality.Low,
avatarName: avatarId,
knowledgeId: knowledgeId, // from labs.heygen.com
// knowledgeBase: knowledgeBase, // your customized prompt content
voice: {
voiceId: voiceId,
rate: 1.5, // 0.5 ~ 1.5
emotion: VoiceEmotion.EXCITED,
// elevenlabsSettings: {} // https://docs.heygen.com/reference/new-session#voicesetting
// model: ElevenLabsModel.MULTILINGUAL, elevenlabs audio model
},
sttSettings: {
provider: STTProvider.DEEPGRAM, // STT provider to use. The default is DEEPGRAM.
confidence: 0.55, // The default is 0.55.
},
language: language,
// disableIdleTimeout: false, // Default is false; enable cautiously.
voiceChatTransport?: VoiceChatTransport.WEBSOCKET, // user input transport. The default is WEBSOCKET
// number of seconds that avatar will wait before closing session after last activity,
// expects value from 30 to 3600 (1h)
activityIdleTimeout?: number, // The default is 120 (2 minutes)
});
// switch to voice chat. in this mode, we will record your voice and keep chatting with avatar in real time.
await streamingAvatar.startVoiceChat({
useSilencePrompt: true, // the default is false. true means you will receive silence prompts.
isInputAudioMuted: true, // the default is false. you can also handle mute by using streamingAvatar.muteInputAudio(), streamingAvatar.unmuteInputAudio().
});
}
// In text mode, please use the speak method (Default TALK type).
streamingAvatar.speak({ text: text, task_type: TaskType.REPEAT, taskMode: TaskMode.SYNC });
// Please note, you can use the speak method in voice chat, but only the TALK type is supported in voice chat mode.
streamingAvatar.speak({ text: text })
// close voice chat, will stop recording your voice.
streamingAvatar.closeVoiceChat();
// close the session
streamingAvatar.stopAvatar();
// keep session alive, will be count as an activity to keep session for additional activityIdleTimeout seconds
// after last activity.
streamingAvatar.keepAlive();
// interrupt the avatar's talking
streamingAvatar.interrupt();
// it's helpful in text mode. startListening will let the avatar switch to listening state.`
streamingAvatar.startListening();
streamingAvatar.stopListening();
To generate your access token you must first have access to your API key. API Keys are reserved for Enterprise customers. You can retrieve either the API Key or Trial Token by logging in to HeyGen and navigating to this page in your settings: https://app.heygen.com/settings?nav=API. Afterwards you can run the following to obtain your access token.
Please note the tokens are one time use.
`
curl -X POST https://api.heygen.com/v1/streaming.create_token -H "x-api-key:
By default, there are several Public Interactive Avatars that can be used. You can find the Avatar IDs for these Avatars by navigating to labs.heygen.com/interactive-avatar and clicking 'Select Avatar'.
You can create your own Interactive Avatar to use with this API by visiting labs.heygen.com/interactive-avatar and clicking 'Create Interactive Avatar' at the bottom of the screen.
Most likely, you are hitting your concurrent session limit. While testing this API with your Trial Token, only 3 concurrent sessions can be created. Please endeavor to close unused sessions with the Close Session endpoint when they are no longer being used; they will automatically close after some minutes.
You can check how many active sessions you have open with the List Sessions endpoint: https://docs.heygen.com/reference/list-sessions