The universal, low-level SDK for Keyframe Labs.
npm install @keyframelabs/sdkThe universal, low-level SDK for Keyframe Labs.
Which package should I use?
- @keyframelabs/sdk (high control)
- You implement the UI, state management, and agent/llm binding yourself
- @keyframelabs/elements (custom UI)
- You implement the UI; we handle the state and agent/llm binding (framework-agnostic)
- @keyframelabs/react: (drop-in)
- We handle the UI, state, and agent/llm binding
``bash`
pnpm add @keyframelabs/sdk
From your backend, use your secret Keyframe API key to create a session.
`typescriptBearer ${process.env.KFL_API_KEY}
// POST https://api.keyframelabs.com/v1/session
const response = await fetch('https://api.keyframelabs.com/v1/session', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': ,
},
body: JSON.stringify({
persona_id: "6efx..." // or persona_slug
}),
});
// Returns { serverUrl, participantToken }
const session = await response.json();
`
Pass the session details to the client SDK.
`typescript
import { createClient } from '@keyframelabs/sdk';
const persona = createClient({
serverUrl: "wss://...",
participantToken: "A6gB...",
onVideoTrack: (track) => {
// Some HTML video element
videoElement.srcObject = new MediaStream([track]);
},
onAudioTrack: (track) => {
// Some HTML audio element
audioElement.srcObject = new MediaStream([track]);
},
});
// Connect to the session
await client.connect();
// Send audio (e.g. from your LLM/Agent)
client.sendAudio(pcmAudioBytes);
// Signal an interruption (clears pending frames)
persona.interrupt();
// Cleanup
await client.close();
`
The SDK handles the real-time transport loop between your agent or real-time LLM and the Keyframe Platform, as well as synced audio/video rendering.
`
+-----------------------+ +-----------------------+
| Browser | | Keyframe Platform |
| | | |
| +-----------------+ | | +-----------------+ |
| | Microphone | | | | AvatarSession | |
| +-----------------+ | | +-----------------+ |
| | | | ^ |
| v | | | |
| +-----------------+ | DataStream | | |
| | Agent / LLM | | (PCM 16kHz) | | |
| +-----------------+ | ----------------------> | | |
| | | | | |
| v | | v |
| +-----------------+ | | +-----------------+ |
| | PersonaSession | | | | Inference | |
| +-----------------+ | | +-----------------+ |
| ^ | | | |
| | | WebRTC | | |
| | | (Audio + Video) | v |
| +-----------------+ | <---------------------- | +-----------------+ |
| | Video Element | | | | Video | |
| +-----------------+ | | +-----------------+ |
| | | |
+-----------------------+ +-----------------------+
The SDK is intentionally minimal—it only handles the avatar connection. You bring your own agent or real-time LLM (e.g, Cartesia, ElevenLabs, Gemini, OpenAI).
Initializes the WebSocket connection and media managers.
| Option | Type | Default | Description |
| ------------------ | ----------------- | -------- | --------------------------------------------------------- |
| serverUrl | string | Required | The WSS URL returned by the /session API endpoint. |participantToken
| | string | Required | The access token returned by the /session API endpoint. |onVideoTrack
| | (track) => void | Required | Fired when the WebRTC video track is ready. |onAudioTrack
| | (track) => void | Required | Fired when the WebRTC audio track is ready. |onStateChange
| | (state) => void | — | Fired when session state changes. |onError
| | (err) => void | — | Fired on errors. |
The client instance returned by createClient().
#### Methods
| Method | Signature | Description |
| ----------- | ---------------------------------------------- | ------------------------------------------------ |
| connect | () => Promise | Connect to the session. |sendAudio
| | (pcmData: ArrayBuffer \| Int16Array) => void | Send 24 kHz 16-bit PCM audio. |interrupt
| | () => void | Signal an interruption and clear pending frames. |close
| | () => void | Close the session and release resources. |
#### Properties
| Property | Type | Description |
| -------- | ---------------------------------------------------------- | ---------------------- |
| state | 'disconnected' \| 'connecting' \| 'connected' \| 'error'` | Current session state. |
MIT