@spatialwalk/avatarkit-rtc Unified RTC adapter for avatarkit - supports LiveKit, Agora and other RTC providers.
š Documentation : https://docs.spatialreal.ai/web-sdk/introduction
> For server-side RTC SDK integration, please refer to the online documentation .
Installation ā ļø IMPORTANT: This package does NOT work standalone. You MUST install the dependencies below.
$3 | Package | Purpose | Required | |---------|---------|----------| | @spatialwalk/avatarkit | Web SDK for avatar rendering | ā
Yes | | @spatialwalk/avatarkit-rtc | This package - RTC adapter | ā
Yes | | livekit-client | LiveKit RTC SDK | ā” Choose one | | agora-rtc-sdk-ng | Agora RTC SDK | ā” Choose one |
$3 ``bashWith LiveKit pnpm add @spatialwalk/avatarkit @spatialwalk/avatarkit-rtc livekit-client
With Agora pnpm add @spatialwalk/avatarkit @spatialwalk/avatarkit-rtc agora-rtc-sdk-ng
`
$3 Step 1: Install the Web SDK (required dependency)
`
bash pnpm add @spatialwalk/avatarkit`
Step 2: Install the RTC adapter
`
bash pnpm add @spatialwalk/avatarkit-rtc`
Step 3: Install your chosen RTC provider SDK
`
bashFor LiveKit pnpm add livekit-client
For Agora pnpm add agora-rtc-sdk-ng`
$3 | Error | Cause | Solution | |-------|-------|----------| |
No matching version found for @spatialwalk/avatarkit-rtc@^0.x.x
| Wrong version number | Use pnpm add @spatialwalk/avatarkit-rtc
(no version) | | No matching version found for @spatialwalk/avatarkit@^0.x.x
| Wrong version number | Use pnpm add @spatialwalk/avatarkit
(no version) | | Cannot find module '@spatialwalk/avatarkit'
| Missing dependency | Install it: pnpm add @spatialwalk/avatarkit
| | Avatar placeholder - SDK integration required
| SDK not initialized | Call AvatarSDK.initialize()
before creating AvatarView |
Get Credentials Before using the SDK, you need to obtain authentication credentials:
$3 - Visit the Developer Platform to create your App - Get your App ID from the platform dashboard - The App ID is required for SDK initialization
$3 - Session Token is required for avatar loading and SDK authentication - Session Token should be obtained from your backend server - Your backend server should request the Session Token from the AvatarKit Server - Session Token has a maximum validity of 1 hour
$3
`
Your Client ā Your Backend Server ā AvatarKit Server ā returns Session Token (max 1 hour)`
> Note: App ID and Session Token are paired and must be used together. Each App ID corresponds to a specific list of avatars that can be driven.
Quick Start When initializing
@spatialwalk/avatarkit
, use DrivingServiceMode.host
since network communication is managed by this SDK.
`
typescript import { AvatarPlayer, LiveKitProvider, AgoraProvider } from '@spatialwalk/avatarkit-rtc'; import { AvatarSDK, AvatarView, AvatarManager, DrivingServiceMode, Environment } from '@spatialwalk/avatarkit';// 0. Initialize SDK (use host mode) await AvatarSDK.initialize(appId, { environment: Environment.cn, // or Environment.intl drivingServiceMode: DrivingServiceMode.host, });
// 1. Set session token (required for avatar loading) AvatarSDK.setSessionToken(sessionToken);
// 2. Create Avatar and AvatarView const avatar = await AvatarManager.shared.load(characterId); const avatarView = new AvatarView(avatar, container);
// 3. Create Provider (choose LiveKit or Agora) const provider = new LiveKitProvider(); // or new AgoraProvider()
// 4. Create Player const player = new AvatarPlayer(provider, avatarView, { logLevel: 'warning', // optional });
// 5. Connect to RTC server await player.connect({ url: 'wss://your-livekit-server.com', token: 'your-token', roomName: 'room-name', });
// 6. Start microphone await player.startPublishing();
// 7. Stop microphone await player.stopPublishing();
// 8. Disconnect await player.disconnect();
`
API Reference
$3 Main entry class that manages RTC connection and avatar rendering.
#### Constructor
`
typescript new AvatarPlayer(provider: RTCProvider, avatarView: AvatarView, options?: AvatarPlayerOptions)`
Parameters:
| Parameter | Type | Description | |-----------|------|-------------| |
provider
| LiveKitProvider \| AgoraProvider
| RTC provider instance | | avatarView
| AvatarView
| AvatarView instance from avatarkit | | options
| AvatarPlayerOptions
| Optional configuration |#### AvatarPlayerOptions
`
typescript interface AvatarPlayerOptions { /* Start speaking transition frames, default 5 (~200ms at 25fps) / transitionStartFrameCount?: number; /* End speaking transition frames, default 40 (~1600ms at 25fps) / transitionEndFrameCount?: number; /* Log level: 'info' | 'warning' | 'error' | 'none', default 'warning' / logLevel?: LogLevel; }`
#### Properties
| Property | Type | Description | |----------|------|-------------| |
isConnected
| boolean
| Whether connected to RTC server |#### Methods
##### connect(config)
Connect to RTC server.
`
typescript await player.connect(config: RTCConnectionConfig): Promise`
LiveKit Config:
`
typescript interface LiveKitConnectionConfig { url: string; // LiveKit server URL (wss://...) token: string; // Auth token roomName: string; // Room name }`
Agora Config:
`
typescript interface AgoraConnectionConfig { appId: string; // Agora App ID channel: string; // Channel name token?: string; // Auth token (required in production) uid?: number; // User ID (optional, 0 = auto assign) }`
##### disconnect()
Disconnect RTC connection and clean up all resources.
`
typescript await player.disconnect(): Promise`
##### reconnect()
Reconnect to RTC server using the last connection configuration. Useful for recovering from connection issues or stream stalls.
`
typescript await player.reconnect(): Promise`
Note: Throws an error if no previous connection exists. Call
connect()
first.##### startPublishing()
Start publishing microphone audio. Automatically requests microphone permission.
`
typescript await player.startPublishing(): Promise`
##### stopPublishing()
Stop publishing microphone audio and release the device.
`
typescript await player.stopPublishing(): Promise`
##### publishAudio(track)
Publish custom audio track (advanced usage). For non-microphone audio sources like audio file playback.
`
typescript await player.publishAudio(track: MediaStreamTrack): Promise`
Supported audio sources:
| Source | How to obtain | |--------|---------------| | š¤ Microphone | Use
startPublishing()
for convenience | | šµ element | audioElement.captureStream() | | š„ļø Screen share audio | getDisplayMedia({ audio: true }) | | š¹ Web Audio API | audioContext.createMediaStreamDestination().stream |Example:
`typescript // Play audio from browser element const audioEl = document.querySelector('audio'); const stream = audioEl.captureStream(); await player.publishAudio(stream.getAudioTracks()[0]); `##### unpublishAudio()
Stop publishing custom audio track.
`typescript await player.unpublishAudio(): Promise`Note:
unpublishAudio() does not stop the track - caller is responsible for managing track lifecycle:
`typescript await player.unpublishAudio(); stream.getTracks().forEach(t => t.stop()); // Clean up externally`##### getConnectionState()
Get current connection state.
`typescript player.getConnectionState(): ConnectionState`ConnectionState enum:
| Value | Description | |-------|-------------| |
'disconnected' | Not connected | | 'connecting' | Connecting | | 'connected' | Connected | | 'reconnecting' | Reconnecting | | 'failed' | Connection failed |##### getNativeClient()
Get underlying RTC client object for platform-specific features.
`typescript player.getNativeClient(): unknown`Return value:
| Provider | Return type | |----------|-------------| | LiveKit |
Room instance (exported as LiveKitRoom) | | Agora | IAgoraRTCClient instance (exported as AgoraClient) |Type safety notes:
- Direct Provider usage : Returns concrete type, no assertion needed - Via AvatarPlayer : Returns
unknown, requires manual type assertionExample:
`typescript // Method 1: Use Provider directly (recommended, full type hints) import { LiveKitProvider, LiveKitRoom } from '@spatialwalk/avatarkit-rtc';const provider = new LiveKitProvider(); const room = provider.getNativeClient(); // Type: LiveKitRoom | null console.log('Remote participants:', room?.remoteParticipants.size);
// Method 2: Via AvatarPlayer (requires assertion) import { AvatarPlayer, LiveKitRoom } from '@spatialwalk/avatarkit-rtc';
const room = player.getNativeClient() as LiveKitRoom | null; console.log('Remote participants:', room?.remoteParticipants.size);
// Agora example import { AgoraProvider, AgoraClient } from '@spatialwalk/avatarkit-rtc';
const provider = new AgoraProvider(); const client = provider.getNativeClient(); // Type: AgoraClient | null console.log('Connection state:', client?.connectionState);
`##### on(event, handler)
Listen to events.
`typescript player.on(event: string, handler: Function): void`Supported events:
| Event | Callback params | Description | |-------|-----------------|-------------| |
'connected' | () | Connection successful | | 'disconnected' | () | Disconnected | | 'error' | (error: Error) | Error occurred | | 'connection-state-changed' | (state: ConnectionState) | Connection state changed | | 'stalled' | () | Data stream stalled (no frames for 3s). Avatar auto-transitions to idle. |Example: Handling stream stalls
`typescript player.on('stalled', async () => { console.log('Stream stalled, attempting reconnection...'); try { await player.reconnect(); } catch (error) { console.error('Reconnection failed:', error); } });`##### off(event, handler)
Remove event listener.
`typescript player.off(event: string, handler: Function): void`---
$3 LiveKit RTC provider implementation.
`typescript import { LiveKitProvider } from '@spatialwalk/avatarkit-rtc';const provider = new LiveKitProvider();
`---
$3 Agora RTC provider implementation.
`typescript import { AgoraProvider } from '@spatialwalk/avatarkit-rtc';const provider = new AgoraProvider({ debugLogging: true, // optional: enable debug logs });
`AgoraProviderOptions:
`typescript interface AgoraProviderOptions { /* Enable verbose debug logging, default false / debugLogging?: boolean; }`---
$3 Type guard functions for config type checking.
`typescript import { isLiveKitConfig, isAgoraConfig } from '@spatialwalk/avatarkit-rtc';if (isLiveKitConfig(config)) { // config is LiveKitConnectionConfig }
if (isAgoraConfig(config)) { // config is AgoraConnectionConfig }
`---
$3 SDK exports type aliases for underlying RTC clients, useful when using
getNativeClient():
`typescript import type { LiveKitRoom, AgoraClient } from '@spatialwalk/avatarkit-rtc';// LiveKitRoom = Room type from livekit-client // AgoraClient = IAgoraRTCClient type from agora-rtc-sdk-ng
`---
Log Levels SDK log level is configured via
AvatarPlayerOptions.logLevel:| Level | Output | |-------|--------| |
'info' | All logs (connection state, frame processing, debug info) | | 'warning' | Warnings + errors (default ) | | 'error' | Errors only | | 'none' | Disable all logs |
`typescript const player = new AvatarPlayer(provider, avatarView, { logLevel: 'info', // Enable for debugging });`---
Complete Example
$3
`typescript import { AvatarPlayer, LiveKitProvider } from '@spatialwalk/avatarkit-rtc'; import { AvatarSDK, AvatarView, AvatarManager, DrivingServiceMode, Environment } from '@spatialwalk/avatarkit';async function initLiveKit() { // Initialize SDK (use host mode) await AvatarSDK.initialize('your-app-id', { environment: Environment.cn, drivingServiceMode: DrivingServiceMode.host, });
// Set session token (required) AvatarSDK.setSessionToken('your-session-token');
// Create Avatar const avatar = await AvatarManager.shared.load('character-id'); const container = document.getElementById('avatar-container')!; const avatarView = new AvatarView(avatar, container);
// Create Player const provider = new LiveKitProvider(); const player = new AvatarPlayer(provider, avatarView, { logLevel: 'info', transitionStartFrameCount: 5, transitionEndFrameCount: 40, });
// Listen to events player.on('connected', () => console.log('Connected!')); player.on('disconnected', () => console.log('Disconnected!')); player.on('error', (err) => console.error('Error:', err));
// Connect await player.connect({ url: 'wss://your-livekit-server.com', token: 'your-token', roomName: 'my-room', });
// Start microphone await player.startPublishing();
// Stop microphone // await player.stopPublishing();
// Disconnect // await player.disconnect(); }
`
$3
`typescript import { AvatarPlayer, AgoraProvider } from '@spatialwalk/avatarkit-rtc'; import { AvatarSDK, AvatarView, AvatarManager, DrivingServiceMode, Environment } from '@spatialwalk/avatarkit';async function initAgora() { // Initialize SDK (use host mode) await AvatarSDK.initialize('your-app-id', { environment: Environment.cn, drivingServiceMode: DrivingServiceMode.host, });
// Set session token (required) AvatarSDK.setSessionToken('your-session-token');
// Create Avatar const avatar = await AvatarManager.shared.load('character-id'); const container = document.getElementById('avatar-container')!; const avatarView = new AvatarView(avatar, container);
// Create Player const provider = new AgoraProvider({ debugLogging: true }); const player = new AvatarPlayer(provider, avatarView);
// Connect await player.connect({ appId: 'your-agora-app-id', channel: 'my-channel', token: 'your-token', // Required in production uid: 0, // 0 = auto assign });
// Start microphone await player.startPublishing(); }
`---
Browser Compatibility | Feature | Chrome | Firefox | Safari | Edge | |---------|--------|---------|--------|------| | LiveKit (VP8 + RTCRtpScriptTransform) | ā
94+ | ā
117+ | ā
15.4+ | ā
94+ | | Agora (H.264 + SEI) | ā
74+ | ā
78+ | ā
14.1+ | ā
79+ |
Note: LiveKit requires browser support for
RTCRtpScriptTransform` API (Safari 15.4+ supported).---
License MIT