Real-Time Voice Inference React SDK

!NPM Version

Install

``bash
yarn add realtime-ai realtime-ai-react


or


npm install realtime-ai realtime-ai-react
`

Quick Start

Instantiate an RTVIClient instance and pass it down to the RTVIClientProvider. Render the component to have audio output setup automatically.

`tsx
import { RTVIClient } from "realtime-ai";
import { RTVIClientAudio, RTVIClientProvider } from "realtime-ai-react";

const client = new RTVIClient({
baseUrl: "https://rtvi.pipecat.bot",
enableMic: true,
});

render(




);
`

We recommend starting the voiceClient from a click of a button, so here's a minimal implementation of to get started:

`tsx
import { useRTVIClient } from "realtime-ai-react";

const MyApp = () => {
const client = useRTVIClient();
return client.start()}>OK Computer;
};
`

Components

$3

The root component for providing RTVI client context to your application.

#### Props

- client (RTVIClient, required): A singleton instance of RTVIClient

`jsx

{/ Child components /}

`

$3

Creates a new

#### Props

No props

`jsx

`

$3

Creates a new

#### Props

- participant ("local" | "bot"): Defines which participant's video track is rendered
- fit ("contain" | "cover", optional): Defines whether the video should be fully contained or cover the box. Default: 'contain'.
- mirror (boolean, optional): Forces the video to be mirrored, if set.
- onResize(dimensions: object) (function, optional): Triggered whenever the video's rendered width or height changes. Returns the video's native width, height and aspectRatio.

`jsx
participant="local"
fit="cover"
mirror
onResize={({ aspectRatio, height, width }) => {
console.log("Video dimensions changed:", { aspectRatio, height, width });
}}
/>
`

$3

Renders a visual representation of audio input levels on a element.
The visualization consists of five vertical bars.

#### Props

- participantType (string, required): The participant type to visualize audio for.
- backgroundColor (string, optional): The background color of the canvas. Default: 'transparent'.
- barColor (string, optional): The color of the audio level bars. Default: 'black'.
- barGap (number, optional): The gap between bars in pixels. Default: 12.
- barWidth (number, optional): The width of each bar in pixels. Default: 30.
- barMaxHeight (number, optional): The maximum height at full volume of each bar in pixels. Default: 120.

`jsx
participantType="local"
backgroundColor="white"
barColor="black"
barGap={1}
barWidth={4}
barMaxHeight={24}
/>
`

Hooks

$3

Provides access to the RTVIClient instance originally passed to RTVIClientProvider.

`jsx
import { useRTVIClient } from "realtime-ai-react";

function MyComponent() {
const rtviClient = useRTVIClient();
}
`

$3

Allows subscribing to RTVI client events.
It is advised to wrap handlers with useCallback.

#### Arguments

- event (RTVIEvent, required)
- handler (function, required)

`jsx
import { useCallback } from "react";
import { RTVIEvent, TransportState } from "realtime-ai";
import { useRTVIClientEvent } from "realtime-ai-react";

function EventListener() {
useRTVIClientEvent(
RTVIEvent.TransportStateChanged,
useCallback((transportState: TransportState) => {
console.log("Transport state changed to", transportState);
}, [])
);
}
`

$3

Manage and list available media devices.

`jsx
import { useRTVIClientMediaDevices } from "realtime-ai-react";

function DeviceSelector() {
const {
availableCams,
availableMics,
selectedCam,
selectedMic,
updateCam,
updateMic,
} = useRTVIClientMediaDevices();

return (
<>
name="cam"
onChange={(ev) => updateCam(ev.target.value)}
value={selectedCam?.deviceId}
>
{availableCams.map((cam) => (

{cam.label}

))}

name="mic"
onChange={(ev) => updateMic(ev.target.value)}
value={selectedMic?.deviceId}
>
{availableMics.map((mic) => (

{mic.label}

))}


);
}
`

$3

Access audio and video tracks.

#### Arguments

- trackType ("audio" | "video", required)
- participantType ("bot" | "local", required)

`jsx
import { useRTVIClientMediaTrack } from "realtime-ai-react";

function MyTracks() {
const localAudioTrack = useRTVIClientMediaTrack("audio", "local");
const botAudioTrack = useRTVIClientMediaTrack("audio", "bot");
}
`

$3

Returns the current transport state.

`jsx
import { useRTVIClientTransportState } from "realtime-ai-react";

function ConnectionStatus() {
const transportState = useRTVIClientTransportState();
}
``

Contributing

We are welcoming contributions to this project in form of issues and pull request. For questions about RTVI head over to the Pipecat discord server and check the #rtvi channel.