@cloudflare/ai-chat

$3

!npm install @cloudflare/ai-chat

Welcome to @cloudflare/ai-chat, a specialized package for building intelligent chat experiences with persistent, stateful AI agents. Built on top of the agents framework and integrated with the AI SDK, this package provides everything you need to create conversational AI applications that remember, learn, and evolve.

#### The Nature of AI Chat

AI chat transcends simple request-response patterns. It's a continuous dialogue where:

- Persistence: Conversations are automatically saved and restored across sessions
- Streaming: Real-time responses that flow naturally to users
- Resumability: Automatic recovery from disconnections without losing context
- Tools: Seamless integration of client and server-side capabilities
- State: Each conversation maintains its own memory and understanding

Built on Cloudflare's global network with Durable Objects, your chat agents persist across time, maintaining context and relationships with users.

#### 💫 Core Principles

1. Automatic Persistence: Messages are saved automatically - no manual storage needed
2. Resumable Streaming: Disconnections don't break the flow - streams resume automatically
3. Tool Harmony: Mix server and client-side tools seamlessly
4. AI SDK Integration: Built on the latest AI SDK v6 for maximum compatibility
5. React First: Designed for modern React applications with hooks

---

$3

Install the package in your project:

``sh
npm install @cloudflare/ai-chat agents ai
`

You'll also need the agents package (for the base Agent class) and ai (for AI SDK integration).

$3

Create a chat agent that remembers and responds:

`ts
import { AIChatAgent } from "@cloudflare/ai-chat";
import { openai } from "@ai-sdk/openai";
import {
streamText,
convertToModelMessages,
createUIMessageStream,
createUIMessageStreamResponse
} from "ai";

export class ChatAgent extends AIChatAgent {
async onChatMessage() {
const stream = createUIMessageStream({
execute: async ({ writer }) => {
const result = streamText({
model: openai("gpt-4o"),
messages: await convertToModelMessages(this.messages)
});

writer.merge(result.toUIMessageStream());
}
});

return createUIMessageStreamResponse({ stream });
}
}
`

That's it! Your agent now:

- Automatically persists all messages
- Streams responses in real-time
- Handles reconnections gracefully
- Maintains conversation context

$3

Define your chat agent in your wrangler.toml:

`jsonc
{
"durable_objects": {
"bindings": [
{
"name": "ChatAgent",
"class_name": "ChatAgent"
}
]
},
"migrations": [
{
"tag": "v1",
"new_sqlite_classes": ["ChatAgent"]
}
]
}
`

$3

#### Streaming Responses (Recommended)

For real-time interaction, use streaming:

`ts
import { AIChatAgent } from "@cloudflare/ai-chat";
import { openai } from "@ai-sdk/openai";
import {
streamText,
convertToModelMessages,
createUIMessageStream,
createUIMessageStreamResponse
} from "ai";

export class StreamingChat extends AIChatAgent {
async onChatMessage(onFinish) {
const stream = createUIMessageStream({
execute: async ({ writer }) => {
const result = streamText({
model: openai("gpt-4o"),
messages: await convertToModelMessages(this.messages),
onFinish
});

writer.merge(result.toUIMessageStream());
}
});

return createUIMessageStreamResponse({ stream });
}
}
`

#### Non-Streaming Responses

For simpler use cases without real-time updates:

`ts
import { AIChatAgent } from "@cloudflare/ai-chat";
import { openai } from "@ai-sdk/openai";
import { generateText, convertToModelMessages } from "ai";

export class SimpleChat extends AIChatAgent {
async onChatMessage() {
const result = await generateText({
model: openai("gpt-4o"),
messages: await convertToModelMessages(this.messages)
});

const message = result.toUIMessage({
metadata: {
model: "gpt-4o",
totalTokens: result.usage?.totalTokens
}
});

return new Response(JSON.stringify(message), {
headers: { "Content-Type": "application/json" }
});
}
}
`

$3

Connect your React application to your chat agent:

`tsx
import { useAgent } from "agents/react";
import { useAgentChat } from "@cloudflare/ai-chat/react";

function ChatInterface() {
// Connect to the agent
const agent = useAgent({
agent: "ChatAgent",
name: "my-chat"
});

// Set up the chat interaction
const { messages, sendMessage, clearHistory, status } = useAgentChat({
agent
});

const [input, setInput] = useState("");

const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault();
if (!input.trim()) return;

await sendMessage({
role: "user",
parts: [{ type: "text", text: input }]
});
setInput("");
};

return (


One of the most powerful features of @cloudflare/ai-chat is automatic resumable streaming. When a client disconnects during a stream, the response automatically resumes when they reconnect.

#### How It Works

1. During streaming: All chunks are automatically persisted to SQLite
2. On disconnect: The stream continues server-side, buffering chunks
3. On reconnect: Client receives all buffered chunks and continues streaming

It just works! No special configuration needed.

#### Example

`tsx
import { useAgent } from "agents/react";
import { useAgentChat } from "@cloudflare/ai-chat/react";

function ResumableChat() {
const agent = useAgent({
agent: "ResumableStreamingChat",
name: "demo"
});

// resume: true is the default - streams automatically resume on reconnect
const { messages, sendMessage, status } = useAgentChat({
agent
// resume: true is the default
});

// Try it: Start a long response, refresh the page, and watch it resume!
return (


@cloudflare/ai-chat supports both server-side and client-side tools, allowing you to build rich interactive experiences.

#### Server-Side Tools

Define tools that execute on the server:

`ts
import { AIChatAgent } from "@cloudflare/ai-chat";
import { openai } from "@ai-sdk/openai";
import { streamText, convertToModelMessages, tool } from "ai";
import { z } from "zod";
import { createUIMessageStream, createUIMessageStreamResponse } from "ai";

export class ToolChat extends AIChatAgent {
async onChatMessage() {
const stream = createUIMessageStream({
execute: async ({ writer }) => {
const result = streamText({
model: openai("gpt-4o"),
messages: await convertToModelMessages(this.messages),
tools: {
getWeather: tool({
description: "Get weather for a city",
parameters: z.object({ city: z.string() }),
execute: async ({ city }) => {
// Fetch weather from your API
const weather = await fetch(https://api.weather.com/${city});
return { temperature: 72, condition: "sunny" };
}
})
}
});

writer.merge(result.toUIMessageStream());
}
});

return createUIMessageStreamResponse({ stream });
}
}
`

#### Client-Side Tools

For tools that need to run in the browser (like DOM manipulation, user interactions):

`tsx
import { useAgent } from "agents/react";
import { useAgentChat } from "@cloudflare/ai-chat/react";
import { tool } from "ai";
import { z } from "zod";

function ChatWithClientTools() {
const agent = useAgent({ agent: "ToolChat" });

const { messages, sendMessage, addToolResult } = useAgentChat({
agent,
onToolCall: async ({ toolCall, addToolOutput }) => {
if (toolCall.toolName === "showAlert") {
alert(toolCall.input.message);
addToolOutput({
toolCallId: toolCall.toolCallId,
output: { success: true }
});
}
}
});

// Server defines the tool without execute:
// showAlert: tool({
// description: "Shows an alert to the user",
// parameters: z.object({ message: z.string() })
// // No execute = client handles it
// })
}
`

#### Human-in-the-Loop Tools

For tools requiring user approval before execution:

`tsx
import { useAgentChat } from "@cloudflare/ai-chat/react";

function HumanInTheLoopChat() {
const agent = useAgent({ agent: "ApprovalChat" });

const { messages, sendMessage, addToolResult } = useAgentChat({
agent,
toolsRequiringConfirmation: ["sendEmail", "deleteFile"],
onToolCall: async ({ toolCall, addToolOutput }) => {
if (toolCall.toolName === "sendEmail") {
const approved = confirm(Send email to ${toolCall.input.recipient}?);
if (approved) {
// Execute the tool
const result = await sendEmail(toolCall.input);
addToolOutput({
toolCallId: toolCall.toolCallId,
output: result,
autoContinue: true // Automatically continue after tool result
});
}
}
}
});
}
`

$3

Attach custom metadata to messages for tracking, analytics, or display purposes:

`ts
import { AIChatAgent } from "@cloudflare/ai-chat";
import { openai } from "@ai-sdk/openai";
import {
streamText,
convertToModelMessages,
createUIMessageStream,
createUIMessageStreamResponse
} from "ai";

export class MetadataChat extends AIChatAgent {
async onChatMessage() {
const startTime = Date.now();

const stream = createUIMessageStream({
execute: async ({ writer }) => {
const result = streamText({
model: openai("gpt-4o"),
messages: await convertToModelMessages(this.messages)
});

writer.merge(
result.toUIMessageStream({
messageMetadata: ({ part }) => {
if (part.type === "start") {
return {
model: "gpt-4o",
createdAt: Date.now(),
messageCount: this.messages.length
};
}
if (part.type === "finish") {
return {
responseTime: Date.now() - startTime,
totalTokens: part.totalUsage?.totalTokens
};
}
}
})
);
}
});

return createUIMessageStreamResponse({ stream });
}
}
`

Access metadata on the client:

`tsx
{
messages.map((message) => (


`tsx
const { messages, sendMessage } = useAgentChat({
agent,
prepareSendMessagesRequest: ({ id, messages }) => ({
body: {
currentUrl: window.location.href,
userTimezone: Intl.DateTimeFormat().resolvedOptions().timeZone
},
headers: {
"X-Widget-Version": "1.0.0",
"X-Request-ID": crypto.randomUUID()
}
})
});
`

#### Automatic Tool Continuation

Enable automatic continuation after tool results for seamless UX:

`tsx
const { messages, addToolResult } = useAgentChat({
agent,
autoContinueAfterToolResult: true, // Automatically continue after tool execution
onToolCall: async ({ toolCall, addToolOutput }) => {
const result = await executeTool(toolCall);
addToolOutput({
toolCallId: toolCall.toolCallId,
output: result,
autoContinue: true // Server will automatically continue the conversation
});
}
});
`

$3

Explore these examples to see @cloudflare/ai-chat in action:

- Basic Chat: examples/playground/src/agents/chat.ts - Simple streaming chat
- Resumable Streaming: examples/resumable-stream-chat/ - Automatic stream resumption
- Human-in-the-Loop: guides/human-in-the-loop/ - Tools requiring user approval

$3

#### AIChatAgent

Base class for chat agents. Extends Agent from the agents package.

Methods:

- onChatMessage(onFinish, options?): Override this method to handle chat messages and return a Response
- persistMessages(messages, excludeBroadcastIds?): Manually persist messages (usually automatic)
- messages: Array of current chat messages

Properties:

- messages: ChatMessage[]: The current conversation messages

#### useAgentChat(options)

React hook for chat interactions.

Options:

- agent: Agent connection from useAgent()
- onToolCall: Callback for handling client-side tool execution
- toolsRequiringConfirmation: Array of tool names that need user approval
- autoContinueAfterToolResult: Automatically continue conversation after tool results
- resume: Enable automatic stream resumption (default: true)
- prepareSendMessagesRequest: Customize request headers/body

Returns:

- messages: Array of chat messages
- sendMessage: Function to send a new message
- clearHistory: Function to clear conversation history
- addToolResult: Function to provide tool output
- status: Current status ("idle" | "streaming" | "error")

$3

@cloudflare/ai-chat is built on top of the agents framework. It extends the base Agent class with chat-specific capabilities:

- Automatic message persistence in SQLite
- WebSocket-based real-time communication
- Resumable streaming infrastructure
- Tool execution coordination

For more information about the underlying agent framework, see the agents README.

$3

We're continuously improving @cloudflare/ai-chat`:

- Enhanced Tool Support: More flexible tool execution patterns
- Better Error Handling: Graceful degradation and recovery
- Performance Optimizations: Faster streaming and lower latency
- Developer Experience: Better TypeScript types and debugging tools

$3

Contributions are welcome! Please:

1. Open an issue to discuss your proposal
2. Ensure your changes align with the package's goals
3. Include tests for new features
4. Update documentation as needed

$3

MIT licensed. See the LICENSE file at the root of this repository for details.