ai-agent-react

Composable React/Vite component library that ships the Multiplayer AI Agents UI primitives – chat, model/context controls, tool discovery, reasoning trace, artifacts, and background-agent management hooks. The package is designed to drop into any React (or Vite) web-app and talk either to Multiplayer's proxy backend or directly to an LLM provider such as OpenRouter/OpenAI.

Highlights

- Dual-transport runtime – switch between proxy (Multiplayer backend) and direct (OpenRouter/OpenAI/custom) without touching UI code.
- Context-aware UI – bind chats to context keys that automatically flip tool availability, default models, and confirmation policies per page/feature.
- Reasoning + artifacts – dedicated sidebar surfaces thinking traces, background plans, and generated artifacts.
- Tool registry – declare tools once, reuse across contexts, and wire custom launchers or approval flows.
- Theme tokens – override a small set of CSS variables or wrap components with your own layout.
- React Query + Zustand – predictable caching/state that can be extended by host apps.

Installation

The package is managed inside the monorepo workspace. From the repo root:

``bash
npm install
npm run dev -w ai-agent-react # optional preview sandbox
npm run build -w ai-agent-react
`

Quick start

`tsx
import { AgentProvider, AgentChatWindow } from '@multiplayer-app/ai-agent-react'

const config = {
appId: 'your-app-id',
workspaceId: 'workspace-123',
contextKeys: [
{ key: 'global', label: 'Global', tools: ['kb-search', 'jira'] },
{
key: 'filing',
label: 'Filing Prep',
tools: ['kb-search', 'pdf-draft'],
defaultModel: 'gpt-4o'
}
],
models: [
{ id: 'gpt-4o', label: 'GPT-4o', reasoning: 'concise' },
{ id: 'sonnet', label: 'Claude Sonnet', reasoning: 'deep' }
],
tools: [
{ name: 'kb-search', label: 'Knowledge Base' },
{ name: 'jira', label: 'Jira' }
],
transport: {
mode: 'proxy',
baseUrl: '/api' // never ship raw API keys to the browser
},
features: {
reasoning: 'panel',
artifactsPanel: true,
modelSwitching: true
}
}

export const YourAppAgent = () => (



)
`

Context attachments (page/form context in-chat)

The UI supports sending attachments with user messages. One special attachment type is:

- AgentAttachmentType.Context: structured “small context” payloads (page snippet, form fields, etc.)

$3

Context attachments can contain sensitive data (emails, names, phone numbers, access tokens, etc.). If you include the optional metadata.security block:

- containsPII: true means “this attachment may contain PII/sensitive data.” This is a signal for your logging/storage/prompt policies; it does not automatically redact or protect anything.
- redactionsApplied: string[] is an audit trail of what you already scrubbed before attaching it, e.g. ['email', 'phone', 'access_token'].

If you set containsPII: true with redactionsApplied: [], you are explicitly saying “may contain sensitive data and we did not redact it.”

$3

- UI: renders attachment chips under messages.
- Composer: includes a built-in “Attach selection” button that captures the current text selection as a webSnippet context attachment (when running in the browser).
- Composer (page context): auto-attaches a pageContext chip (URL + metadata) by default. Users can remove it; if removed, the composer shows an “Attach page context” button.
- Runtime: forwards attachments in SendMessagePayload to your transport (proxy or direct).

$3

If you want to inject host-specific metadata into the built-in pageContext attachment (e.g. tenant id, current record id, feature flags), pass:

`ts
const config = {
// ...
features: { pageContextAttachments: true },
pageContext: {
name: 'Page',
getData: ({ url, route, title }) => ({
url,
route,
title,
tenantId: window.__TENANT_ID__
})
}
}
`

The return value is merged into attachment.metadata.data. Keep it fast and avoid heavy DOM scraping.

$3

Sometimes you don’t have access to the config module where getData is defined. Use the runtime:

`ts
import { useAgentRuntime } from '@multiplayer-app/ai-agent-react'

const runtime = useAgentRuntime()
const data = runtime.getPageContextData() // calls config.pageContext.getData(...)
const attachment = runtime.buildPageContextAttachment()
`

$3

Most “page context” is naturally owned by the page/route (record id, active tab, filters). Don’t force that into global config.
Instead, register it from the page:

`ts
import { usePageContext } from '@multiplayer-app/ai-agent-react'
import { useParams } from 'react-router-dom'
import { useCallback } from 'react'

export function CustomerPage() {
const { customerId } = useParams()
const { customerName } = useCustomer(customerId)

usePageContext(
() => ({
customerId,
customerName
}),
[customerId, customerName]
)

// ...
}
`

This data is merged into the built-in pageContext attachment’s metadata.data whenever it is attached/refreshed.

$3

You generally don’t call transports directly. You attach context by:

1. Building AgentAttachment[]
2. Passing them through the composer send flow (or your own UI that calls runtime send with attachments)

The underlying contract is SendMessagePayload.attachments.

#### Public API: mutate the composer draft from anywhere

The library exposes a small hook that lets you push attachments into the current composer draft (active chat or draft chat):

`ts
import { useComposerDraft } from '@multiplayer-app/ai-agent-react'

const { addAttachments, removeAttachment, clearDraft, draft } = useComposerDraft()
`

This is the intended way for host apps to attach page/form context from “different places” in the UI.

Runtime events (host integrations)

AgentRuntime exposes a small per-runtime event bus at runtime.events for host integrations (e.g. “apply these form values”).

$3

Use this to subscribe to runtime.events with proper cleanup and without re-subscribing every render:

`ts
import { useRuntimeEventListener } from '@multiplayer-app/ai-agent-react'
import { FormApplyRequestedEventType, type FormApplyRequestedEvent } from '@multiplayer-app/ai-agent-react'

useRuntimeEventListener(FormApplyRequestedEventType, (e) => {
console.log('apply requested', e.detail)
})
`

$3

The built-in propose_form_values renderer emits a FormApplyRequestedEventType event when the user clicks Apply/Reject.
In a host app, you should listen for that and mutate your local form state:

`ts
import { useProposeFormValues } from '@multiplayer-app/ai-agent-react'

useProposeFormValues({
formId: 'customerOnboarding',
onApply: ({ fields }) => {
// setState(...) based on fields
}
})
`

#### Helper component: AttachContextButton

For convenience, the library also exports an unstyled helper button:

`tsx
import { AttachContextButton } from '@multiplayer-app/ai-agent-react'
; // Prefer context so boilerplate is filled under the hood.
context={() => ({
kind: 'crmRecord',
name: 'CRM context',
summary: 'Renewal in 14 days. Billing escalation open.',
data: { accountId: 'acc_123' }
})}
>
Attach CRM context

`

If you need full control, you can also provide a complete attachment object:

`tsx
import { AttachContextButton } from '@multiplayer-app/ai-agent-react'
import { nanoid } from 'nanoid'
import { AgentAttachmentType } from '@multiplayer-app/ai-agent-types'
; attachment={() => ({
id: nanoid(),
type: AgentAttachmentType.Context,
name: 'CRM context (manual)',
metadata: {
schemaVersion: 1,
kind: 'crmRecord',
capturedAt: new Date().toISOString(),
summary: 'Renewal in 14 days. Billing escalation open.',
data: { accountId: 'acc_123' }
}
})}
>
Attach CRM context (manual)

`

#### Helper: createContextAttachment (fills boilerplate under the hood)

If you don’t want to manually set id, type, metadata.schemaVersion, metadata.capturedAt, and source.url, use:

`ts
import { createContextAttachment } from '@multiplayer-app/ai-agent-react'

const att = createContextAttachment({
kind: 'webSnippet',
url: window.location.href,
selectedText: '…'
})
`

This produces a valid AgentAttachment with defaults filled in.

#### Example: create a custom context attachment

`ts
import { nanoid } from 'nanoid'
import { AgentAttachmentType, SendMessagePayloadSchema } from '@multiplayer-app/ai-agent-types'

const attachments = [
{
id: nanoid(),
type: AgentAttachmentType.Context,
name: 'CRM context',
metadata: {
schemaVersion: 1,
kind: 'crmRecord', // custom kind
capturedAt: new Date().toISOString(),
title: 'Account: Acme Inc.',
summary: 'Renewal in 14 days. Open billing escalation.',
data: { accountId: 'acc_123', health: 'yellow' }
}
}
]

// Optional: validate locally before sending.
SendMessagePayloadSchema.parse({
content: 'Draft a renewal follow-up email.',
contextKey: 'support',
attachments
})
`

#### Example: create a built-in webSnippet attachment

`ts
import { nanoid } from 'nanoid'
import { AgentAttachmentType } from '@multiplayer-app/ai-agent-types'

const selectedText = window.getSelection?.()?.toString()?.trim() ?? ''
const url = window.location.href

const attachments = selectedText
? [
{
id: nanoid(),
type: AgentAttachmentType.Context,
name: 'Selection',
url,
metadata: {
schemaVersion: 1,
kind: 'webSnippet',
capturedAt: new Date().toISOString(),
selectedText,
source: { app: 'browser', url }
}
}
]
: []
`

$3

`ts
// app/api/agents/route.ts
import { NextResponse } from 'next/server'

const AGENT_API = 'https://agents.api.multiplayer.app'
const AGENT_KEY = process.env.AGENTS_API_KEY

export async function POST(req: Request) {
if (!AGENT_KEY) {
return NextResponse.json({ error: 'Missing AGENTS_API_KEY' }, { status: 500 })
}

const upstream = await fetch(${AGENT_API}/proxy, {
method: 'POST',
headers: {
'content-type': 'application/json',
authorization: Bearer ${AGENT_KEY}
},
body: await req.text()
})

return NextResponse.json(await upstream.json(), { status: upstream.status })
}
`

Host apps should hit /api/agents from the browser and keep provider credentials on the server (or issue short-lived tokens) so the public bundle never contains secrets.

Agent runtime loop

The UI is a thin layer over the runtime primitives inside src/runtime. Every send follows the same deterministic loop:

1. Payload assembly (AgentComposer.tsx) – builds a SendMessagePayload with the active contextKey, composer overrides, and the existing chatId. Pending assistant state is queued in AgentStore so the transcript can optimistically render “Thinking…” bubbles.
2. Transport execution (ProxyTransport or DirectTransport) – the runtime picks the correct transport based on config.transport.mode. ProxyTransport streams Server-Sent Events and enforces timeouts/headers; DirectTransport fabricates an in-memory chat, forwards OpenAI/OpenRouter style payloads, and stitches streaming deltas into a pending assistant message.
3. Store reconciliation (useAgentStore) – incoming chunks get merged via consumeStreamChunk, reasoning traces are appended per chat, and once the transport resolves the final AgentChat, upsertChat hydrates the full transcript + artifacts.
4. Tool call hydration (ToolCallCard.tsx) – any AgentMessage.toolCalls entries are rendered immediately. If you registered a renderer via AgentRuntime.registerToolRenderer, the UI swaps the JSON viewer for your custom card.

The runtime never mutates chats outside of these store actions, so you can safely swap in your own components as long as they call the same selectors/actions.

$3

`
User input
│
▼
AgentComposer ──┐
│ payload (SendMessagePayload)
▼
AgentRuntime
│ picks transport
│
│──────────► ProxyTransport → Multiplayer backend (SSE/event stream)
│
└──────────► DirectTransport → Provider API (OpenAI/OpenRouter/custom)
│
▼
AgentStore (Zustand)
│
├─ updates chats/reasoning/tool calls
└─ exposes selectors
│
▼
AgentMessageList / AgentToolShelf / AgentSidebar (render)
`

$3

`
User selects tool chip / runtime.tools.invoke(...)
│
▼
AgentRuntime.invokeTool
│ validates chat context
▼
ToolRegistry.execute
│ runs handler (client) or awaits backend status
▼
AgentStore.upsertToolCall
│
▼
ToolCallCard renderer
`

$3

1. Config validation – keep the config object colocated with a Zod schema (or reuse Multiplayer's built-in schema) so CI fails on breaking changes.
2. Transport secret management – never inline API keys; wire them through env vars or a server-side token exchange endpoint.
3. Context discipline – prefer <10 contextKeys. Each key should map to a clear product surface to avoid combinatorial explosion in tool policies.
4. Tool UX – every tool entry needs label, description, and ideally icon + confirmation. Missing metadata means degraded UX.

Configuration surface

| Field | Purpose |
| ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| appId | Tenant/application identifier issued by Multiplayer. Used for auth and metrics. |
| workspaceId | Optional per-user workspace scoping. Leave undefined if the backend infers it from the session. |
| debug | Enables extra debug UI in the React client (currently: tool-call details for tools without a registered renderer). |
| contextKeys | Array of { key, label, tools, defaultModel?, autoConfirmTools? } describing per-surface policies. Mirrors backend contextKey rules. |
| defaultContextKey | Forces the initial chat context; otherwise the first item in contextKeys is used. |
| tools | Optional registry with metadata (description, icon, category, confirmation, handler). Powers tool shelves and runtime overrides. |
| models | Optional catalog with per-model reasoning depth + allowed contexts. Drives the model switcher + default-model hints. |
| transport | { mode: 'proxy', baseUrl, headers?, timeoutMs? } to hit Multiplayer backend, or { mode: 'direct', provider, endpoint?, apiKey?, model? } for raw LLMs. |
| features | Fine-grained toggles: reasoning panel location, artifact visibility, multi-agent controls, sandbox switches, etc. |
| theme | Partial overrides for the default theme tokens (background, accent, font, radius, etc.). |
| toolRenderers | Map tool names to React components when tool output needs rich visuals (tables, charts, custom viewers). |
| user | Optional { id, displayName, email, avatarUrl } shape. If provided, user metadata is attached to tool calls/history. |

All configs are validated through zod at runtime, so invalid configurations fail fast.

Theming

`tsx
config={{
...config,
theme: { background: '#050b16', accent: '#2FE6FF', radius: 16 }
}}
>


`

Under the hood AgentThemeProvider exposes CSS variables (--mp-agents-*). If your product already has a design system, wrap AgentChatWindow and override the variables to match the host theme.

Tooling integration

$3

Tool metadata is strongly typed and validated in src/config/types.ts:

`ts
export interface AgentToolDefinition {
name: string
label: string
description?: string
icon?: string
schema?: Record
confirmation?: 'auto' | 'manual'
category?: string
handler?: AgentToolInvoke
}
`

`ts
export type AgentToolCall = {
id: string
name: string
input: Record
status: 'pending' | 'running' | 'succeeded' | 'failed'
output?: Record
error?: string
requiresConfirmation?: boolean
}
`

The UI never guesses fields—if handler is omitted, the tool is display-only; if the backend emits requiresConfirmation, the shelf can branch into your own approval UX.

$3

1. AgentToolShelf filters the runtime registry against the active contextKey -> tools list. Chips are pure metadata.
2. When the backend/LLM emits a tool call, the transport serializes it into AgentMessage.toolCalls. The store keeps those calls as-is (including status, error), which means you must propagate the tool status from the server.
3. ToolCallCard asks AgentRuntime for a renderer. If none exists, it falls back to a JSON viewer so the transcript always stays auditable.
4. If you register a handler via runtime.tools.register, the invocation happens purely on the client (synchronous or async) and the resulting AgentToolCall is returned to the caller. This is ideal for local helpers (opening drawers, querying browser APIs) but you should keep privileged work on the server.

$3

`ts
// tooling/registry.ts
import type { AgentToolDefinition } from '@multiplayer-app/ai-agent-react'

export const baseTools: AgentToolDefinition[] = [
{
name: 'kb-search',
label: 'Knowledge Base',
description: 'Semantic search across product docs',
icon: 'search',
confirmation: 'auto'
},
{
name: 'jira',
label: 'Jira Issues',
description: 'Open, triage, or update tickets',
icon: 'jira',
confirmation: 'manual'
}
]
`

Feed the array into config.tools. Context gating still happens via each ContextKeyConfig.tools array, so a tool never surfaces outside the contexts you list there.

To register ad-hoc client helpers, reach for runtime.registerTool (a thin wrapper over ToolRegistry):

`ts
runtime.registerTool({
name: 'copy-last-answer',
label: 'Copy answer to clipboard',
handler: async (_input, ctx) => {
const chat = runtime.store.getState().chats[ctx.chatId]
const lastAssistant = chat?.messages
.slice()
.reverse()
.find((m) => m.role === 'assistant')
if (lastAssistant) {
await navigator.clipboard.writeText(lastAssistant.content)
ctx.appendSystemMessage('Copied last assistant response to clipboard.')
}
}
})
`

$3

`ts
// pages/api/agents/tools/jira.ts
import type { NextApiRequest, NextApiResponse } from 'next'
import { jiraClient } from '@/lib/jira'

export default async function handler(req: NextApiRequest, res: NextApiResponse) {
if (req.method !== 'POST') return res.status(405).end()
const { input, contextKey, actor } = req.body

if (contextKey !== 'filing') {
return res.status(403).json({ error: 'Jira disabled for this context' })
}

try {
const issue = await jiraClient.search(input.query, { assignee: actor.id })
return res.json({
output: issue,
observations: [Returned ${issue.total} issues]
})
} catch (err) {
console.error(err)
return res.status(502).json({
error: 'Jira search failed',
observations: [err instanceof Error ? err.message : 'Unknown Jira error']
})
}
}
`

When using the Multiplayer proxy, mirror the same policies server-side via your /tools definitions so the UI never drifts from backend enforcement.

$3

`tsx
import { AgentProvider, AgentRuntime, AgentSidebar } from '@multiplayer-app/ai-agent-react'
import { baseTools } from './tooling/registry'

const runtime = new AgentRuntime({ ...config, tools: baseTools })
runtime.tools.register({
name: 'feature-flags',
label: 'Toggle FF',
description: 'Flip LaunchDarkly flags scoped to the current workspace',
onInvoke: async ({ input, context }) => {
try {
const resp = await fetch('/api/feature-flags/toggle', {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ ...input, context })
})

if (!resp.ok) {
throw new Error(Toggle failed with ${resp.status})
}

return resp.json()
} catch (error) {
context.appendSystemMessage(
Feature-flag tool failed: ${error instanceof Error ? error.message : 'Unknown error'}
)
throw error
}
}
})

export function AgentsPanel() {
return (



)
}
`

runtime.tools.register accepts inline handlers (onInvoke) for front-end only tools (e.g., opening a modal, reading local storage) or can proxy to your server routes.

$3

AgentToolShelf exposes the selected AgentToolDefinition before execution so you can gate, hydrate, or redirect calls:

`tsx
import { AgentToolShelf, useAgentRuntime } from '@multiplayer-app/ai-agent-react'

export const ToolShelfWithReview = () => {
const runtime = useAgentRuntime()

const handleToolSelected = async (tool) => {
if (tool.name === 'prod-deploy') {
const ok = window.confirm('Ship to production?')
if (!ok) return
}
runtime.tools.invoke(tool.name, { target: 'production' })
}

return
}
`

$3

The transcript renders every tool call as a dedicated card. Provide custom components when you want richer visuals (charts, tables, external viewers):

`tsx
import type { ToolRendererProps } from '@multiplayer-app/ai-agent-react';

const FilingChartRenderer = ({ call }: ToolRendererProps) => {
const rows = call.output?.rows as Array<{ label: string; value: number }>;
if (!rows) return null;
return (


const config = {
...,
toolRenderers: {
'filing-stats': FilingChartRenderer
}
};
`

At runtime you can register or override a renderer:

`ts
const runtime = new AgentRuntime(config)
runtime.registerToolRenderer('jira-search', JiraTableRenderer)
`

If no renderer is provided, the library falls back to a simple tool status panel.

#### Debug: tool-call input/output inspector (no renderer case)

When config.debug: true and a tool has no renderer, the tool card shows a collapsible Details panel with pretty-printed JSON for:

- input
- output
- status / error

$3

1. Build a Storybook story with mocked AgentRuntime handlers (msw highly recommended).
2. Record golden transcripts (JSON) and feed them into visual regression tests to lock UX.
3. Exercise the same handler via Playwright/E2E in the host app to guarantee auth + transport policies stay aligned.

$3

- Tool handlers must handle authorization before running side effects. Front-end context checks are advisory; the backend is the source of truth.
- Always wrap outbound API calls in try/catch and emit actionable observations so the agent transcript preserves failure context.
- Convert third-party errors to deterministic messages (res.status(502).json({ error: 'Jira search failed' })) to avoid leaking stack traces.
- From the runtime, bubble surfaced errors to the UI via appendSystemMessage or runtime.store.setError(...) so operators know a tool degraded.
- Return HTTP 4xx for policy violations and 5xx for execution failures; the Multiplayer proxy replays those codes directly to the UI.

$3

- Single-agent focus – the React package ships UI primitives only. Handoffs/agent orchestration are expected to happen server-side; surface them to the UI by emitting intermediary messages or tool cards.
- Transparent tool cards – instead of wrapping tool execution in SDK magic, every tool call is rendered verbatim (input/output/error) unless you override it with a renderer. This favors auditing over abstraction.
- Bring-your-own loop – Multiplayer does not expose a runner like run(agent, prompt). Your backend decides when to call tools, switch contexts, or stop the loop, and the UI simply mirrors whatever the transport returns.

If you need OpenAI-style agent handoffs, build them into your backend transport first—once your API emits the intermediate messages/tool calls, the React package will visualize them without extra work.

Direct vs proxy transports

| Mode | Recommended when | Notes |
| -------- | -------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- |
| proxy | The host app can reach Multiplayer's backend (preferred). | Enables background agents, artifacts, storage APIs, and server-side tool execution. |
| direct | On-prem or hybrid deployments that must talk straight to a provider. | The library stores chats in-memory and calls OpenRouter/OpenAI style APIs. Ideal for single-page use cases or POCs. |

Runtime transport config updates (headers, API keys, baseUrl)

The runtime is intentionally created once inside AgentProvider and reused. For production integrations you often need to update transport headers at runtime (auth refresh, tenant switch, tracing).

Use the runtime APIs instead of trying to mutate the original config object.

$3

`ts
// Replace (or clear) the headers map
runtime.setTransportHeaders({ Authorization: Bearer ${token}, 'x-tenant': tenantId })
runtime.setTransportHeaders(undefined) // clears

// Merge/override a few keys
runtime.mergeTransportHeaders({ 'x-trace-id': traceId })

// Remove keys
runtime.removeTransportHeaders(['Authorization', 'x-trace-id'])
`

$3

`ts
runtime.updateTransportConfig({
...runtime.config.transport,
// proxy example
headers: { ...(runtime.config.transport.headers ?? {}), Authorization: Bearer ${token} }
})
`

$3

- Proxy mode Socket.IO reconnects when apiKey / headers / baseUrl change so realtime updates stay authorized.
- In-flight requests/streams are not retroactively updated; changes apply to subsequent requests.
- Switching transport.mode (proxy ↔ direct) replaces the transport instance (active streams are cancelled).

Extending the UI

- Drop in your own message renderer by composing AgentMessageList and handing it custom markdown/render logic.
- Compose AgentComposer, AgentToolShelf, or AgentSidebar individually when embedding within an existing layout.
- Use useAgentRuntime() + useAgentStore() to build custom widgets (e.g., active-agent dashboards, analytics, or workspace switchers).

Roadmap

- SSE-first streaming pathway once the backend contract is finalized.
- Background agent monitor panel (list/stop/resume) backed by /agents` APIs.
- Pluggable artifact viewers (render notebooks, diff viewers, etc.).
- Cypress/storybook fixtures to harden component behaviors.