ui-agent-annotation

AI-powered annotation toolkit for React and React Native apps. Enables component inspection, annotation, and screenshot capture for AI-assisted development workflows.

Demo


▶️ Click the image above to watch the demo video

Features

- 🔍 Component Inspector - Hover/tap to identify React components
- 📝 Annotations - Add notes to components for AI context
- 📷 Screenshots - Capture component screenshots (web + native)
- 🖱️ Draggable Toolbar - Non-intrusive floating UI
- 🌐 Cross-platform - Works on Web, iOS, and Android
- 🧩 Chrome Extension - Annotate any website!

Installation

``bash


Using npm

npm install ui-agent-annotation

Using yarn

yarn add ui-agent-annotation

Using bun

bun add ui-agent-annotation
`

$3

For Web:
`bash
npm install framer-motion html2canvas lucide-react
`

For React Native:
`bash
npm install react-native-view-shot

Optional for clipboard support

npm install expo-clipboard

Optional for saving to camera roll

npm install expo-media-library
`

Usage

$3

`tsx
import { AgentAnnotationProvider } from 'ui-agent-annotation';

function App() {
return (



);
}
`

$3

`tsx
import { AgentAnnotationProvider } from 'ui-agent-annotation/native';
// or
import { AgentAnnotationProvider } from 'ui-agent-annotation';

function App() {
return (



);
}
`

API

$3

#### AgentAnnotationProvider

Wraps your app to provide annotation functionality.

`tsx

{children}

`

#### Toolbar

The floating toolbar with annotation controls. Automatically rendered by the provider.

#### Highlighter

Component inspector overlay. Highlights components on hover (web) or tap (native).

Native-specific props:
`tsx
interface HighlighterProps {
onInspect?: (event: GestureResponderEvent) => {
name: string;
rect: { x: number; y: number; width: number; height: number };
} | null;
}
`

#### ErrorBoundary

A customizable error boundary component with copy-to-clipboard functionality for error reports.

`tsx
import { ErrorBoundary } from 'ui-agent-annotation';




`

Props:

| Prop | Type | Default | Description |
|------|------|---------|-------------|
| children | ReactNode | - | Child components to wrap |
| fallback | React.ComponentType | - | Fully custom fallback component |
| fallbackElement | ReactNode | - | Simple static fallback element |
| title | string | "Something went wrong" | Error page title |
| subtitle | string | "An unexpected error..." | Error description |
| showErrorDetails | boolean | true | Show/hide error stack trace |
| showCopyButton | boolean | true | Show/hide copy error button |
| showRetryButton | boolean | true | Show/hide retry button |
| retryButtonLabel | string | "Try Again" | Retry button text |
| copyButtonLabel | string | "Copy Error Details" | Copy button text |
| customButtons | ErrorButtonConfig[] | - | Additional custom buttons |
| containerStyle | CSSProperties | - | Override container styles |
| errorDetailsStyle | CSSProperties | - | Override error details styles |
| onError | (error, errorInfo) => void | - | Callback when error occurs |
| onReset | () => void | - | Callback when reset/retry |

Custom Fallback Example:

`tsx
fallback={({ error, errorInfo, reset, copyToClipboard, copied }) => (



)}
>


`

Custom Buttons Example:

`tsx
title="Application Error"
subtitle="Please try again or contact support"
customButtons={[
{
label: 'Go Home',
onClick: () => window.location.href = '/',
style: { backgroundColor: '#4CAF50' }
},
{
label: 'Contact Support',
onClick: () => window.open('mailto:support@example.com')
}
]}
>


`

Minimal Error Page:

`tsx
showErrorDetails={false}
showCopyButton={false}
title="Oops!"
subtitle="Something went wrong. Please refresh the page."
>


`

$3

#### useAgentAnnotation()

Access annotation state and dispatch actions.

`tsx
const { state, dispatch } = useAgentAnnotation();

// State shape
interface State {
mode: 'disabled' | 'inspecting';
annotations: Annotation[];
hoveredElement: any;
hoveredComponentInfo: { name: string } | null;
isMinimized: boolean;
showList: boolean;
}

// Actions
dispatch({ type: 'SET_MODE', payload: 'inspecting' });
dispatch({ type: 'ADD_ANNOTATION', payload: { id, componentName, note, timestamp } });
dispatch({ type: 'REMOVE_ANNOTATION', payload: annotationId });
dispatch({ type: 'CLEAR_ALL_ANNOTATIONS' });
dispatch({ type: 'TOGGLE_MINIMIZED' });
dispatch({ type: 'TOGGLE_LIST' });
`

$3

#### captureScreenshot(element, options)

Capture a screenshot of an element/view.

Web:
`tsx
import { captureScreenshot } from 'ui-agent-annotation';

const result = await captureScreenshot(htmlElement, {
scale: 2,
copyToClipboard: true,
download: false,
});
`

Native:
`tsx
import { captureScreenshot } from 'ui-agent-annotation/native';

const viewRef = useRef(null);

const result = await captureScreenshot(viewRef, {
scale: 2,
format: 'png',
copyToClipboard: true,
saveToCameraRoll: false,
});
`

#### Platform Detection

`tsx
import { isWeb, isNative, getPlatform, isTauriEnv } from 'ui-agent-annotation';

if (isWeb) {
// Web-specific code
}

if (isNative) {
// React Native-specific code
}

const platform = getPlatform(); // 'web' | 'ios' | 'android' | 'native'
`

Toolbar Controls

| Button | Description |
|--------|-------------|
| ⋮⋮ | Drag handle - move the toolbar |
| 👆/🚫 | Toggle inspection mode |
| 📋 | View annotation list |
| 📋 | Copy annotations as JSON |
| ➖/⬜ | Minimize/expand toolbar |

Workflow

1. Enable Inspection - Click the inspection toggle
2. Select Component - Hover (web) or tap (native) on a component
3. Lock Selection - Click to lock the selection (web)
4. Add Annotation - Click the + button to add a note
5. Copy Annotations - Copy all annotations as JSON for AI context

Tauri Support

The package includes special handling for Tauri apps (web apps running in Tauri):

`tsx
import { isTauriEnv, captureScreenshot } from 'ui-agent-annotation';

if (isTauriEnv()) {
// Uses @tauri-apps/plugin-clipboard-manager for clipboard
}
`

Chrome Extension

Use the annotation toolkit on any website as a Chrome extension!

$3

`bash
cd packages/ui-ai-anotation
bun install
bun run build:extension
`

$3

1. Go to chrome://extensions/
2. Enable Developer mode (toggle in top right)
3. Click Load unpacked
4. Select the dist/` folder

$3

1. Click the extension icon to open the popup
2. Toggle "Enable Inspector" to activate on the current page
3. Hover over elements to highlight them
4. Click to add annotations
5. Use the floating toolbar to manage annotations

See extension/README.md for more details.

License

MIT