Client-side SAM (Segment Anything Model) for click-to-segment with WebGPU acceleration
npm install sam-webClient-side Segment Anything Model (SAM) for the browser with WebGPU acceleration.
Click-to-segment in 3 lines of code. Works with any web framework (React, Vue, vanilla JS).
- 🚀 WebGPU Acceleration - Fast inference with automatic CPU fallback
- 📦 Two Models - MobileSAM (fast, 45MB) and SAM2 (accurate, 151MB)
- 🎯 Simple API - Encode once, segment many times
- 💾 Model Caching - OPFS caching for instant subsequent loads
- 🔧 Framework Agnostic - Works with React, Vue, Svelte, vanilla JS
- 📐 Normalized Coordinates - Use 0-1 coordinates, not pixels
``bash`
npm install sam-web onnxruntime-web
`typescript
import { SAMClient } from 'sam-web';
// Create client with model choice
const sam = new SAMClient({
model: 'mobilesam', // or 'sam2' for higher quality
onProgress: (stage) => console.log(stage), // 'downloading' | 'loading' | 'encoding' | 'decoding' | 'ready'
});
// Initialize worker (required)
await sam.initialize(
new URL('sam-web/worker', import.meta.url)
);
// Load and encode image (one-time per image, ~300-700ms)
await sam.setImage(imageElement);
// Click to segment (fast, ~50ms per click)
const mask = await sam.segment({
points: [
{ x: 0.5, y: 0.5, label: 1 }, // foreground point (normalized 0-1)
],
});
// Use the result
console.log(mask.score); // IoU confidence
console.log(mask.bitmap); // ImageBitmap for canvas
console.log(mask.data); // Float32Array raw mask
console.log(mask.bounds); // { x, y, width, height } normalized
// Cleanup
sam.dispose();
`
Main class for click-to-segment functionality.
#### Constructor
`typescript`
new SAMClient(options?: SAMClientOptions)
Options:
- model: Model to use - 'mobilesam' | 'sam2' | 'sam2_tiny' | custom ModelConfigdevice
- : Preferred device - 'webgpu' | 'cpu' | 'auto' (default: 'auto')onProgress
- : Callback for progress updates
#### Methods
##### initialize(workerUrl: URL | string): Promise
Initialize the worker. Required before using setImage or segment.
`typescript`
await sam.initialize(
new URL('sam-web/worker', import.meta.url)
);
##### setImage(image: ImageInput): Promise
Load and encode an image. Call once per image.
`typescript`
// Accepts: HTMLImageElement, HTMLCanvasElement, ImageBitmap, ImageData
await sam.setImage(document.getElementById('myImage'));
##### segment(options: SegmentOptions): Promise
Segment based on point/box prompts.
`typescript
const mask = await sam.segment({
// Point prompts (normalized 0-1 coordinates)
points: [
{ x: 0.5, y: 0.5, label: 1 }, // foreground
{ x: 0.2, y: 0.2, label: 0 }, // background
],
// Optional: box prompt
box: { x1: 0.1, y1: 0.1, x2: 0.9, y2: 0.9 },
// Optional: previous mask for refinement
previousMask: previousResult,
});
`
##### dispose(): void
Cleanup resources.
Result from segment():
`typescript`
interface SegmentResult {
bitmap: ImageBitmap; // For canvas rendering
data: Float32Array; // Raw mask (0-1 values)
shape: [number, number]; // [height, width]
score: number; // IoU confidence
bounds: { // Normalized bounding box
x: number;
y: number;
width: number;
height: number;
};
}
`typescript
// Check browser capabilities
const caps = await SAMClient.checkCapabilities();
// { webgpu: true, opfs: true, workers: true, recommended: 'sam2_tiny' }
// Get available models
const models = SAMClient.getAvailableModels();
`
| Model | Size | Encode Time | Quality | Best For |
|-------|------|-------------|---------|----------|
| mobilesam | 45 MB | ~345ms | Good | Interactive use, mobile |sam2_tiny
| | 151 MB | ~700ms | Better | Accuracy-critical tasks |
`tsx
import { useEffect, useRef, useState } from 'react';
import { SAMClient } from 'sam-web';
function SegmentImage({ imageSrc }) {
const samRef = useRef
const [ready, setReady] = useState(false);
useEffect(() => {
const sam = new SAMClient({ model: 'mobilesam' });
samRef.current = sam;
sam.initialize(
new URL('sam-web/worker', import.meta.url)
).then(() => setReady(true));
return () => sam.dispose();
}, []);
const handleImageLoad = async (img: HTMLImageElement) => {
if (samRef.current && ready) {
await samRef.current.setImage(img);
}
};
const handleClick = async (e: React.MouseEvent
if (!samRef.current) return;
const rect = e.currentTarget.getBoundingClientRect();
const x = (e.clientX - rect.left) / rect.width;
const y = (e.clientY - rect.top) / rect.height;
const mask = await samRef.current.segment({
points: [{ x, y, label: 1 }],
});
// Draw mask.bitmap to canvas
};
return (
);
}
`
`html`
| Feature | Chrome | Firefox | Safari | Edge |
|---------|--------|---------|--------|------|
| WebGPU | ✅ 113+ | ⚠️ Flag | ❌ | ✅ 113+ |
| CPU Fallback | ✅ | ✅ | ✅ | ✅ |
| OPFS Cache | ✅ | ✅ | ⚠️ Partial | ✅ |
For Next.js projects, add this webpack configuration to next.config.js:
`javascript
const path = require('path');
module.exports = {
webpack: (config) => {
config.resolve.alias = {
...config.resolve.alias,
'onnxruntime-web/all': path.join(
__dirname,
'node_modules/onnxruntime-web/dist/ort.all.bundle.min.mjs'
),
};
return config;
},
};
`
Note: For Next.js, use a local worker file instead of importing from sam-web/worker directly. See sam-web-demo for a complete example.
`typescript
import { SAMClient, ModelConfig } from 'sam-web';
const customConfig: ModelConfig = {
id: 'my-model',
name: 'My Custom SAM',
description: 'Custom SAM model',
encoderUrl: 'https://my-cdn.com/encoder.onnx',
decoderUrl: 'https://my-cdn.com/decoder.onnx',
imageSize: { w: 1024, h: 1024 },
maskSize: { w: 256, h: 256 },
modelType: 'sam2',
encoderInputName: 'image',
useBatchDimension: true,
tensorFormat: 'CHW',
};
const sam = new SAMClient({ model: customConfig });
`
For advanced control, use the core classes directly:
`typescript
import { SAM2, SAMWorker } from 'sam-web';
// Direct ONNX inference (no worker)
const sam2 = new SAM2('mobilesam');
await sam2.downloadModels();
await sam2.createSessions();
await sam2.encodeImage(tensor);
const result = await sam2.decode(points);
`
1. Encode Once - Call setImage() once, then segment() multiple timespreviousMask` for better results
2. Use MobileSAM - 2x faster encoding, sufficient for most uses
3. Preload Models - Models are cached after first download
4. Iterative Refinement - Use
MIT