camera.ui rust detector
npm install @camera.ui/rust-detectorA high-performance Rust-based motion detection library with GPU acceleration using WebGPU, optimized for real-time video surveillance and computer vision applications.
- 🚀 GPU Acceleration - Hardware-accelerated motion detection using WebGPU compute shaders
- 📹 Motion Detection - Advanced frame differencing with Gaussian blur and morphological operations
- ⚡ SIMD Optimization - CPU-based SIMD acceleration for fallback operations
- 🎯 Bounding Box Detection - Automatic detection and tracking of moving objects
- 🔧 Multi-Backend Support - Vulkan, Metal, DirectX12, and OpenGL
- 📊 Real-time Performance - Optimized for streaming and live video processing
- 🔄 Adaptive Processing - Automatic fallback from GPU to CPU when needed
- 🎛️ Configurable Parameters - Adjustable threshold, blur, and dilation settings
``bash`
npm install @camera.ui/rust-detector
`typescript
import { ImageProcessor } from '@camera.ui/rust-detector';
const processor = new ImageProcessor(1920, 1080);
// Process frame and detect motion
const boundingBoxes = processor.processImage(
frameData, // Uint8Array grayscale frame
30, // threshold (0-255)
5, // blur kernel size
3, // dilation size
100 // minimum area
);
// Each bounding box: [x, y, x+width, y+height]
boundingBoxes.forEach(box => {
console.log(Motion detected at: x=${box[0]}, y=${box[1]}, w=${box[2]-box[0]}, h=${box[3]-box[1]});`
});
`typescript
import { GpuImageProcessor } from '@camera.ui/rust-detector';
const processor = new GpuImageProcessor(1920, 1080);
// Initialize with preferred GPU backend
const adapterDetails = await processor.initialize('vulkan');
console.log(Using GPU: ${adapterDetails.name} (${adapterDetails.backend}));
// Process frame with GPU acceleration
const boundingBoxes = processor.processImage(
frameData, // grayscale frame buffer
30, // threshold
5, // blur kernel size
3, // dilation size
100, // minimum area
false // debug mode (optional)
);
`
`typescript
import { GpuImageProcessor, ImageProcessor, getGpuBackends } from '@camera.ui/rust-detector';
class MotionDetector {
private gpuProcessor?: GpuImageProcessor;
private cpuProcessor: ImageProcessor;
private useGpu = false;
constructor(width: number, height: number) {
this.cpuProcessor = new ImageProcessor(width, height);
}
async initialize(preferredBackend: string = 'auto') {
try {
const availableBackends = getGpuBackends();
console.log('Available GPU backends:', availableBackends);
this.gpuProcessor = new GpuImageProcessor(1920, 1080);
const adapter = await this.gpuProcessor.initialize(preferredBackend);
console.log(GPU acceleration enabled: ${adapter.name});
this.useGpu = true;
return adapter;
} catch (error) {
console.warn('GPU initialization failed, falling back to CPU:', error);
this.useGpu = false;
return null;
}
}
detectMotion(frame: Uint8Array, config: MotionConfig) {
const processor = this.useGpu ? this.gpuProcessor : this.cpuProcessor;
if (!processor?.initialized) {
throw new Error('Processor not initialized');
}
return processor.processImage(
frame,
config.threshold,
config.blurKernel,
config.dilationSize,
config.minArea,
config.debug
);
}
cleanup() {
this.gpuProcessor?.resetState();
this.cpuProcessor.resetState();
}
}
interface MotionConfig {
threshold: number; // Motion sensitivity (0-255)
blurKernel: number; // Gaussian blur kernel size
dilationSize: number; // Morphological dilation size
minArea: number; // Minimum area for motion detection
debug?: boolean; // Enable debug output
}
`
#### Initialization
`typescript`
class GpuImageProcessor {
readonly processorType: string; // "GPU"
readonly initialized: boolean;
constructor(width: number, height: number);
// Initialize GPU with preferred backend
initialize(preferredBackend?: string): Promise
// Reset GPU state and free resources
resetState(): void;
}
#### Motion Detection
`typescript`
// GPU-accelerated motion detection
processImage(
frame: Buffer, // Grayscale frame data
threshold: number, // Motion threshold (0-255)
kernelSize: number, // Gaussian blur kernel size
dilateSize: number, // Morphological dilation size
minArea: number, // Minimum area for detection
debug?: boolean // Enable debug output
): number[][]; // Array of bounding boxes [x1, y1, x2, y2]
#### Initialization
`typescript`
class ImageProcessor {
readonly processorType: string; // "CPU"
readonly initialized: boolean;
constructor(width: number, height: number);
// Reset processor state
resetState(): void;
}
#### Motion Detection
`typescript`
// CPU-based motion detection
processImage(
frame: Uint8Array, // Grayscale frame data
threshold: number, // Motion threshold (0-255)
kernelSize: number, // Gaussian blur kernel size
dilateSize: number, // Morphological dilation size
minArea: number // Minimum area for detection
): number[][]; // Array of bounding boxes [x1, y1, x2, y2]
#### GPU Backend Options
`typescript
type GpuBackend = 'vulkan' | 'metal' | 'directx12' | 'opengl' | 'auto';
// Get available GPU backends
getGpuBackends(): string[];
`
#### Adapter Details
`typescript`
interface AdapterDetails {
name: string; // GPU name
vendor: number; // Vendor ID
device: number; // Device ID
deviceType: string; // Device type
driver: string; // Driver name
driverInfo: string; // Driver info
backend: string; // Backend type
}
#### Bounding Box Format
`typescript``
type BoundingBox = [number, number, number, number]; // [x1, y1, x2, y2]
1. Gaussian Blur - Reduces noise and smooths the image
2. Frame Differencing - Compares current frame with previous blurred frame
3. Thresholding - Converts differences to binary motion mask
4. Morphological Dilation - Fills gaps and expands motion regions
5. Connected Component Analysis - Finds connected motion regions
6. Bounding Box Extraction - Calculates bounding boxes for each region
The GPU implementation uses WebGPU compute shaders with optimized workgroup sizes:
- Horizontal Blur Pass - Separable Gaussian blur (horizontal)
- Vertical Blur Pass - Separable Gaussian blur (vertical)
- Difference & Dilate Pass - Combined frame differencing and morphological dilation
The CPU implementation uses SIMD optimization for:
- Frame Differencing - Vectorized pixel comparison
- Connected Components - Efficient flood-fill algorithm
- Gaussian Blur - SIMD-accelerated convolution
- Windows - x64 (DirectX12, Vulkan, OpenGL)
- macOS - Universal (Metal, Vulkan via MoltenVK)
- Linux - x64, ARM64 (Vulkan, OpenGL)
- FreeBSD - x64 (Vulkan, OpenGL)
| Platform | Vulkan | Metal | DirectX12 | OpenGL |
|----------|--------|-------|-----------|--------|
| Windows | ✅ | ❌ | ✅ | ✅ |
| macOS | ✅* | ✅ | ❌ | ✅ |
| Linux | ✅ | ❌ | ❌ | ✅ |
| FreeBSD | ✅ | ❌ | ❌ | ✅ |
*Vulkan on macOS requires MoltenVK
- Modern GPU with WebGPU support (for GPU acceleration)
- Platform-specific requirements:
- Windows: DirectX 12 or Vulkan drivers
- macOS: Metal support (macOS 10.14+)
- Linux: Vulkan or OpenGL drivers
Contributions are welcome! Please read our contributing guidelines and submit pull requests to our repository.
MIT
---
*Part of the camera.ui ecosystem - A comprehensive camera management solution